Added a variant of the test that uses HashiCorp Vault to manage te

Certificate Authorities.
2026-10-17 11:55:27 +02:00
1 parent 7f07e47138
commit 777b86ce33
@@ -7,3 +7,7 @@ package-lock.json
 *.downloaded.md
 iam_policy.yaml
 env.sh
+vault-data/
+vault-init.txt
+vault.hcl
+*.csr
@@ -5,6 +5,7 @@ This short guide provides step by step instructions how to replace a JSON key wi
 ## Prerequisites

 - `openssl` 3.x
+- `vault` (only for the Vault based CA in step 1a)
 - Node.js with `npm`
 - `gcloud` authenticated as an administrator of the target project

@@ -50,6 +51,97 @@ The `.sh` files are meant as references for copying and pasting the commands one
 bash make_certs.sh
 ```

+## 1a. Alternative: create the certificates with Vault
+
+HashiCorp Vault Community Edition can replace the OpenSSL based CA. The PKI secrets engine hosts the root CA and the issuing CA, and a role issues the client certificate. The commands below produce the same files as `make_certs.sh`: `ca_cert.pem`, `issuing_ca_cert.pem`, `client_key.pem` and `client_cert.pem`. Continue with step 2 afterwards.
+
+Prerequisites: the `vault` command installed on the workstation.
+
+### Start a local Vault server
+
+Vault dev mode keeps all data in memory, and a dev server cannot be restarted on existing file storage. The local server therefore runs in normal mode with file storage in the gitignored `vault-data` directory.
+
+Create `vault.hcl`, replacing the path with the absolute path of the `vault-data` directory:
+
+```hcl
+storage "file" {
+  path = "/path/to/repository/vault-data"
+}
+
+listener "tcp" {
+  address     = "127.0.0.1:8200"
+  tls_disable = true
+}
+
+api_addr      = "http://127.0.0.1:8200"
+disable_mlock = true
+ui            = true
+```
+
+Start the server in a separate terminal:
+
+```bash
+vault server -config=vault.hcl
+```
+
+The web UI is available at `http://127.0.0.1:8200/ui`.
+
+Initialize the server once with a single unseal key, unseal it, and set `VAULT_TOKEN` to the generated root token. Replace `<initial-root-token>` with the initial root token printed in `vault-init.txt`:
+
+```bash
+export VAULT_ADDR=http://127.0.0.1:8200
+vault operator init -key-shares=1 -key-threshold=1 > vault-init.txt
+vault operator unseal
+export VAULT_TOKEN=<initial-root-token>
+vault status
+```
+
+`vault operator unseal` asks for the unseal key from `vault-init.txt`. After every server restart, run `vault operator unseal` again and set `VAULT_ADDR` and `VAULT_TOKEN` in new terminals. The data is kept in `vault-data`.
+
+> **Note:** The unseal key and the root token stored in a file are suitable only for a local, disposable server.
+
+### Configure the CA hierarchy
+
+`vault_configure.sh` performs the following steps in the terminal where `VAULT_ADDR` and `VAULT_TOKEN` are set:
+
+1. Enables the PKI secrets engine for the root CA and for the issuing CA, and sets the maximum TTLs.
+2. Generates the root CA inside Vault.
+3. Generates the issuing CA key and CSR, signs it with the root CA, and installs the signed certificate.
+4. Creates a role that issues client authentication certificates.
+5. Exports the root CA and the issuing CA certificates (`ca_cert.pem`, `issuing_ca_cert.pem`).
+
+```bash
+bash vault_configure.sh
+```
+
+### Create the client certificate
+
+`vault_make_client_cert.sh` performs the following steps in the same terminal:
+
+1. Generates the client key and CSR locally with `openssl`. The key is never sent to Vault.
+2. Signs the client CSR with the issuing CA. The certificate subject CN (`test-client`) becomes the WiF subject.
+
+```bash
+bash vault_make_client_cert.sh
+```
+
+The client certificate is valid for 365 days, has the Digital Signature and Key Encipherment key usages, and the TLS Web Client Authentication extended key usage. Unlike the certificate from `make_certs.sh`, it has no `basicConstraints` extension.
+
+### Update a WiF provider created with the OpenSSL CA
+
+If the provider was already created by `make_wif.sh` with the OpenSSL based CA, `vault_gcp_update.sh` switches it to the Vault CA. The client certificate keeps the subject CN `test-client`, so the service account binding and `cred.json` stay valid. The script performs the following steps:
+
+1. Generates `trust_store.yaml` from the Vault CA certificates with `make_trust_store.mjs`.
+2. Replaces the trust store of the existing X.509 provider.
+3. Shows the provider configuration.
+4. Tests the federation with the client certificate issued by Vault.
+
+```bash
+bash vault_gcp_update.sh
+```
+
+For a new setup, continue with step 2 instead.
+
 ## 2. Create the trust store

 `make_trust_store.mjs` creates `trust_store.yaml`, a YAML file that holds the root CA as the trust anchor and the issuing CA as an intermediate CA. It uses the `yaml` npm package, declared in `package.json`:
@@ -81,18 +173,27 @@ The PEM files must contain only the certificate block. A trailing newline after
 4. Grants the service account read access to the project.
 5. Writes `cred.json`.
 6. Creates a storage bucket and grants the service account read and write access to its objects.
-7. Enables Data Access audit logs for Cloud Storage.
-8. Creates a separate gcloud configuration and logs in with `cred.json`.
-9. Checks the access to the project and the bucket, including an upload and a download of a file.
-10. Switches back to the original gcloud configuration.
-11. Makes the `public/` managed folder of the bucket publicly readable.
-12. Lists the audit log of the service account operations.
-13. Optionally switches back to the service account configuration.
+7. Makes the `public/` managed folder of the bucket publicly readable.
+8. Enables Data Access audit logs for Cloud Storage.

 ```bash
 bash make_wif.sh
 ```

+## 4. Test the federation
+
+`test_wif.sh` performs the following steps:
+
+1. Creates a separate gcloud configuration and logs in with `cred.json`.
+2. Checks the access to the project and the bucket, including an upload and a download of a file.
+3. Switches back to the original gcloud configuration.
+4. Lists the audit log of the service account operations.
+5. Optionally switches back to the service account configuration.
+
+```bash
+bash test_wif.sh
+```
+
 Data Access audit logs are enabled for the whole project, not per bucket, by editing the project IAM policy. `add_audit_config.mjs` adds the `auditConfigs` section to the policy file fetched by gcloud.

 The audit log lists only Cloud Storage operations of the service account. Resource Manager calls such as `projects describe` are not covered by the Data Access configuration.
@@ -49,50 +49,13 @@ gcloud storage buckets add-iam-policy-binding "gs://${BUCKET}" \
    --member="serviceAccount:${SA_EMAIL}" \
    --role=roles/storage.objectUser

-# Enable Data Access audit logs for Cloud Storage (project-wide, not per bucket)
-gcloud projects get-iam-policy "${PROJECT_ID}" --format=yaml > iam_policy.yaml
-node add_audit_config.mjs
-gcloud projects set-iam-policy "${PROJECT_ID}" iam_policy.yaml
-
-# Create a new gcloud configuration for the workload identity federation test
-gcloud config configurations create wif-test
-
-# Authenticate gcloud with the credentials configuration file
-gcloud auth login --cred-file=cred.json
-
-# Set the project
-gcloud config set project "${PROJECT_ID}"
-
-# Test the authentication by listing the project details
-gcloud auth list
-gcloud projects describe "${PROJECT_ID}"
-gcloud projects get-iam-policy "${PROJECT_ID}" \
-    --flatten="bindings[].members" \
-    --format="table(bindings.role, bindings.members)"
-
-# Test the bucket access by uploading and downloading a file
-gcloud storage ls "gs://${BUCKET}"
-gcloud storage cp README.md "gs://${BUCKET}/README.md"
-gcloud storage ls "gs://${BUCKET}"
-gcloud storage cp "gs://${BUCKET}/README.md" README.downloaded.md
-cmp README.md README.downloaded.md
-
-# Then, switch back to your original account
-gcloud config configurations activate default
-
 # Create a managed folder and make the objects in it publicly readable
 gcloud storage managed-folders create "gs://${BUCKET}/public/"
 gcloud storage managed-folders add-iam-policy-binding "gs://${BUCKET}/public/" \
    --member=allUsers \
    --role=roles/storage.objectViewer

-# List the audit log of the operations made by the service account
-gcloud logging read \
-    "protoPayload.authenticationInfo.principalEmail=\"${SA_EMAIL}\"" \
-    --project="${PROJECT_ID}" \
-    --freshness=1h \
-    --order=asc \
-    --format="table(timestamp, protoPayload.serviceName, protoPayload.methodName, protoPayload.authenticationInfo.principalEmail:label=EMAIL, protoPayload.authenticationInfo.principalSubject:label=SUBJECT, protoPayload.authenticationInfo.serviceAccountDelegationInfo[0].principalSubject:label=DELEGATED_BY)"
-
-# optionally switch back to the service account for further testing
-# gcloud config configurations activate wif-test
+# Enable Data Access audit logs for Cloud Storage (project-wide, not per bucket)
+gcloud projects get-iam-policy "${PROJECT_ID}" --format=yaml > iam_policy.yaml
+node add_audit_config.mjs
+gcloud projects set-iam-policy "${PROJECT_ID}" iam_policy.yaml
@@ -0,0 +1,38 @@
+source ./env.sh
+
+# Create a new gcloud configuration for the workload identity federation test
+gcloud config configurations create wif-test
+
+# Authenticate gcloud with the credentials configuration file
+gcloud auth login --cred-file=cred.json
+
+# Set the project
+gcloud config set project "${PROJECT_ID}"
+
+# Test the authentication by listing the project details
+gcloud auth list
+gcloud projects describe "${PROJECT_ID}"
+gcloud projects get-iam-policy "${PROJECT_ID}" \
+    --flatten="bindings[].members" \
+    --format="table(bindings.role, bindings.members)"
+
+# Test the bucket access by uploading and downloading a file
+gcloud storage ls "gs://${BUCKET}"
+gcloud storage cp README.md "gs://${BUCKET}/README.md"
+gcloud storage ls "gs://${BUCKET}"
+gcloud storage cp "gs://${BUCKET}/README.md" README.downloaded.md
+cmp README.md README.downloaded.md
+
+# Then, switch back to your original account
+gcloud config configurations activate default
+
+# List the audit log of the operations made by the service account
+gcloud logging read \
+    "protoPayload.authenticationInfo.principalEmail=\"${SA_EMAIL}\"" \
+    --project="${PROJECT_ID}" \
+    --freshness=1h \
+    --order=asc \
+    --format="table(timestamp, protoPayload.serviceName, protoPayload.methodName, protoPayload.authenticationInfo.principalEmail:label=EMAIL, protoPayload.authenticationInfo.principalSubject:label=SUBJECT, protoPayload.authenticationInfo.serviceAccountDelegationInfo[0].principalSubject:label=DELEGATED_BY)"
+
+# optionally switch back to the service account for further testing
+# gcloud config configurations activate wif-test
@@ -0,0 +1,49 @@
+# Enable the PKI secrets engine for the root CA and for the issuing CA
+vault secrets enable pki
+vault secrets enable -path=pki_int pki
+
+# Set the maximum TTLs of the CAs
+vault secrets tune -max-lease-ttl=87600h pki
+vault secrets tune -max-lease-ttl=43800h pki_int
+
+# Generate the root CA inside Vault
+vault write pki/root/generate/internal \
+    common_name="Test CA" \
+    issuer_name=root \
+    key_type=rsa \
+    key_bits=4096 \
+    ttl=87600h \
+    max_path_length=1
+
+# Generate the issuing CA key and CSR
+vault write -field=csr pki_int/intermediate/generate/internal \
+    common_name="Issuing Test CA" \
+    key_type=rsa \
+    key_bits=4096 > issuing_ca.csr
+
+# Sign the issuing CA CSR with the root CA
+vault write -field=certificate pki/root/sign-intermediate \
+    csr=@issuing_ca.csr \
+    format=pem_bundle \
+    ttl=43800h \
+    max_path_length=0 > issuing_ca_signed.pem
+
+# Install the signed issuing CA certificate
+vault write pki_int/intermediate/set-signed certificate=@issuing_ca_signed.pem
+
+# Create a role that issues client authentication certificates
+vault write pki_int/roles/wif-client \
+    allow_any_name=true \
+    enforce_hostnames=false \
+    server_flag=false \
+    client_flag=true \
+    key_type=rsa \
+    key_bits=4096 \
+    key_usage=DigitalSignature,KeyEncipherment \
+    ttl=8760h \
+    max_ttl=8760h \
+    no_store=true
+
+# Export the CA certificates
+vault read -field=certificate pki/cert/ca > ca_cert.pem
+vault read -field=certificate pki_int/cert/ca > issuing_ca_cert.pem
@@ -0,0 +1,20 @@
+source ./env.sh
+
+# Generate the trust store from the Vault CA certificates
+node make_trust_store.mjs
+
+# Replace the trust store of the existing X.509 provider
+gcloud iam workload-identity-pools providers update-x509 "${PROVIDER_ID}" \
+  --project="${PROJECT_ID}" \
+  --location=global \
+  --workload-identity-pool="${POOL_ID}" \
+  --trust-store-config-path=trust_store.yaml
+
+# Show the provider configuration
+gcloud iam workload-identity-pools providers describe "${PROVIDER_ID}" \
+  --project="${PROJECT_ID}" \
+  --location=global \
+  --workload-identity-pool="${POOL_ID}"
+
+# Test the federation with the client certificate issued by Vault
+gcloud storage ls "gs://${BUCKET}" --configuration=wif-test
@@ -0,0 +1,13 @@
+# Generate the client key and CSR locally, the key is never sent to Vault
+openssl req \
+    -newkey rsa:4096 \
+    -keyout client_key.pem \
+    -noenc \
+    -subj "/CN=test-client" \
+    -out client.csr
+
+# Sign the client CSR
+vault write -field=certificate pki_int/sign/wif-client \
+    csr=@client.csr \
+    common_name=test-client \
+    ttl=8760h > client_cert.pem