diff --git a/.gitignore b/.gitignore index ab2a144..fc066be 100644 --- a/.gitignore +++ b/.gitignore @@ -7,3 +7,7 @@ package-lock.json *.downloaded.md iam_policy.yaml env.sh +vault-data/ +vault-init.txt +vault.hcl +*.csr diff --git a/README.md b/README.md index 35a6555..fd2fffb 100644 --- a/README.md +++ b/README.md @@ -5,6 +5,7 @@ This short guide provides step by step instructions how to replace a JSON key wi ## Prerequisites - `openssl` 3.x +- `vault` (only for the Vault based CA in step 1a) - Node.js with `npm` - `gcloud` authenticated as an administrator of the target project @@ -50,6 +51,97 @@ The `.sh` files are meant as references for copying and pasting the commands one bash make_certs.sh ``` +## 1a. Alternative: create the certificates with Vault + +HashiCorp Vault Community Edition can replace the OpenSSL based CA. The PKI secrets engine hosts the root CA and the issuing CA, and a role issues the client certificate. The commands below produce the same files as `make_certs.sh`: `ca_cert.pem`, `issuing_ca_cert.pem`, `client_key.pem` and `client_cert.pem`. Continue with step 2 afterwards. + +Prerequisites: the `vault` command installed on the workstation. + +### Start a local Vault server + +Vault dev mode keeps all data in memory, and a dev server cannot be restarted on existing file storage. The local server therefore runs in normal mode with file storage in the gitignored `vault-data` directory. + +Create `vault.hcl`, replacing the path with the absolute path of the `vault-data` directory: + +```hcl +storage "file" { + path = "/path/to/repository/vault-data" +} + +listener "tcp" { + address = "127.0.0.1:8200" + tls_disable = true +} + +api_addr = "http://127.0.0.1:8200" +disable_mlock = true +ui = true +``` + +Start the server in a separate terminal: + +```bash +vault server -config=vault.hcl +``` + +The web UI is available at `http://127.0.0.1:8200/ui`. + +Initialize the server once with a single unseal key, unseal it, and set `VAULT_TOKEN` to the generated root token. Replace `` with the initial root token printed in `vault-init.txt`: + +```bash +export VAULT_ADDR=http://127.0.0.1:8200 +vault operator init -key-shares=1 -key-threshold=1 > vault-init.txt +vault operator unseal +export VAULT_TOKEN= +vault status +``` + +`vault operator unseal` asks for the unseal key from `vault-init.txt`. After every server restart, run `vault operator unseal` again and set `VAULT_ADDR` and `VAULT_TOKEN` in new terminals. The data is kept in `vault-data`. + +> **Note:** The unseal key and the root token stored in a file are suitable only for a local, disposable server. + +### Configure the CA hierarchy + +`vault_configure.sh` performs the following steps in the terminal where `VAULT_ADDR` and `VAULT_TOKEN` are set: + +1. Enables the PKI secrets engine for the root CA and for the issuing CA, and sets the maximum TTLs. +2. Generates the root CA inside Vault. +3. Generates the issuing CA key and CSR, signs it with the root CA, and installs the signed certificate. +4. Creates a role that issues client authentication certificates. +5. Exports the root CA and the issuing CA certificates (`ca_cert.pem`, `issuing_ca_cert.pem`). + +```bash +bash vault_configure.sh +``` + +### Create the client certificate + +`vault_make_client_cert.sh` performs the following steps in the same terminal: + +1. Generates the client key and CSR locally with `openssl`. The key is never sent to Vault. +2. Signs the client CSR with the issuing CA. The certificate subject CN (`test-client`) becomes the WiF subject. + +```bash +bash vault_make_client_cert.sh +``` + +The client certificate is valid for 365 days, has the Digital Signature and Key Encipherment key usages, and the TLS Web Client Authentication extended key usage. Unlike the certificate from `make_certs.sh`, it has no `basicConstraints` extension. + +### Update a WiF provider created with the OpenSSL CA + +If the provider was already created by `make_wif.sh` with the OpenSSL based CA, `vault_gcp_update.sh` switches it to the Vault CA. The client certificate keeps the subject CN `test-client`, so the service account binding and `cred.json` stay valid. The script performs the following steps: + +1. Generates `trust_store.yaml` from the Vault CA certificates with `make_trust_store.mjs`. +2. Replaces the trust store of the existing X.509 provider. +3. Shows the provider configuration. +4. Tests the federation with the client certificate issued by Vault. + +```bash +bash vault_gcp_update.sh +``` + +For a new setup, continue with step 2 instead. + ## 2. Create the trust store `make_trust_store.mjs` creates `trust_store.yaml`, a YAML file that holds the root CA as the trust anchor and the issuing CA as an intermediate CA. It uses the `yaml` npm package, declared in `package.json`: @@ -81,18 +173,27 @@ The PEM files must contain only the certificate block. A trailing newline after 4. Grants the service account read access to the project. 5. Writes `cred.json`. 6. Creates a storage bucket and grants the service account read and write access to its objects. -7. Enables Data Access audit logs for Cloud Storage. -8. Creates a separate gcloud configuration and logs in with `cred.json`. -9. Checks the access to the project and the bucket, including an upload and a download of a file. -10. Switches back to the original gcloud configuration. -11. Makes the `public/` managed folder of the bucket publicly readable. -12. Lists the audit log of the service account operations. -13. Optionally switches back to the service account configuration. +7. Makes the `public/` managed folder of the bucket publicly readable. +8. Enables Data Access audit logs for Cloud Storage. ```bash bash make_wif.sh ``` +## 4. Test the federation + +`test_wif.sh` performs the following steps: + +1. Creates a separate gcloud configuration and logs in with `cred.json`. +2. Checks the access to the project and the bucket, including an upload and a download of a file. +3. Switches back to the original gcloud configuration. +4. Lists the audit log of the service account operations. +5. Optionally switches back to the service account configuration. + +```bash +bash test_wif.sh +``` + Data Access audit logs are enabled for the whole project, not per bucket, by editing the project IAM policy. `add_audit_config.mjs` adds the `auditConfigs` section to the policy file fetched by gcloud. The audit log lists only Cloud Storage operations of the service account. Resource Manager calls such as `projects describe` are not covered by the Data Access configuration. diff --git a/make_wif.sh b/make_wif.sh index a190d01..953af21 100755 --- a/make_wif.sh +++ b/make_wif.sh @@ -49,50 +49,13 @@ gcloud storage buckets add-iam-policy-binding "gs://${BUCKET}" \ --member="serviceAccount:${SA_EMAIL}" \ --role=roles/storage.objectUser -# Enable Data Access audit logs for Cloud Storage (project-wide, not per bucket) -gcloud projects get-iam-policy "${PROJECT_ID}" --format=yaml > iam_policy.yaml -node add_audit_config.mjs -gcloud projects set-iam-policy "${PROJECT_ID}" iam_policy.yaml - -# Create a new gcloud configuration for the workload identity federation test -gcloud config configurations create wif-test - -# Authenticate gcloud with the credentials configuration file -gcloud auth login --cred-file=cred.json - -# Set the project -gcloud config set project "${PROJECT_ID}" - -# Test the authentication by listing the project details -gcloud auth list -gcloud projects describe "${PROJECT_ID}" -gcloud projects get-iam-policy "${PROJECT_ID}" \ - --flatten="bindings[].members" \ - --format="table(bindings.role, bindings.members)" - -# Test the bucket access by uploading and downloading a file -gcloud storage ls "gs://${BUCKET}" -gcloud storage cp README.md "gs://${BUCKET}/README.md" -gcloud storage ls "gs://${BUCKET}" -gcloud storage cp "gs://${BUCKET}/README.md" README.downloaded.md -cmp README.md README.downloaded.md - -# Then, switch back to your original account -gcloud config configurations activate default - # Create a managed folder and make the objects in it publicly readable gcloud storage managed-folders create "gs://${BUCKET}/public/" gcloud storage managed-folders add-iam-policy-binding "gs://${BUCKET}/public/" \ --member=allUsers \ --role=roles/storage.objectViewer -# List the audit log of the operations made by the service account -gcloud logging read \ - "protoPayload.authenticationInfo.principalEmail=\"${SA_EMAIL}\"" \ - --project="${PROJECT_ID}" \ - --freshness=1h \ - --order=asc \ - --format="table(timestamp, protoPayload.serviceName, protoPayload.methodName, protoPayload.authenticationInfo.principalEmail:label=EMAIL, protoPayload.authenticationInfo.principalSubject:label=SUBJECT, protoPayload.authenticationInfo.serviceAccountDelegationInfo[0].principalSubject:label=DELEGATED_BY)" - -# optionally switch back to the service account for further testing -# gcloud config configurations activate wif-test +# Enable Data Access audit logs for Cloud Storage (project-wide, not per bucket) +gcloud projects get-iam-policy "${PROJECT_ID}" --format=yaml > iam_policy.yaml +node add_audit_config.mjs +gcloud projects set-iam-policy "${PROJECT_ID}" iam_policy.yaml diff --git a/test_wif.sh b/test_wif.sh new file mode 100644 index 0000000..34c348e --- /dev/null +++ b/test_wif.sh @@ -0,0 +1,38 @@ +source ./env.sh + +# Create a new gcloud configuration for the workload identity federation test +gcloud config configurations create wif-test + +# Authenticate gcloud with the credentials configuration file +gcloud auth login --cred-file=cred.json + +# Set the project +gcloud config set project "${PROJECT_ID}" + +# Test the authentication by listing the project details +gcloud auth list +gcloud projects describe "${PROJECT_ID}" +gcloud projects get-iam-policy "${PROJECT_ID}" \ + --flatten="bindings[].members" \ + --format="table(bindings.role, bindings.members)" + +# Test the bucket access by uploading and downloading a file +gcloud storage ls "gs://${BUCKET}" +gcloud storage cp README.md "gs://${BUCKET}/README.md" +gcloud storage ls "gs://${BUCKET}" +gcloud storage cp "gs://${BUCKET}/README.md" README.downloaded.md +cmp README.md README.downloaded.md + +# Then, switch back to your original account +gcloud config configurations activate default + +# List the audit log of the operations made by the service account +gcloud logging read \ + "protoPayload.authenticationInfo.principalEmail=\"${SA_EMAIL}\"" \ + --project="${PROJECT_ID}" \ + --freshness=1h \ + --order=asc \ + --format="table(timestamp, protoPayload.serviceName, protoPayload.methodName, protoPayload.authenticationInfo.principalEmail:label=EMAIL, protoPayload.authenticationInfo.principalSubject:label=SUBJECT, protoPayload.authenticationInfo.serviceAccountDelegationInfo[0].principalSubject:label=DELEGATED_BY)" + +# optionally switch back to the service account for further testing +# gcloud config configurations activate wif-test diff --git a/vault_configure.sh b/vault_configure.sh new file mode 100644 index 0000000..e4312e3 --- /dev/null +++ b/vault_configure.sh @@ -0,0 +1,49 @@ +# Enable the PKI secrets engine for the root CA and for the issuing CA +vault secrets enable pki +vault secrets enable -path=pki_int pki + +# Set the maximum TTLs of the CAs +vault secrets tune -max-lease-ttl=87600h pki +vault secrets tune -max-lease-ttl=43800h pki_int + +# Generate the root CA inside Vault +vault write pki/root/generate/internal \ + common_name="Test CA" \ + issuer_name=root \ + key_type=rsa \ + key_bits=4096 \ + ttl=87600h \ + max_path_length=1 + +# Generate the issuing CA key and CSR +vault write -field=csr pki_int/intermediate/generate/internal \ + common_name="Issuing Test CA" \ + key_type=rsa \ + key_bits=4096 > issuing_ca.csr + +# Sign the issuing CA CSR with the root CA +vault write -field=certificate pki/root/sign-intermediate \ + csr=@issuing_ca.csr \ + format=pem_bundle \ + ttl=43800h \ + max_path_length=0 > issuing_ca_signed.pem + +# Install the signed issuing CA certificate +vault write pki_int/intermediate/set-signed certificate=@issuing_ca_signed.pem + +# Create a role that issues client authentication certificates +vault write pki_int/roles/wif-client \ + allow_any_name=true \ + enforce_hostnames=false \ + server_flag=false \ + client_flag=true \ + key_type=rsa \ + key_bits=4096 \ + key_usage=DigitalSignature,KeyEncipherment \ + ttl=8760h \ + max_ttl=8760h \ + no_store=true + +# Export the CA certificates +vault read -field=certificate pki/cert/ca > ca_cert.pem +vault read -field=certificate pki_int/cert/ca > issuing_ca_cert.pem diff --git a/vault_gcp_update.sh b/vault_gcp_update.sh new file mode 100644 index 0000000..c16e4c7 --- /dev/null +++ b/vault_gcp_update.sh @@ -0,0 +1,20 @@ +source ./env.sh + +# Generate the trust store from the Vault CA certificates +node make_trust_store.mjs + +# Replace the trust store of the existing X.509 provider +gcloud iam workload-identity-pools providers update-x509 "${PROVIDER_ID}" \ + --project="${PROJECT_ID}" \ + --location=global \ + --workload-identity-pool="${POOL_ID}" \ + --trust-store-config-path=trust_store.yaml + +# Show the provider configuration +gcloud iam workload-identity-pools providers describe "${PROVIDER_ID}" \ + --project="${PROJECT_ID}" \ + --location=global \ + --workload-identity-pool="${POOL_ID}" + +# Test the federation with the client certificate issued by Vault +gcloud storage ls "gs://${BUCKET}" --configuration=wif-test diff --git a/vault_make_client_cert.sh b/vault_make_client_cert.sh new file mode 100644 index 0000000..cdc6e40 --- /dev/null +++ b/vault_make_client_cert.sh @@ -0,0 +1,13 @@ +# Generate the client key and CSR locally, the key is never sent to Vault +openssl req \ + -newkey rsa:4096 \ + -keyout client_key.pem \ + -noenc \ + -subj "/CN=test-client" \ + -out client.csr + +# Sign the client CSR +vault write -field=certificate pki_int/sign/wif-client \ + csr=@client.csr \ + common_name=test-client \ + ttl=8760h > client_cert.pem