gcp-wif-test/README.md

How to replace JSON key with X.509 WiF

This short guide provides step by step instructions how to replace a JSON key with a Workload Identity Federation (WiF) X.509 certificate.

Prerequisites

• openssl 3.x
• vault (only for the Vault based CA in step 1a)
• Node.js with npm
• gcloud authenticated as an administrator of the target project

Configuration

The scripts read the names of the resources from env.sh. Replace the example values with your own before running them. Run all scripts from the repository directory.

env.sh:

# Project
PROJECT_ID=my-project
PROJECT_NUMBER=123456789012

# Workload identity pool and provider
POOL_ID=my-pool
POOL_DISPLAY_NAME="My Pool"
PROVIDER_ID=my-x509

# Service account
SA_EMAIL="sa-test@${PROJECT_ID}.iam.gserviceaccount.com"

# Storage bucket
BUCKET=my-project-wif-test
BUCKET_LOCATION=europe-central2

About the shell scripts

The .sh files are meant as references for copying and pasting the commands one by one, so that each step can be inspected and adapted. Running them directly with bash also works.

Note: The scripts are not idempotent. Running a whole script with bash a second time may fail, because the resources created by the first run already exist, for example the workload identity pool, the provider, the service account, the bucket and the wif-test gcloud configuration.

1. Create the certificates

make_certs.sh performs the following steps:

1. Creates a root CA certificate and key (ca_cert.pem, ca_key.pem).
2. Creates an issuing CA certificate and key (issuing_ca_cert.pem, issuing_ca_key.pem), signed by the root CA.
3. Creates a client certificate and key (client_cert.pem, client_key.pem), signed by the issuing CA. Its subject CN (test-client) becomes the WiF subject.

bash make_certs.sh

1a. Alternative: create the certificates with Vault

HashiCorp Vault Community Edition can replace the OpenSSL based CA. The PKI secrets engine hosts the root CA and the issuing CA, and a role issues the client certificate. The commands below produce the same files as make_certs.sh: ca_cert.pem, issuing_ca_cert.pem, client_key.pem and client_cert.pem. Continue with step 2 afterwards.

Prerequisites: the vault command installed on the workstation.

Start a local Vault server

Vault dev mode keeps all data in memory, and a dev server cannot be restarted on existing file storage. The local server therefore runs in normal mode with file storage in the gitignored vault-data directory.

Create vault.hcl, replacing the path with the absolute path of the vault-data directory:

storage "file" {
  path = "/path/to/repository/vault-data"
}

listener "tcp" {
  address     = "127.0.0.1:8200"
  tls_disable = true
}

api_addr      = "http://127.0.0.1:8200"
disable_mlock = true
ui            = true

Start the server in a separate terminal:

vault server -config=vault.hcl

The web UI is available at http://127.0.0.1:8200/ui.

Initialize the server once with a single unseal key, unseal it, and set VAULT_TOKEN to the generated root token. Replace <initial-root-token> with the initial root token printed in vault-init.txt:

export VAULT_ADDR=http://127.0.0.1:8200
vault operator init -key-shares=1 -key-threshold=1 > vault-init.txt
vault operator unseal
export VAULT_TOKEN=<initial-root-token>
vault status

vault operator unseal asks for the unseal key from vault-init.txt. After every server restart, run vault operator unseal again and set VAULT_ADDR and VAULT_TOKEN in new terminals. The data is kept in vault-data.

Note: The unseal key and the root token stored in a file are suitable only for a local, disposable server.

Configure the CA hierarchy

vault_configure.sh performs the following steps in the terminal where VAULT_ADDR and VAULT_TOKEN are set:

1. Enables the PKI secrets engine for the root CA and for the issuing CA, and sets the maximum TTLs.
2. Generates the root CA inside Vault.
3. Generates the issuing CA key and CSR, signs it with the root CA, and installs the signed certificate.
4. Creates a role that issues client authentication certificates.
5. Exports the root CA and the issuing CA certificates (ca_cert.pem, issuing_ca_cert.pem).

bash vault_configure.sh

Create the client certificate

vault_make_client_cert.sh performs the following steps in the same terminal:

1. Generates the client key and CSR locally with openssl. The key is never sent to Vault.
2. Signs the client CSR with the issuing CA. The certificate subject CN (test-client) becomes the WiF subject.

bash vault_make_client_cert.sh

The client certificate is valid for 365 days, has the Digital Signature and Key Encipherment key usages, and the TLS Web Client Authentication extended key usage. Unlike the certificate from make_certs.sh, it has no basicConstraints extension.

Update a WiF provider created with the OpenSSL CA

If the provider was already created by make_wif.sh with the OpenSSL based CA, vault_gcp_update.sh switches it to the Vault CA. The client certificate keeps the subject CN test-client, so the service account binding and cred.json stay valid. The script performs the following steps:

1. Generates trust_store.yaml from the Vault CA certificates with make_trust_store.mjs.
2. Replaces the trust store of the existing X.509 provider.
3. Shows the provider configuration.
4. Tests the federation with the client certificate issued by Vault.

bash vault_gcp_update.sh

For a new setup, continue with step 2 instead.

2. Create the trust store

make_trust_store.mjs creates trust_store.yaml, a YAML file that holds the root CA as the trust anchor and the issuing CA as an intermediate CA. It uses the yaml npm package, declared in package.json:

{
  "dependencies": {
    "yaml": ">=2.9.1"
  }
}

Install the dependency and generate trust_store.yaml:

npm install
node make_trust_store.mjs

The PEM files must contain only the certificate block. A trailing newline after -----END CERTIFICATE----- makes gcloud fail with Certificate is not in PEM format, which is why the script trims it.

3. Create the pool, provider and credentials

make_wif.sh performs the following steps:

1. Creates the workload identity pool and the X.509 provider.
2. Creates a service account.
3. Allows the client certificate subject to impersonate the service account.
4. Grants the service account read access to the project.
5. Writes cred.json.
6. Creates a storage bucket and grants the service account read and write access to its objects.
7. Makes the public/ managed folder of the bucket publicly readable.
8. Enables Data Access audit logs for Cloud Storage.

bash make_wif.sh

4. Test the federation

test_wif.sh performs the following steps:

1. Creates a separate gcloud configuration and logs in with cred.json.
2. Checks the access to the project and the bucket, including an upload and a download of a file.
3. Switches back to the original gcloud configuration.
4. Lists the audit log of the service account operations.
5. Optionally switches back to the service account configuration.

bash test_wif.sh

Data Access audit logs are enabled for the whole project, not per bucket, by editing the project IAM policy. add_audit_config.mjs adds the auditConfigs section to the policy file fetched by gcloud.

The audit log lists only Cloud Storage operations of the service account. Resource Manager calls such as projects describe are not covered by the Data Access configuration.

Result

cred.json is a credentials configuration file that references the client certificate and private key. It contains no secret itself, but client_key.pem does and must be protected like a JSON key.