gcp-wif-test/README.md

# How to replace JSON key with X.509 WiF

This short guide provides step by step instructions how to replace a JSON key with a Workload Identity Federation (WiF) X.509 certificate.

## Prerequisites

- `openssl` 3.x
- `vault` (only for the Vault based CA in step 1a)
- Node.js with `npm`
- `gcloud` authenticated as an administrator of the target project

## Configuration

The scripts read the names of the resources from `env.sh`. Replace the example values with your own before running them. Run all scripts from the repository directory.

`env.sh`:

```bash
# Project
PROJECT_ID=my-project
PROJECT_NUMBER=123456789012

# Workload identity pool and provider
POOL_ID=my-pool
POOL_DISPLAY_NAME="My Pool"
PROVIDER_ID=my-x509

# Service account
SA_EMAIL="sa-test@${PROJECT_ID}.iam.gserviceaccount.com"

# Storage bucket
BUCKET=my-project-wif-test
BUCKET_LOCATION=europe-central2
```

## About the shell scripts

The `.sh` files are meant as references for copying and pasting the commands one by one, so that each step can be inspected and adapted. Running them directly with `bash` also works.

> **Note:** The scripts are not idempotent. Running a whole script with `bash` a second time may fail, because the resources created by the first run already exist, for example the workload identity pool, the provider, the service account, the bucket and the `wif-test` gcloud configuration.

## 1. Create the certificates

`make_certs.sh` performs the following steps:

1. Creates a root CA certificate and key (`ca_cert.pem`, `ca_key.pem`).
2. Creates an issuing CA certificate and key (`issuing_ca_cert.pem`, `issuing_ca_key.pem`), signed by the root CA.
3. Creates a client certificate and key (`client_cert.pem`, `client_key.pem`), signed by the issuing CA. Its subject CN (`test-client`) becomes the WiF subject.

```bash
bash make_certs.sh
```

## 1a. Alternative: create the certificates with Vault

HashiCorp Vault Community Edition can replace the OpenSSL based CA. The PKI secrets engine hosts the root CA and the issuing CA, and a role issues the client certificate. The commands below produce the same files as `make_certs.sh`: `ca_cert.pem`, `issuing_ca_cert.pem`, `client_key.pem` and `client_cert.pem`. Continue with step 2 afterwards.

Prerequisites: the `vault` command installed on the workstation.

### Start a local Vault server

The local Vault server stores its data in files in the gitignored `vault-data` directory, so the CA survives restarts. The server starts sealed, is initialized once, and must be unsealed after every start.

Create `vault.hcl`, replacing the path with the absolute path of the `vault-data` directory:

```hcl
storage "file" {
  path = "/path/to/repository/vault-data"
}

listener "tcp" {
  address     = "127.0.0.1:8200"
  tls_disable = true
}

api_addr      = "http://127.0.0.1:8200"
disable_mlock = true
ui            = true
```

Start the server in a separate terminal:

```bash
vault server -config=vault.hcl
```

The web UI is available at `http://127.0.0.1:8200/ui`.

Initialize the server once with a single unseal key, unseal it, and set `VAULT_TOKEN` to the generated root token. Replace `<initial-root-token>` with the initial root token printed in `vault-init.txt`:

```bash
export VAULT_ADDR=http://127.0.0.1:8200
vault operator init -key-shares=1 -key-threshold=1 > vault-init.txt
vault operator unseal
export VAULT_TOKEN=<initial-root-token>
vault status
```

`vault operator unseal` asks for the unseal key from `vault-init.txt`. After every server restart, run `vault operator unseal` again and set `VAULT_ADDR` and `VAULT_TOKEN` in new terminals. The data is kept in `vault-data`.

> **Note:** The unseal key and the root token stored in a file are suitable only for a local, disposable server.

### Configure the CA hierarchy

`vault_configure.sh` performs the following steps in the terminal where `VAULT_ADDR` and `VAULT_TOKEN` are set:

1. Enables the PKI secrets engine for the root CA and for the issuing CA, and sets the maximum TTLs.
2. Generates the root CA inside Vault.
3. Generates the issuing CA key and CSR, signs it with the root CA, and installs the signed certificate.
4. Creates a role that issues client authentication certificates.
5. Exports the root CA and the issuing CA certificates (`ca_cert.pem`, `issuing_ca_cert.pem`).

```bash
bash vault_configure.sh
```

### Create the client certificate

`vault_make_client_cert.sh` performs the following steps in the same terminal:

1. Generates the client key and CSR locally with `openssl`. The key is never sent to Vault.
2. Signs the client CSR with the issuing CA. The certificate subject CN (`test-client`) becomes the WiF subject.

```bash
bash vault_make_client_cert.sh
```

The client certificate is valid for 365 days, has the Digital Signature and Key Encipherment key usages, and the TLS Web Client Authentication extended key usage. Unlike the certificate from `make_certs.sh`, it has no `basicConstraints` extension.

### Update a WiF provider created with the OpenSSL CA

If the provider was already created by `make_wif.sh` with the OpenSSL based CA, `vault_gcp_update.sh` switches it to the Vault CA. The client certificate keeps the subject CN `test-client`, so the service account binding and `cred.json` stay valid. The script performs the following steps:

1. Generates `trust_store.yaml` from the Vault CA certificates with `make_trust_store.mjs`.
2. Replaces the trust store of the existing X.509 provider.
3. Shows the provider configuration.
4. Tests the federation with the client certificate issued by Vault.

```bash
bash vault_gcp_update.sh
```

For a new setup, continue with step 2 instead.

Requesting the client certificate from a remote Vault server with a user account is described in `docs/vault-userpass.md` (user and password) and `docs/vault-entra-id.md` (Microsoft Entra ID).

## 2. Create the trust store

`make_trust_store.mjs` creates `trust_store.yaml`, a YAML file that holds the root CA as the trust anchor and the issuing CA as an intermediate CA. It uses the `yaml` npm package, declared in `package.json`:

```json
{
  "dependencies": {
    "yaml": ">=2.9.1"
  }
}
```

Install the dependency and generate `trust_store.yaml`:

```bash
npm install
node make_trust_store.mjs
```

The PEM files must contain only the certificate block. A trailing newline after `-----END CERTIFICATE-----` makes gcloud fail with `Certificate is not in PEM format`, which is why the script trims it.

## 3. Create the pool, provider and credentials

`make_wif.sh` performs the following steps:

1. Creates the workload identity pool and the X.509 provider.
2. Creates a service account.
3. Allows the client certificate subject to impersonate the service account.
4. Grants the service account read access to the project.
5. Writes `cred.json`.
6. Creates a storage bucket and grants the service account read and write access to its objects.
7. Makes the `public/` managed folder of the bucket publicly readable.
8. Enables Data Access audit logs for Cloud Storage.

```bash
bash make_wif.sh
```

## 4. Test the federation

`test_wif.sh` performs the following steps:

1. Creates a separate gcloud configuration and logs in with `cred.json`.
2. Checks the access to the project and the bucket, including an upload and a download of a file.
3. Switches back to the original gcloud configuration.
4. Lists the audit log of the service account operations.
5. Optionally switches back to the service account configuration.

```bash
bash test_wif.sh
```

Data Access audit logs are enabled for the whole project, not per bucket, by editing the project IAM policy. `add_audit_config.mjs` adds the `auditConfigs` section to the policy file fetched by gcloud.

The audit log lists only Cloud Storage operations of the service account. Resource Manager calls such as `projects describe` are not covered by the Data Access configuration.

## Result

`cred.json` is a credentials configuration file that references the client certificate and private key. It contains no secret itself, but `client_key.pem` does and must be protected like a JSON key.