diff --git a/docs/storage-transfer.md b/docs/storage-transfer.md index 212439d..ac4dda4 100644 --- a/docs/storage-transfer.md +++ b/docs/storage-transfer.md @@ -16,7 +16,7 @@ Agent host, for sections 3 and 4: - Linux on AMD64 with a container runtime, Python 3, and the external storage mounted at `/mnt/ext-storage`. - A copy of this repository, a `git clone` or an archive unpacked as described in the section "Copy the repository to a server" of the main `README.md`, with `env.sh` created as described in the Configuration section of the main `README.md`. The commands run from its directory. -- The client certificate renewed in `/var/lib/vault-cert/wif/` and `cred.json` for it in the repository directory, as described in section 4 and Next steps of `docs/vault-approle.md`. +- With the Vault CA, the client certificate renewed in `/var/lib/vault-cert/wif/`, as described in section 4 of `docs/vault-approle.md`. With the OpenSSL CA, the certificate is installed in section 3. The names come from `env.sh`. @@ -27,7 +27,7 @@ A transfer from a filesystem is agent based. It involves three identities: | Identity | Principal | Purpose | Roles | |---|---|---|---| | Job creator | `sa-test` through the `wif-test` configuration | Creates and runs transfer jobs | `roles/storagetransfer.user`, `roles/iam.roleViewer` | -| Transfer agent | `sa-test` through Application Default Credentials (ADC) with the `cred.json` of the agent host in the agent container | Reads the files and writes the objects | `roles/storagetransfer.transferAgent`, `roles/storage.objectAdmin` on the bucket | +| Transfer agent | `sa-test` through Application Default Credentials (ADC) with `/etc/sts-agent/cred.json` in the agent container | Reads the files and writes the objects | `roles/storagetransfer.transferAgent`, `roles/storage.objectAdmin` on the bucket | | Service agent | `project-PROJECT_NUMBER@storage-transfer-service.iam.gserviceaccount.com` | Manages the Pub/Sub topics of the pool, checks the transferred objects and writes the transfer logs | `roles/storagetransfer.serviceAgent`, `roles/storage.legacyBucketReader`, `roles/storage.objectViewer` and `roles/storage.objectCreator` on the bucket | The service agent is managed by Google and exists once per project. It cannot be replaced with a user-managed service account in agent based transfers. @@ -52,18 +52,44 @@ bash make_sts.sh Role: **Server administrator** -`make_sts_agent_creds.py` runs on the agent host. It creates the directory `/etc/sts-agent` with the following files, which hold no secrets: +### Install the client certificate -- `cred.json`, an unchanged copy of the `cred.json` of the agent host. The agent finds it through the `GOOGLE_APPLICATION_CREDENTIALS` environment variable. It refers to the default certificate configuration, which the agent finds through the `GOOGLE_API_CERTIFICATE_CONFIG` environment variable. -- `certificate_config.json`, with the paths of the client certificate and key as seen inside the container, where the host directory `/var/lib/vault-cert/wif/` is mounted at `/transfer_root/var/lib/vault-cert/wif/`. +The agent reads the client certificate and key from `/var/lib/vault-cert/wif/`. With the Vault CA, `vault-cert-renew@wif.timer` keeps them there and this step is skipped. + +With the OpenSSL CA, `make_certs.sh` created `client_cert.pem` and `client_key.pem` in the repository directory on the admin workstation. Copy them to the agent host, replacing `agent-host` with its name: ```bash -sudo ./make_sts_agent_creds.py +scp client_cert.pem client_key.pem agent-host: ``` -The script accepts the options `--cred` (default `cred.json`), `--cert` and `--key` (host paths, default the files in `/var/lib/vault-cert/wif/`), `--output` (default `/etc/sts-agent`) and `--prefix` (container path of the host paths, default `/transfer_root`). +Install them on the agent host and delete the copies: -The certificate and key are not copied. The agent reads the files renewed by `vault-cert-renew@wif.timer` from the mounted directory. +```bash +sudo install -d -m 0750 /var/lib/vault-cert/wif +sudo install -m 0644 ~/client_cert.pem /var/lib/vault-cert/wif/client_cert.pem +sudo install -m 0600 ~/client_key.pem /var/lib/vault-cert/wif/client_key.pem +rm ~/client_cert.pem ~/client_key.pem +``` + +### Create the credentials configuration + +`make_sts_agent_creds.py` runs on the agent host. It checks that the client certificate and key exist, and creates the directory `/etc/sts-agent` with the following files, which hold no secrets: + +- `cred.json`, the credentials configuration for the X.509 provider and the impersonation of the service account. The agent finds it through the `GOOGLE_APPLICATION_CREDENTIALS` environment variable. It refers to the default certificate configuration, which the agent finds through the `GOOGLE_API_CERTIFICATE_CONFIG` environment variable. +- `certificate_config.json`, with the paths of the client certificate and key. + +```bash +source ./env.sh +sudo ./make_sts_agent_creds.py \ + --project-number="${PROJECT_NUMBER}" \ + --pool="${POOL_ID}" \ + --provider="${PROVIDER_ID}" \ + --service-account="${SA_EMAIL}" +``` + +The options `--cert` and `--key` (default the files in `/var/lib/vault-cert/wif/`) and `--output` (default `/etc/sts-agent`) change the paths. + +The certificate and key are not copied into `/etc/sts-agent`. The agent reads them from the mounted directory `/var/lib/vault-cert/wif/`, so certificates renewed there by `vault-cert-renew@wif.timer` are used without changes to the configuration. ## 4. Start the transfer agent @@ -74,11 +100,11 @@ The agent container runs with the following settings: | Setting | Value | |---|---| | Image | `gcr.io/cloud-ingest/tsop-agent:latest` | -| Mount | `/mnt/ext-storage` to `/mnt/ext-storage`, read only | -| Mount | `/etc/sts-agent` to `/transfer_root/etc/sts-agent`, read only | -| Mount | `/var/lib/vault-cert/wif` to `/transfer_root/var/lib/vault-cert/wif`, read only | -| Environment | `GOOGLE_APPLICATION_CREDENTIALS=/transfer_root/etc/sts-agent/cred.json` | -| Environment | `GOOGLE_API_CERTIFICATE_CONFIG=/transfer_root/etc/sts-agent/certificate_config.json` | +| Mount | `/mnt/ext-storage`, read only | +| Mount | `/etc/sts-agent`, read only | +| Mount | `/var/lib/vault-cert/wif`, read only | +| Environment | `GOOGLE_APPLICATION_CREDENTIALS=/etc/sts-agent/cred.json` | +| Environment | `GOOGLE_API_CERTIFICATE_CONFIG=/etc/sts-agent/certificate_config.json` | | Locked memory limit | `64000000` bytes | | Arguments | `--project-id=${PROJECT_ID} --hostname=$(hostname) --agent-pool=${STS_AGENT_POOL}` | @@ -99,7 +125,7 @@ cd sts-agent sudo docker compose up -d ``` -The source directory is mounted at the same path as on the host, because the transfer job refers to it by that path. +The directories are mounted at the same paths as on the host. The transfer job refers to the source directory by that path, and `certificate_config.json` holds the host paths of the certificate and key. The agent authenticates with ADC, not with `--creds-file`. The `--creds-file` option accepts only a service account key or user credentials, not a Workload Identity Federation credential configuration. diff --git a/make_sts_agent_creds.py b/make_sts_agent_creds.py index 1cec5f7..b1d53aa 100755 --- a/make_sts_agent_creds.py +++ b/make_sts_agent_creds.py @@ -2,31 +2,54 @@ import argparse import json import os -import shutil +import sys parser = argparse.ArgumentParser(description="Create the credential configuration directory for the Storage Transfer agent container.") -parser.add_argument("--cred", default="cred.json", help="credentials configuration file") -parser.add_argument("--cert", default="/var/lib/vault-cert/wif/client_cert.pem", help="client certificate on the host") -parser.add_argument("--key", default="/var/lib/vault-cert/wif/client_key.pem", help="client private key on the host") -parser.add_argument("--output", default="/etc/sts-agent", help="credential configuration directory on the host") -parser.add_argument("--prefix", default="/transfer_root", help="container path under which the host paths are mounted") +parser.add_argument("--project-number", required=True, help="number of the project of the workload identity pool") +parser.add_argument("--pool", required=True, help="workload identity pool ID") +parser.add_argument("--provider", required=True, help="X.509 provider ID") +parser.add_argument("--service-account", required=True, help="email of the impersonated service account") +parser.add_argument("--cert", default="/var/lib/vault-cert/wif/client_cert.pem", help="client certificate") +parser.add_argument("--key", default="/var/lib/vault-cert/wif/client_key.pem", help="client private key") +parser.add_argument("--output", default="/etc/sts-agent", help="credential configuration directory") args = parser.parse_args() +cert = os.path.abspath(args.cert) +key = os.path.abspath(args.key) + +# The container mounts the certificate directory, so the files must exist on the host. +for path in (cert, key): + if not os.path.isfile(path): + sys.exit(f"{path} does not exist") + # Credential configuration directory mounted read only in the agent container, its files hold no secrets. os.makedirs(args.output, mode=0o755, exist_ok=True) -shutil.copyfile(args.cred, os.path.join(args.output, "cred.json")) +cred = { + "universe_domain": "googleapis.com", + "type": "external_account", + "audience": f"//iam.googleapis.com/projects/{args.project_number}/locations/global/workloadIdentityPools/{args.pool}/providers/{args.provider}", + "subject_token_type": "urn:ietf:params:oauth:token-type:mtls", + "token_url": "https://sts.mtls.googleapis.com/v1/token", + "credential_source": { + "certificate": { + "use_default_certificate_config": True, + } + }, + "service_account_impersonation_url": f"https://iamcredentials.mtls.googleapis.com/v1/projects/-/serviceAccounts/{args.service_account}:generateAccessToken", +} -# Certificate and key paths as seen inside the container, where the host certificate directory is mounted under the prefix. +# The container mounts the certificate directory at the same path as on the host. certificate_config = { "cert_configs": { "workload": { - "cert_path": args.prefix + os.path.abspath(args.cert), - "key_path": args.prefix + os.path.abspath(args.key), + "cert_path": cert, + "key_path": key, } } } -with open(os.path.join(args.output, "certificate_config.json"), "w") as file: - json.dump(certificate_config, file, indent=2) - file.write("\n") +for name, content in (("cred.json", cred), ("certificate_config.json", certificate_config)): + with open(os.path.join(args.output, name), "w") as file: + json.dump(content, file, indent=2) + file.write("\n") diff --git a/sts-agent/compose.yaml b/sts-agent/compose.yaml index 24c8bb2..b33238d 100644 --- a/sts-agent/compose.yaml +++ b/sts-agent/compose.yaml @@ -5,11 +5,11 @@ services: memlock: 64000000 volumes: - /mnt/ext-storage:/mnt/ext-storage:ro - - /etc/sts-agent:/transfer_root/etc/sts-agent:ro - - /var/lib/vault-cert/wif:/transfer_root/var/lib/vault-cert/wif:ro + - /etc/sts-agent:/etc/sts-agent:ro + - /var/lib/vault-cert/wif:/var/lib/vault-cert/wif:ro environment: - GOOGLE_APPLICATION_CREDENTIALS: /transfer_root/etc/sts-agent/cred.json - GOOGLE_API_CERTIFICATE_CONFIG: /transfer_root/etc/sts-agent/certificate_config.json + GOOGLE_APPLICATION_CREDENTIALS: /etc/sts-agent/cred.json + GOOGLE_API_CERTIFICATE_CONFIG: /etc/sts-agent/certificate_config.json command: - --project-id=${PROJECT_ID} - --hostname=${HOSTNAME}