slawek/gcp-wif-test
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Change the way STS agent is authenticated.

Browse Source
This commit is contained in:
slawek committed 2026-10-19 14:03:01 +02:00
1 parent 8e679a0753
commit 40d889d946
2 files changed
+8 -5
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/storage-transfer.md
+7 -4
View File
@@ -27,7 +27,7 @@ A transfer from a filesystem is agent based. It involves three identities:
| Identity | Principal | Purpose | Roles |
|---|---|---|---|
| Job creator | `sa-test` through the `wif-test` configuration | Creates and runs transfer jobs | `roles/storagetransfer.user`, `roles/iam.roleViewer` |
| Transfer agent | `sa-test` through the `cred.json` of the agent host in the agent container | Reads the files and writes the objects | `roles/storagetransfer.transferAgent`, `roles/storage.objectAdmin` on the bucket |
| Transfer agent | `sa-test` through Application Default Credentials (ADC) with the `cred.json` of the agent host in the agent container | Reads the files and writes the objects | `roles/storagetransfer.transferAgent`, `roles/storage.objectAdmin` on the bucket |
| Service agent | `project-PROJECT_NUMBER@storage-transfer-service.iam.gserviceaccount.com` | Manages the Pub/Sub topics of the pool, checks the transferred objects and writes the transfer logs | `roles/storagetransfer.serviceAgent`, `roles/storage.legacyBucketReader`, `roles/storage.objectViewer` and `roles/storage.objectCreator` on the bucket |
The service agent is managed by Google and exists once per project. It cannot be replaced with a user-managed service account in agent based transfers.
@@ -54,7 +54,7 @@ Role: **Server administrator**
`make_sts_agent_creds.py` runs on the agent host. It creates the directory `/etc/sts-agent` with the following files, which hold no secrets:
- `cred.json`, an unchanged copy of the `cred.json` of the agent host. It refers to the default certificate configuration, which the agent finds through the `GOOGLE_API_CERTIFICATE_CONFIG` environment variable.
- `cred.json`, an unchanged copy of the `cred.json` of the agent host. The agent finds it through the `GOOGLE_APPLICATION_CREDENTIALS` environment variable. It refers to the default certificate configuration, which the agent finds through the `GOOGLE_API_CERTIFICATE_CONFIG` environment variable.
- `certificate_config.json`, with the paths of the client certificate and key as seen inside the container, where the host directory `/var/lib/vault-cert/wif/` is mounted at `/transfer_root/var/lib/vault-cert/wif/`.
```bash
@@ -77,9 +77,10 @@ The agent container runs with the following settings:
| Mount | `/mnt/ext-storage` to `/mnt/ext-storage`, read only |
| Mount | `/etc/sts-agent` to `/transfer_root/etc/sts-agent`, read only |
| Mount | `/var/lib/vault-cert/wif` to `/transfer_root/var/lib/vault-cert/wif`, read only |
| Environment | `GOOGLE_APPLICATION_CREDENTIALS=/transfer_root/etc/sts-agent/cred.json` |
| Environment | `GOOGLE_API_CERTIFICATE_CONFIG=/transfer_root/etc/sts-agent/certificate_config.json` |
| Locked memory limit | `64000000` bytes |
| Arguments | `--project-id=${PROJECT_ID} --creds-file=/transfer_root/etc/sts-agent/cred.json --hostname=$(hostname) --agent-pool=${STS_AGENT_POOL}` |
| Arguments | `--project-id=${PROJECT_ID} --hostname=$(hostname) --agent-pool=${STS_AGENT_POOL}` |
`sts-agent/compose.yaml` holds these settings for Docker Compose. Compose substitutes `PROJECT_ID`, `STS_AGENT_POOL` and `HOSTNAME` from `sts-agent/.env` into the agent arguments; they are not passed to the container as environment variables. Create the file with the values of `env.sh` and the host name printed by `hostname`:
@@ -98,7 +99,9 @@ cd sts-agent
sudo docker compose up -d
```
The source directory is mounted at the same path as on the host, because the transfer job refers to it by that path. The agent requires the `--creds-file` path to start with `/transfer_root` unless the whole host filesystem is mounted with `--enable-mount-directory`.
The source directory is mounted at the same path as on the host, because the transfer job refers to it by that path.
The agent authenticates with ADC, not with `--creds-file`. The `--creds-file` option accepts only a service account key or user credentials, not a Workload Identity Federation credential configuration.
The connected agents of the pool are listed on the Agent pools page of the Google Cloud console. An agent appears there only after it has authenticated.