slawek/gcp-wif-test
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Added IMDS emulator container.

Browse Source
This commit is contained in:
slawek committed 2026-10-19 19:57:38 +02:00
1 parent f7466a0ac2
commit b5cb5f0952
6 files changed
+290 -47
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/storage-transfer.md
+38 -17
View File
@@ -27,7 +27,7 @@ A transfer from a filesystem is agent based. It involves three identities:
| Identity | Principal | Purpose | Roles |
|---|---|---|---|
| Job creator | `sa-test` through the `wif-test` configuration | Creates and runs transfer jobs | `roles/storagetransfer.user`, `roles/iam.roleViewer` |
| Transfer agent | `sa-test` through Application Default Credentials (ADC) with `/etc/sts-agent/cred.json` in the agent container | Reads the files and writes the objects | `roles/storagetransfer.transferAgent`, `roles/storage.objectAdmin` on the bucket |
| Transfer agent | `sa-test` through tokens of the IMDS emulator, which authenticates with Application Default Credentials (ADC) and `/etc/sts-agent/cred.json` | Reads the files and writes the objects | `roles/storagetransfer.transferAgent`, `roles/storage.objectAdmin` on the bucket |
| Service agent | `project-PROJECT_NUMBER@storage-transfer-service.iam.gserviceaccount.com` | Manages the Pub/Sub topics of the pool, checks the transferred objects and writes the transfer logs | `roles/storagetransfer.serviceAgent`, `roles/storage.legacyBucketReader`, `roles/storage.objectViewer` and `roles/storage.objectCreator` on the bucket |
The service agent is managed by Google and exists once per project. It cannot be replaced with a user-managed service account in agent based transfers.
@@ -95,31 +95,37 @@ The agent reads the certificate and key from the mounted directory, so certifica
Role: **Server administrator**
The agent container runs with the following settings:
The agent cannot use the X.509 credentials itself. Its launcher is a statically linked binary that cannot load the `cryptography` package, which certificate-based authentication requires, and the `--creds-file` option accepts only a service account key or user credentials. The agent therefore gets its tokens from the IMDS emulator, a Compute Engine metadata server emulator in a second container. The emulator authenticates with ADC and the X.509 credentials in `/etc/sts-agent`, and serves the access tokens of `sa-test` on the metadata token endpoint. Without `GOOGLE_APPLICATION_CREDENTIALS`, ADC in the agent falls back to the metadata server, which `GCE_METADATA_HOST` and `GCE_METADATA_IP` point to the emulator.
| Setting | Value |
|---|---|
| Image | built from `sts-agent/Dockerfile` on `gcr.io/cloud-ingest/tsop-agent:latest` |
| Mount | `/mnt/ext-storage`, read only |
| Mount | `/etc/sts-agent`, read only |
| Environment | `GOOGLE_APPLICATION_CREDENTIALS=/etc/sts-agent/cred.json` |
| Environment | `GOOGLE_API_CERTIFICATE_CONFIG=/etc/sts-agent/certificate_config.json` |
| Locked memory limit | `64000000` bytes |
| Arguments | `--project-id=${PROJECT_ID} --hostname=$(hostname) --agent-pool=${STS_AGENT_POOL}` |
The containers run with the following settings:
`sts-agent/compose.yaml` holds these settings for Docker Compose. Compose substitutes `PROJECT_ID`, `STS_AGENT_POOL` and `HOSTNAME` from `sts-agent/.env` into the agent arguments; they are not passed to the container as environment variables. Create the file with the values of `env.sh` and the host name printed by `hostname`:
| Container | Setting | Value |
|---|---|---|
| `imds-emulator` | Image | built from `imds-emulator/Dockerfile` |
| | Mount | `/etc/sts-agent`, read only |
| | Environment | `GOOGLE_APPLICATION_CREDENTIALS=/etc/sts-agent/cred.json` |
| | Environment | `GOOGLE_API_CERTIFICATE_CONFIG=/etc/sts-agent/certificate_config.json` |
| | Environment | `PROJECT_ID`, `PROJECT_NUMBER` |
| `sts-agent` | Image | `gcr.io/cloud-ingest/tsop-agent:latest` |
| | Mount | `/mnt/ext-storage`, read only |
| | Environment | `GCE_METADATA_HOST=imds-emulator`, `GCE_METADATA_IP=imds-emulator` |
| | Locked memory limit | `64000000` bytes |
| | Arguments | `--project-id=${PROJECT_ID} --hostname=$(hostname) --agent-pool=${STS_AGENT_POOL}` |
The emulator publishes no port; only the containers of the Compose project reach it. The agent container must not set `GOOGLE_API_CERTIFICATE_CONFIG`, because the metadata token code of the agent parses the certificate it points to with `cryptography`.
`sts-agent/compose.yaml` holds these settings for Docker Compose. Compose substitutes `PROJECT_ID`, `PROJECT_NUMBER`, `STS_AGENT_POOL` and `HOSTNAME` from `sts-agent/.env`. Create the file with the values of `env.sh` and the host name printed by `hostname`:
`sts-agent/.env`:
```bash
PROJECT_ID=my-project
PROJECT_NUMBER=123456789012
STS_AGENT_POOL=my-agents
HOSTNAME=agent-host
```
The image of Google lacks the shared glibc libraries that the `cryptography` package needs for certificate-based authentication. The agent launcher is statically linked with GRTE v5, the Google Run Time Environment based on glibc 2.27, which loads shared libraries from `/usr/grte/v5/lib64`. The libraries must come from the same GRTE build, because GRTE changes internal structures of the dynamic loader. The first stage of `sts-agent/Dockerfile` builds them from the GRTE v5 source at `https://third-party-mirror.googlesource.com/GRTEv5` on `ubuntu:18.04`, and the second stage adds them to the image of Google.
Build the image and start the agent:
Build the emulator and start both containers:
```bash
cd sts-agent
@@ -127,9 +133,24 @@ docker compose build
docker compose up -d
```
The directories are mounted at the same paths as on the host. The transfer job refers to the source directory by that path, and `certificate_config.json` holds the host paths of the certificate and key.
The emulator logs one line per request with the path and the status, never the token:
The agent authenticates with ADC, not with `--creds-file`. The `--creds-file` option accepts only a service account key or user credentials, not a Workload Identity Federation credential configuration.
```bash
docker compose logs imds-emulator
```
Requests for paths that the emulator does not serve are logged with the status 404.
`imds-test` in the emulator container tests the emulator. It checks the metadata endpoints and the rejection of requests without the `Metadata-Flavor` header or with `X-Forwarded-For`. It then authenticates with the Compute Engine credentials of `google-auth` against the emulator, the same way as the agent, and checks the token with the Google token information endpoint. With `--bucket`, it also lists the objects of the bucket with the token:
```bash
source ../env.sh
docker compose exec imds-emulator imds-test --bucket "${BUCKET}"
```
It prints `PASS` or `FAIL` for each check and exits with status 1 when a check fails.
The source directory is mounted at the same path as on the host, because the transfer job refers to it by that path.
The connected agents of the pool are listed on the Agent pools page of the Google Cloud console. An agent appears there only after it has authenticated.