Rewrote IMDS emulator in Go, added unit tests.

docs/storage-transfer.md

@@ -95,20 +95,21 @@ The agent reads the certificate and key from the mounted directory, so certifica
 
 Role: **Server administrator**
 
-The agent cannot use the X.509 credentials itself. Its launcher is a statically linked binary that cannot load the `cryptography` package, which certificate-based authentication requires, and the `--creds-file` option accepts only a service account key or user credentials. The agent therefore gets its tokens from the IMDS emulator, a Compute Engine metadata server emulator in a second container. The emulator authenticates with ADC and the X.509 credentials in `/etc/sts-agent`, and serves the access tokens of `sa-test` on the metadata token endpoint. Without `GOOGLE_APPLICATION_CREDENTIALS`, ADC in the agent falls back to the metadata server, which `GCE_METADATA_HOST` and `GCE_METADATA_IP` point to the emulator.
+The agent cannot use the X.509 credentials itself. Its launcher is a statically linked binary that cannot load the `cryptography` package, which certificate-based authentication requires, and the `--creds-file` option accepts only a service account key or user credentials. The agent therefore gets its tokens from the IMDS emulator, a Compute Engine metadata server emulator in a second container. The emulator is a Go program that authenticates with ADC and the X.509 credentials in `/etc/sts-agent`, and serves the access tokens of `sa-test` on the metadata token endpoint. Without `GOOGLE_APPLICATION_CREDENTIALS`, ADC in the agent falls back to the metadata server, which `GCE_METADATA_HOST` and `GCE_METADATA_IP` point to the emulator.
 
 The containers run with the following settings:
 
 | Container | Setting | Value |
 |---|---|---|
-| `imds-emulator` | Image | built from `sts-agent/Dockerfile` |
+| `imds-emulator` | Image | built from `sts-agent/Dockerfile`: a static Go binary on `gcr.io/distroless/static-debian13` |
+| | Port | `8080`, set by `IMDS_PORT` |
 | | Mount | `/etc/sts-agent`, read only |
 | | Environment | `GOOGLE_APPLICATION_CREDENTIALS=/etc/sts-agent/cred.json` |
 | | Environment | `GOOGLE_API_CERTIFICATE_CONFIG=/etc/sts-agent/certificate_config.json` |
 | | Environment | `PROJECT_ID`, `PROJECT_NUMBER` |
 | `sts-agent` | Image | `gcr.io/cloud-ingest/tsop-agent:latest` |
 | | Mount | `/mnt/ext-storage`, read only |
-| | Environment | `GCE_METADATA_HOST=imds-emulator`, `GCE_METADATA_IP=imds-emulator` |
+| | Environment | `GCE_METADATA_HOST=imds-emulator:8080`, `GCE_METADATA_IP=imds-emulator:8080` |
 | | Locked memory limit | `64000000` bytes |
 | | Arguments | `--project-id=${PROJECT_ID} --hostname=$(hostname) --agent-pool=${STS_AGENT_POOL}` |
 
@@ -141,15 +142,15 @@ docker compose logs imds-emulator
 
 Requests for paths that the emulator does not serve are logged with the status 404.
 
-`imds_test.py` in the emulator container tests the emulator. It checks the metadata endpoints and the rejection of requests without the `Metadata-Flavor` header or with `X-Forwarded-For`. It then authenticates with the Compute Engine credentials of `google-auth` against the emulator, the same way as the agent, and checks the token with the Google token information endpoint. With `--bucket`, it also lists the objects of the bucket with the token:
+The emulator is a static binary without dependencies on files of the system: it embeds the root certificates and the time zone data, and it is the first process (PID 1) of its container. On `SIGTERM` or `SIGINT`, which `docker compose stop` sends, it stops accepting connections, lets the requests in progress finish for up to 5 seconds, and exits with status 0. A second signal makes it exit at once.
+
+The unit tests of the emulator, `sts-agent/main_test.go`, cover the request checks, the metadata endpoints, the token response, the extraction of the service account email from the credentials file, the shutdown, and a run of the emulator as a process stopped with real signals. They run during the build of the image, so a failing test stops it. To run them on a workstation with Go:
 
 ```bash
-source ../env.sh
-docker compose exec imds-emulator /venv/bin/python3 /app/imds_test.py --bucket "${BUCKET}"
+cd sts-agent
+go test -race ./...
 ```
 
-It prints `PASS` or `FAIL` for each check and exits with status 1 when a check fails.
-
 The source directory is mounted at the same path as on the host, because the transfer job refers to it by that path.
 
 The connected agents of the pool are listed on the Agent pools page of the Google Cloud console. An agent appears there only after it has authenticated.

docs/vault-approle.md

@@ -151,7 +151,7 @@ sudo install -d -m 0750 /etc/vault-cert
 }
 ```
 
-The `caddy` instance sets the signing path of the `web-server` role and a hook. The `wif` instance uses the default signing path and has no hook. It keeps the certificate in `/etc/sts-agent`, the credential directory of the Storage Transfer agent that authenticates with it (`docs/storage-transfer.md`).
+The `caddy` instance sets the signing path of the `web-server` role and a hook. The `wif` instance uses the default signing path and has no hook. It keeps the certificate in `/etc/sts-agent`, the credential directory of that agent (`docs/storage-transfer.md`).
 
 ```bash
 sudo /usr/local/sbin/vault-unwrap-secret-id /etc/vault-cert/caddy.json