diff --git a/.gitignore b/.gitignore index b9d84b7..0c6ac91 100644 --- a/.gitignore +++ b/.gitignore @@ -20,3 +20,4 @@ oidc-role.json **/tmp/* __pycache__/ *.tar.gz +.env diff --git a/docs/storage-transfer.md b/docs/storage-transfer.md index 1077d70..75c3f6b 100644 --- a/docs/storage-transfer.md +++ b/docs/storage-transfer.md @@ -87,6 +87,38 @@ Run the agent container with the following settings: | Locked memory limit | `64000000` bytes | | Arguments | `--project-id=${PROJECT_ID} --creds-file=/transfer_root/etc/sts-agent/cred.json --hostname=$(hostname) --agent-pool=${STS_AGENT_POOL}` | +With Docker: + +```bash +sudo docker run --ulimit memlock=64000000 -d --rm \ + -v /mnt/ext-storage:/mnt/ext-storage:ro \ + -v /etc/sts-agent:/transfer_root/etc/sts-agent:ro \ + -v /var/lib/vault-cert/wif:/transfer_root/var/lib/vault-cert/wif:ro \ + --env GOOGLE_API_CERTIFICATE_CONFIG=/transfer_root/etc/sts-agent/certificate_config.json \ + gcr.io/cloud-ingest/tsop-agent:latest \ + --project-id="${PROJECT_ID}" \ + --creds-file=/transfer_root/etc/sts-agent/cred.json \ + --hostname="$(hostname)" \ + --agent-pool="${STS_AGENT_POOL}" +``` + +With Docker Compose, `sts-agent/compose.yaml` holds the same settings. It reads `PROJECT_ID`, `STS_AGENT_POOL` and `HOSTNAME` from `sts-agent/.env`. Create the file with the values of `env.sh` and the host name printed by `hostname`: + +`sts-agent/.env`: + +```bash +PROJECT_ID=my-project +STS_AGENT_POOL=my-agents +HOSTNAME=agent-host +``` + +Start the agent: + +```bash +cd sts-agent +sudo docker compose up -d +``` + The source directory is mounted at the same path as on the host, because the transfer job refers to it by that path. The agent requires the `--creds-file` path to start with `/transfer_root` unless the whole host filesystem is mounted with `--enable-mount-directory`. The connected agents of the pool are listed on the Agent pools page of the Google Cloud console. An agent appears there only after it has authenticated. diff --git a/sts-agent/compose.yaml b/sts-agent/compose.yaml new file mode 100644 index 0000000..fc1289c --- /dev/null +++ b/sts-agent/compose.yaml @@ -0,0 +1,16 @@ +services: + sts-agent: + image: gcr.io/cloud-ingest/tsop-agent:latest + ulimits: + memlock: 64000000 + volumes: + - /mnt/ext-storage:/mnt/ext-storage:ro + - /etc/sts-agent:/transfer_root/etc/sts-agent:ro + - /var/lib/vault-cert/wif:/transfer_root/var/lib/vault-cert/wif:ro + environment: + GOOGLE_API_CERTIFICATE_CONFIG: /transfer_root/etc/sts-agent/certificate_config.json + command: + - --project-id=${PROJECT_ID} + - --creds-file=/transfer_root/etc/sts-agent/cred.json + - --hostname=${HOSTNAME} + - --agent-pool=${STS_AGENT_POOL}