Build missing STS container libraries using original sources.

.gitignore

@@ -21,3 +21,4 @@ oidc-role.json
 __pycache__/
 *.tar.gz
 .env
+sts-agent/grte/

docs/storage-transfer.md

@@ -117,13 +117,25 @@ STS_AGENT_POOL=my-agents
 HOSTNAME=agent-host
 ```
 
-The image of Google lacks the shared glibc libraries that the `cryptography` package needs for certificate-based authentication. The agent launcher is statically linked with the Google runtime glibc 2.27 (GRTE v5), which loads shared libraries from `/usr/grte/v5/lib64`. `sts-agent/Dockerfile` adds the glibc 2.27 libraries of Ubuntu 18.04 there.
+The image of Google lacks the shared glibc libraries that the `cryptography` package needs for certificate-based authentication. The agent launcher is statically linked with GRTE v5, the Google Run Time Environment based on glibc 2.27, which loads shared libraries from `/usr/grte/v5/lib64`. The libraries must come from the same GRTE build, because GRTE changes internal structures of the dynamic loader. `sts-agent/build-grte.sh` builds them from the GRTE v5 source at `https://third-party-mirror.googlesource.com/GRTEv5` in a temporary `ubuntu:18.04` container, and `sts-agent/Dockerfile` adds them to the image.
+
+Build the GRTE v5 libraries into `sts-agent/grte`:
+
+```bash
+cd sts-agent
+docker run --rm --security-opt apparmor=unconfined \
+    -v "$PWD/build-grte.sh:/build-grte.sh:ro" \
+    -v "$PWD/grte:/out" \
+    ubuntu:18.04 /build-grte.sh
+```
+
+`--security-opt apparmor=unconfined` is needed only where Docker cannot load its AppArmor profile, for example inside an LXC container.
 
 Build the image and start the agent:
 
 ```bash
-cd sts-agent
+docker compose build
-sudo docker compose up -d --build
+docker compose up -d
 ```
 
 The directories are mounted at the same paths as on the host. The transfer job refers to the source directory by that path, and `certificate_config.json` holds the host paths of the certificate and key.

sts-agent/Dockerfile

@@ -1,9 +1,4 @@
-FROM ubuntu:18.04 AS glibc
-
 FROM gcr.io/cloud-ingest/tsop-agent:latest
 
-# Shared glibc 2.27 libraries in the library directory of the GRTE v5 glibc 2.27 the launcher is statically linked with.
+# GRTE v5 shared libraries built by build-grte.sh, matching the GRTE v5 glibc the launcher is statically linked with.
-COPY --from=glibc /lib/x86_64-linux-gnu/libc-2.27.so /usr/grte/v5/lib64/libc.so.6
+COPY grte/usr/grte/v5/lib64/ /usr/grte/v5/lib64/
-COPY --from=glibc /lib/x86_64-linux-gnu/libm-2.27.so /usr/grte/v5/lib64/libm.so.6
-COPY --from=glibc /lib/x86_64-linux-gnu/libpthread-2.27.so /usr/grte/v5/lib64/libpthread.so.0
-COPY --from=glibc /lib/x86_64-linux-gnu/ld-2.27.so /usr/grte/v5/lib64/ld-linux-x86-64.so.2

sts-agent/build-grte.sh

@@ -0,0 +1,30 @@
+#!/bin/bash
+set -euo pipefail
+
+prefix=/usr/grte/v5
+libdir="${prefix}/lib64"
+
+# Install the build tools
+export DEBIAN_FRONTEND=noninteractive
+apt-get update -qq
+apt-get install -y -qq --no-install-recommends build-essential gawk bison python3 git texinfo ca-certificates > /dev/null
+
+# Get the GRTE v5 source, Google's glibc 2.27
+git clone --depth 1 https://third-party-mirror.googlesource.com/GRTEv5 /src
+
+# Build the libraries in the library directory searched by the GRTE v5 loader
+mkdir /build
+cd /build
+cat > configparms <<EOF
+slibdir=${libdir}
+rtlddir=${libdir}
+EOF
+/src/google3/third_party/grte/v5_src/glibc-2.27/configure \
+    --prefix="${prefix}" \
+    --libdir="${libdir}" \
+    --disable-werror \
+    --enable-kernel=3.2
+make -j"$(nproc)"
+make install DESTDIR=/out
+
+ls -l "/out${libdir}/libc.so.6" "/out${libdir}/libm.so.6" "/out${libdir}/libpthread.so.0" "/out${libdir}/ld-linux-x86-64.so.2"