App Role guide update.
Documents issuing both client and server certificates. WIF relies on client certificates. Server example documents an example of internal Web service using an autorenewing internal certificate.
This commit is contained in:
slawek committed
1 parent
c21ebd7725
commit
db8a3aac8a
7 files changed
+164
-73
No files matched your search
+103
-73
@@ -1,10 +1,12 @@
|
||||
# Automatic renewal of a client certificate on Ubuntu with Vault AppRole
|
||||
# Automatic renewal of certificates on Ubuntu with Vault AppRole
|
||||
|
||||
This guide shows how an unattended Ubuntu server gets a client certificate from a remote Vault server with the AppRole authentication method, and keeps it fresh. A systemd timer starts a renewal service, the service renews the certificate before it expires, and after a successful renewal it runs a configurable Bash hook that signals the application to reread the certificate or restarts it. The renewal script is written in Python 3 and talks to Vault through the Vault HTTP API, so the server does not need the Vault CLI.
|
||||
This guide shows how an unattended Ubuntu server gets certificates from a remote Vault server with the AppRole authentication method, and keeps them fresh. A systemd timer starts a renewal service, the service renews a certificate before it expires, and after a successful renewal it runs a configurable Bash hook that signals the application to reread the certificate or restarts it. The renewal script is written in Python 3 and talks to Vault through the Vault HTTP API, so the server does not need the Vault CLI.
|
||||
|
||||
The renewal service is a systemd template, so one server can renew several certificates, each with its own settings. The guide first tests the whole process, including the hook, with a server certificate served by Caddy (section 3). Then it sets up the client certificate for the WIF tests (section 4).
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- The Vault PKI configured as described in step 1a of the main `README.md`: the `pki_int` mount and the `wif-client` role (`vault_configure.sh`).
|
||||
- The Vault PKI configured as described in step 1a of the main `README.md`: the `pki_int` mount and the `wif-client` and `web-server` roles (`vault_configure.sh`).
|
||||
- A Vault server reachable over HTTPS. The examples use `https://vault.koszewscy.waw.pl`.
|
||||
- Ubuntu Server 24.04 LTS or 26.04 LTS with systemd and `sudo` access.
|
||||
- The IP address of the server as seen by Vault. The examples use `203.0.113.10`.
|
||||
@@ -22,7 +24,7 @@ export VAULT_ADDR=https://vault.koszewscy.waw.pl
|
||||
export VAULT_TOKEN=<admin-token>
|
||||
```
|
||||
|
||||
1. Create a policy that allows signing certificate requests with the `wif-client` role and nothing else. Skip this step when the policy already exists.
|
||||
1. Create a policy that allows signing certificate requests with the `wif-client` and `web-server` roles and nothing else. When the policy already exists, the command replaces it, so the file must list all the paths.
|
||||
2. Enable the AppRole authentication method.
|
||||
3. Create the `cert-renew-<server_name>` role. It issues batch tokens valid for 10 minutes, the secret ID does not expire so that the server can renew unattended, and both the secret ID and the tokens work only from the IP address of the server.
|
||||
4. Create the `cert-renew` role. It has the same settings, but the secret ID and the tokens work from any address.
|
||||
@@ -35,6 +37,10 @@ export VAULT_TOKEN=<admin-token>
|
||||
path "pki_int/sign/wif-client" {
|
||||
capabilities = ["update"]
|
||||
}
|
||||
|
||||
path "pki_int/sign/web-server" {
|
||||
capabilities = ["update"]
|
||||
}
|
||||
```
|
||||
|
||||
```bash
|
||||
@@ -85,7 +91,7 @@ sudo apt install -y python3 openssl
|
||||
|
||||
### Store the credentials
|
||||
|
||||
1. Create the directories for the configuration and for the certificate.
|
||||
1. Create the directories for the configuration and for the certificates.
|
||||
2. Create the credential files with the access limited to root before writing into them.
|
||||
3. Write the role ID.
|
||||
4. Store the secret ID with the script. It asks for the wrapping token without echo, so the token does not appear in the process list or in the shell history. The configuration file is not read in this step, so pass the address of Vault. The wrapping token can be unwrapped only once. When the unwrap fails, treat the wrapping token as compromised and ask for a new one.
|
||||
@@ -119,17 +125,30 @@ A `permission denied` answer means that the role ID or the secret ID is wrong, o
|
||||
|
||||
### Configure the renewal
|
||||
|
||||
`/etc/vault-cert/vault-cert.env`:
|
||||
Each certificate has an instance name and a configuration file `/etc/vault-cert/<instance>.env`. This guide uses the instances `caddy` for the server certificate of the test site (section 3) and `wif` for the client certificate of the WIF tests (section 4).
|
||||
|
||||
`/etc/vault-cert/caddy.env`:
|
||||
|
||||
```text
|
||||
VAULT_ADDR=https://vault.koszewscy.waw.pl
|
||||
CERT_CN=web.example.com
|
||||
CERT_DIR=/var/lib/vault-cert/caddy
|
||||
SIGN_PATH=pki_int/sign/web-server
|
||||
HOOK=/etc/vault-cert/caddy-hook.sh
|
||||
```
|
||||
|
||||
`/etc/vault-cert/wif.env`:
|
||||
|
||||
```text
|
||||
VAULT_ADDR=https://vault.koszewscy.waw.pl
|
||||
CERT_CN=test-client
|
||||
CERT_DIR=/var/lib/vault-cert
|
||||
CERT_DIR=/var/lib/vault-cert/wif
|
||||
RENEW_BEFORE_DAYS=30
|
||||
CERT_TTL=8760h
|
||||
HOOK=/etc/vault-cert/hook.sh
|
||||
```
|
||||
|
||||
The `caddy` instance sets the signing path of the `web-server` role and a hook. The `wif` instance uses the defaults for the signing path and has no hook.
|
||||
|
||||
The settings are:
|
||||
|
||||
| Variable | Default | Meaning |
|
||||
@@ -145,100 +164,111 @@ The settings are:
|
||||
| `CERT_TTL` | `8760h` | Requested lifetime of the new certificate. |
|
||||
| `HOOK` | none | Script run after a successful renewal. |
|
||||
|
||||
### Write the hook
|
||||
|
||||
The hook runs as root after the new files are in place. It gets the paths of the certificate and the key in the `CERT_FILE` and `KEY_FILE` environment variables. A failing hook fails the service, the failure is visible in the journal and in `systemctl status`.
|
||||
|
||||
`/etc/vault-cert/hook.sh`, mode `0750`. The example copies the files to the application and signals it to reread them:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
|
||||
install -m 0644 -o root -g root "${CERT_FILE}" /etc/myapp/client_cert.pem
|
||||
install -m 0640 -o root -g myapp "${KEY_FILE}" /etc/myapp/client_key.pem
|
||||
systemctl reload myapp.service
|
||||
```
|
||||
|
||||
Replace `reload` with `restart` when the application cannot reread the certificate. Leave `HOOK` unset to renew without any action. When `HOOK` names a file that does not exist or is not executable, the renewal still succeeds and the script logs a warning and skips the hook.
|
||||
The hook runs as root after the new files are in place. It gets the paths of the certificate and the key in the `CERT_FILE` and `KEY_FILE` environment variables. A failing hook fails the service, the failure is visible in the journal and in `systemctl status`. Leave `HOOK` unset to renew without any action. When `HOOK` names a file that does not exist or is not executable, the renewal still succeeds and the script logs a warning and skips the hook.
|
||||
|
||||
> **Note:** The certificate is already renewed when the hook runs. When the hook fails, the next timer run finds a valid certificate and does not run the hook again. Fix the hook and run it by hand with the environment variables set, or remove `client_cert.pem` to force a new renewal.
|
||||
|
||||
### Create the systemd units
|
||||
|
||||
`/etc/systemd/system/vault-cert-renew.service`:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=Renew the client certificate from Vault
|
||||
Wants=network-online.target
|
||||
After=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
EnvironmentFile=/etc/vault-cert/vault-cert.env
|
||||
ExecStart=/usr/local/sbin/vault-cert-renew
|
||||
PrivateTmp=true
|
||||
NoNewPrivileges=true
|
||||
ProtectHome=true
|
||||
```
|
||||
|
||||
`/etc/systemd/system/vault-cert-renew.timer`:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=Check the client certificate expiry four times a day
|
||||
|
||||
[Timer]
|
||||
OnCalendar=*-*-* 00,06,12,18:00:00
|
||||
RandomizedDelaySec=30min
|
||||
Persistent=true
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
```
|
||||
|
||||
`Persistent=true` runs a missed check after the server was powered off, and `RandomizedDelaySec` spreads the load when many servers share the configuration.
|
||||
|
||||
Enable the timer:
|
||||
The directory `systemd` in the repository contains the service `vault-cert-renew@.service` and the timer `vault-cert-renew@.timer`. The service runs the renewal script once with the settings of the instance. The timer starts it four times a day. `Persistent=true` runs a missed check after the server was powered off, and `RandomizedDelaySec` spreads the load when many servers share the configuration. Install the units:
|
||||
|
||||
```bash
|
||||
sudo install -m 0644 systemd/vault-cert-renew@.service /etc/systemd/system/vault-cert-renew@.service
|
||||
sudo install -m 0644 systemd/vault-cert-renew@.timer /etc/systemd/system/vault-cert-renew@.timer
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now vault-cert-renew.timer
|
||||
```
|
||||
|
||||
## 3. Verify and operate
|
||||
## 3. Test the renewal and the hook with Caddy
|
||||
|
||||
Role: **Server administrator**
|
||||
|
||||
Caddy serves a test page over TLS with a server certificate issued by the `web-server` role. The hook copies the renewed certificate to Caddy and reloads it, so the serial number of the served certificate shows whether the hook works.
|
||||
|
||||
### Install Caddy, the test site and the hook
|
||||
|
||||
Install Caddy from the Ubuntu repository (the `universe` component):
|
||||
|
||||
```bash
|
||||
sudo apt install -y caddy
|
||||
```
|
||||
|
||||
The directory `web-test` in the repository contains these files:
|
||||
|
||||
- `index.html`, the test page.
|
||||
- `Caddyfile`, the configuration of a site on port 443 that uses the certificate files `/etc/caddy/server_cert.pem` and `/etc/caddy/server_key.pem`.
|
||||
- `caddy-hook.sh`, the hook that copies the renewed files to those paths and reloads Caddy, or starts it when it is not running. The reload makes Caddy read the certificate files again.
|
||||
|
||||
Install them. The hook goes to the path of `HOOK` in `caddy.env`, and the certificate subject `CERT_CN` in that file is the name of the site.
|
||||
```bash
|
||||
sudo install -d -m 0755 /var/www/web-test
|
||||
sudo install -m 0644 web-test/index.html /var/www/web-test/index.html
|
||||
sudo install -m 0644 web-test/Caddyfile /etc/caddy/Caddyfile
|
||||
sudo install -m 0750 web-test/caddy-hook.sh /etc/vault-cert/caddy-hook.sh
|
||||
```
|
||||
|
||||
### Run the renewal and check the result
|
||||
|
||||
1. Run the first renewal by hand and read its log. The log shows the login, the signing and the hook.
|
||||
|
||||
```bash
|
||||
sudo systemctl start vault-cert-renew@caddy.service
|
||||
sudo journalctl -u vault-cert-renew@caddy.service --no-pager
|
||||
```
|
||||
|
||||
2. Check the issued certificate. It has the subject and the SAN `web.example.com` and the extended key usage `TLS Web Server Authentication`.
|
||||
|
||||
```bash
|
||||
sudo openssl x509 -in /var/lib/vault-cert/caddy/client_cert.pem -noout -subject -issuer -serial -enddate -ext subjectAltName,extendedKeyUsage
|
||||
```
|
||||
|
||||
3. Request the page and read the serial number of the certificate that Caddy serves. It equals the serial number from the previous step. The `-k` option skips the verification of the certificate chain.
|
||||
|
||||
```bash
|
||||
curl -k https://localhost/
|
||||
openssl s_client -connect localhost:443 -servername web.example.com < /dev/null 2> /dev/null | openssl x509 -noout -serial -enddate
|
||||
```
|
||||
|
||||
4. Force a renewal by removing the certificate, and read the serial number again. It differs from the previous one, so Caddy serves the new certificate and the hook works.
|
||||
|
||||
```bash
|
||||
sudo rm /var/lib/vault-cert/caddy/client_cert.pem
|
||||
sudo systemctl start vault-cert-renew@caddy.service
|
||||
openssl s_client -connect localhost:443 -servername web.example.com < /dev/null 2> /dev/null | openssl x509 -noout -serial -enddate
|
||||
```
|
||||
|
||||
A run that finds a valid certificate logs `nothing to do`, a renewal logs the new expiry date and the hook. To run the renewal on schedule, enable the timer of the instance as in section 4.
|
||||
|
||||
## 4. Renew the WIF client certificate
|
||||
|
||||
Role: **Server administrator**
|
||||
|
||||
The client certificate for the WIF tests is issued by the `wif-client` role and needs no hook. Its settings are in `/etc/vault-cert/wif.env`.
|
||||
|
||||
1. Run the first renewal by hand and read its log.
|
||||
|
||||
```bash
|
||||
sudo systemctl start vault-cert-renew.service
|
||||
sudo journalctl -u vault-cert-renew.service --no-pager
|
||||
sudo systemctl start vault-cert-renew@wif.service
|
||||
sudo journalctl -u vault-cert-renew@wif.service --no-pager
|
||||
```
|
||||
|
||||
2. Check the certificate.
|
||||
|
||||
```bash
|
||||
sudo openssl x509 -in /var/lib/vault-cert/client_cert.pem -noout -subject -issuer -enddate
|
||||
sudo openssl x509 -in /var/lib/vault-cert/wif/client_cert.pem -noout -subject -issuer -enddate
|
||||
```
|
||||
|
||||
3. Check the timer schedule.
|
||||
3. Enable the timer.
|
||||
|
||||
```bash
|
||||
systemctl list-timers vault-cert-renew.timer
|
||||
sudo systemctl enable --now vault-cert-renew@wif.timer
|
||||
```
|
||||
|
||||
4. Force a renewal and the hook by removing the certificate.
|
||||
4. Check the timer schedule.
|
||||
|
||||
```bash
|
||||
sudo rm /var/lib/vault-cert/client_cert.pem
|
||||
sudo systemctl start vault-cert-renew.service
|
||||
systemctl list-timers 'vault-cert-renew@*'
|
||||
```
|
||||
|
||||
The service logs to the journal. A run that finds a valid certificate logs `nothing to do`, a renewal logs the new expiry date and the hook.
|
||||
|
||||
## Next steps
|
||||
|
||||
1. Create the credentials configuration `cred.json` on the server for the certificate and the key managed by the renewal service. The names come from `env.sh`.
|
||||
@@ -247,8 +277,8 @@ The service logs to the journal. A run that finds a valid certificate logs `noth
|
||||
gcloud iam workload-identity-pools create-cred-config \
|
||||
"projects/${PROJECT_NUMBER}/locations/global/workloadIdentityPools/${POOL_ID}/providers/${PROVIDER_ID}" \
|
||||
--service-account="${SA_EMAIL}" \
|
||||
--credential-cert-path=/var/lib/vault-cert/client_cert.pem \
|
||||
--credential-cert-private-key-path=/var/lib/vault-cert/client_key.pem \
|
||||
--credential-cert-path=/var/lib/vault-cert/wif/client_cert.pem \
|
||||
--credential-cert-private-key-path=/var/lib/vault-cert/wif/client_key.pem \
|
||||
--output-file=cred.json
|
||||
```
|
||||
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
[Unit]
|
||||
Description=Renew the %i certificate from Vault
|
||||
Wants=network-online.target
|
||||
After=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
EnvironmentFile=/etc/vault-cert/%i.env
|
||||
ExecStart=/usr/local/sbin/vault-cert-renew
|
||||
PrivateTmp=true
|
||||
NoNewPrivileges=true
|
||||
ProtectHome=true
|
||||
@@ -0,0 +1,10 @@
|
||||
[Unit]
|
||||
Description=Check the %i certificate expiry four times a day
|
||||
|
||||
[Timer]
|
||||
OnCalendar=*-*-* 00,06,12,18:00:00
|
||||
RandomizedDelaySec=30min
|
||||
Persistent=true
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
@@ -44,6 +44,19 @@ vault write pki_int/roles/wif-client \
|
||||
max_ttl=8760h \
|
||||
no_store=true
|
||||
|
||||
# Create a role that issues server authentication certificates
|
||||
vault write pki_int/roles/web-server \
|
||||
allow_any_name=true \
|
||||
enforce_hostnames=true \
|
||||
server_flag=true \
|
||||
client_flag=false \
|
||||
key_type=rsa \
|
||||
key_bits=4096 \
|
||||
key_usage=DigitalSignature,KeyEncipherment \
|
||||
ttl=8760h \
|
||||
max_ttl=8760h \
|
||||
no_store=true
|
||||
|
||||
# Export the CA certificates
|
||||
vault read -field=certificate pki/cert/ca > ca_cert.pem
|
||||
vault read -field=certificate pki_int/cert/ca > issuing_ca_cert.pem
|
||||
@@ -0,0 +1,5 @@
|
||||
:443 {
|
||||
tls /etc/caddy/server_cert.pem /etc/caddy/server_key.pem
|
||||
root * /var/www/web-test
|
||||
file_server
|
||||
}
|
||||
Executable
+6
@@ -0,0 +1,6 @@
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
|
||||
install -m 0644 -o root -g root "${CERT_FILE}" /etc/caddy/server_cert.pem
|
||||
install -m 0640 -o root -g caddy "${KEY_FILE}" /etc/caddy/server_key.pem
|
||||
systemctl reload-or-restart caddy.service
|
||||
@@ -0,0 +1,15 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Hook test</title>
|
||||
<script src="https://cdn.jsdelivr.net/npm/@tailwindcss/browser@4"></script>
|
||||
</head>
|
||||
<body class="min-h-screen bg-slate-100 flex items-center justify-center">
|
||||
<main class="max-w-md rounded-xl bg-white p-8 shadow-lg">
|
||||
<h1 class="text-3xl font-bold text-slate-900">Hook test</h1>
|
||||
<p class="mt-4 text-slate-600">This page is served over TLS with the certificate managed by the renewal service.</p>
|
||||
</main>
|
||||
</body>
|
||||
</html>
|
||||
Reference in new issue
Block a user