fix: the PKI is kept in ou=pki of the router's directory instead of files under /etc/vpn-router/pki, and the router's site-to-site certificate names only the identities its connections present, and /ca/bundle gives the certificate bundle of a CA.

ADR.md

@@ -27,28 +27,40 @@ port is reachable from the external interface only when it is in `exposed_ports`
 in `DOCKER-USER` are not touched, because the package owns only its own chain. Restarting Docker
 restores the rules. The rules cover IPv4 only.
 
-## ADR-2: A small status web service, bound to the internal address
+## ADR-2: A small status web service, on the interfaces web.listen chooses
 
 Status: Accepted
 
 Context: The router has no way to see its state without a shell. A check from the protected
-network, by a person or by a monitor, needs a page or an endpoint. The router is an
-internet-facing host, so what the service shows and where it listens matter.
+network, by a person or by a monitor, needs a page or an endpoint, and so does the platform that
+deploys the router, which reaches it on its public address. The router is an internet-facing
+host, so what the service shows and where it listens matter.
 
 Decision: The `vpn-router-web` unit runs one Python script on the standard library's
 `http.server`, with the package's own modules and the Python packages of the Ubuntu archive that
-the package depends on, and no web framework. It listens on the address of the internal interface and on
-`web.port`, 7081 by default, and is on unless `web.enabled` is `false` (ADR-7). `/` is for
-people, in plain text for `curl` and `wget` and as a Tailwind CSS page for a browser, and
-`/status` is the same status as JSON. It shows the state of the services, the package version
-and the state of the site-to-site, road-warrior, OpenVPN and WireGuard features, and nothing from
-the configuration. `vpn-router-setup` opens the port on the internal interface in UFW and writes
-the features and the facts it resolves to `/run/vpn-router/web.json`.
+the package depends on, and no web framework. It listens on `web.port`, 7081 by default, on the
+address of the internal interface, of the external interface or of both, as `web.listen` sets
+(`internal`, `external` or `both`, the default), with a server for each address, and is on
+unless `web.enabled` is `false` (ADR-7). `/` is for people, in plain text for `curl` and `wget`
+and as a Tailwind CSS page for a browser, and `/status` is the same status as JSON. Both show as
+much as `web.status_content` says, since the router is reachable from the internet and the
+operator decides how much it tells strangers: `basic`, the default because it tells the least,
+the overall status and the Client IP, `standard` also the package versions and the number of
+active and other services, and `full` also each service and the state of the site-to-site, road-warrior, OpenVPN and
+WireGuard features. The Client IP is the address the request came from, through the trusted
+proxies, so a person can see the address the router sees. Nothing comes from the configuration.
+`/status` names its content, so clients such as manage-ui and the Router Manager show what it
+has. `vpn-router-setup` opens the
+port on both interfaces in UFW, so the choice of `web.listen` is made by the service alone, and
+writes the features and the facts it resolves to `/run/vpn-router/web.json`.
 
-Consequences: Nothing outside the internal network can reach it, since it listens on the internal
-address only and the firewall opens the port on the internal interface only. `/` and `/status`
-have no authentication, which is acceptable because they are read only and show no
-configuration. It runs as root, because `swanctl` and `wg` need it, with `NoNewPrivileges`,
+Consequences: By default the service is reachable from the internet on the external address;
+`web.listen` set to `internal` keeps it to the internal network. It serves plain HTTP itself;
+HTTPS is Caddy's (ADR-21), so on the plain HTTP port the passwords of `/auth` and the tokens
+cross the internet unencrypted. A change of `web.listen`
+needs no run of setup, since the firewall does not depend on it; the package's upgrade runs
+setup, which writes the rules. `/` and `/status` have no authentication, which is acceptable because they are
+read only and show no configuration. It runs as root, because `swanctl` and `wg` need it, with `NoNewPrivileges`,
 `ProtectSystem=strict`, `ProtectHome` and `PrivateTmp`. The Tailwind style comes from a CDN, so a
 browser on a network without internet shows the unstyled page, and `curl` is unaffected.
 
@@ -60,12 +72,12 @@ Context: Clients such as the Router Manager, and people with `curl`, need the co
 the users and the groups of the router, which `/` and `/status` must not show. Both people and
 programs sign in with a user of the directory. A secret shared with the platform that deploys
 the router would let whoever holds it, and every copy of the deployment's state, make tokens.
-The service listens on plain HTTP on the internal network.
+The service listens on plain HTTP.
 
 Decision: `/config`, `/users` and `/groups` need a JWT in an `Authorization: Bearer` header,
 HS256, verified with `hmac` and `hashlib`; no other algorithm is accepted. The key is
 `/etc/vpn-router/web.key`, random, made by the web service on its first run and readable by root
-only; it is never configured or output, and `web.jwt_secret` is ignored. The router itself issues
+only; it is never configured or output. The router itself issues
 the tokens: `POST /auth` checks a user's password and TOTP code with a bind (ADR-14) and answers
 an access token naming the user in `sub`, valid for 5 minutes, and a refresh token valid for 8
 hours (ADR-15); `vpn-router-token` makes tokens with the `admin` claim (ADR-13). A token without
@@ -109,25 +121,30 @@ and it is backed up with `slapcat`, not by copying files.
 
 Status: Accepted
 
-Context: The router CA issues client certificates for IKEv2. OpenVPN can check a certificate, a
-password, or both. With both, a certificate alone and a password alone are not enough, but
-`openvpn-auth-ldap` checks only the password of the user name given at login, which need not be
-the user of the certificate.
+Context: The P2S issuing CA (ADR-27) issues the users' client certificates, for IKEv2 and
+OpenVPN. OpenVPN can check a certificate, a password, or both. With both, a certificate alone and
+a password alone are not enough, but `openvpn-auth-ldap` checks only the password of the user name
+given at login, which need not be the user of the certificate. OpenVPN trusts every chain to a
+CA of its `ca` file and has no option that accepts one intermediate CA only, while the root also
+signs the site-to-site and the general-purpose CAs.
 
-Decision: OpenVPN checks the certificate against the CA and the CRL, the password with
-`openvpn-auth-ldap`, and runs `openvpn-same-user` as `auth-user-pass-verify` with `via-env`,
-which accepts the login only when the user name equals the certificate's common name. OpenVPN
-calls both and needs both to pass. A user's certificate has the user name as its common name, is
-issued when the user is added and revoked when the user is removed. The CRL is created with the
-CA, so that `crl-verify` has a file from the start, and is valid for ten years, because the router
-renews it only when it revokes a certificate. The server uses the router's own certificate, whose
-common name is `wan.local_fqdn`; the client profile checks that name (`verify-x509-name`), since
-the CA's certificates carry both the server and the client purpose.
+Decision: OpenVPN checks the certificate against the root and the P2S CA and their CRLs, which
+it needs for every CA of the chain, the password with `openvpn-auth-ldap`, and runs
+`openvpn-same-user` as `auth-user-pass-verify` with `via-env`, which accepts the login only when
+the user name equals the certificate's common name. `openvpn-issuer`, as `tls-verify`, accepts
+the certificate at depth 1 only when its SHA-256 fingerprint, which OpenVPN gives the script,
+is the P2S CA's. OpenVPN calls all of them and needs all to pass. A user's certificate has the
+user name as its common name, is issued with the template `vpn-user`, client authentication
+only, when the user is added, and revoked when the user is removed. The server uses the router's
+VPN server certificate, whose common name is `wan.local_fqdn`; the client profile checks that
+name (`verify-x509-name`).
 
 Consequences: Disabling a user locks the password, which stops OpenVPN but not IKEv2, which
-checks certificates only; removing a user revokes the certificate for both. A user's
-certificate expires after the CA's default of a year and is renewed by removing and adding the
-user again.
+checks certificates only; removing a user revokes the certificate for both. A certificate of
+another issuing CA with a user's name does not reach the password check. A user's certificate
+expires after the days of `vpn-user` and is renewed by removing and adding the user again. An
+expired root CRL stops every OpenVPN client, so an offline root needs its CRL renewed in time
+(ADR-28).
 
 ## ADR-6: The web service and vpn-router-user share one module for users
 
@@ -135,19 +152,24 @@ Status: Accepted
 
 Context: Users are managed from the command line, by an operator at a terminal, and through
 `/users` of the web service, by the Router Manager. Both must apply the same rules to names,
-passwords and person attributes, and both change the directory and the CA. `simple-ca` keeps
-the CA's state in one file, so two changes must not run at the same time. The web unit is
-sandboxed with `ProtectSystem=strict`.
+passwords and person attributes, and both change the directory and the PKI, whose CRL numbers and
+CRLs two changes must not write at the same time. The web unit is sandboxed with
+`ProtectSystem=strict`.
 
 Decision: `vpnrouter_users.py` holds the user operations, the checks and a session that takes a
 lock and the connection of the user manager account. The web service imports it and maps a
 `Failure` to the HTTP status; `vpn-router-user` imports it and adds the argument parsing and
 the output, text for people and JSON with `--json`. The web unit allows writing the paths the
-operations need: `/run/vpn-router` for the lock, `/etc/vpn-router/pki` for the certificates and
-the CA's state, and `/etc/swanctl/x509crl` for the published CRL.
+operations need: `/run/vpn-router` for the lock, its `RuntimeDirectory`, which systemd creates
+before the service starts, also before setup has run, and keeps when it stops, as setup's state
+files are there too, and `/etc/swanctl/x509crl`, `/etc/openvpn/server` and
+`/var/lib/vpn-router/pki-publish` for the copies of the CRLs a revocation signs; the PKI itself is
+in the directory (ADR-32). The PKI module, `vpnrouter_pki.py`, works the same way for `/ca`,
+`vpn-router-ca` and `vpn-router-setup`, and its lock is the users' lock, as adding and removing
+a user issue and revoke a certificate.
 
-Consequences: The rules exist once. A request starts no process, and a password never leaves
-the process that received it. The web service imports `python3-ldap` and the package's
+Consequences: The rules exist once. A request starts no process but `swanctl --load-creds` after
+a revocation, and a password never leaves the process that received it. The web service imports `python3-ldap` and the package's
 configuration module. The API is protected by the tokens of ADR-3, so only members of `admins`
 and the holders of an admin token manage users.
 
@@ -156,22 +178,23 @@ and the holders of an admin token manage users.
 Status: Accepted
 
 Context: The status page is the tool to see a problem of `vpn-router-setup`, so it must run
-whatever state setup is in, also before setup has run and while it fails. The internal address,
-`web.port` and `web.enabled` can change at any time.
+whatever state setup is in, also before setup has run and while it fails. The addresses,
+`web.listen`, `web.port` and `web.enabled` can change at any time.
 
 Decision: The service depends on nothing to start and does not end on its own. A discovery
-thread works out, every 30 seconds, the internal address from `vpn-router.json` as
-`general.mode` directs, with the discovery code in `vpnrouter_net.py`, which `vpnrouter.py` uses
-too, and `web.port` and `web.enabled` from the same file. It runs the HTTP server in a thread of
-its own at that address and port, stops it and starts a new one when they change, and runs none
-while the service is disabled, the address is not known or it cannot be bound. Each request
+thread works out, every 30 seconds, the addresses of the interfaces `web.listen` names from
+`vpn-router.json` as `general.mode` directs, with the discovery code in `vpnrouter_net.py`,
+which `vpnrouter.py` uses too, and `web.port` and `web.enabled` from the same file. It runs an
+HTTP server in a thread of its own at each address and the port, stops one and starts a new one
+when they change, and runs none while the service is disabled or no address is known; an address
+that cannot be bound is tried again the next round. Each request
 reads the signing key and `vpn-router.json`, and the features and resolved facts from
 `web.json`, which setup writes when it applies the configuration, so the service shows what setup
 applied; while that file is missing the router shows no features. The unit is not
 ordered after `vpn-router-setup`, and setup does not restart it.
 
-Consequences: The status page answers as soon as the internal address exists, whatever state
-setup is in, and follows a changed address, port or `web.enabled` within 30 seconds without a
+Consequences: The status page answers as soon as an address exists, whatever state setup is in,
+and follows a changed address, `web.listen`, port or `web.enabled` within 30 seconds without a
 restart. A new signing key applies to the next request. The interfaces are read every 30
 seconds, which costs a few `ip` commands.
 
@@ -184,10 +207,15 @@ that a secret is set tells a reader more than it needs.
 
 Decision: The configuration that `vpn-router-setup` writes for the web service has no secret
 options at all. They are recognised by name: `psk`, `database`, whose connection string may hold
-a password, and options with `secret`, `password`, `token` or a `_key` ending in the name.
+a password, and options with `secret`, `password`, `token` or a `_key` ending in the name. The
+names are checked at every level, so the settings of each site-to-site connection lose their
+`psk` too; an object is kept whatever its name, so a connection named like a secret is still
+shown.
 
-Consequences: No secret reaches the web service's state file or any answer of the API. A client
-cannot tell whether a secret is set. A new secret option needs a name that matches.
+Consequences: No secret reaches the web service's state file or any answer of the API. Through
+`/config` a client cannot tell whether a secret is set; `/site-to-site` says whether each
+connection has a key (ADR-25), as an admin managing connections needs to know. A new secret
+option needs a name that matches.
 
 ## ADR-9: The Router Manager is a separate package with a small Python server
 
@@ -207,9 +235,9 @@ router's `/auth`, keeps their tokens in a server-side session in PostgreSQL behi
 session's access token, refreshing it as needed; it holds no logic of the router's own and no
 key of it. It reads its settings from `/etc/vpn-router-manager/manager.json` at each request; on
 the router the settings not given come from `vpn-router.json`, and it listens on the internal
-address like the status web service, with the same discovery thread. `vpn-router-setup` opens
-the manager's port on the internal interface while the manager is installed, so the firewall
-stays in one place.
+address, the external address or both, as its own `listen` sets, like the status web service,
+with the same discovery thread. `vpn-router-setup` opens the manager's port on the internal and
+the external interface while the manager is installed, so the firewall stays in one place.
 
 Consequences: The router needs no Node.js; the package needs only `python3` and packages of the
 Ubuntu archive. Tokens never reach the browser, and sessions survive a restart of the manager.
@@ -225,12 +253,13 @@ Context: Users are prepared before a P2S system is switched on, and the Router M
 them on any configured router, with or without one.
 
 Decision: Once a platform is chosen, `vpn-router-setup` always provisions the user directory in
-slapd and the CA with its CRL; the server certificate follows once `wan.local_fqdn` is set.
-`/users` and `vpn-router-user` rely only on the directory and the CA. Only the OpenVPN client
-profile needs OpenVPN to be enabled.
+slapd and the PKI (ADR-27) with its CRLs; the router's certificates follow once `wan.local_fqdn`
+is set. `/users` and `vpn-router-user` rely only on the directory and the P2S CA. Only the
+OpenVPN client profile needs OpenVPN to be enabled.
 
-Consequences: slapd always holds the directory and the CA always exists on a configured router,
-even when no P2S system uses them. An operator who wants a CA of their own must place it before
+Consequences: slapd always holds the directory and the CAs exist on a configured router, even
+when no P2S system uses them; with an offline root the P2S CA, and so the users' certificates,
+wait for its certificate. An operator who wants a root CA of their own must place it before
 choosing the platform.
 
 ## ADR-11: Groups are groupOfNames entries that never become empty
@@ -318,8 +347,8 @@ within 15 minutes; a successful bind clears the count; `vpn-router-user unlock`
 `/auth` answers the same 401 for a wrong name, password or code and for a locked or disabled
 user. `/auth` and `/auth/refresh` allow 10 calls a minute per client address, counted in memory,
 and answer 429 with `Retry-After` beyond that. The client address is the peer, or the last
-`X-Forwarded-For` entry for a peer in `web.trusted_proxies`, by default the address the service
-listens on, where a manager on the router calls from.
+`X-Forwarded-For` entry for a peer in `web.trusted_proxies`, by default the address the call came
+in on, which is the address a manager on the router calls from.
 
 Consequences: One lockout covers OpenVPN and `/auth`, kept by slapd alone. Someone who knows a
 user name can lock the user out for 15 minutes at a time. The rate limit restarts with the
@@ -348,26 +377,521 @@ no usable token. Changing a user's password, disabling, unlocking or removing th
 signing out, end the user's sessions at once; an access token already issued lives out its
 5 minutes.
 
-## ADR-16: Each service creates what it needs on its first run and shows its starting phase
+## ADR-16: Each service creates what it needs on its first run
 
 Status: Accepted
 
 Context: The package manager configures packages in an order of its own; on a new VM the Router
 Manager was configured before the PostgreSQL server, so a database created at installation was
-missing. PostgreSQL may also start after the services at boot. A router whose setup fails can
-be unreachable, and the reason must be visible without a shell where possible.
+missing. PostgreSQL may also start after the services at boot.
 
 Decision: The web service makes its signing key and creates the refresh tokens' database itself,
 and the Router Manager its sessions database; neither `vpn-router-setup` nor a maintainer script
-creates them. Each service starts with a starting phase: an initialization thread does the work
-and retries every 15 seconds until it succeeds, and meanwhile the service listens as usual but
-serves only a page of its phase and the failure of the last attempt, answering 503. The web
-service also shows the state, the step and the failure of the last run of `vpn-router-setup`,
-which setup records in `/run/vpn-router/setup.json` at every step; `vpnrouter.fail` ends with a
-`SystemExit` carrying its message, so setup records it. Once the initialization succeeds, the
-service serves its real pages.
+creates them. Each service does it before it listens, retrying every 15 seconds and logging each
+failure, then listens. `vpnrouter_auth` turns every failure of the database into
+`DatabaseError`, the only database exception the web service and `vpn-router-user` handle.
 
-Consequences: The order of installation and of the start of PostgreSQL does not matter; a
-database that cannot be created shows why on the status page and is created once the cause is
-gone. The web unit may write `/etc/vpn-router` for the key and the database password. While the
-web service starts, `/status` is not the router's status, and clients see 503.
+psycopg (Ubuntu ships only its pure-Python implementation) finds libpq through the dynamic linker
+cache, which dpkg refreshes with its deferred `ldconfig` only at the end of an installation, so a
+service started during the installation cannot import it yet; a failed import cannot be repeated
+in the same process either, as psycopg caches the lookup. The services therefore import psycopg
+plainly, and their units wait instead: `ExecStartPre=wait-for-libpq`, a shell script of each
+package, waits until `ldconfig -p` lists `libpq.so.5`, within `TimeoutStartSec=5min`. The
+maintainer scripts start the units with `systemctl --no-block` (debhelper's start is turned off),
+because a start that waited would hold dpkg before its `ldconfig`.
+
+Consequences: The order of installation and of the start of PostgreSQL and of the linker cache
+does not matter; a database that cannot be created is retried until the cause is gone, and the
+reason is in the service's log. Outside an installation the wait ends at once. The web unit may
+write `/etc/vpn-router` for the key and the database password.
+
+## ADR-17: Bootstrap diagnostics run only until the first setup succeeds
+
+Status: Accepted
+
+Context: A new VM is reachable only once its setup has worked, and the reason of a failure must
+be readable from the internal network before an SSH session is possible. A service that reports
+the setup cannot be one of the services being set up: those import packages that may not load yet
+(psycopg could not find libpq before `ldconfig` ran), and they would fail before reporting
+anything. Once the router has been set up, SSH is the way to diagnose it. The more a diagnostic
+page does, the more ways it has to fail itself.
+
+Decision: Each package ships a bootstrap service of the Python standard library alone, a script
+that answers every request with the plain-text output of a fixed list of commands: the setup
+state, the services, the failed units, the addresses and the end of the logs. The router's
+listens on 17081 and the Router Manager's on 17080, beside the ports of the services, 7081 and
+7080, on every address. The units are never enabled and not started at installation;
+`vpn-router-setup` starts them while `/var/lib/vpn-router/setup.done` does not exist, and opens
+their ports on the internal interface. The first successful run writes that file, closes the ports
+and stops them; a failed run leaves them running, so the reason stays readable. Setup records its
+state, step and failure in `/run/vpn-router/setup.json` at every step; `vpnrouter.fail` ends with
+a `SystemExit` carrying its message, so setup records it.
+
+Consequences: The diagnostics need nothing the setup provides and cannot collide with the real
+services. After the first success they never run again, also not after an upgrade or a reboot.
+The page shows log lines to whoever reaches the internal network during the first setup.
+
+## ADR-18: Manager-only accounts are a second kind of user; the package can create the first admin
+
+Status: Accepted
+
+Context: The Router Manager needs a first administrator before anyone has added users, as other
+packages create a default user from their debconf answers. Such an account, and others that only
+manage the router, have no person behind them and need no VPN access. A VPN user is an
+`inetOrgPerson`, which requires a last name and a common name, and its certificate needs the
+names. The Manager is configured before the router's directory exists during an installation.
+
+Decision: A user is one of two kinds in `ou=users`, fixed when it is added: a VPN user, an
+`inetOrgPerson` with the three person attributes and a certificate, or a Manager-only account,
+an `account`, `simpleSecurityObject` and `vpnRouterAccount` (ADR-19) with the name, the password,
+an optional e-mail address for notifications, and no certificate. `add` with all three person
+attributes adds a VPN user and without the first and the last name a Manager-only account; one
+name without the rest is refused. The CLI and the API add both kinds; the Manager's dialog
+requires the names unless Manager access only is chosen, which offers the e-mail address only.
+Answers carry `manager_only`, and `update` changes only the e-mail address of a Manager-only
+account. The Manager's package asks `admin_username` and `admin_password`; with both
+it writes them to a root-only file, clears the answers, and `vpn-router-setup` creates the account
+after it provisions the directory, adds it to `admins` (creating the group when missing) and
+removes the file. Without a name nothing is created. `vpnrouter_users` knows only the directory,
+so setup can use it before PostgreSQL's driver can be loaded; `web` and `vpn-router-user` revoke
+refresh tokens after the operations of `vpnrouter_auth.REVOKING`.
+
+Consequences: A Manager-only account cannot become a VPN user, as LDAP cannot change an entry's
+structural class and its password hash cannot be copied; it is removed and added again instead.
+It has no certificate, so OpenVPN and road-warrior access refuse it. The administrator's password
+is on disk only until setup has run.
+
+## ADR-19: The package has an LDAP schema of its own under a UUID OID
+
+Status: Accepted
+
+Context: A Manager-only account (ADR-18) is an `account`, which allows no `mail`, yet notifications
+need an address for it; no standard auxiliary class allows `mail` except `extensibleObject`, which
+allows every attribute. Authorization scopes for users of both kinds may follow. An attribute or
+class of its own needs an OID, and the package has no registered arc.
+
+Decision: The package defines its schema, `cn=vpn-router,cn=schema,cn=config`, which
+`vpnrouter_ldap.provision` adds when missing and keeps as `SCHEMA_ATTRIBUTE_TYPES` and
+`SCHEMA_OBJECT_CLASSES`. Its OID is `2.25.98477352595735834532404256531620013001`, made from a
+random UUID under the arc 2.25 (ITU-T X.667), which needs no registration; attribute types are
+under `.1` and object classes under `.2`. It has the attribute `vpnRouterScope` (multi-valued,
+case-insensitive strings) and the auxiliary class `vpnRouterAccount`,
+`MAY ( mail $ vpnRouterScope $ displayName $ description )`; `displayName` and `description`, the
+standard attributes, are the display name and the description of a service account, which
+`account` cannot hold. Manager-only accounts have the class from their creation, so an e-mail
+address can be added later without changing their classes; VPN users get it once scopes are used.
+`vpnrouter_ldap.provision` replaces the definitions in `cn=config` that differ, so an extended class
+reaches an existing directory at the next setup run.
+
+Consequences: The schema check stays strict. New attributes extend `vpnRouterAccount`, and their
+OIDs continue the arcs. The scope `service` marks a service account (ADR-23). The OID must never
+be reused for anything else.
+
+## ADR-20: Users manage their own account through /account, proving who they are again
+
+Status: Accepted
+
+Context: Users who are not members of `admins` can sign in but get 403 on every path, so they
+cannot change their password or enrol an authenticator without an administrator. A session
+token alone is a weak proof for such changes: whoever holds an open session could lock the user
+out. A wrong current password must not look like an ended session to the Manager, which signs a
+user out on 401.
+
+Decision: `/account` serves the user named in the token's `sub`, admin or not: `GET` the user
+and whether they are a member of `admins`, `PUT /account/password`, and `POST` and
+`PUT /account/otp` to enrol or replace the TOTP enrolment. The routes do not take a user name.
+Each change sends the current password and, for an enrolled user, a current code, checked with a
+bind as the user like `/auth`, so it counts toward the lockout (ADR-14); a wrong one is 403. A
+new password is checked before the bind, so an invalid one does not use up a code. A user cannot
+remove their own enrolment; `/users` and `vpn-router-user` do. Tokens of `vpn-router-token` name
+no user and get 403 there.
+
+Consequences: slapd's `otp` overlay accepts each code once, so a change right after signing in
+needs the next code. Changing the password revokes the user's refresh tokens, so the user's
+Manager session ends with its access token. An enrolled user who has lost the authenticator
+needs an administrator.
+
+## ADR-21: Caddy from the Ubuntu archive serves HTTPS with a certificate from Let's Encrypt
+
+Status: Accepted
+
+Context: The status web service and the Router Manager are reachable from the internet (ADR-2),
+and `/auth` and the Manager's sign-in carry passwords. Both are small servers on the standard
+library's `http.server`, which would need certificates, their renewal and a TLS setup of their
+own. Browsers trust a public CA, not the router's.
+
+Decision: Caddy, the `caddy` package of the Ubuntu archive, is the HTTPS proxy. While
+`https.enabled` is `true` (the default) and `wan.local_fqdn` is set, `vpn-router-setup` writes
+`/etc/caddy/vpn-router.Caddyfile`: a site for `wan.local_fqdn` on `https.web_port` (7444) for the
+web service and, while the Router Manager is installed, one on `https.manager_port` (7443) for
+the manager, each passing calls to the service on `127.0.0.1`. Caddy obtains and renews the
+certificate from Let's Encrypt with the HTTP challenge on port 80, which the firewall opens on
+the external interface; `auto_https disable_redirects` keeps port 80 to the challenge. The
+package's drop-in for `caddy.service` points it at that file and keeps it from starting before
+the file exists, so `/etc/caddy/Caddyfile`, a configuration file of the `caddy` package, is left
+unchanged. Both services always listen on `127.0.0.1` as well as on the interfaces their
+`listen` settings name, and trust `X-Forwarded-For` from it, which Caddy sets from the peer and
+never takes from the client.
+
+Consequences: HTTPS needs `wan.local_fqdn` in public DNS pointing at the router and port 80
+reachable from the internet; until then Caddy retries and the HTTPS ports do not answer. The
+certificate and the ACME account are in `/var/lib/caddy`. Plain HTTP stays as the `listen`
+settings set it, so `internal` keeps passwords off the internet. Caddy's admin endpoint listens
+on `localhost:2019`, which `caddy reload` uses. Port 80 and the HTTPS ports are not available
+to Docker containers while HTTPS is on. The Manager's HTTPS port is a setting of the router, not
+of the manager.
+
+## ADR-22: Metrics are kept in PostgreSQL and served in the Prometheus format, pushed with OTLP
+
+Status: Accepted
+
+Context: The router should keep its CPU, memory, disk, network, load and VPN client metrics for
+hour, day, month and year views, as Proxmox VE does, and expose them to OpenTelemetry. Ubuntu
+noble has no OpenTelemetry Python SDK and no OpenTelemetry Collector package. OTLP itself is
+push-only; a collector reads a scraped endpoint in the Prometheus format with its `prometheus`
+receiver. The router already runs PostgreSQL for the refresh tokens (ADR-15).
+
+Decision: A service of its own, `vpn-router-metrics`, takes a sample every minute with `psutil`
+(`python3-psutil`) and the VPN readers it shares with the web service (`vpnrouter_status`). It
+keeps the samples in the router's database, in four detail tables (`metrics_system`,
+`metrics_network`, `metrics_vpn`, `metrics_site_to_site`, see ADR-25). Each detail table has hourly, daily, monthly and yearly summary
+tables, with the average and the maximum of each value per period, written when the clock passes a
+period from the level below with `INSERT ... ON CONFLICT DO NOTHING`, so a stopped service catches
+up. Garbage collection runs on start and once a day, not with every write, and deletes the rows
+older than the retention of their level, `metrics.keep_*`, PostgreSQL intervals. The service serves
+the last sample as `/metrics` in the Prometheus text format with `prometheus_client`
+(`python3-prometheus-client`) and pushes it as OTLP/HTTP JSON, built with the standard library, to
+`metrics.otlp_endpoint`. The names follow the OpenTelemetry semantic conventions. The web service
+answers the views from the tables at `/metrics/history`, protected like `/config`, for the Router
+Manager.
+
+Consequences: The metrics live in the database of the refresh tokens and are dropped with it on
+purge. A view shows only what its level keeps, so a shorter retention empties the older part of a
+view. A month view needs the hourly rows and a year view the monthly rows. `/metrics` has only the
+last sample; history is not scraped.
+
+## ADR-23: /metrics takes HTTP Basic authentication of an internal service account the router manages
+
+Status: Accepted
+
+Context: An OpenTelemetry Collector's `prometheus` receiver authenticates with HTTP Basic, not with
+the router's tokens, which last minutes (ADR-3). The scraper should not be able to sign in to the
+API or the Manager, and its password should be checked and locked out like any other. The account
+belongs to the package's metrics, not to the people using the router.
+
+Decision: A service account is a Manager-only account (ADR-18) with `vpnRouterScope` `service`,
+which another system signs in with to use a router service. Administrators add service accounts
+with `vpn-router-user add --service` and the web service's `/service-accounts`.
+`/service-accounts` is a path of its own, apart from `/users`, which does not list or serve them:
+a service account has no names, certificate, profile or TOTP, and is not a person an
+administrator manages with the users. A service account has a display name and a description
+besides its name, so an administrator can tell what each is for. Internal service accounts are
+those the router creates and manages for its own services (`vpnrouter_users.internal_accounts`),
+with every property, display name and description included, so the system, not an
+administrator, owns them; at present that is the metrics service's, which setup names Metrics. `/service-accounts` marks
+them as `internal` and only lists them: every other request on one, and putting one in or taking
+one out of a group at `/groups`, gets 403, whatever the token. The router changes them through the
+directory's user-manager account, not the API, and root may still with `vpn-router-user`. For the
+metrics service, `vpn-router-setup` adds the account `metrics.service_account` names, `metrics` by
+default, while metrics are enabled, with a random password it writes to
+`/etc/vpn-router/metrics.secret` (root only), and sets a new random password, ending a lockout,
+when the file is missing or no longer holds the password. `/auth` refuses every service account with the message of a wrong password. `/metrics` accepts HTTP Basic only for that account, and
+only when it is a service account; the password is checked with a bind as the account, so the
+password policy's lockout applies (ADR-14). TLS is Caddy's, on `metrics.https_port` (ADR-21);
+plain HTTP is served on the interfaces `metrics.listen` names, `internal` by default.
+
+Consequences: Every scrape binds to slapd once. The collector's operator copies the password from
+the router; deleting the file and running setup changes it. A user who already has the name is not
+taken over; setup fails until `metrics.service_account` names another. An account root disabled
+stays disabled. A scraper on the external interface over plain HTTP sends the password in the clear, so
+`metrics.listen` stays `internal` unless HTTPS is used.
+
+## ADR-24: The Router Manager upgrades the router's packages in a transient unit outside its own
+
+Status: Accepted
+
+Context: An admin should upgrade `vpn-router` and `vpn-router-manager` from the Router Manager. The
+upgrade restarts the process that starts it: the manager's postinst restarts
+`vpn-router-manager.service`, and the router's restarts `vpn-router-web` and `vpn-router-setup`. A
+child of the manager would be stopped with the manager's cgroup in the middle of dpkg, leaving
+packages half configured, and the manager's unit cannot run apt at all, since `ProtectSystem=strict`
+makes `/usr` and `/var/lib/dpkg` read-only. Ubuntu's `apt-daily` timer updates the package lists
+twice a day, so a release newer than its last run is not seen until the next one.
+
+Decision: The manager starts the helper `/usr/lib/vpn-router-manager/upgrade` with `systemd-run` as
+the transient unit `vpn-router-manager-upgrade`, of its own cgroup and without a sandbox, and does
+not wait for it. The helper:
+
+- finishes an interrupted dpkg run with `dpkg --configure -a`;
+- runs `apt-get install --only-upgrade` of the two packages, non-interactive, keeping changed configuration files, with `needrestart` suspended, and waiting for the dpkg lock that `apt-daily` or `unattended-upgrades` may hold;
+- records the state and the result in `/var/lib/vpn-router-manager/upgrade.json`, replaced by a rename, and the output in `upgrade.log`.
+
+A restarted manager reads them again. A state of running without an active unit is shown as interrupted. The manager reboots the router the same way, as a transient timer `vpn-router-manager-reboot` 5 seconds later, so the answer reaches the browser first. It checks for updates the same way too: Check update runs `apt-get update` in the transient unit `vpn-router-manager-check`, which touches apt's update stamp when it succeeds, so an admin need not wait for the `apt-daily` timer. Only members of `admins`, as the router's `/account` answers, may start any of them.
+
+Consequences: The fixed unit name allows one upgrade at a time. The page loses the manager and the
+router's web service for a while during an upgrade and reads again until they answer. Sessions
+survive in PostgreSQL. The manager can upgrade only the system it runs on, so a manager on another
+host upgrades that host's packages, not the router's. The operating system's other packages are
+left to unattended-upgrades.
+
+## ADR-25: Site-to-site connections are a map of named connections, edited through the API and applied by setup
+
+Status: Accepted
+
+Context: The router had one IPsec site-to-site tunnel, the `remote` section, run as the swanctl
+connection `remote-site`, with the local identity mode in `wan` and the local subnets in `local`.
+A router often links more than one site, each with its own peer, subnets and key, and an admin
+should add them without a shell. The install answers and the Cloud Platform describe one
+connection, and the platform rewrites `vpn-router.json` when it rebuilds the VM.
+
+Decision: `site_to_site` maps a connection name to its settings: `addrs`, `id`, `cidrs`,
+`local_cidrs`, `local_id_mode`, `psk` and `psk_file`. `remote` and `wan.local_id_mode` are gone,
+without a fallback for the old format. The name, a lowercase letter followed by at most 30
+lowercase letters, digits and hyphens, names the swanctl connection, its child and its secret
+`ike-psk-<name>`, so it is a valid swanctl section name and stays the same while the connection
+lives. The install answers and the Cloud Platform write the connection `remote-site`, the first
+one, so its name is the one the tunnel always had.
+
+- **Local settings per connection:** each connection has its own `local_cidrs` and
+  `local_id_mode`, as each peer expects its own traffic selectors and identity. `wan.local_fqdn`
+  stays the router's one name, used by the `fqdn` mode, resolved once for the `public_ip` mode,
+  and used by HTTPS, road-warrior and OpenVPN. `local.cidrs` stays the subnets the router
+  routes, masquerades and offers to road-warrior and OpenVPN clients.
+- **One file:** `vpn-router-setup` writes every connection that is ready to
+  `/etc/swanctl/conf.d/site-to-site.conf`, with the proposals and timers the one tunnel had, and
+  removes the file when none is. The NAT exemptions pair each connection's `local_cidrs` with its
+  `cidrs`.
+- **Editing:** the status web service serves `/site-to-site` to members of `admins` and admin
+  tokens, like `/users`. It never returns a key, only whether one is set, and takes a new key only
+  when one is given. `vpn-config-edit` edits the same map. Both read, check with the validation
+  of `vpn-router-setup` and write `vpn-router.json` holding one lock, replacing the file in one
+  step, so neither loses the other's change.
+- **Applying:** the web service restarts `vpn-router-setup` without waiting for it, the one
+  place that renders, loads and opens the firewall, so a change through the API takes the same
+  path as a change by hand.
+- **Metrics per connection:** the metrics service keeps, per connection, whether it is up, the
+  bytes and packets per second through its child SAs and the seconds its IKE SA has been up, in
+  a detail table of its own, `metrics_site_to_site`, keyed by the connection's name, with the
+  summaries and retention of the other tables (ADR-22). One row per connection keeps the values
+  of a tunnel together and lets the set of connections change without changing the schema;
+  `metrics_vpn` counts only clients. swanctl's child SA counters start again at a rekey, so the
+  service keeps rates computed between two samples, treating a lower counter as counted from
+  zero, and `/metrics` and the OTLP push serve them as gauges rather than counters that would
+  appear to reset.
+
+Consequences: A configuration with `remote` is refused, so the package and a configuration in
+the new format go together. The status shows each connection. A rate across a rekey misses the
+traffic of the old child SA since the previous sample. A renamed connection starts a new series.
+A connection added through the API lives in
+`vpn-router.json` only; the Cloud Platform's rebuild of the VM writes the file again with its
+one connection, dropping the others. A change through the API is answered before setup has
+applied it, so a configuration that setup cannot apply shows in the setup state, not in the
+answer. A connection with a key file keeps it until a key is given through the API, which then
+replaces the file.
+
+## ADR-26: Backups are a staged, consistent copy, archived locally with tar and zstd or sent to a Proxmox Backup Server
+
+Status: Accepted
+
+Context: An image backup of the VM, such as Azure Backup, restores the whole disk. It cannot be
+exported as a file, read on another system, or restored onto a new VM. The state that matters is
+small, but it is spread over several places: `vpn-router.json` and the secrets, the PKI, the
+WireGuard keys and peers, the firewall rules, the slapd directory and, optionally, the
+PostgreSQL database. Only part of it can be made again: the CA, the password hashes and the TOTP
+secrets cannot. The API and the command-line tools change the configuration, the PKI and the
+directory while the router runs.
+
+Decision: `vpn-router-backup run`, started by a timer on `backup.schedule`, stages a copy of that
+state in one directory and then archives or sends it.
+
+- **Consistency:** the files are copied and the directory is written with `slapcat` while the
+  configuration lock and the users lock are held, the locks every change of the API and the tools
+  takes, so a user, their certificate and the CRL are copied together. `slapcat` reads the MDB
+  database consistently while slapd runs (ADR-4), and `pg_dump` reads a snapshot. Files that
+  `vpn-router-setup` regenerates are left out.
+- **Local target:** a tar archive compressed with zstd, both from the Ubuntu archive, so a local
+  backup needs nothing else and the archive can be read on any Linux system. A `pxar` archive
+  would need Proxmox's client and is not compressed.
+- **Retention:** local archives are pruned by the rules of Proxmox Backup Server's prune, so the
+  settings mean the same for both targets and can be checked with Proxmox's prune simulator.
+  Settings finer than the schedule do not apply.
+- **Cloud copy:** rclone, from the Ubuntu archive, copies the archives to any storage it
+  supports. The remote is defined in `/etc/vpn-router/rclone.conf` with `rclone config`, since
+  each backend has its own settings. `vpn-router.json` names only the remote, and the
+  configuration file is part of the backup. `sync` makes retention apply to the remote as well;
+  `copy` leaves it to the storage's lifecycle rules. Only this host's archives are included, so a
+  shared remote path is safe.
+- **Proxmox Backup Server:** the static `proxmox-backup-client` sends the staged directory as one
+  `pxar` archive of a host snapshot. Deduplication makes frequent backups cheap. The server's
+  prune jobs keep the snapshots, so the token needs only `DatastoreBackup`, which cannot delete.
+  The client is a suggestion, not a dependency, because it comes from Proxmox's repository, which
+  an Ubuntu system does not know. Without it, only a PBS backup fails.
+- **Restore:** only on the command line, as root. It replaces the directory and the database, so
+  it stops their services, keeps what it replaces and lets `vpn-router-setup` rebuild the rest.
+  The API offers no restore, so a stolen admin session cannot replace the router's keys.
+
+Consequences: A backup holds every secret of the router, including the keys of the issuing CAs
+and the root key while it is on the router, in plain form
+unless a PBS key file encrypts it, so where it is kept must be as safe as the router. A restore
+onto another VM brings back the old interface names and addresses, which may need changing. The
+timer's schedules are systemd's shorthands, so all backups of one schedule start at the same
+moment. A change of the users between the end of the copy and the next backup is in no backup
+until then.
+
+## ADR-27: A two-level CA with dedicated issuing CAs, whose root may be offline
+
+Status: Accepted
+
+Context: One self-signed CA signed every certificate of the router with the same server and
+client purposes, so a certificate for any use was also a VPN client certificate, and the key
+that signed everything had to stay on the router. The router needs CAs for the VPN clients, for
+site-to-site certificates and for the administrator's own uses, and a root whose key can be kept
+elsewhere. The `simple-ca` script handled one level and ran as a separate process with its own
+state file.
+
+Decision: The root signs only issuing CAs, path length 1; the issuing CAs, path length 0, sign
+the certificates. `p2s` signs the router's VPN server certificate and the users' certificates,
+`s2s` the site-to-site certificates, and the CAs of `pki.issuing_cas` what their templates
+allow. A template fixes the key, the lifetime, the key usages, the extended key usages, the kind
+of common name and the subject alternative names a certificate may have; `vpn-server` and
+`vpn-user` are issued by the router only, so no certificate for another use comes from the P2S
+CA. The module `vpnrouter_pki.py`, with `python3-cryptography` from the Ubuntu archive, builds
+the keys, certificates, CSRs and CRLs in the processes that need them, as the user module does
+(ADR-6); the settings are the `pki` section of `vpn-router.json`, and the CAs and the
+certificates are kept in the router's directory (ADR-32). Keeping the root key off the router is recommended, not required: while it is
+on the router a new issuing CA is signed at once, and while it is not the CA gets its key and a
+CSR, signed offline and imported with `vpn-router-ca` or `/ca`. The root key itself crosses
+neither the API nor the Manager. For a certificate of something other than the router, the router
+signs the requester's CSR, or makes the key and returns it once in a PKCS#12 file, keeping only
+the certificate, so a key it gave away cannot leak from it later. The PKCS#12 file uses 3DES and
+SHA-1 MACs, the encryption the key stores of Windows and macOS import. strongSwan reads only the
+first certificate of a PEM file, so each CA has a file of its own in `/etc/swanctl/x509ca`;
+road-warrior and site-to-site name their CA in `cacerts`, which restricts the trust chain to it.
+
+Consequences: The P2S CA's certificates are accepted by road-warrior and OpenVPN only, the site-to-site CA's by
+site-to-site only. With an offline root, new issuing CAs and the root CRL need the offline step,
+and a router whose P2S CA waits has no VPN server certificate and so no P2S access. The router
+keeps the users' keys, as the OpenVPN profile embeds them.
+
+## ADR-28: CA certificates and CRLs are published at configurable endpoints, and CRLs are short-lived
+
+Status: Accepted
+
+Context: Clients outside the router, such as Windows and servers using the general-purpose CAs,
+find the issuer's certificate and the CRL through the AIA and CRL distribution point extensions,
+and cache a CRL until its next update. The router's CRLs were valid for ten years and published
+nowhere. Caddy serves files itself, and the web service already answers on the status port.
+
+Decision: `pki.publish` lists the endpoints, in the order the URLs appear in the certificates:
+plain HTTP on the status port, served by the web service; plain HTTP on port 80, HTTPS on the
+status HTTPS port and HTTPS on port 443, served by Caddy's `file_server`. The files are DER, as
+RFC 5280 asks of these URLs, copied to `/var/lib/vpn-router/pki-publish`, outside the 0700 PKI
+directory, so that Caddy's user reads them. Ports 80 and 443 serve only the CA files and are opened
+on both interfaces when chosen. The CRLs of the issuing CAs are valid for `pki.crl_days`, a week
+by default, signed again at every revocation and by a daily timer once half their validity has
+passed; the root CRL has its own `pki.root_crl_days`, half a year by default, so an offline root
+needs its CRL signed twice a year, not weekly.
+
+Consequences: A client that caches a CRL sees a revocation within a week. The URLs are fixed in a
+certificate when it is signed, so a change of the endpoints or of `wan.local_fqdn` applies to
+certificates signed afterwards. HTTPS URLs are offered for clients that cannot use plain HTTP,
+although a client checking the revocation of the HTTPS server's own certificate cannot use them;
+Caddy's certificate is Let's Encrypt's, so the router's endpoints do not depend on themselves.
+
+## ADR-29: Site-to-site connections authenticate with a pre-shared key or certificates
+
+Status: Accepted
+
+Context: Site-to-site connections used pre-shared keys only. Peers that are routers of this
+package, or have a PKI of their own, can authenticate with certificates, which leaves no shared
+secret to distribute. Azure's VPN gateway authenticates site-to-site connections with pre-shared
+keys only.
+
+Decision: A connection's `auth` is `psk`, the default, or `certificate`. With `certificate` the
+router presents one site-to-site certificate of its `s2s` CA, whose names are `wan.local_fqdn` and
+the local identities the connections with `certificate` use, the public or the internal address of
+their `local_id_mode`, and no address no connection presents, such as an internal address the peers
+never see; the peer's
+certificate must name the connection's `id` and be issued by the router's `s2s` CA or, when the
+connection has `remote_ca`, by that CA, its PEM certificate kept in the connection and copied to
+`/etc/swanctl/x509ca/s2s-<name>.pem`. A peer without a PKI gets its certificate from the router.
+
+Consequences: The connection Azure's gateway needs stays `psk`, so the platform writes nothing
+new. The router's site-to-site certificate is issued again when the set of names it needs changes,
+such as when a connection's `local_id_mode` or its address changes. The
+router does not check a remote CA's revocations unless strongSwan can fetch the CRL its
+certificates name.
+
+## ADR-30: A general-purpose issuing CA can back an ACME server of Caddy
+
+Status: Accepted
+
+Context: Servers that use a general-purpose CA need their certificates renewed, which ACME clients
+automate. Caddy, which the router runs for HTTPS, has an ACME server built on a CA of its `pki`
+app. Caddy 2.6.2 of the Ubuntu archive needs the key file of that CA's root, never renews an
+intermediate given to it, keeps no CRL and takes only the CA in the `acme_server` directive,
+issuing certificates valid for 12 hours. Its ACME handler answers every path under `/acme/`.
+
+Decision: An issuing CA of `pki.issuing_cas` with `acme` makes Caddy serve an ACME server for it
+at `/acme/<name>/` of the HTTPS status port, each in a `handle` block of its own. The issuing CA's
+certificate and key are given to Caddy as both the root and the intermediate of a `pki` CA, copied
+to `/etc/caddy/vpn-router-pki/<name>/` for Caddy's group, so that the router's root key never
+reaches Caddy and may be offline.
+
+Consequences: The certificates Caddy issues chain to the router's root through the issuing CA, are
+not in the router's inventory, follow no template and are not revoked; their short lifetime takes
+the place of revocation. An issuing CA waiting for the offline root has no ACME server.
+
+## ADR-31: vpn-router-setup installs Caddy from the Ubuntu archive or from Caddy's repository
+
+Status: Accepted
+
+Context: Caddy 2.6.2 of the Ubuntu archive lacks later features, such as a pki root without its key
+and the settings of the ACME server. Caddy's own repository has current versions of the package of
+the same name. A dependency on `caddy` installs the archive's package with `vpn-router`. apt cannot
+run inside `vpn-router-setup` when the package's installation runs setup, as dpkg holds its lock.
+
+Decision: `vpn-router` only suggests `caddy`. While HTTPS is on, setup has
+`vpn-router-caddy.service` install it from `https.caddy_source`: `ubuntu`, or `upstream`, for
+which the unit writes the deb822 source and the key it downloads over HTTPS from Caddy's
+repository. The unit waits for dpkg's lock and runs setup again when Caddy is in place.
+`vpn-router-caddy use-upstream` switches an installed router. Upstream's versions sort above the
+archive's, so once the repository is configured apt keeps upstream's Caddy without a pin, and
+`ubuntu` has no effect; nothing downgrades Caddy on its own, as a downgrade may break the
+configuration and is the administrator's decision. Only the CLI switches the source; the Router
+Manager shows which one is in effect.
+
+Consequences: The first setup run with HTTPS starts the installation and serves HTTPS on the run
+that follows it. The key of Caddy's repository is trusted as downloaded over HTTPS, the way Caddy's
+installation instructions do it. Going back to the archive's Caddy is a manual downgrade.
+
+## ADR-32: The PKI is kept in the router's directory
+
+Status: Accepted
+
+Context: The CAs, their keys and CRLs, the inventory of the certificates issued and their
+revocations, and the router's and the users' certificates and keys need a store. Files with a
+JSON state file cannot be searched, and are copied apart from the users the certificates belong
+to. The router runs slapd, which holds the users and is provisioned on every
+configured router (ADR-10), and PostgreSQL, which holds the refresh tokens and the metrics, may
+be another server (`web.database`) and is in a backup only with `backup.metrics`. RFC 4523
+defines `pkiCA` with `cACertificate` and `certificateRevocationList`, and `userCertificate`, which
+OpenLDAP's core schema includes.
+
+Decision: The PKI is held in `ou=pki` of the router's directory. A CA is an entry of the package's
+class `vpnRouterAuthority` with `cACertificate` and `certificateRevocationList`, its key, its CSR,
+its CRL number, the source of the root CRL and the time it was retired; a certificate is an entry
+of `vpnRouterCertificate` named by its serial number under its CA, with `userCertificate`, its
+kind, template, names, expiry and revocation, and the key of a router or user certificate. A
+user's certificate is also the `userCertificate` of the user's entry. An ACL lets no account read
+`vpnRouterPrivateKey`; only root, the database's root DN, does, through the local socket. A
+retired CA keeps its entry, renamed, so its certificates stay in the inventory. A root supplied
+from outside, by cloud-init or by hand, is placed as files before the first setup with a platform,
+which imports and removes them. strongSwan, OpenVPN, Caddy and the system's trust store read files,
+so setup and every revocation write them from the directory. Setup provisions the directory before
+the PKI; without the directory it leaves the PKI and its files as they are and configures the rest.
+
+Consequences: `slapcat` writes the users and the PKI into every backup in one consistent copy, and a
+restore brings both back together. The PKI depends on slapd, as the users already do, and not on
+PostgreSQL. The directory holds the CA keys, so it is as sensitive as the CA itself. The schema
+grows by the package's PKI attributes and classes, which setup adds to slapd's configuration.

debian-package/debian/changelog

@@ -1,3 +1,480 @@
+vpn-router (0.21.3-1) unstable; urgency=medium
+
+  fix: the PKI is kept in ou=pki of the router's directory instead of files under /etc/vpn-router/pki, and the router's site-to-site certificate names only the identities its connections present, and /ca/bundle gives the certificate bundle of a CA.
+
+  * The CAs with their certificates, keys, CSRs and CRLs, and every
+    certificate issued with its revocation, are entries of the
+    package's schema in ou=pki; a user's certificate is also the
+    userCertificate of the user's entry, and only root reads a key.
+  * slapcat puts the PKI into every backup with the users.
+  * A root CA supplied as /etc/vpn-router/pki/root/cert.pem and key.pem
+    before the first setup is imported into the directory, and the files
+    are removed; vpn-router-ca root import-cert is gone.
+  * Setup provisions the directory before the configuration; without it,
+    the PKI and its files are left as they are.
+  * The router's site-to-site certificate names wan.local_fqdn and the
+    public or internal address a connection with auth certificate uses
+    as its identity, not every address of the router, and is issued
+    again when those names change.
+  * GET /ca/bundle/<name> of the status web service answers the PEM
+    certificates a client trusts a CA with: the root alone, or an issuing
+    CA followed by the root.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 01 Nov 2026 10:00:00 +0100
+
+vpn-router (0.21.2-1) unstable; urgency=medium
+
+  fix: pki.root_key_warning silences the warning of vpn-router-ca status and /ca about the root key on the router.
+
+  * pki.root_key_warning, true by default, set to false silences the
+    warning that the root key is on the router.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 31 Oct 2026 10:00:00 +0100
+
+vpn-router (0.21.1-1) unstable; urgency=medium
+
+  fix: vpn-router-ca status and /ca warn about a CRL the router signs only once vpn-router-pki.timer has not renewed it in time.
+
+  * A CRL whose CA key is on the router is reported once past half of its
+    validity, when the timer should have signed it again; a root CRL
+    signed offline is still reported 30 days before it expires.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Fri, 30 Oct 2026 10:00:00 +0100
+
+vpn-router (0.21.0-1) unstable; urgency=medium
+
+  feature: the router has a two-level CA with an optional offline root, issuing CAs for P2S, site-to-site and general use, templates, published certificates and CRLs and an ACME server, site-to-site connections authenticate with certificates, and setup installs Caddy from Ubuntu or Caddy's repository.
+
+  * The root CA signs the issuing CAs p2s, which issues the router's VPN
+    server certificate and the users' certificates, s2s, which issues
+    the site-to-site certificates, and the general-purpose CAs of
+    pki.issuing_cas; road-warrior and OpenVPN accept certificates of p2s
+    only.
+  * The pki section of vpn-router.json sets the names, lifetimes and
+    key type of the CAs, the validity of the CRLs, where certificates
+    and CRLs are published, the general-purpose CAs and the certificate
+    templates; p2s.ca_name and the vpn-router/p2s_ca_name question are
+    replaced by pki.root_name and vpn-router/pki_root_name.
+  * Without the root key on the router, new issuing CAs wait with a CSR
+    for an offline signature; vpn-router-ca exports, removes and imports
+    the root key, imports the signed certificates and a root CRL, issues
+    certificates from a CSR or into a PFX file, lists and revokes them.
+  * Certificates name their issuer's certificate and CRL at the
+    endpoints of pki.publish: the status port over HTTP, or Caddy on port
+    80, on the HTTPS status port or on port 443; the CRLs are signed
+    again at every revocation and by vpn-router-pki.timer.
+  * A general-purpose CA with acme backs an ACME server of Caddy.
+  * The router renews its own certificates before they expire.
+  * A site-to-site connection with auth certificate authenticates with
+    the router's site-to-site certificate and trusts the s2s CA or the
+    connection's remote_ca.
+  * /ca of the status web service shows and changes the PKI; /pki/
+    serves the published certificates and CRLs; /config shows the
+    installed Caddy and its source; request bodies may be 64 KiB.
+  * vpn-router no longer depends on caddy: vpn-router-setup installs it
+    from https.caddy_source, the Ubuntu archive or Caddy's repository,
+    and vpn-router-caddy use-upstream switches an installed router to
+    Caddy's repository for good.
+  * The package depends on python3-cryptography instead of openssl, and
+    simple-ca is gone.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Thu, 29 Oct 2026 10:00:00 +0100
+
+vpn-router (0.20.0-1) unstable; urgency=medium
+
+  feature: the router makes scheduled backups to a local folder, copied to the cloud with rclone, or to a Proxmox Backup Server, and vpn-router-backup restores them.
+
+  * The backup section of vpn-router.json switches on backups and sets
+    the schedule (hourly, daily or weekly), the target (local or pbs),
+    the folder, the retention of local archives, whether the metrics
+    database is included, the rclone remote and mode, and the Proxmox
+    Backup Server repository, token secret, fingerprint, namespace and
+    key file.
+  * A backup holds /etc/vpn-router, /etc/wireguard, the ufw rules, the
+    Router Manager's settings, PSK files outside /etc/vpn-router, the
+    user directory as slapcat writes it and, optionally, the database as
+    pg_dump writes it, copied while the configuration and the users are
+    locked.
+  * Local backups are tar archives compressed with zstd. Archives the
+    retention settings no longer keep are deleted by the rules of Proxmox
+    Backup Server's prune, and rclone can sync or copy the archives to a
+    remote of /etc/vpn-router/rclone.conf.
+  * Backups to a Proxmox Backup Server are made with
+    proxmox-backup-client-static, which the package suggests; the server
+    keeps them.
+  * vpn-router-backup runs a backup, lists the backups and restores a
+    local archive or a snapshot; vpn-router-backup.timer runs it on the
+    schedule.
+  * /backup of the status web service shows and changes the backup
+    settings, shows the last run, the local archives and the next run,
+    and starts a backup.
+  * The package depends on zstd and suggests rclone and
+    proxmox-backup-client-static.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Wed, 28 Oct 2026 10:00:00 +0100
+
+vpn-router (0.19.1-3) unstable; urgency=medium
+
+  update: ADR-24 names Ubuntu's apt-daily timer, not a cron job, as what updates the package lists.
+
+  * ADR-24 says that Ubuntu's apt-daily timer updates the package lists
+    twice a day, and that apt-daily or unattended-upgrades may hold the
+    dpkg lock the upgrade waits for.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Tue, 27 Oct 2026 10:00:00 +0100
+
+vpn-router (0.19.1-2) unstable; urgency=medium
+
+  update: ADR-24 says that the Router Manager checks for updates on demand.
+
+  * ADR-24 describes Check update of the Router Manager, which updates
+    apt's package lists in the transient unit vpn-router-manager-check.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Tue, 27 Oct 2026 09:00:00 +0100
+
+vpn-router (0.19.1-1) unstable; urgency=medium
+
+  fix: /metrics/history gives the availability of each site-to-site connection as the percentage of time it was up in slots of the view.
+
+  * site_to_site_up of /metrics/history divides the view into slots of 5
+    minutes for an hour, 1 hour for a day, 1 day for a month and 1 month
+    for a year, and gives for each slot and connection the percentage of
+    the samples in which the connection was up.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Mon, 26 Oct 2026 21:00:00 +0100
+
+vpn-router (0.19.0-1) unstable; urgency=medium
+
+  feature: service accounts have a display name and a description, which the router sets for its internal ones.
+
+  * vpnRouterAccount of the package's schema also takes displayName and
+    description; vpn-router-setup updates the schema of an existing
+    directory.
+  * /service-accounts shows display_name and description, takes them when
+    adding a service account and changes them with PUT; "" removes one.
+    /users shows them as null.
+  * vpn-router-user add --service and update take --display-name and
+    --description.
+  * vpn-router-setup gives the metrics service's internal account the
+    display name Metrics and its description, and sets them again when
+    they differ.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Mon, 26 Oct 2026 20:00:00 +0100
+
+vpn-router (0.18.3-1) unstable; urgency=medium
+
+  fix: internal service accounts are the router's own, of which the metrics service's is the one at present, and the documentation says what a service account is.
+
+  * /service-accounts, /groups and the internal field of the answers
+    take every internal service account, the accounts the router creates
+    and manages for its own services, instead of only the metrics
+    service's; the 403 answers say that the router manages them.
+  * The README, the ADR and the users module describe a service account
+    as the account another system, such as an OpenTelemetry Collector
+    reading /metrics, signs in with, and the internal ones as managed by
+    the router.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Mon, 26 Oct 2026 19:00:00 +0100
+
+vpn-router (0.18.2-1) unstable; urgency=medium
+
+  fix: service accounts are managed at /service-accounts, not /users, and the internal one of the metrics service is only listed.
+
+  * /service-accounts, for members of admins and admin tokens, lists the
+    service accounts, adds one with a password and an optional e-mail
+    address, changes the e-mail address, sets the password, disables,
+    enables, unlocks and removes them.
+  * /users no longer lists service accounts or takes "service" on POST,
+    and answers 404 to a request naming one.
+  * /service-accounts answers 403 to every request but the list on the
+    account metrics.service_account names while metrics are enabled, and
+    /groups to putting it in or taking it out of a group;
+    vpn-router-setup and root's vpn-router-user still change it.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Mon, 26 Oct 2026 18:00:00 +0100
+
+vpn-router (0.18.1-1) unstable; urgency=medium
+
+  fix: vpn-router-setup owns only the creation of the metrics service's account, which is marked internal, and administrators add service accounts with vpn-router-user add --service and /users.
+
+  * POST /users takes "service": true and adds a service account, which
+    cannot sign in to /auth; vpn-router-user add takes --service.
+  * The users of /users and vpn-router-user have internal, true for the
+    account metrics.service_account names while metrics are enabled;
+    vpn-router-user list shows its kind as internal.
+  * PUT /users/<name>/password refuses the internal service account,
+    whose password vpn-router-setup keeps.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Mon, 26 Oct 2026 17:00:00 +0100
+
+vpn-router (0.18.0-2) unstable; urgency=medium
+
+  update: release.py tags a Debian revision other than 1 as x.y.z-n.
+
+  * release.py tags the newest entry x.y.z for the Debian revision 1 and
+    x.y.z-n for a later revision, so a packaging change gets a release
+    of its own.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Mon, 26 Oct 2026 14:00:00 +0100
+
+vpn-router (0.18.0-1) unstable; urgency=medium
+
+  feature: site_to_site holds any number of named site-to-site connections, which /site-to-site lists, adds, changes and removes.
+
+  * site_to_site maps a connection name to addrs, id, cidrs, local_cidrs,
+    local_id_mode, psk and psk_file; it replaces remote and
+    wan.local_id_mode, and a configuration with them is refused.
+  * The install answers fill the connection remote-site; the local subnets
+    fill both its local_cidrs and local.cidrs.
+  * Every ready connection is written to
+    /etc/swanctl/conf.d/site-to-site.conf as the swanctl connection and
+    child of its name with the secret ike-psk-<name>; remote-site.conf is
+    no longer written.
+  * Traffic from each connection's local_cidrs to its cidrs is not
+    masqueraded.
+  * /site-to-site, for members of admins and admin tokens, lists the
+    connections without their keys, adds, changes and removes them, and
+    restarts vpn-router-setup to apply the change.
+  * vpn-config-edit sets site_to_site.NAME.option, creating the
+    connection, and removes site_to_site.NAME; it and the web service
+    write vpn-router.json under one lock.
+  * The status shows site_to_site.connections, whether each connection is
+    established.
+  * The metrics service keeps, per site-to-site connection, whether it is
+    up, the bytes and packets per second received and sent, and the
+    seconds its IKE SA has been up, in metrics_site_to_site and its
+    summaries; /metrics/history has them in site_to_site of each point.
+  * /metrics serves vpn_router_site_to_site_up,
+    vpn_router_site_to_site_uptime_seconds,
+    vpn_router_site_to_site_io_bytes_per_second and
+    vpn_router_site_to_site_packets_per_second by connection, and the
+    OTLP push the vpn_router.site_to_site gauges; vpn_router_clients and
+    metrics_vpn no longer have the kind site_to_site.
+  * /config shows each connection in network.site_to_site, and no psk at
+    any level of the configuration.
+  * Removing the package terminates every site-to-site connection; purging
+    removes site-to-site.conf.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 23:00:00 +0100
+
+vpn-router (0.17.0-1) unstable; urgency=medium
+
+  feature: /config shows the router's PostgreSQL clusters and their settings, and /postgresql their databases with their sizes.
+
+  * /config has a postgresql section: each cluster of pg_lsclusters with
+    its version, name, port, status and data directory, and the main
+    settings of each online cluster.
+  * /postgresql, protected like /config, lists the databases of each
+    online cluster with owner, encoding and size, and the total size.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 21:00:00 +0200
+
+vpn-router (0.16.0-1) unstable; urgency=medium
+
+  feature: the router keeps its metrics in PostgreSQL, serves them for an OpenTelemetry Collector and pushes them over OTLP.
+
+  * vpn-router-metrics takes a sample every minute: CPU, memory, free
+    space of /, bytes per second on the external and the internal
+    interface, the load averages, and the VPN clients of each enabled
+    system.
+  * The samples are kept in metrics_system, metrics_network and
+    metrics_vpn, and summarized hourly, daily, monthly and yearly in
+    their _hourly, _daily, _monthly and _yearly tables; once a day the
+    rows older than metrics.keep_details, keep_hourly, keep_daily,
+    keep_monthly and keep_yearly are deleted.
+  * /metrics serves the last sample in the Prometheus format on the
+    interfaces metrics.listen names (internal by default) at
+    metrics.port, and over HTTPS by Caddy at metrics.https_port, with
+    HTTP Basic authentication of the service account
+    metrics.service_account names (metrics by default).
+  * metrics.otlp_endpoint pushes every sample as OTLP/HTTP JSON, with
+    metrics.otlp_username and metrics.otlp_password.
+  * /metrics/history of the web service answers the hour, day, month
+    and year views for the Router Manager.
+  * vpn-router-setup adds and keeps that service account, which cannot
+    sign in to /auth, with a random password in
+    /etc/vpn-router/metrics.secret, set anew when the file is missing.
+  * Depends on python3-psutil and python3-prometheus-client.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 19:00:00 +0200
+
+vpn-router (0.15.0-1) unstable; urgency=medium
+
+  feature: vpn-config-edit shows and changes the router's configuration.
+
+  * vpn-config-edit show prints /etc/vpn-router/vpn-router.json as
+    indented JSON.
+  * vpn-config-edit set SECTION.OPTION VALUE converts the value to the
+    setting's type, checks the whole configuration as vpn-router-setup
+    does, writes the file indented with mode 0600, and reminds that
+    vpn-router-setup applies it.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 18:00:00 +0200
+
+vpn-router (0.14.1-1) unstable; urgency=medium
+
+  fix: web.status_content is basic by default.
+
+  * Without web.status_content, or with a value that is not allowed, /
+    and /status show only the overall status, the time and the Client
+    IP; set it to standard or full to show more.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 17:00:00 +0200
+
+vpn-router (0.14.0-1) unstable; urgency=medium
+
+  feature: the status page shows the Client IP, and web.status_content sets how much it shows.
+
+  * / and /status show the Client IP, the address the request came from:
+    the last X-Forwarded-For entry for a request from a trusted proxy, so
+    behind Caddy or the Router Manager the browser's; /status has it as
+    client_ip.
+  * web.status_content sets what / and /status show: basic the overall
+    status, the time and the Client IP; standard also the package
+    versions and the number of active and failed services; full, the
+    default, everything as before. /status names it in content.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 16:00:00 +0200
+
+vpn-router (0.13.0-1) unstable; urgency=medium
+
+  feature: Caddy serves the status page and the Router Manager over HTTPS with a certificate from Let's Encrypt.
+
+  * The package depends on caddy. While https.enabled is true, the
+    default, and wan.local_fqdn is set, vpn-router-setup writes
+    /etc/caddy/vpn-router.Caddyfile and runs Caddy, which serves the
+    status page on https.web_port (7444) and the Router Manager, when it
+    is installed, on https.manager_port (7443) for wan.local_fqdn, and
+    obtains the certificate from Let's Encrypt with the HTTP challenge.
+  * The package's drop-in for caddy.service makes Caddy read that file
+    and keeps it from starting before the file exists. The file belongs
+    to root and the group caddy, which Caddy runs as, with mode 0640.
+  * The firewall opens port 80 on the external interface and the HTTPS
+    ports on both interfaces while HTTPS is on.
+  * The web service also listens on 127.0.0.1, where Caddy calls it.
+  * docker.exposed_ports may not take port 80 or the HTTPS ports while
+    HTTPS is on.
+  * Removing the package stops and disables caddy; purging it removes
+    /etc/caddy/vpn-router.Caddyfile.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 15:00:00 +0200
+
+vpn-router (0.12.3-1) unstable; urgency=medium
+
+  fix: the web service listens on the internal address, the external address or both.
+
+  * web.listen in vpn-router.json sets the interfaces the web service
+    listens on: internal, external or both, the default. It is read
+    every 30 seconds, so a change takes effect without running setup,
+    at the latest when vpn-router-web restarts.
+  * The firewall opens the ports of the web service and of the Router
+    Manager on the internal and the external interface.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 13:00:00 +0200
+
+vpn-router (0.12.2-1) unstable; urgency=medium
+
+  fix: the web service manages users also when it started before setup.
+
+  * vpn-router-web creates /run/vpn-router when it starts, so the user
+    operations take their lock there; started before vpn-router-setup had
+    made the directory, they failed with "Read-only file system".
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 12:00:00 +0200
+
+vpn-router (0.12.1-1) unstable; urgency=medium
+
+  fix: the status page shows whether the Router Manager runs.
+
+  * When vpn-router-manager is installed, the services show the state of
+    vpn-router-manager, and status is degraded while it is not active.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 11:00:00 +0200
+
+vpn-router (0.12.0-1) unstable; urgency=medium
+
+  feature: every user manages their own account through /account.
+
+  * /account serves the signed-in user, admin or not: GET shows the user
+    and whether they are a member of admins; PUT /account/password sets
+    the password and POST and PUT /account/otp enrol or replace the TOTP
+    enrolment, given the current password and, when enrolled, a current
+    code. A wrong one is 403 and counts as a failed login. Tokens of
+    vpn-router-token get 403 there.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 10:00:00 +0200
+
+vpn-router (0.11.0-1) unstable; urgency=medium
+
+  feature: the status page shows the Router Manager's version.
+
+  * / and /status show the installed version of vpn-router-manager, as
+    manager_version in JSON, or that it is not installed.
+  * The services show the state of each PostgreSQL cluster, as
+    postgresql@<version>-<cluster>, or postgresql: no cluster without one.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 09:00:00 +0200
+
+vpn-router (0.10.0-1) unstable; urgency=medium
+
+  feature: Manager-only accounts, and the Router Manager's first administrator.
+
+  * vpn-router-user add without the person options, and POST /users without
+    them, add a Manager-only account: a name and a password, no names, no
+    e-mail address and no certificate, so no VPN access. Giving only some of
+    the three is refused. The answers show the kind as manager_only, list
+    shows a KIND column, and update refuses a Manager-only account.
+  * vpn-router-setup creates the administrator the Router Manager's package
+    leaves in /etc/vpn-router-manager/admin.json as a Manager-only account in
+    the group admins, which it creates when missing, and removes the file.
+  * The directory has the package's own schema, with the auxiliary class
+    vpnRouterAccount and the attribute vpnRouterScope. A Manager-only account
+    can have an e-mail address: add --email, update --email ('' removes it),
+    and email in POST and PUT /users.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 08:00:00 +0200
+
+vpn-router (0.9.2-1) unstable; urgency=medium
+
+  fix: the web service starts once libpq can be found.
+
+  * The web service unit waits, with wait-for-libpq, until libpq is in the
+    linker cache before it starts the service, for at most 5 minutes; the
+    service imports psycopg at start again.
+  * The package starts the web service without waiting for it.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 06:00:00 +0200
+
+vpn-router (0.9.1-1) unstable; urgency=medium
+
+  fix: the web service waits for the PostgreSQL driver instead of failing at start.
+
+  * The web service imports psycopg as the first step of its
+    initialization, retrying every 15 seconds and logging the failure while
+    libpq cannot be found yet during an installation.
+  * vpn-router-user and vpn-router-token load without psycopg; a command
+    that needs the token database then reports that the driver cannot be
+    loaded, as it reports every failure of that database.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 05:00:00 +0200
+
+vpn-router (0.9.0-1) unstable; urgency=medium
+
+  feature: bootstrap diagnostics on port 17081 during the first setup.
+
+  * vpn-router-bootstrap serves, on port 17081, the setup state, the
+    services, the failed units, the addresses, cloud-init status and the end
+    of the logs of setup and the web service, as plain text.
+  * vpn-router-setup starts it, and the Router Manager's bootstrap on port
+    17080, until a run has succeeded, and opens both ports on the internal
+    interface; the first successful run writes
+    /var/lib/vpn-router/setup.done, closes the ports and stops them. A
+    failed run leaves them running.
+  * The web service no longer has a starting phase: it creates its key and
+    database, then listens.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 04:00:00 +0200
+
 vpn-router (0.8.0-1) unstable; urgency=medium
 
   feature: the web service creates its key and database itself and shows its starting phase.

debian-package/debian/config

@@ -51,6 +51,7 @@ db_input high vpn-router/remote_addrs || true
 db_input high vpn-router/remote_id || true
 db_input high vpn-router/remote_cidrs || true
 db_input high vpn-router/psk || true
+db_input high vpn-router/pki_root_name || true
 db_input high vpn-router/p2s_enabled || true
 db_input high vpn-router/wg_enabled || true
 db_go || true
@@ -58,7 +59,6 @@ db_go || true
 db_get vpn-router/p2s_enabled
 if [ "$RET" = "true" ]; then
     db_input high vpn-router/p2s_address_pool || true
-    db_input high vpn-router/p2s_ca_name || true
     db_go || true
 fi
 

debian-package/debian/control

@@ -17,24 +17,37 @@ Depends: ${misc:Depends},
          ufw,
          debconf,
          ucf,
-         openssl,
          python3,
+         python3-cryptography,
          python3-jinja2,
          python3-ldap,
          python3-pyotp,
          python3-psycopg,
+         python3-psutil,
+         python3-prometheus-client,
          postgresql,
          slapd,
          openvpn,
          openvpn-auth-ldap,
          iproute2,
-         systemd-resolved
+         systemd-resolved,
+         zstd
+Suggests: caddy,
+          rclone,
+          proxmox-backup-client-static
 Description: Linux VPN router with IPSec, road-warrior, OpenVPN and WireGuard support
  Configures a Linux host or virtual machine as a VPN router: site-to-site
- IKEv2 IPSec (strongSwan swanctl), optional road-warrior access using EAP-TLS,
- optional OpenVPN with a certificate and a password per user, kept in a local
- LDAP directory, optional WireGuard, and the routing, NAT, MSS clamping and
- firewall rules a router needs. Nothing in the package is specific to a cloud
+ IKEv2 IPSec (strongSwan swanctl) with pre-shared keys or certificates,
+ optional road-warrior access using EAP-TLS, optional OpenVPN with a
+ certificate and a password per user, kept in a local LDAP directory, optional
+ WireGuard, and the routing, NAT, MSS clamping and firewall rules a router
+ needs. Its two-level CA, whose root may be kept offline, has issuing CAs for
+ the VPN clients, for site-to-site and for general use, and publishes their
+ certificates and CRLs. Caddy, installed by the package from the Ubuntu
+ archive or from Caddy's repository, serves HTTPS. It keeps metrics of the
+ router in PostgreSQL and serves them for an OpenTelemetry Collector, and
+ makes scheduled backups to a local folder, copied to the cloud with rclone,
+ or to a Proxmox Backup Server. Nothing in the package is specific to a cloud
  provider.
  .
  Settings live in /etc/vpn-router/vpn-router.json, which is created on first

debian-package/debian/install

@@ -2,18 +2,31 @@ src/etc/sysctl.d/99-vpn-router.conf                   etc/sysctl.d/
 src/usr/lib/vpn-router/vpnrouter.py                   usr/lib/vpn-router/
 src/usr/lib/vpn-router/vpnrouter_ldap.py              usr/lib/vpn-router/
 src/usr/lib/vpn-router/vpnrouter_net.py               usr/lib/vpn-router/
+src/usr/lib/vpn-router/vpnrouter_serve.py             usr/lib/vpn-router/
 src/usr/lib/vpn-router/vpnrouter_users.py             usr/lib/vpn-router/
 src/usr/lib/vpn-router/vpnrouter_auth.py              usr/lib/vpn-router/
+src/usr/lib/vpn-router/vpnrouter_metrics.py           usr/lib/vpn-router/
+src/usr/lib/vpn-router/vpnrouter_status.py            usr/lib/vpn-router/
+src/usr/lib/vpn-router/vpnrouter_backup.py            usr/lib/vpn-router/
+src/usr/lib/vpn-router/vpnrouter_pki.py               usr/lib/vpn-router/
 src/usr/lib/vpn-router/openvpn-same-user              usr/lib/vpn-router/
+src/usr/lib/vpn-router/openvpn-issuer                 usr/lib/vpn-router/
 src/usr/sbin/vpn-router-user                          usr/sbin/
 src/usr/sbin/vpn-router-group                         usr/sbin/
 src/usr/sbin/vpn-router-token                         usr/sbin/
-src/usr/lib/vpn-router/simple-ca                      usr/lib/vpn-router/
+src/usr/sbin/vpn-config-edit                          usr/sbin/
+src/usr/sbin/vpn-router-backup                        usr/sbin/
+src/usr/sbin/vpn-router-ca                            usr/sbin/
+src/usr/sbin/vpn-router-caddy                         usr/sbin/
 src/usr/lib/vpn-router/configure                      usr/lib/vpn-router/
 src/usr/lib/vpn-router/docker-rules                   usr/lib/vpn-router/
 src/usr/lib/vpn-router/setup                          usr/lib/vpn-router/
 src/usr/lib/vpn-router/web                            usr/lib/vpn-router/
+src/usr/lib/vpn-router/metrics                        usr/lib/vpn-router/
+src/usr/lib/vpn-router/bootstrap                      usr/lib/vpn-router/
+src/usr/lib/vpn-router/wait-for-libpq                 usr/lib/vpn-router/
 src/usr/lib/vpn-router/generate-config                usr/lib/vpn-router/
 src/usr/lib/vpn-router/vpnrouter_platforms/*.py       usr/lib/vpn-router/vpnrouter_platforms/
 src/usr/share/vpn-router/templates/*                  usr/share/vpn-router/templates/
+src/usr/lib/systemd/system/caddy.service.d/vpn-router.conf usr/lib/systemd/system/caddy.service.d/
 src/usr/share/doc/vpn-router/vpn-router.json.example  usr/share/doc/vpn-router/

debian-package/debian/postinst

@@ -31,7 +31,7 @@ case "$1" in
         answer vpn-router/psk;              VPN_ROUTER_PSK="$ANSWER"
         answer vpn-router/p2s_enabled;      VPN_ROUTER_P2S_ENABLED="$ANSWER"
         answer vpn-router/p2s_address_pool; VPN_ROUTER_P2S_ADDRESS_POOL="$ANSWER"
-        answer vpn-router/p2s_ca_name;      VPN_ROUTER_P2S_CA_NAME="$ANSWER"
+        answer vpn-router/pki_root_name;    VPN_ROUTER_PKI_ROOT_NAME="$ANSWER"
         answer vpn-router/wg_enabled;       VPN_ROUTER_WG_ENABLED="$ANSWER"
         answer vpn-router/wg_address;       VPN_ROUTER_WG_ADDRESS="$ANSWER"
         answer vpn-router/wg_listen_port;   VPN_ROUTER_WG_LISTEN_PORT="$ANSWER"
@@ -44,7 +44,7 @@ case "$1" in
                VPN_ROUTER_REMOTE_ADDRS VPN_ROUTER_REMOTE_ID \
                VPN_ROUTER_REMOTE_CIDRS VPN_ROUTER_PSK \
                VPN_ROUTER_P2S_ENABLED VPN_ROUTER_P2S_ADDRESS_POOL \
-               VPN_ROUTER_P2S_CA_NAME VPN_ROUTER_WG_ENABLED \
+               VPN_ROUTER_PKI_ROOT_NAME VPN_ROUTER_WG_ENABLED \
                VPN_ROUTER_WG_ADDRESS VPN_ROUTER_WG_LISTEN_PORT
 
         # --- Hand a candidate configuration to ucf ---
@@ -70,6 +70,12 @@ case "$1" in
         # The firewall is not touched here. Enabling it, allowing SSH and
         # setting the forward policy all belong to vpn-router-setup, which does
         # them only once the configuration says this host is a router.
+
+        # The web service waits for libpq in the linker cache, which dpkg
+        # refreshes after this script, so it is started without waiting for it.
+        if [ -d /run/systemd/system ]; then
+            systemctl --no-block restart vpn-router-web.service >/dev/null 2>&1 || true
+        fi
         ;;
 esac
 

debian-package/debian/postrm

@@ -11,10 +11,10 @@ strip_ufw_blocks() {
 
     tmp="$(mktemp)"
     awk '
-        /^# (IPSEC|WIREGUARD|OPENVPN|P2S DNS|WEB|MANAGER|ROUTER FORWARD|ROUTER NAT|ROUTER MSS) RULES START$/ {
+        /^# (IPSEC|WIREGUARD|OPENVPN|P2S DNS|WEB|BOOTSTRAP|MANAGER|HTTPS|METRICS|ROUTER FORWARD|ROUTER NAT|ROUTER MSS) RULES START$/ {
             skip = 1; pending = 0; next
         }
-        /^# (IPSEC|WIREGUARD|OPENVPN|P2S DNS|WEB|MANAGER|ROUTER FORWARD|ROUTER NAT|ROUTER MSS) RULES END$/ {
+        /^# (IPSEC|WIREGUARD|OPENVPN|P2S DNS|WEB|BOOTSTRAP|MANAGER|HTTPS|METRICS|ROUTER FORWARD|ROUTER NAT|ROUTER MSS) RULES END$/ {
             skip = 0; next
         }
         skip { next }
@@ -43,14 +43,24 @@ case "$1" in
             ucfr --purge vpn-router /etc/vpn-router/vpn-router.json
         fi
 
-        rm -f /etc/swanctl/conf.d/remote-site.conf \
+        rm -f /etc/swanctl/conf.d/site-to-site.conf \
               /etc/swanctl/conf.d/road-warrior.conf \
               /etc/systemd/resolved.conf.d/p2s-forwarder.conf \
               /etc/wireguard/wg0.conf \
               /etc/openvpn/server/vpn-router.conf \
               /etc/openvpn/server/vpn-router-ldap.conf \
+              /etc/caddy/vpn-router.Caddyfile \
               /etc/vpn-router/vpn-router.json \
-              /etc/vpn-router/web.key
+              /etc/vpn-router/web.key \
+              /etc/vpn-router/metrics.secret \
+              /etc/systemd/system/vpn-router-backup.timer.d/schedule.conf \
+              /var/lib/vpn-router/setup.done \
+              /var/lib/vpn-router/backup.json
+        rmdir --ignore-fail-on-non-empty /etc/systemd/system/vpn-router-backup.timer.d 2>/dev/null || true
+        rm -rf /var/lib/vpn-router/backup/staging /var/lib/vpn-router/backup/restore \
+               /var/lib/vpn-router/pki-publish /etc/caddy/vpn-router-pki \
+               /etc/openvpn/server/vpn-router-ca.pem /etc/openvpn/server/vpn-router-crl.pem
+        rmdir --ignore-fail-on-non-empty /var/lib/vpn-router/backup /var/lib/vpn-router 2>/dev/null || true
 
         # The refresh tokens' database and role, when the package created them.
         if [ -f /etc/vpn-router/database.secret ] && command -v psql >/dev/null 2>&1; then
@@ -60,8 +70,11 @@ case "$1" in
         rm -f /etc/vpn-router/database.secret
 
         # Key material and users are not package state: /etc/vpn-router/pki,
-        # /etc/vpn-router/ldap, the user directory in slapd, the WireGuard key
-        # pair and everything under /etc/swanctl are left in place.
+        # /etc/vpn-router/ldap, /etc/vpn-router/rclone.conf, the user directory
+        # in slapd, the WireGuard key pair, everything under /etc/swanctl, the
+        # backups and what a restore replaced are left in place. Caddy stays
+        # installed, so its repository, which vpn-router-caddy configures, stays
+        # too.
         rmdir --ignore-fail-on-non-empty /etc/vpn-router 2>/dev/null || true
         ;;
 esac

debian-package/debian/prerm

@@ -6,15 +6,22 @@ case "$1" in
         # Stop offering the tunnels, but leave strongSwan itself enabled: it is
         # a shared service and may be in use by something other than this
         # package. The connection files are removed on purge.
+        # The site-to-site connections are the names of site_to_site in vpn-router.json.
         if [ -d /run/systemd/system ]; then
-            swanctl --terminate --ike remote-site >/dev/null 2>&1 || true
+            connections="$(python3 -c 'import json, sys; print(" ".join(json.load(open(sys.argv[1])).get("site_to_site", {})))' \
+                /etc/vpn-router/vpn-router.json 2>/dev/null || true)"
+            for connection in $connections; do
+                swanctl --terminate --ike "$connection" >/dev/null 2>&1 || true
+            done
             swanctl --terminate --ike road-warrior >/dev/null 2>&1 || true
         fi
 
-        # This package enabled wg-quick@wg0 and openvpn-server@vpn-router, so
-        # it takes them down again. Use the debhelper wrappers rather than
+        # vpn-router-setup enabled wg-quick@wg0, openvpn-server@vpn-router,
+        # caddy, vpn-router-metrics, the backup timer and the PKI timer, so the
+        # package takes them down again. Use the debhelper wrappers rather than
         # systemctl, so the script behaves on a machine without systemd.
-        for unit in wg-quick@wg0.service openvpn-server@vpn-router.service; do
+        for unit in wg-quick@wg0.service openvpn-server@vpn-router.service caddy.service vpn-router-metrics.service \
+                vpn-router-backup.timer vpn-router-pki.timer; do
             if command -v deb-systemd-invoke >/dev/null 2>&1; then
                 deb-systemd-invoke stop "$unit" >/dev/null 2>&1 || true
             fi

debian-package/debian/rules

@@ -5,4 +5,9 @@
 override_dh_installsystemd:
 	dh_installsystemd --name=vpn-router-setup
 	dh_installsystemd --name=vpn-router-docker
-	dh_installsystemd --name=vpn-router-web
+	dh_installsystemd --name=vpn-router-web --no-start
+	dh_installsystemd --name=vpn-router-metrics --no-enable --no-start
+	dh_installsystemd --name=vpn-router-bootstrap --no-enable --no-start
+	dh_installsystemd --name=vpn-router-backup --no-enable --no-start
+	dh_installsystemd --name=vpn-router-pki --no-enable --no-start
+	dh_installsystemd --name=vpn-router-caddy --no-enable --no-start

debian-package/debian/templates

@@ -66,7 +66,8 @@ Type: select
 Choices: fqdn, public_ip, internal_ip
 Default: fqdn
 Description: IKE local identity mode:
- How to derive the IKE identity advertised to the remote site:
+ How to derive the IKE identity the first site-to-site connection
+ advertises to the remote site:
  .
  fqdn - use the FQDN, which must match what the peer expects.
  .
@@ -78,8 +79,9 @@ Template: vpn-router/local_cidrs
 Type: string
 Default:
 Description: Local subnet CIDR(s):
- Comma-separated list of local subnet CIDRs to advertise into the
- site-to-site tunnel (for example 10.0.0.0/24 or 10.0.0.0/24,10.0.1.0/24).
+ Comma-separated list of local subnet CIDRs routed by this router and
+ advertised into the first site-to-site connection (for example
+ 10.0.0.0/24 or 10.0.0.0/24,10.0.1.0/24).
 
 Template: vpn-router/int_gateway_ip
 Type: string
@@ -94,25 +96,26 @@ Type: string
 Default:
 Description: Remote site WAN IP address(es):
  Comma-separated list of remote gateway addresses or FQDNs for the
- site-to-site IPSec tunnel.
+ first site-to-site connection.
 
 Template: vpn-router/remote_id
 Type: string
 Default:
 Description: Remote site IKE identity:
- IKE identity of the remote peer, without a leading @.
+ IKE identity of the remote peer of the first site-to-site connection,
+ without a leading @.
 
 Template: vpn-router/remote_cidrs
 Type: string
 Default:
 Description: Remote subnet CIDR(s):
  Comma-separated list of remote subnet CIDRs reachable through the
- site-to-site tunnel (for example 192.168.0.0/24).
+ first site-to-site connection (for example 192.168.0.0/24).
 
 Template: vpn-router/psk
 Type: password
 Description: Pre-shared key (PSK):
- Pre-shared key for the site-to-site IKEv2 tunnel. Must match the value
+ Pre-shared key of the first site-to-site connection. Must match the value
  configured on the remote peer. Stored in the configuration file and cleared
  from the debconf database after installation.
 
@@ -121,8 +124,7 @@ Type: boolean
 Default: false
 Description: Enable road-warrior (P2S) access?
  If enabled, this router accepts IKEv2 EAP-TLS connections from individual
- clients. Certificates are taken from /etc/vpn-router/pki, and a local
- certificate authority is created there if that directory is empty.
+ clients, with certificates of the router's P2S issuing CA.
 
 Template: vpn-router/p2s_address_pool
 Type: string
@@ -130,12 +132,12 @@ Default:
 Description: Road-warrior address pool:
  CIDR block assigned to road-warrior clients (for example 172.16.0.0/24).
 
-Template: vpn-router/p2s_ca_name
+Template: vpn-router/pki_root_name
 Type: string
-Default: VPN Router CA
-Description: Road-warrior CA name:
- Common name for the certificate authority created in /etc/vpn-router/pki
- when that directory is empty. Ignored when certificates are supplied.
+Default: VPN Router Root CA
+Description: Root CA name:
+ Common name of the root certificate authority the router creates in
+ /etc/vpn-router/pki/root. Ignored when a root certificate is placed there.
 
 Template: vpn-router/wg_enabled
 Type: boolean

debian-package/debian/vpn-router.vpn-router-backup.service

@@ -0,0 +1,11 @@
+[Unit]
+Description=VPN Router backup
+Documentation=file:/usr/share/doc/vpn-router/README.md
+After=network-online.target slapd.service postgresql.service
+Wants=network-online.target
+
+[Service]
+Type=oneshot
+ExecStart=/usr/sbin/vpn-router-backup run
+Nice=10
+IOSchedulingClass=idle

debian-package/debian/vpn-router.vpn-router-backup.timer

@@ -0,0 +1,10 @@
+[Unit]
+Description=VPN Router backup schedule
+Documentation=file:/usr/share/doc/vpn-router/README.md
+
+[Timer]
+OnCalendar=daily
+Persistent=true
+
+[Install]
+WantedBy=timers.target

debian-package/debian/vpn-router.vpn-router-bootstrap.service

@@ -0,0 +1,14 @@
+[Unit]
+Description=VPN Router bootstrap diagnostics page
+Documentation=file:/usr/share/doc/vpn-router/README.md
+
+# Never enabled: vpn-router-setup starts it until a setup has succeeded, and stops it after that.
+[Service]
+Type=simple
+ExecStart=/usr/bin/python3 /usr/lib/vpn-router/bootstrap
+Restart=on-failure
+RestartSec=5
+NoNewPrivileges=yes
+ProtectSystem=strict
+ProtectHome=yes
+PrivateTmp=yes

debian-package/debian/vpn-router.vpn-router-caddy.service

@@ -0,0 +1,10 @@
+[Unit]
+Description=VPN Router Caddy installation
+Documentation=file:/usr/share/doc/vpn-router/README.md
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=oneshot
+ExecStart=/usr/sbin/vpn-router-caddy apply
+TimeoutStartSec=20min

debian-package/debian/vpn-router.vpn-router-metrics.service

@@ -0,0 +1,22 @@
+[Unit]
+Description=VPN Router metrics service
+Documentation=file:/usr/share/doc/vpn-router/README.md
+After=network-online.target postgresql.service vpn-router-web.service
+Wants=network-online.target
+
+[Service]
+Type=simple
+ExecStartPre=/usr/lib/vpn-router/wait-for-libpq
+ExecStart=/usr/bin/python3 /usr/lib/vpn-router/metrics
+TimeoutStartSec=5min
+Restart=on-failure
+RestartSec=15
+NoNewPrivileges=yes
+ProtectSystem=strict
+RuntimeDirectory=vpn-router
+RuntimeDirectoryPreserve=yes
+ProtectHome=yes
+PrivateTmp=yes
+
+[Install]
+WantedBy=multi-user.target

debian-package/debian/vpn-router.vpn-router-pki.service

@@ -0,0 +1,7 @@
+[Unit]
+Description=VPN Router PKI maintenance
+Documentation=file:/usr/share/doc/vpn-router/README.md
+
+[Service]
+Type=oneshot
+ExecStart=/usr/sbin/vpn-router-ca maintain

debian-package/debian/vpn-router.vpn-router-pki.timer

@@ -0,0 +1,10 @@
+[Unit]
+Description=VPN Router PKI maintenance schedule
+Documentation=file:/usr/share/doc/vpn-router/README.md
+
+[Timer]
+OnCalendar=daily
+Persistent=true
+
+[Install]
+WantedBy=timers.target

debian-package/debian/vpn-router.vpn-router-web.service

@@ -6,12 +6,16 @@ Wants=network-online.target
 
 [Service]
 Type=simple
+ExecStartPre=/usr/lib/vpn-router/wait-for-libpq
 ExecStart=/usr/bin/python3 /usr/lib/vpn-router/web
+TimeoutStartSec=5min
 Restart=on-failure
 RestartSec=15
 NoNewPrivileges=yes
 ProtectSystem=strict
-ReadWritePaths=-/run/vpn-router -/etc/vpn-router -/etc/swanctl/x509crl
+RuntimeDirectory=vpn-router
+RuntimeDirectoryPreserve=yes
+ReadWritePaths=-/etc/vpn-router -/etc/swanctl/x509crl -/etc/openvpn/server -/var/lib/vpn-router/pki-publish
 ProtectHome=yes
 PrivateTmp=yes
 

debian-package/src/usr/lib/systemd/system/caddy.service.d/vpn-router.conf

@@ -0,0 +1,10 @@
+# Caddy is the HTTPS proxy of vpn-router: it reads the configuration vpn-router-setup writes, and
+# does not start until that exists.
+[Unit]
+ConditionPathExists=/etc/caddy/vpn-router.Caddyfile
+
+[Service]
+ExecStart=
+ExecStart=/usr/bin/caddy run --environ --adapter caddyfile --config /etc/caddy/vpn-router.Caddyfile
+ExecReload=
+ExecReload=/usr/bin/caddy reload --adapter caddyfile --config /etc/caddy/vpn-router.Caddyfile --force

debian-package/src/usr/lib/vpn-router/bootstrap

@@ -0,0 +1,55 @@
+#!/usr/bin/python3
+"""Serve diagnostics of the router's first setup on PORT, with the Python standard library alone.
+
+Run by vpn-router-bootstrap.service, which vpn-router-setup starts until a setup has succeeded and
+then stops. Every request on any address answers, as plain text, the output of COMMANDS run at that
+moment: the state of the last setup run, the state of the services, the addresses, and the end of
+the logs.
+"""
+
+import subprocess
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+
+PORT = 17081
+
+#: The title of each section and its command.
+COMMANDS = (
+    ('Setup state', ['cat', '/run/vpn-router/setup.json']),
+    ('Services', ['systemctl', 'is-active', 'vpn-router-setup', 'vpn-router-web', 'slapd', 'postgresql',
+                  'strongswan', 'openvpn-server@vpn-router', 'wg-quick@wg0']),
+    ('Failed units', ['systemctl', '--failed', '--no-legend', '--plain']),
+    ('Addresses', ['ip', '-brief', 'address']),
+    ('Routes', ['ip', 'route']),
+    ('cloud-init', ['cloud-init', 'status', '--long']),
+    ('vpn-router-setup log', ['journalctl', '--unit', 'vpn-router-setup', '--lines', '60', '--no-pager', '--output', 'short-iso']),
+    ('vpn-router-web log', ['journalctl', '--unit', 'vpn-router-web', '--lines', '30', '--no-pager', '--output', 'short-iso']),
+)
+
+
+def report():
+    """The output of every command of COMMANDS under its title."""
+    sections = []
+    for title, command in COMMANDS:
+        try:
+            result = subprocess.run(command, capture_output=True, text=True, timeout=10)
+            text = (result.stdout + result.stderr).rstrip()
+        except (OSError, subprocess.TimeoutExpired) as exc:
+            text = f'cannot run {command[0]}: {exc}'
+        sections.append(f'== {title} ==\n{text}\n')
+    return '\n'.join(sections)
+
+
+class Handler(BaseHTTPRequestHandler):
+    def do_GET(self):
+        body = report().encode()
+        self.send_response(200)
+        self.send_header('Content-Type', 'text/plain; charset=utf-8')
+        self.send_header('Content-Length', str(len(body)))
+        self.send_header('Cache-Control', 'no-store')
+        self.send_header('Refresh', '10')
+        self.end_headers()
+        self.wfile.write(body)
+
+
+if __name__ == '__main__':
+    ThreadingHTTPServer(('', PORT), Handler).serve_forever()

debian-package/src/usr/lib/vpn-router/generate-config

@@ -18,7 +18,9 @@ on the next dpkg-reconfigure.
 
 Answers arrive as VPN_ROUTER_* environment variables and are converted to the
 type of the setting they fill: comma-separated lists become arrays, "true" and
-"false" become booleans and digits become integers.
+"false" become booleans and digits become integers. The site-to-site answers
+fill the connection vpnrouter.FIRST_CONNECTION of site_to_site; the local
+subnets fill both its local_cidrs and local.cidrs.
 """
 
 import json
@@ -29,27 +31,29 @@ sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
 
 import vpnrouter
 
-# environment variable -> (section, option)
+FIRST = ('site_to_site', vpnrouter.FIRST_CONNECTION)
+
+# environment variable -> the settings it fills, each (section, option) or (site_to_site, name, option)
 ANSWERS = {
-    'VPN_ROUTER_PLATFORM':        ('general', 'platform'),
-    'VPN_ROUTER_MODE':            ('general', 'mode'),
-    'VPN_ROUTER_INT_ADDR':        ('local', 'int_addr'),
-    'VPN_ROUTER_EXTERNAL_INTERFACE': ('interfaces', 'external'),
-    'VPN_ROUTER_INTERNAL_INTERFACE': ('interfaces', 'internal'),
-    'VPN_ROUTER_LOCAL_FQDN':      ('wan', 'local_fqdn'),
-    'VPN_ROUTER_LOCAL_ID_MODE':   ('wan', 'local_id_mode'),
-    'VPN_ROUTER_LOCAL_CIDRS':     ('local', 'cidrs'),
-    'VPN_ROUTER_INT_GATEWAY_IP':  ('local', 'int_gateway_ip'),
-    'VPN_ROUTER_REMOTE_ADDRS':    ('remote', 'addrs'),
-    'VPN_ROUTER_REMOTE_ID':       ('remote', 'id'),
-    'VPN_ROUTER_REMOTE_CIDRS':    ('remote', 'cidrs'),
-    'VPN_ROUTER_PSK':             ('remote', 'psk'),
-    'VPN_ROUTER_P2S_ENABLED':     ('p2s', 'enabled'),
-    'VPN_ROUTER_P2S_ADDRESS_POOL': ('p2s', 'address_pool'),
-    'VPN_ROUTER_P2S_CA_NAME':     ('p2s', 'ca_name'),
-    'VPN_ROUTER_WG_ENABLED':      ('wireguard', 'enabled'),
-    'VPN_ROUTER_WG_ADDRESS':      ('wireguard', 'address'),
-    'VPN_ROUTER_WG_LISTEN_PORT':  ('wireguard', 'listen_port'),
+    'VPN_ROUTER_PLATFORM':        (('general', 'platform'),),
+    'VPN_ROUTER_MODE':            (('general', 'mode'),),
+    'VPN_ROUTER_INT_ADDR':        (('local', 'int_addr'),),
+    'VPN_ROUTER_EXTERNAL_INTERFACE': (('interfaces', 'external'),),
+    'VPN_ROUTER_INTERNAL_INTERFACE': (('interfaces', 'internal'),),
+    'VPN_ROUTER_LOCAL_FQDN':      (('wan', 'local_fqdn'),),
+    'VPN_ROUTER_LOCAL_ID_MODE':   ((*FIRST, 'local_id_mode'),),
+    'VPN_ROUTER_LOCAL_CIDRS':     (('local', 'cidrs'), (*FIRST, 'local_cidrs')),
+    'VPN_ROUTER_INT_GATEWAY_IP':  (('local', 'int_gateway_ip'),),
+    'VPN_ROUTER_REMOTE_ADDRS':    ((*FIRST, 'addrs'),),
+    'VPN_ROUTER_REMOTE_ID':       ((*FIRST, 'id'),),
+    'VPN_ROUTER_REMOTE_CIDRS':    ((*FIRST, 'cidrs'),),
+    'VPN_ROUTER_PSK':             ((*FIRST, 'psk'),),
+    'VPN_ROUTER_P2S_ENABLED':     (('p2s', 'enabled'),),
+    'VPN_ROUTER_P2S_ADDRESS_POOL': (('p2s', 'address_pool'),),
+    'VPN_ROUTER_PKI_ROOT_NAME':   (('pki', 'root_name'),),
+    'VPN_ROUTER_WG_ENABLED':      (('wireguard', 'enabled'),),
+    'VPN_ROUTER_WG_ADDRESS':      (('wireguard', 'address'),),
+    'VPN_ROUTER_WG_LISTEN_PORT':  (('wireguard', 'listen_port'),),
 }
 
 
@@ -74,23 +78,28 @@ def main():
             vpnrouter.fail(f'cannot parse {vpnrouter.CONFIG_FILE}: the top level is not an object')
 
     answered = False
-    for variable, (section, option) in ANSWERS.items():
-        value = os.environ.get(variable)
-        if not value:
+    for variable, paths in ANSWERS.items():
+        answer = os.environ.get(variable)
+        if not answer:
             continue
-        # A value that does not convert is kept as a string for validate() to report.
-        default = vpnrouter.SCHEMA[section][option]
-        if isinstance(default, bool):
-            value = value == 'true'
-        elif isinstance(default, int):
-            value = int(value) if value.isdigit() else value
-        elif isinstance(default, list):
-            value = [item.strip() for item in value.split(',') if item.strip()]
-        if not isinstance(cfg.get(section, {}), dict):
-            vpnrouter.fail(f'cannot parse {vpnrouter.CONFIG_FILE}: {section} is not an object')
-        if cfg.setdefault(section, {}).get(option) != value:
-            cfg[section][option] = value
-            answered = True
+        for path in paths:
+            # A value that does not convert is kept as a string for validate() to report.
+            default = vpnrouter.CONNECTION[path[2]] if path[0] == 'site_to_site' else vpnrouter.SCHEMA[path[0]][path[1]]
+            value = answer
+            if isinstance(default, bool):
+                value = value == 'true'
+            elif isinstance(default, int):
+                value = int(value) if value.isdigit() else value
+            elif isinstance(default, list):
+                value = [item.strip() for item in value.split(',') if item.strip()]
+            target = cfg
+            for key in path[:-1]:
+                if not isinstance(target.setdefault(key, {}), dict):
+                    vpnrouter.fail(f'cannot parse {vpnrouter.CONFIG_FILE}: {".".join(path[:-1])} is not an object')
+                target = target[key]
+            if target.get(path[-1]) != value:
+                target[path[-1]] = value
+                answered = True
 
     # Without a change the candidate is the current file byte for byte, as ucf compares checksums.
     with open(output, 'w', encoding='utf-8') as handle:

debian-package/src/usr/lib/vpn-router/metrics

@@ -0,0 +1,456 @@
+#!/usr/bin/python3
+"""Collect the metrics of the router, keep them in PostgreSQL, serve them for scraping and push them with OTLP.
+
+Run by vpn-router-metrics.service while metrics.enabled is true; vpn-router-setup enables and
+disables the unit.
+
+Every SAMPLE_SECONDS, on the minute, a sample is taken with psutil: the CPU utilization since the
+previous sample, the memory used and in total, the free and the total space of /, the bytes
+received and sent per second on the external and the internal interface, and the load averages;
+and the VPN clients of each enabled VPN system (vpnrouter_status): road-warrior (p2s) and
+OpenVPN clients and WireGuard peers with a recent handshake; and of each site-to-site connection
+that is set up, whether it is up, the bytes and packets per second through its child SAs since
+the previous sample, and how long its IKE SA has been up. The interfaces, the enabled systems and
+the connections are those vpn-router-setup resolved, read from /run/vpn-router/web.json.
+
+The sample is written to the detail tables of vpnrouter_metrics. When the clock has passed into
+a new hour, day, month or year, the complete periods are summarized, and once a day, after the
+daily summary, the rows older than their retention are deleted; both also run on start. A
+failure of the database is logged and tried again with the next sample.
+
+GET /metrics answers the last sample in the Prometheus text format, written by
+prometheus_client, with OpenTelemetry semantic convention names in their Prometheus form, for an
+OpenTelemetry Collector's prometheus receiver. It needs HTTP Basic authentication as the service
+account metrics.service_account names, which vpn-router-setup keeps with the password in
+/etc/vpn-router/metrics.secret, checked with a bind to the router's directory, so failed attempts
+lock the account as for any user. The service listens on the interfaces metrics.listen
+names and on the loopback address, where Caddy serves it over HTTPS at metrics.https_port; the
+address of a call through Caddy is the last entry of its X-Forwarded-For header.
+
+When metrics.otlp_endpoint is set, every sample is also pushed there as an OTLP/HTTP JSON
+ExportMetricsServiceRequest, with HTTP Basic authentication when metrics.otlp_username is set.
+"""
+
+import base64
+import datetime
+import json
+import os
+import signal
+import socket
+import sys
+import threading
+import time
+import urllib.error
+import urllib.request
+from http.server import BaseHTTPRequestHandler
+
+sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
+
+import ldap
+import psutil
+from prometheus_client import CONTENT_TYPE_LATEST, generate_latest
+from prometheus_client.core import CollectorRegistry, CounterMetricFamily, GaugeMetricFamily
+
+import vpnrouter
+import vpnrouter_metrics
+import vpnrouter_net
+import vpnrouter_serve
+import vpnrouter_status
+import vpnrouter_users
+
+STATE_FILE = '/run/vpn-router/web.json'
+CONFIG_FILE = '/etc/vpn-router/vpn-router.json'
+
+SAMPLE_SECONDS = 60
+
+#: Seconds between two discoveries of where the service should listen.
+DISCOVERY_SECONDS = 30
+
+#: Seconds an OTLP push may take.
+OTLP_TIMEOUT = 10
+
+#: The mount point whose space is measured.
+MOUNTPOINT = '/'
+
+#: The last sample, as sample() makes it, or None before the first.
+_latest = {'sample': None}
+_latest_lock = threading.Lock()
+
+
+def log(message):
+    print(f'vpn-router-metrics: {message}', file=sys.stderr, flush=True)
+
+
+def read_json(path):
+    """The object in a JSON file, or {} when the file is missing or holds no object."""
+    try:
+        with open(path, encoding='utf-8') as handle:
+            data = json.load(handle)
+    except (OSError, ValueError):
+        return {}
+    return data if isinstance(data, dict) else {}
+
+
+def section(configuration, name):
+    """A section of the configuration, or {} when it is missing or not an object."""
+    value = configuration.get(name)
+    return value if isinstance(value, dict) else {}
+
+
+def clients(state):
+    """The VPN clients of each enabled VPN system, None where it cannot be asked."""
+    counts = {}
+    if state.get('p2s'):
+        counts['p2s'] = vpnrouter_status.established('road-warrior')
+    if state.get('openvpn'):
+        counts['openvpn'] = vpnrouter_status.openvpn_clients()
+    if state.get('wireguard'):
+        counts['wireguard'] = vpnrouter_status.wireguard_connected()
+    return counts
+
+
+def sample(previous):
+    """A sample of the router, with the rates since the previous sample, None for the first."""
+    state = read_json(STATE_FILE)
+    network = section(state, 'network')
+    names = [name for name in dict.fromkeys((network.get('external_interface'), network.get('internal_interface')))
+             if isinstance(name, str) and name]
+    counters = psutil.net_io_counters(pernic=True)
+    memory = psutil.virtual_memory()
+    disk = psutil.disk_usage(MOUNTPOINT)
+    load1, load5, load15 = os.getloadavg()
+    connections = state.get('site_to_site')
+    tunnels = vpnrouter_status.site_to_site_tunnels(connections) if isinstance(connections, list) and connections else {}
+    now = time.time()
+    current = {
+        'time': now,
+        'monotonic': time.monotonic(),
+        'system': {
+            'cpu': psutil.cpu_percent(interval=None),
+            'memory_used': memory.total - memory.available,
+            'memory_total': memory.total,
+            'disk_free': disk.free,
+            'disk_total': disk.total,
+            'load1': load1,
+            'load5': load5,
+            'load15': load15,
+        },
+        'counters': {name: (counters[name].bytes_recv, counters[name].bytes_sent) for name in names if name in counters},
+        'clients': clients(state),
+        'rates': {},
+        'tunnels': tunnels or {},
+        'site_to_site': {},
+    }
+    elapsed = current['monotonic'] - previous['monotonic'] if previous is not None else 0
+    if elapsed > 0:
+        for name, (received, sent) in current['counters'].items():
+            if name in previous['counters']:
+                before_received, before_sent = previous['counters'][name]
+                current['rates'][name] = (max(received - before_received, 0) / elapsed, max(sent - before_sent, 0) / elapsed)
+    # A connection's rates need its counters of the previous sample; a counter that went down was rekeyed and counts from zero.
+    for name, tunnel in current['tunnels'].items():
+        before = previous['tunnels'].get(name) if elapsed > 0 else None
+        values = {'up': tunnel['up'], 'uptime': tunnel['uptime']}
+        for value, counter in (('rx', 'bytes_in'), ('tx', 'bytes_out'), ('packets_rx', 'packets_in'), ('packets_tx', 'packets_out')):
+            if before is None:
+                values[value] = None
+            else:
+                count = tunnel[counter]
+                values[value] = (count - before[counter] if count >= before[counter] else count) / elapsed
+        current['site_to_site'][name] = values
+    return current
+
+
+def periods(moment):
+    """The hour, the day, the month and the year of a UTC moment, keyed by the levels of vpnrouter_metrics.LEVELS."""
+    return {'hourly': moment.strftime('%Y-%m-%dT%H'), 'daily': moment.strftime('%Y-%m-%d'),
+            'monthly': moment.strftime('%Y-%m'), 'yearly': moment.strftime('%Y')}
+
+
+def store(cfg, current, done):
+    """Write the sample, summarize the periods the clock has passed and collect the garbage once a day.
+
+    done holds the periods summarized so far, and garbage the day of the last garbage collection;
+    it is updated only for what succeeded.
+    """
+    now = periods(datetime.datetime.now(datetime.timezone.utc))
+    with vpnrouter_metrics.database(cfg) as connection:
+        vpnrouter_metrics.write(connection, current['system'], current['rates'], current['clients'], current['site_to_site'])
+        for level, _, _ in vpnrouter_metrics.LEVELS:
+            if done.get(level) != now[level]:
+                vpnrouter_metrics.summarize(connection, level)
+                done[level] = now[level]
+        if done.get('garbage') != now['daily']:
+            vpnrouter_metrics.collect_garbage(connection, cfg)
+            done['garbage'] = now['daily']
+
+
+def attributes(**values):
+    """OTLP attributes of string values."""
+    return [{'key': key, 'value': {'stringValue': value}} for key, value in values.items()]
+
+
+def otlp_request(current, started):
+    """The OTLP/HTTP JSON ExportMetricsServiceRequest of a sample; started is when the counters were first read, in Unix seconds."""
+    stamp = str(int(current['time'] * 1e9))
+    start = str(int(started * 1e9))
+    system = current['system']
+
+    def gauge(name, unit, points):
+        return {'name': name, 'unit': unit, 'gauge': {'dataPoints': [
+            {'timeUnixNano': stamp, 'asDouble': float(value), 'attributes': attributes(**labels)}
+            for value, labels in points if value is not None]}}
+
+    metrics = [
+        gauge('system.cpu.utilization', '1', [(system['cpu'] / 100, {})]),
+        gauge('system.memory.usage', 'By', [(system['memory_used'], {'state': 'used'})]),
+        gauge('system.memory.limit', 'By', [(system['memory_total'], {})]),
+        gauge('system.filesystem.usage', 'By', [(system['disk_free'], {'mountpoint': MOUNTPOINT, 'state': 'free'}),
+                                               (system['disk_total'] - system['disk_free'], {'mountpoint': MOUNTPOINT, 'state': 'used'})]),
+        gauge('system.filesystem.utilization', '1',
+              [(1 - system['disk_free'] / system['disk_total'] if system['disk_total'] else None, {'mountpoint': MOUNTPOINT})]),
+        gauge('system.cpu.load_average.1m', '{thread}', [(system['load1'], {})]),
+        gauge('system.cpu.load_average.5m', '{thread}', [(system['load5'], {})]),
+        gauge('system.cpu.load_average.15m', '{thread}', [(system['load15'], {})]),
+        gauge('vpn_router.clients', '{client}', [(count, {'kind': kind}) for kind, count in current['clients'].items()]),
+        gauge('vpn_router.site_to_site.up', '1', [(values['up'], {'connection': name})
+                                                 for name, values in current['site_to_site'].items()]),
+        gauge('vpn_router.site_to_site.uptime', 's', [(values['uptime'], {'connection': name})
+                                                     for name, values in current['site_to_site'].items()]),
+        gauge('vpn_router.site_to_site.io', 'By/s', [(values[value], {'connection': name, 'direction': direction})
+                                                    for name, values in current['site_to_site'].items()
+                                                    for value, direction in (('rx', 'receive'), ('tx', 'transmit'))]),
+        gauge('vpn_router.site_to_site.packets', '{packet}/s',
+              [(values[value], {'connection': name, 'direction': direction})
+               for name, values in current['site_to_site'].items()
+               for value, direction in (('packets_rx', 'receive'), ('packets_tx', 'transmit'))]),
+        {'name': 'system.network.io', 'unit': 'By', 'sum': {
+            'aggregationTemporality': 2, 'isMonotonic': True, 'dataPoints': [
+                {'startTimeUnixNano': start, 'timeUnixNano': stamp, 'asInt': str(value),
+                 'attributes': attributes(device=name, direction=direction)}
+                for name, counters in current['counters'].items()
+                for value, direction in zip(counters, ('receive', 'transmit'))]}},
+    ]
+    return {'resourceMetrics': [{
+        'resource': {'attributes': attributes(**{'service.name': 'vpn-router', 'host.name': socket.gethostname()})},
+        'scopeMetrics': [{'scope': {'name': 'vpn-router-metrics'}, 'metrics': metrics}],
+    }]}
+
+
+def push(metrics, current, started):
+    """Push the sample to metrics.otlp_endpoint; raises OSError when the collector does not take it."""
+    request = urllib.request.Request(metrics['otlp_endpoint'], data=json.dumps(otlp_request(current, started)).encode(),
+                                     headers={'Content-Type': 'application/json'}, method='POST')
+    if metrics['otlp_username']:
+        credentials = f"{metrics['otlp_username']}:{metrics['otlp_password']}".encode()
+        request.add_header('Authorization', 'Basic ' + base64.b64encode(credentials).decode())
+    with urllib.request.urlopen(request, timeout=OTLP_TIMEOUT) as response:
+        response.read()
+
+
+def collector_loop():
+    """Take a sample on every minute, store it and push it; every new failure is logged once."""
+    previous, done, reported = None, {}, {}
+    started = time.time()
+    psutil.cpu_percent(interval=None)
+
+    def report(kind, message):
+        if message != reported.get(kind):
+            if message:
+                log(message)
+            elif reported.get(kind):
+                log(f'{kind} works again')
+            reported[kind] = message
+
+    while True:
+        time.sleep(SAMPLE_SECONDS - time.time() % SAMPLE_SECONDS)
+        try:
+            cfg = vpnrouter.load_config()
+        except SystemExit as exc:
+            report('configuration', str(exc.code))
+            continue
+        report('configuration', '')
+        current = sample(previous)
+        previous = current
+        with _latest_lock:
+            _latest['sample'] = current
+        try:
+            store(cfg, current, done)
+            report('database', '')
+        except vpnrouter_metrics.DatabaseError as exc:
+            report('database', str(exc))
+        metrics = cfg['metrics']
+        if metrics['otlp_endpoint']:
+            try:
+                push(metrics, current, started)
+                report('OTLP push', '')
+            except (OSError, ValueError) as exc:
+                report('OTLP push', f"pushing to {metrics['otlp_endpoint']} failed: {exc}")
+
+
+class Sample:
+    """A prometheus_client collector of a sample."""
+
+    def __init__(self, current):
+        self.current = current
+
+    def collect(self):
+        system = self.current['system']
+        yield GaugeMetricFamily('system_cpu_utilization', 'CPU utilization since the previous sample, 0 to 1',
+                                value=system['cpu'] / 100)
+        memory = GaugeMetricFamily('system_memory_usage_bytes', 'Memory in use', labels=['state'])
+        memory.add_metric(['used'], system['memory_used'])
+        yield memory
+        yield GaugeMetricFamily('system_memory_limit_bytes', 'Total memory', value=system['memory_total'])
+        usage = GaugeMetricFamily('system_filesystem_usage_bytes', 'Space of the file system', labels=['mountpoint', 'state'])
+        usage.add_metric([MOUNTPOINT, 'free'], system['disk_free'])
+        usage.add_metric([MOUNTPOINT, 'used'], system['disk_total'] - system['disk_free'])
+        yield usage
+        if system['disk_total']:
+            utilization = GaugeMetricFamily('system_filesystem_utilization', 'Used part of the file system, 0 to 1',
+                                            labels=['mountpoint'])
+            utilization.add_metric([MOUNTPOINT], 1 - system['disk_free'] / system['disk_total'])
+            yield utilization
+        for period in ('1m', '5m', '15m'):
+            yield GaugeMetricFamily(f'system_cpu_load_average_{period}', f'Load average over {period}',
+                                    value=system[f'load{period[:-1]}'])
+        network = CounterMetricFamily('system_network_io_bytes', 'Bytes received and sent', labels=['device', 'direction'])
+        for name, (received, sent) in self.current['counters'].items():
+            network.add_metric([name, 'receive'], received)
+            network.add_metric([name, 'transmit'], sent)
+        yield network
+        vpn = GaugeMetricFamily('vpn_router_clients', 'Connected VPN clients', labels=['kind'])
+        for kind, count in self.current['clients'].items():
+            if count is not None:
+                vpn.add_metric([kind], count)
+        yield vpn
+        tunnels = self.current['site_to_site']
+        up = GaugeMetricFamily('vpn_router_site_to_site_up', 'Whether the site-to-site connection is established, 1 or 0',
+                               labels=['connection'])
+        uptime = GaugeMetricFamily('vpn_router_site_to_site_uptime_seconds', 'Seconds the IKE SA of the site-to-site connection has been up',
+                                   labels=['connection'])
+        io = GaugeMetricFamily('vpn_router_site_to_site_io_bytes_per_second',
+                               'Bytes per second received and sent through the site-to-site connection', labels=['connection', 'direction'])
+        packets = GaugeMetricFamily('vpn_router_site_to_site_packets_per_second',
+                                    'Packets per second received and sent through the site-to-site connection',
+                                    labels=['connection', 'direction'])
+        for name, values in tunnels.items():
+            up.add_metric([name], values['up'])
+            if values['uptime'] is not None:
+                uptime.add_metric([name], values['uptime'])
+            for family, received, sent in ((io, 'rx', 'tx'), (packets, 'packets_rx', 'packets_tx')):
+                for value, direction in ((received, 'receive'), (sent, 'transmit')):
+                    if values[value] is not None:
+                        family.add_metric([name, direction], values[value])
+        yield from (up, uptime, io, packets)
+
+
+def authorized(header, account):
+    """Whether an Authorization header holds the password of the service account; LDAP errors propagate."""
+    if not header.startswith('Basic '):
+        return False
+    try:
+        name, _, password = base64.b64decode(header[6:].strip(), validate=True).decode().partition(':')
+    except ValueError:
+        return False
+    if name != account:
+        return False
+    try:
+        with vpnrouter_users.session() as connection:
+            if not vpnrouter_users.service(vpnrouter_users.entry(connection, name)):
+                return False
+        vpnrouter_users.login(name, password, None)
+    except vpnrouter_users.Failure:
+        return False
+    return True
+
+
+class Handler(BaseHTTPRequestHandler):
+    protocol_version = 'HTTP/1.1'
+
+    def client(self):
+        """The client address: the last X-Forwarded-For entry of a call through Caddy on the loopback address, else the peer."""
+        peer = self.client_address[0]
+        forwarded = self.headers.get('X-Forwarded-For', '')
+        if peer == vpnrouter_net.LOOPBACK and forwarded.strip():
+            return forwarded.split(',')[-1].strip()
+        return peer
+
+    def answer(self, send_body):
+        self.principal = None
+        extra = []
+        path = self.path.split('?', 1)[0]
+        account = str(section(read_json(CONFIG_FILE), 'metrics').get('service_account',
+                                                                       vpnrouter.SCHEMA['metrics']['service_account'])).strip()
+        with _latest_lock:
+            current = _latest['sample']
+        if path != '/metrics':
+            body, kind, code = b'Not found\n', 'text/plain; charset=utf-8', 404
+        elif not account:
+            body, kind, code = b'metrics.service_account is not set\n', 'text/plain; charset=utf-8', 503
+        else:
+            try:
+                allowed = authorized(self.headers.get('Authorization', ''), account)
+            except (ldap.LDAPError, OSError) as exc:
+                allowed = None
+                body, kind, code = f'the user directory failed: {exc}\n'.encode(), 'text/plain; charset=utf-8', 500
+            if allowed is False:
+                body, kind, code = b'Unauthorized\n', 'text/plain; charset=utf-8', 401
+                extra.append(('WWW-Authenticate', 'Basic realm="vpn-router metrics", charset="UTF-8"'))
+            elif allowed:
+                self.principal = account
+                if current is None:
+                    body, kind, code = b'no sample yet\n', 'text/plain; charset=utf-8', 503
+                else:
+                    registry = CollectorRegistry()
+                    registry.register(Sample(current))
+                    body, kind, code = generate_latest(registry), CONTENT_TYPE_LATEST, 200
+        self.send_response(code)
+        self.send_header('Content-Type', kind)
+        self.send_header('Content-Length', str(len(body)))
+        self.send_header('Cache-Control', 'no-store')
+        for name, value in extra:
+            self.send_header(name, value)
+        self.end_headers()
+        if send_body:
+            self.wfile.write(body)
+
+    def do_GET(self):
+        self.answer(True)
+
+    def do_HEAD(self):
+        self.answer(False)
+
+    def log_message(self, format, *args):
+        principal = getattr(self, 'principal', None) or '-'
+        sys.stderr.write('%s - %s [%s] %s\n' % (self.client(), principal, self.log_date_time_string(), format % args))
+
+
+def wanted():
+    """Where the service listens: the addresses of the interfaces metrics.listen names and LOOPBACK, at metrics.port."""
+    configuration = read_json(CONFIG_FILE)
+    metrics = section(configuration, 'metrics')
+    defaults = vpnrouter.SCHEMA['metrics']
+    if metrics.get('enabled', defaults['enabled']) is False:
+        return [], 'metrics are disabled in the configuration'
+    port = metrics.get('port', defaults['port'])
+    if isinstance(port, bool) or not isinstance(port, int) or not 1 <= port <= 65535:
+        port = defaults['port']
+    addresses = vpnrouter_net.listen_addresses(configuration, metrics.get('listen', defaults['listen']))
+    if not addresses:
+        return [], 'the addresses to listen on are not known yet'
+    return [(address, port) for address in [*addresses, vpnrouter_net.LOOPBACK]], ''
+
+
+def main():
+    signal.signal(signal.SIGTERM, lambda *_: sys.exit(0))
+    threading.Thread(target=collector_loop, name='collector', daemon=True).start()
+    try:
+        vpnrouter_serve.serve(wanted, Handler, log, DISCOVERY_SECONDS)
+    except KeyboardInterrupt:
+        pass
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main())

debian-package/src/usr/lib/vpn-router/openvpn-issuer

@@ -0,0 +1,24 @@
+#!/usr/bin/python3
+"""Accept an OpenVPN client certificate only when the P2S issuing CA signed it.
+
+Run by the OpenVPN server as tls-verify for each certificate of the client's chain, with its depth and subject as the
+arguments. At depth 1, the CA that signed the client's certificate, the SHA-256 fingerprint OpenVPN gives in
+tls_digest_sha256_1 must be that of P2S_CA; the other depths are accepted.
+"""
+
+import hashlib
+import os
+import ssl
+import sys
+
+P2S_CA = '/etc/swanctl/x509ca/p2s.pem'
+
+if len(sys.argv) < 2 or sys.argv[1] != '1':
+    sys.exit(0)
+try:
+    with open(P2S_CA, encoding='ascii') as handle:
+        expected = hashlib.sha256(ssl.PEM_cert_to_DER_cert(handle.read())).hexdigest()
+except (OSError, ValueError):
+    sys.exit(1)
+given = os.environ.get('tls_digest_sha256_1', '').replace(':', '').lower()
+sys.exit(0 if given == expected else 1)

debian-package/src/usr/lib/vpn-router/setup

@@ -5,15 +5,24 @@ Regenerates the configuration files first, then does everything that needs
 the live system: routes, DNS-derived identity, WireGuard keys, firewall rules
 and service reloads. Idempotent - a second run changes nothing.
 
+Until a run has succeeded (SETUP_DONE), every run starts the bootstrap
+services, BOOTSTRAP_UNITS, which show the progress and the failure recorded in
+SETUP_STATE on BOOTSTRAP_PORTS, and the firewall opens those ports on the
+internal interface. The first successful run writes SETUP_DONE, closes the
+ports and stops the bootstrap services; a failed run leaves them running.
+
 Interface names come from the configuration and are never guessed. The
 firewall and NAT rules are written against them, so choosing the wrong one
 would be a silent and security-relevant mistake.
 """
 
+import grp
 import json
 import os
 import pathlib
 import re
+import secrets
+import shutil
 import socket
 import subprocess
 import sys
@@ -21,23 +30,59 @@ import time
 
 sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
 
+import ldap
+
 import vpnrouter
 import vpnrouter_ldap
-from vpnrouter import fail
+import vpnrouter_pki
+import vpnrouter_users
+from vpnrouter import fail, warn
 
 BEFORE_RULES = pathlib.Path('/etc/ufw/before.rules')
 UFW_DEFAULTS = pathlib.Path('/etc/default/ufw')
 WG_DIR       = pathlib.Path('/etc/wireguard')
 WEB_STATE    = pathlib.Path('/run/vpn-router/web.json')
 
-#: The state, the current step and the failure of the last run, which the web service shows while it starts.
+#: The state, the current step and the failure of the last run, which the bootstrap services show.
 SETUP_STATE  = pathlib.Path('/run/vpn-router/setup.json')
 
+#: Written once a run has succeeded; until then setup runs the bootstrap services and opens their ports.
+SETUP_DONE   = pathlib.Path('/var/lib/vpn-router/setup.done')
+
+#: The units of the bootstrap services, and their ports: the Router Manager's and the router's.
+BOOTSTRAP_UNITS = ('vpn-router-bootstrap.service', 'vpn-router-manager-bootstrap.service')
+BOOTSTRAP_PORTS = (17080, 17081)
+
 #: The Router Manager, vpn-router-manager, when it is installed, and its settings.
 MANAGER        = pathlib.Path('/usr/lib/vpn-router-manager/manager')
 MANAGER_CONFIG = pathlib.Path('/etc/vpn-router-manager/manager.json')
+
+#: The default admin of the Router Manager, {"username", "password"}, which its package writes from
+#: its debconf answers; setup creates the account and removes the file.
+MANAGER_ADMIN  = pathlib.Path('/etc/vpn-router-manager/admin.json')
 MANAGER_PORT   = 7080
 
+#: The password of the metrics service account, which setup makes and keeps (root only).
+METRICS_SECRET = pathlib.Path('/etc/vpn-router/metrics.secret')
+
+#: The schedule of the backup timer, written from backup.schedule.
+BACKUP_SCHEDULE = pathlib.Path('/etc/systemd/system/vpn-router-backup.timer.d/schedule.conf')
+
+#: The configuration of Caddy, the HTTPS proxy, which the package's drop-in for caddy.service names;
+#: Caddy runs as the user and group CADDY_GROUP, which the caddy package creates.
+CADDYFILE   = pathlib.Path('/etc/caddy/vpn-router.Caddyfile')
+CADDY_UNIT  = 'caddy.service'
+CADDY_GROUP = 'caddy'
+
+#: The unit that installs Caddy from https.caddy_source with vpn-router-caddy, outside setup.
+CADDY_INSTALL_UNIT = 'vpn-router-caddy.service'
+
+#: The certificates and keys of the issuing CAs behind Caddy's ACME server, <name>/cert.pem and key.pem.
+CADDY_PKI_DIR = pathlib.Path('/etc/caddy/vpn-router-pki')
+
+#: The timer of vpn-router-ca maintain, which keeps the CRLs fresh.
+PKI_TIMER = 'vpn-router-pki.timer'
+
 FILTER_ANCHOR = '# End required lines'
 
 #: Configuration options left out wherever the configuration is shown.
@@ -49,7 +94,10 @@ BLOCKS = (
     ('# OPENVPN RULES START',       '# OPENVPN RULES END'),
     ('# P2S DNS RULES START',       '# P2S DNS RULES END'),
     ('# WEB RULES START',           '# WEB RULES END'),
+    ('# BOOTSTRAP RULES START',     '# BOOTSTRAP RULES END'),
     ('# MANAGER RULES START',       '# MANAGER RULES END'),
+    ('# HTTPS RULES START',         '# HTTPS RULES END'),
+    ('# METRICS RULES START',       '# METRICS RULES END'),
     ('# ROUTER FORWARD RULES START', '# ROUTER FORWARD RULES END'),
     ('# ROUTER NAT RULES START',    '# ROUTER NAT RULES END'),
     ('# ROUTER MSS RULES START',    '# ROUTER MSS RULES END'),
@@ -194,23 +242,60 @@ def filter_block(ctx):
                 ]
         lines += ['# P2S DNS RULES END']
 
-    if ctx['web_enabled'] and ctx['int_iface']:
+    # The web service, the Router Manager and the metrics service choose their interfaces themselves, so their ports are open on both.
+    interfaces = [name for name in (ctx['int_iface'], ctx['wan_iface']) if name]
+    if ctx['web_enabled'] and interfaces:
         lines += [
             '',
             '# WEB RULES START',
-            f'-A ufw-before-input -i {ctx["int_iface"]} -p tcp --dport {ctx["web_port"]} -j ACCEPT',
+            *(f'-A ufw-before-input -i {name} -p tcp --dport {ctx["web_port"]} -j ACCEPT' for name in interfaces),
             '# WEB RULES END',
         ]
 
-    if MANAGER.exists() and ctx['int_iface']:
+    if not SETUP_DONE.exists() and ctx['int_iface']:
+        lines += [
+            '',
+            '# BOOTSTRAP RULES START',
+            *(f'-A ufw-before-input -i {ctx["int_iface"]} -p tcp --dport {port} -j ACCEPT' for port in BOOTSTRAP_PORTS),
+            '# BOOTSTRAP RULES END',
+        ]
+
+    if MANAGER.exists() and interfaces:
         port = manager_port()
         lines += [
             '',
             '# MANAGER RULES START',
-            f'-A ufw-before-input -i {ctx["int_iface"]} -p tcp --dport {port} -j ACCEPT',
+            *(f'-A ufw-before-input -i {name} -p tcp --dport {port} -j ACCEPT' for name in interfaces),
             '# MANAGER RULES END',
         ]
 
+    if vpnrouter.metrics_ready(ctx) and interfaces:
+        lines += [
+            '',
+            '# METRICS RULES START',
+            *(f'-A ufw-before-input -i {name} -p tcp --dport {ctx["metrics_port"]} -j ACCEPT' for name in interfaces),
+            '# METRICS RULES END',
+        ]
+
+    # Let's Encrypt reaches the HTTP challenge on the external interface, and the CA files published on port 80 are
+    # read on both; the HTTPS ports are open on both.
+    if vpnrouter.https_ready(ctx) and interfaces:
+        publish = ctx['pki_publish']
+        ports = [port for wanted, port in ((ctx['web_enabled'] or 'https' in publish or ctx['pki_acme'],
+                                            ctx['https_web_port']),
+                                           (MANAGER.exists(), ctx['https_manager_port']),
+                                           (vpnrouter.metrics_ready(ctx), ctx['metrics_https_port']),
+                                           ('http-80' in publish, vpnrouter.ACME_HTTP_PORT),
+                                           ('https-default' in publish, vpnrouter.HTTPS_DEFAULT_PORT)) if wanted]
+        lines += [
+            '',
+            '# HTTPS RULES START',
+            *([f'-A ufw-before-input -i {ctx["wan_iface"]} -p tcp --dport {vpnrouter.ACME_HTTP_PORT} -j ACCEPT']
+              if ctx['wan_iface'] and 'http-80' not in publish else []),
+            *(f'-A ufw-before-input -i {name} -p tcp --dport {port} -j ACCEPT' for name in interfaces for port in ports),
+            '# HTTPS RULES END',
+        ]
+
     if vpnrouter.routing_ready(ctx):
         lines += ['', '# ROUTER FORWARD RULES START']
         for subnet in ctx['local_cidr_list']:
@@ -228,7 +313,6 @@ def nat_and_mangle_block(ctx):
 
     if vpnrouter.routing_ready(ctx):
         subnets = ctx['local_cidr_list']
-        remote = ctx['remote_cidr_list']
         lines += [
             '',
             '# ROUTER NAT RULES START',
@@ -236,11 +320,14 @@ def nat_and_mangle_block(ctx):
             ':POSTROUTING ACCEPT [0:0]',
             '-F POSTROUTING',
         ]
-        # Every RETURN precedes every MASQUERADE, so tunnel-bound traffic
-        # escapes NAT whichever local subnet it came from.
-        for subnet in subnets:
-            for cidr in remote:
-                lines.append(f'-A POSTROUTING -s {subnet} -d {cidr} -j RETURN')
+        # Every RETURN precedes every MASQUERADE, so traffic bound for a tunnel
+        # escapes NAT; each ready connection pairs its local and remote subnets.
+        for connection in ctx['connections']:
+            if not connection['ready']:
+                continue
+            for subnet in connection['local_cidr_list']:
+                for cidr in connection['remote_cidr_list']:
+                    lines.append(f'-A POSTROUTING -s {subnet} -d {cidr} -j RETURN')
         for subnet in subnets:
             lines.append(
                 f'-A POSTROUTING -s {subnet} -o {ctx["wan_iface"]} -j MASQUERADE')
@@ -362,6 +449,86 @@ def manage_directory():
     return ''
 
 
+def manage_admin():
+    """Create the Router Manager's default admin from MANAGER_ADMIN, then remove the file; the problem, or ''.
+
+    The admin is a Manager-only account and a member of the admins group, which is created with it
+    when it does not exist.
+    """
+    if not MANAGER_ADMIN.exists():
+        return ''
+    try:
+        answers = json.loads(MANAGER_ADMIN.read_text(encoding='utf-8'))
+        name, password = answers['username'], answers['password']
+        with vpnrouter_users.session() as connection:
+            vpnrouter_users.add(connection, name, password, {})
+            try:
+                vpnrouter_users.group_member_add(connection, vpnrouter_users.ADMIN_GROUP, name)
+            except vpnrouter_users.Failure as failure:
+                if failure.status != 3:
+                    raise
+                vpnrouter_users.group_add(connection, vpnrouter_users.ADMIN_GROUP, 'Router administrators', [name])
+        MANAGER_ADMIN.unlink()
+    except (OSError, ValueError, KeyError, TypeError, ldap.LDAPError, vpnrouter_users.Failure) as exc:
+        return f'the Router Manager admin cannot be created from {MANAGER_ADMIN}: {exc}'
+    return ''
+
+
+def manage_metrics_account(ctx):
+    """Keep the service account metrics.service_account names, whose password is in METRICS_SECRET; the problem, or ''.
+
+    While metrics are enabled, setup owns the account, an internal service account: it adds it as a service account
+    with a random password and the labels of vpnrouter_users.internal_accounts when it is missing, sets those labels
+    when they differ, and gives it a new random password, ending any lockout, when METRICS_SECRET is missing or does
+    not hold its password. The password of a disabled account is left as it is. An account of that name that is not a
+    service account is left alone and is a problem.
+    """
+    if not vpnrouter.metrics_ready(ctx):
+        return ''
+    name = ctx['metrics_service_account']
+    wanted = vpnrouter_users.internal_accounts().get(name, {})
+    try:
+        vpnrouter_users.check_name(name)
+        try:
+            password = METRICS_SECRET.read_text(encoding='utf-8').strip()
+        except OSError:
+            password = ''
+        with vpnrouter_users.session() as connection:
+            try:
+                current = vpnrouter_users.entry(connection, name)
+            except vpnrouter_users.Failure as failure:
+                if failure.status != 3:
+                    raise
+                current = None
+            if current is not None and not vpnrouter_users.service(current):
+                return f'metrics.service_account: {name} is a user, not a service account'
+            if current is not None and any(vpnrouter_users.summary(current)[key] != value for key, value in wanted.items()):
+                vpnrouter_users.update_service(connection, name, wanted)
+            # An administrator who disabled the account meant it; its password is left as it is.
+            if current is not None and not vpnrouter_users.summary(current)['enabled']:
+                return ''
+            valid = False
+            if current is not None and password:
+                try:
+                    vpnrouter_users.login(name, password, None)
+                    valid = True
+                except vpnrouter_users.Failure:
+                    pass
+            if not valid:
+                password = secrets.token_urlsafe(32)
+                METRICS_SECRET.touch(mode=0o600)
+                METRICS_SECRET.chmod(0o600)
+                METRICS_SECRET.write_text(password + '\n', encoding='utf-8')
+                if current is None:
+                    vpnrouter_users.add(connection, name, password, wanted, service=True)
+                else:
+                    vpnrouter_users.passwd(connection, name, password)
+                    vpnrouter_users.unlock(connection, name)
+    except (OSError, ldap.LDAPError, vpnrouter_users.Failure) as exc:
+        return f'the metrics service account cannot be kept: {exc}'
+    return ''
+
+
 def manage_openvpn(ctx, changed):
     """Run the OpenVPN server when it is configured, and stop it when it is not."""
     unit = 'openvpn-server@vpn-router'
@@ -377,6 +544,106 @@ def manage_openvpn(ctx, changed):
         run(['systemctl', 'restart', unit], check=False)
 
 
+def manage_caddy(cfg, ctx):
+    """Run Caddy, the HTTPS proxy of the web service, the Router Manager and the metrics service, while HTTPS is ready, and stop it when not.
+
+    CADDYFILE serves the web service on https.web_port, while the manager is installed the
+    manager on https.manager_port, and while metrics are enabled the metrics service on
+    metrics.https_port, all for wan.local_fqdn with a certificate from Let's Encrypt, and all
+    calling the service on its loopback address. It also serves the CA files of
+    vpnrouter_pki.PUBLISH_DIR at /pki/ on the endpoints of pki.publish that are Caddy's, and an
+    ACME server at /acme/<name>/ on https.web_port for each issuing CA with acme and a
+    certificate, whose certificate and key are copied to CADDY_PKI_DIR.
+
+    Caddy is installed by CADDY_INSTALL_UNIT, outside setup, when it is missing or when
+    https.caddy_source is upstream and Caddy's repository is not configured; the unit restarts
+    setup, whose next run serves with it.
+    """
+    if not vpnrouter.https_ready(ctx):
+        vpnrouter.remove(CADDYFILE)
+        if unit_is_active(CADDY_UNIT):
+            run(['systemctl', 'disable', '--now', CADDY_UNIT], check=False)
+        return
+
+    source = cfg['https']['caddy_source']
+    if shutil.which('caddy') is None or (source == 'upstream' and not vpnrouter.CADDY_APT_SOURCE.is_file()):
+        run(['systemctl', 'start', '--no-block', CADDY_INSTALL_UNIT], check=False)
+        warn(f'Caddy is installed from the {source} source by {CADDY_INSTALL_UNIT}, which runs setup again')
+        return
+    if source == 'ubuntu' and vpnrouter.CADDY_APT_SOURCE.is_file():
+        warn("https.caddy_source ubuntu is ignored: Caddy's repository is configured, and Caddy stays upstream's")
+
+    try:
+        group = grp.getgrnam(CADDY_GROUP).gr_gid
+    except KeyError:
+        fail(f'the group {CADDY_GROUP} does not exist; is the caddy package installed?')
+    try:
+        credentials = {name: vpnrouter_pki.authority_files(name) for name in ctx['pki_acme']}
+    except ldap.LDAPError as exc:
+        warn(f'the ACME servers are left out: the PKI cannot be read: {exc}')
+        credentials = {}
+    acme = [name for name, files in credentials.items() if files is not None]
+    copied = False
+    for folder in (CADDY_PKI_DIR.iterdir() if CADDY_PKI_DIR.is_dir() else []):
+        if folder.name not in acme:
+            shutil.rmtree(folder)
+    for name in acme:
+        folder = CADDY_PKI_DIR / name
+        for path in (CADDY_PKI_DIR, folder):
+            path.mkdir(mode=0o750, exist_ok=True)
+            os.chown(path, 0, group)
+        for file, data in zip(('cert.pem', 'key.pem'), credentials[name]):
+            copied |= vpnrouter_pki.write(folder / file, data, 0o640)
+            os.chown(folder / file, 0, group)
+    authorities = vpnrouter.issuing_cas(cfg)
+    caddy_ctx = dict(ctx, manager_port=manager_port() if MANAGER.exists() else 0,
+                     pki_publish_dir=str(vpnrouter_pki.PUBLISH_DIR),
+                     acme_cas=[{'id': name, 'name': authorities[name]['name'].strip(),
+                                'cert': str(CADDY_PKI_DIR / name / 'cert.pem'),
+                                'key': str(CADDY_PKI_DIR / name / 'key.pem')} for name in acme])
+    changed = vpnrouter.render(vpnrouter._jinja_env(), caddy_ctx, 'Caddyfile.j2', CADDYFILE, 0o640)
+    os.chown(CADDYFILE, 0, group)
+    if not unit_is_active(CADDY_UNIT):
+        run(['systemctl', 'enable', '--now', CADDY_UNIT], check=False)
+    elif changed or copied:
+        run(['systemctl', 'reload', CADDY_UNIT], check=False)
+
+
+def manage_metrics(ctx):
+    """Run the metrics service while metrics are enabled, and stop it when not.
+
+    The service reads its settings itself, so it is restarted only to run the code of a new package version, which
+    every run of setup after an upgrade does.
+    """
+    unit = 'vpn-router-metrics.service'
+    if not vpnrouter.metrics_ready(ctx):
+        run(['systemctl', 'disable', '--now', unit], check=False)
+        return
+    run(['systemctl', 'enable', unit], check=False)
+    run(['systemctl', 'restart', unit], check=False)
+
+
+def manage_backup(cfg):
+    """Run the backup timer on backup.schedule while backups are enabled, and stop it when not.
+
+    The schedule is a drop-in of the timer, BACKUP_SCHEDULE, which replaces the OnCalendar of the shipped unit.
+    """
+    unit = 'vpn-router-backup.timer'
+    if not cfg['backup']['enabled']:
+        run(['systemctl', 'disable', '--now', unit], check=False)
+        if BACKUP_SCHEDULE.exists():
+            BACKUP_SCHEDULE.unlink()
+            run(['systemctl', 'daemon-reload'], check=False)
+        return
+    text = f'[Timer]\nOnCalendar=\nOnCalendar={cfg["backup"]["schedule"]}\n'
+    if not BACKUP_SCHEDULE.exists() or BACKUP_SCHEDULE.read_text(encoding='utf-8') != text:
+        BACKUP_SCHEDULE.parent.mkdir(parents=True, exist_ok=True)
+        BACKUP_SCHEDULE.write_text(text, encoding='utf-8')
+        run(['systemctl', 'daemon-reload'], check=False)
+        run(['systemctl', 'restart', unit], check=False)
+    run(['systemctl', 'enable', '--now', unit], check=False)
+
+
 def run_platform_setup(cfg, ctx):
     """Let the platform module apply its own state, if it defines any."""
     module = vpnrouter.platform_module(cfg)
@@ -408,31 +675,37 @@ def refresh_web(cfg, ctx):
     the configuration without its secret options and the facts resolved from
     the system, which it cannot work out with the standard library alone. The
     file is readable by root only. The service follows a changed address or
-    port itself.
+    port itself. Secret options are left out at every level, so also from each
+    site-to-site connection.
     """
-    public = {
-        section: {option: value for option, value in options.items() if not SECRET_OPTION.search(option)}
-        for section, options in cfg.items()
-    }
+    def public(options):
+        return {option: public(value) if isinstance(value, dict) else value
+                for option, value in options.items()
+                if isinstance(value, dict) or not SECRET_OPTION.search(option)}
+
     WEB_STATE.parent.mkdir(parents=True, exist_ok=True)
     WEB_STATE.touch(mode=0o600)
     WEB_STATE.chmod(0o600)
     WEB_STATE.write_text(json.dumps({
-        'site_to_site': bool(vpnrouter.s2s_ready(ctx)),
+        'site_to_site': [connection['name'] for connection in ctx['connections'] if connection['ready']],
         'p2s': bool(vpnrouter.p2s_ready(ctx)),
         'wireguard': bool(vpnrouter.wg_ready(ctx)),
         'openvpn': bool(vpnrouter.openvpn_ready(ctx)),
-        'configuration': public,
+        'configuration': public(cfg),
         'network': {
             'external_interface': ctx['wan_iface'],
             'internal_interface': ctx['int_iface'],
             'external_addresses': ctx['local_addr_list'],
             'internal_address': ctx['int_addr'],
             'internal_gateway': ctx['int_gateway_ip'],
-            'local_identity': ctx['local_id'],
-            'remote_identity': ctx['remote_id'],
             'local_cidrs': ctx['local_cidr_list'],
-            'remote_cidrs': ctx['remote_cidr_list'],
+            'site_to_site': [{
+                'name': connection['name'],
+                'local_identity': connection['local_id'],
+                'remote_identity': connection['remote_id'],
+                'local_cidrs': connection['local_cidr_list'],
+                'remote_cidrs': connection['remote_cidr_list'],
+            } for connection in ctx['connections']],
         },
     }) + '\n', encoding='utf-8')
 
@@ -448,25 +721,50 @@ def progress(state, step, error=None):
     }) + '\n', encoding='utf-8')
 
 
+def start_bootstrap():
+    """Start the bootstrap services while no run has succeeded; a unit that is not installed is skipped."""
+    if not SETUP_DONE.exists():
+        for unit in BOOTSTRAP_UNITS:
+            run(['systemctl', 'start', '--no-block', unit], check=False)
+
+
+def end_bootstrap(ctx):
+    """After the first successful run: write SETUP_DONE, close the bootstrap ports and stop the bootstrap services."""
+    if SETUP_DONE.exists():
+        return
+    SETUP_DONE.parent.mkdir(parents=True, exist_ok=True)
+    SETUP_DONE.touch()
+    if setup_ufw(ctx):
+        run(['ufw', 'reload'], check=False)
+    for unit in BOOTSTRAP_UNITS:
+        run(['systemctl', 'stop', unit], check=False)
+
+
 def main():
+    start_bootstrap()
     progress('running', 'configuration')
     cfg = vpnrouter.load_config()
-    ctx, changed = vpnrouter.configure(cfg)
-
-    if ctx is None:
+    if vpnrouter.deferred(cfg):
         print('vpn-router: platform is none, configuration deferred; '
               f'set general.platform in {vpnrouter.CONFIG_FILE}')
         progress('deferred', 'configuration', f'general.platform is none; set it in {vpnrouter.CONFIG_FILE}')
         return 0
 
+    # The directory holds the PKI, which the configuration needs; its problem ends setup only after every other step.
+    failed_step = 'user directory'
+    progress('running', failed_step)
+    problem = manage_directory()
+    progress('running', 'configuration')
+    ctx, changed = vpnrouter.configure(cfg)
+
     progress('running', 'routes')
     if vpnrouter.routing_ready(ctx):
         apply_routes(ctx)
 
-    if ctx['local_id_mode'] == 'public_ip' and ctx['local_fqdn']:
+    if ctx['local_fqdn'] and any(connection['local_id_mode'] == 'public_ip' for connection in ctx['connections']):
         progress('running', 'public address')
         address = resolve_public_ip(ctx['local_fqdn'])
-        ctx, more = vpnrouter.configure(cfg, local_id_override=address)
+        ctx, more = vpnrouter.configure(cfg, public_ip=address)
         changed |= more
 
     progress('running', 'WireGuard keys')
@@ -490,11 +788,20 @@ def main():
     progress('running', 'WireGuard')
     manage_wireguard(ctx, changed)
 
-    # The tunnels are up before the directory, whose problem ends setup only after every other step.
-    progress('running', 'user directory')
-    problem = manage_directory()
+    if not problem:
+        failed_step = 'Router Manager admin'
+        progress('running', failed_step)
+        problem = manage_admin()
+    if not problem:
+        failed_step = 'metrics service account'
+        progress('running', failed_step)
+        problem = manage_metrics_account(ctx)
     progress('running', 'OpenVPN')
     manage_openvpn(ctx, changed)
+    progress('running', 'HTTPS proxy')
+    manage_caddy(cfg, ctx)
+    progress('running', 'PKI timer')
+    run(['systemctl', 'enable', '--now', PKI_TIMER], check=False)
 
     if 'resolved' in changed:
         progress('running', 'resolver')
@@ -506,11 +813,16 @@ def main():
     refresh_docker_rules()
     progress('running', 'web service state')
     refresh_web(cfg, ctx)
+    progress('running', 'metrics service')
+    manage_metrics(ctx)
+    progress('running', 'backup timer')
+    manage_backup(cfg)
     if problem:
-        progress('failed', 'user directory', problem)
+        progress('failed', failed_step, problem)
         print(f'vpn-router: error: {problem}', file=sys.stderr)
         return 1
     progress('done', 'done')
+    end_bootstrap(ctx)
     return 0
 
 

debian-package/src/usr/lib/vpn-router/simple-ca

@@ -1,611 +0,0 @@
-#!/usr/bin/env python3
-# MIT License
-#
-# Copyright (c) 2026 Sławomir Koszewski
-#
-# Permission is hereby granted, free of charge, to any person obtaining a copy
-# of this software and associated documentation files (the "Software"), to deal
-# in the Software without restriction, including without limitation the rights
-# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-# copies of the Software, and to permit persons to whom the Software is
-# furnished to do so, subject to the following conditions:
-#
-# The above copyright notice and this permission notice shall be included in all
-# copies or substantial portions of the Software.
-#
-# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-# SOFTWARE.
-
-# This module requires Python 3.8+ and OpenSSL to be installed on the system.
-
-import argparse
-import datetime
-import json
-import os
-import re
-import subprocess
-import sys
-import tempfile
-
-OPENSSL = "openssl"
-
-
-class Config:
-    FILE = "simple-ca.json"
-
-    def __init__(self, ca_dir: str):
-        self._path = os.path.join(ca_dir, self.FILE)
-        self._data: dict = {}
-        if not os.path.isfile(self._path):
-            return
-        try:
-            with open(self._path, "r") as f:
-                self._data = json.load(f)
-        except (json.JSONDecodeError, OSError) as e:
-            print(f"WARNING: could not read {self._path}: {e}", file=sys.stderr)
-
-    def get(self, key, default=None):
-        return self._data.get(key, default)
-
-    def update(self, patch: dict):
-        file_missing = not os.path.isfile(self._path)
-        changed = any(self._data.get(k) != v for k, v in patch.items())
-        if file_missing or changed:
-            self._data.update(patch)
-            self._save()
-
-    def append_history(self, ca_key: str, entry: dict):
-        history = self._data.get("history", {})
-        history.setdefault(ca_key, []).append(entry)
-        self._data["history"] = history
-        self._save()
-
-    def revoke_in_history(self, ca_key: str, serial: str, revoked_at: str):
-        """Mark a certificate as revoked. Returns True if newly revoked,
-        False if already revoked, None if serial not found."""
-        history = self._data.get("history", {})
-        for entry in history.get(ca_key, []):
-            if entry.get("serial") == serial:
-                if "revoked" in entry:
-                    return False
-                entry["revoked"] = revoked_at
-                self._data["history"] = history
-                self._save()
-                return True
-        return None
-
-    def _save(self):
-        with open(self._path, "w") as f:
-            json.dump(self._data, f, indent=2)
-            f.write("\n")
-
-
-_config: Config
-
-
-def _now() -> str:
-    return datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
-
-
-def _err(msg):
-    print(f"ERROR: {msg}", file=sys.stderr)
-
-
-def _rebuild_ca_bundle(ca_dir):
-    bundle_path = os.path.join(ca_dir, "ca_bundle.pem")
-    parts = []
-    root = os.path.join(ca_dir, "ca_cert.pem")
-    if os.path.isfile(root):
-        with open(root, "rb") as f:
-            parts.append(f.read())
-    for name in sorted(_config.get("subordinates", [])):
-        sub_cert = os.path.join(ca_dir, name, "ca_cert.pem")
-        if os.path.isfile(sub_cert):
-            with open(sub_cert, "rb") as f:
-                parts.append(f.read())
-    with open(bundle_path, "wb") as f:
-        f.write(b"".join(parts))
-
-
-def _read_serial(cert_path) -> str:
-    result = subprocess.run(
-        [OPENSSL, "x509", "-in", cert_path, "-noout", "-serial"],
-        capture_output=True, text=True,
-    )
-    return result.stdout.strip().split("=", 1)[-1]
-
-
-def _read_expiry(cert_path) -> str:
-    """Return the certificate notAfter date as an ISO 8601 UTC string."""
-    result = subprocess.run(
-        [OPENSSL, "x509", "-in", cert_path, "-noout", "-enddate"],
-        capture_output=True, text=True,
-    )
-    raw = result.stdout.strip().split("=", 1)[-1]
-    raw = " ".join(raw.split())  # normalize whitespace ("May  4" -> "May 4")
-    dt = datetime.datetime.strptime(raw, "%b %d %H:%M:%S %Y GMT")
-    return dt.strftime("%Y-%m-%dT%H:%M:%SZ")
-
-
-def _iso_to_asn1(iso: str) -> str:
-    """Convert "2026-05-24T14:28:10Z" -> "260524142810Z" for OpenSSL index.txt."""
-    dt = datetime.datetime.strptime(iso, "%Y-%m-%dT%H:%M:%SZ")
-    return dt.strftime("%y%m%d%H%M%SZ")
-
-
-_IP_RE = re.compile(r"^[0-9]{1,3}(\.[0-9]{1,3}){3}$")
-_DNS_RE = re.compile(r"^[a-z0-9-]+(\.[a-z0-9-]+)*$")
-
-
-def _is_ip(value):
-    return bool(_IP_RE.match(value))
-
-
-def _is_dns(value):
-    return bool(_DNS_RE.match(value))
-
-
-def _pipe(cmd1, cmd2):
-    p1 = subprocess.Popen(cmd1, stdout=subprocess.PIPE)
-    p2 = subprocess.Popen(cmd2, stdin=p1.stdout)
-    p1.stdout.close()
-    p2.communicate()
-    p1.wait()
-    return p1.returncode == 0 and p2.returncode == 0
-
-
-def make_ca(ca_dir, ca_name, days=3650, issuing_ca=None, ca_publish_base_url=None):
-    if issuing_ca == "ca":
-        _err("--issuing-ca cannot be 'ca' as it is reserved for the root CA.")
-        return False
-
-    if not ca_dir or not os.path.isdir(ca_dir):
-        _err(f"Certificate directory {ca_dir} does not exist.")
-        return False
-
-    if not ca_name:
-        _err("CA name is required.")
-        return False
-
-    root_ca_cert_path = os.path.join(ca_dir, "ca_cert.pem")
-    root_ca_key_path  = os.path.join(ca_dir, "ca_key.pem")
-
-    if not os.path.isfile(root_ca_cert_path) or not os.path.isfile(root_ca_key_path):
-        if issuing_ca:
-            _err(
-                f"Cannot create issuing CA '{ca_name}' without existing root CA "
-                "certificate and key. Please create the root CA first."
-            )
-            return False
-
-        print(f"Generating CA certificate '{ca_name}' and key...")
-        # Path length constraint of 1: allows one level of issuing CAs.
-        cmd = [
-            OPENSSL, "req",
-            "-x509",
-            "-newkey", "rsa:4096",
-            "-keyout", root_ca_key_path,
-            "-out", root_ca_cert_path,
-            "-days", str(days),
-            "-noenc",
-            "-subj", f"/CN={ca_name}",
-            "-text",
-            "-addext", "basicConstraints=critical,CA:TRUE,pathlen:1",
-            "-addext", "keyUsage=critical,keyCertSign,cRLSign",
-        ]
-        if subprocess.run(cmd).returncode != 0:
-            _err("Failed to generate CA certificate and key.")
-            return False
-
-        _rebuild_ca_bundle(ca_dir)
-        patch = {"name": ca_name, "created": _now()}
-        if ca_publish_base_url:
-            patch["ca_publish_base_url"] = ca_publish_base_url
-        _config.update(patch)
-        return True
-
-    issuing_ca_dir  = os.path.join(ca_dir, issuing_ca)
-    issuing_ca_cert = os.path.join(issuing_ca_dir, "ca_cert.pem")
-    issuing_ca_key  = os.path.join(issuing_ca_dir, "ca_key.pem")
-
-    if not os.path.isfile(issuing_ca_cert) or not os.path.isfile(issuing_ca_key):
-        print(f"Generating issuing CA certificate '{ca_name}' and key...")
-        os.makedirs(issuing_ca_dir, exist_ok=True)
-        req_cmd = [
-            OPENSSL, "req",
-            "-newkey", "rsa:4096",
-            "-keyout", issuing_ca_key,
-            "-noenc",
-            "-subj", f"/CN={ca_name}",
-            "-addext", "basicConstraints=critical,CA:TRUE,pathlen:0",
-            "-addext", "keyUsage=critical,keyCertSign,cRLSign",
-        ]
-        if ca_publish_base_url:
-            req_cmd += [
-                "-addext", f"authorityInfoAccess=caIssuers;URI:{ca_publish_base_url}/ca_cert.crt",
-                "-addext", f"crlDistributionPoints=URI:{ca_publish_base_url}/crl.pem",
-            ]
-        x509_cmd = [
-            OPENSSL, "x509",
-            "-req",
-            "-CA", root_ca_cert_path,
-            "-CAkey", root_ca_key_path,
-            "-copy_extensions", "copyall",
-            "-days", str(days),
-            "-text",
-            "-out", issuing_ca_cert,
-        ]
-        if not _pipe(req_cmd, x509_cmd):
-            _err("Failed to generate issuing CA certificate and key.")
-            return False
-
-    patch = {"ca_publish_base_url": ca_publish_base_url} if ca_publish_base_url else {}
-    subs = _config.get("subordinates", [])
-    if issuing_ca not in subs:
-        patch["subordinates"] = subs + [issuing_ca]
-    _config.update(patch)
-    _rebuild_ca_bundle(ca_dir)
-    return True
-
-
-def make_cert(cert_subject_name, sans=None, ca_dir=None, cert_dir=None,
-              issuing_ca=None, days=365, ca_publish_base_url=None):
-    if issuing_ca == "ca":
-        _err("--issuing-ca cannot be 'ca' as it is reserved for the root CA.")
-        return False
-
-    if not ca_dir or not os.path.isdir(ca_dir):
-        _err(f"CA directory {ca_dir} does not exist.")
-        return False
-
-    if not cert_subject_name:
-        _err("Subject name is required.")
-        return False
-
-    if not _is_dns(cert_subject_name):
-        _err(f"Invalid subject name '{cert_subject_name}'. Must be a valid DNS name.")
-        return False
-
-    signing_dir  = os.path.join(ca_dir, issuing_ca) if issuing_ca else ca_dir
-    cert_dir     = cert_dir or signing_dir
-
-    if not os.path.isdir(cert_dir):
-        _err(f"Certificate directory {cert_dir} does not exist.")
-        return False
-    ca_cert_path = os.path.join(signing_dir, "ca_cert.pem")
-    ca_key_path  = os.path.join(signing_dir, "ca_key.pem")
-    if not os.path.isfile(ca_cert_path) or not os.path.isfile(ca_key_path):
-        _err(
-            f"Signing CA certificate and key not found in {signing_dir}. "
-            "Please set up a signing CA first."
-        )
-        return False
-
-    aia_url = cdp_url = ""
-    if ca_publish_base_url:
-        if issuing_ca:
-            aia_url = f"{ca_publish_base_url}/{issuing_ca}/ca_cert.crt"
-            cdp_url = f"{ca_publish_base_url}/{issuing_ca}/crl.pem"
-        else:
-            aia_url = f"{ca_publish_base_url}/ca_cert.crt"
-            cdp_url = f"{ca_publish_base_url}/crl.pem"
-
-    cert_name = cert_subject_name.split(".", 1)[0]
-
-    san_entries = [f"DNS:{cert_subject_name}"]
-    for entry in sans or []:
-        if _is_ip(entry):
-            san_entries.append(f"IP:{entry}")
-        elif _is_dns(entry):
-            san_entries.append(f"DNS:{entry}")
-        else:
-            _err(f"Invalid SAN entry '{entry}'")
-            return False
-
-    sans_ext = "subjectAltName=" + ",".join(san_entries)
-
-    print(f"Generating server certificate for '{cert_subject_name}' with SANs:")
-    for san in san_entries:
-        print(f"  - {san}")
-
-    cert_out = os.path.join(cert_dir, f"{cert_name}_cert.pem")
-    key_out  = os.path.join(cert_dir, f"{cert_name}_key.pem")
-
-    if not os.path.isfile(cert_out) or not os.path.isfile(key_out):
-        print("Generating server certificate and key...")
-        req_cmd = [
-            OPENSSL, "req",
-            "-newkey", "rsa:4096",
-            "-keyout", key_out,
-            "-noenc",
-            "-subj", f"/CN={cert_subject_name}",
-            "-addext", "basicConstraints=critical,CA:FALSE",
-            "-addext", "keyUsage=critical,digitalSignature,keyEncipherment",
-            "-addext", "extendedKeyUsage=serverAuth,clientAuth",
-            "-addext", sans_ext,
-        ]
-        if aia_url:
-            req_cmd += ["-addext", f"authorityInfoAccess=caIssuers;URI:{aia_url}"]
-        if cdp_url:
-            req_cmd += ["-addext", f"crlDistributionPoints=URI:{cdp_url}"]
-        x509_cmd = [
-            OPENSSL, "x509",
-            "-req",
-            "-CA", ca_cert_path,
-            "-CAkey", ca_key_path,
-            "-copy_extensions", "copyall",
-            "-days", str(days),
-            "-text",
-            "-out", cert_out,
-        ]
-        if not _pipe(req_cmd, x509_cmd):
-            _err("Failed to generate server certificate and key.")
-            return False
-
-        _config.append_history(issuing_ca or "ca", {
-            "name": cert_subject_name,
-            "serial": _read_serial(cert_out),
-            "created": _now(),
-            "expires": _read_expiry(cert_out),
-        })
-
-    return True
-
-
-def make_pfx(cert_path, ca_dir, issuing_ca=None, password=None, apple_openssl=False):
-    if issuing_ca == "ca":
-        _err("--issuing-ca cannot be 'ca' as it is reserved for the root CA.")
-        return False
-
-    cert_dir      = os.path.dirname(cert_path)
-    cert_basename = os.path.basename(cert_path)
-    cert_name     = cert_basename[:-len("_cert.pem")] if cert_basename.endswith("_cert.pem") else cert_basename
-    key_path      = os.path.join(cert_dir, f"{cert_name}_key.pem")
-
-    if not cert_dir or not os.path.isdir(cert_dir):
-        _err(f"Certificate directory {cert_dir} does not exist.")
-        return False
-
-    if not ca_dir or not os.path.isdir(ca_dir):
-        _err(f"CA directory {ca_dir} does not exist.")
-        return False
-
-    if not os.path.isfile(cert_path) or not os.path.isfile(key_path):
-        _err("Server certificate or key not found.")
-        return False
-
-    root_ca_cert_path = os.path.join(ca_dir, "ca_cert.pem")
-    root_ca_key_path  = os.path.join(ca_dir, "ca_key.pem")
-    if not os.path.isfile(root_ca_cert_path) or not os.path.isfile(root_ca_key_path):
-        _err(f"CA certificate or key not found in {ca_dir}.")
-        return False
-
-    if issuing_ca:
-        issuing_ca_cert_path = os.path.join(ca_dir, issuing_ca, "ca_cert.pem")
-        if not os.path.isfile(issuing_ca_cert_path):
-            _err(f"Issuing CA certificate not found: {issuing_ca_cert_path}.")
-            return False
-
-    if not password:
-        password = "changeit"
-
-    pfx_path = os.path.join(cert_dir, f"{cert_name}.pfx")
-    if os.path.isfile(pfx_path):
-        print("PKCS#12 (PFX) file already exists, aborting generation.")
-        return False
-
-    print("Generating PKCS#12 (PFX) file...", end="")
-
-    chain_bytes = b""
-    with open(root_ca_cert_path, "rb") as f:
-        chain_bytes += f.read()
-    if issuing_ca:
-        with open(issuing_ca_cert_path, "rb") as f:
-            chain_bytes += f.read()
-
-    chain_fd, chain_file = tempfile.mkstemp()
-    try:
-        with os.fdopen(chain_fd, "wb") as f:
-            f.write(chain_bytes)
-        cmd = [
-            "/usr/bin/openssl" if apple_openssl else OPENSSL, "pkcs12",
-            "-export", "-out", pfx_path,
-            "-inkey", key_path,
-            "-in", cert_path,
-            "-certfile", chain_file,
-            "-password", f"pass:{password}",
-        ]
-        if subprocess.run(cmd).returncode != 0:
-            _err("Failed to generate PKCS#12 (PFX) file.")
-            return False
-    finally:
-        if os.path.exists(chain_file):
-            os.remove(chain_file)
-
-    print("done.")
-    return True
-
-
-def make_crl(ca_dir, issuing_ca=None, days=30):
-    signing_dir = os.path.join(ca_dir, issuing_ca) if issuing_ca else ca_dir
-    ca_cert     = os.path.join(signing_dir, "ca_cert.pem")
-    ca_key      = os.path.join(signing_dir, "ca_key.pem")
-
-    if not os.path.isfile(ca_cert) or not os.path.isfile(ca_key):
-        _err(f"CA certificate or key not found in {signing_dir}.")
-        return False
-
-    crl_path = os.path.join(signing_dir, "crl.pem")
-
-    history_key = issuing_ca or "ca"
-    revoked_entries = [
-        e for e in _config.get("history", {}).get(history_key, [])
-        if "revoked" in e and "expires" in e
-    ]
-
-    with tempfile.TemporaryDirectory() as tmp:
-        index_txt = os.path.join(tmp, "index.txt")
-        crlnumber = os.path.join(tmp, "crlnumber")
-        cnf_path  = os.path.join(tmp, "openssl.cnf")
-
-        with open(index_txt, "w") as f:
-            for e in revoked_entries:
-                expires_asn1 = _iso_to_asn1(e["expires"])
-                revoked_asn1 = _iso_to_asn1(e["revoked"])
-                f.write(f"R\t{expires_asn1}\t{revoked_asn1}\t{e['serial']}\tunknown\t/CN={e['name']}\n")
-        with open(crlnumber, "w") as f:
-            f.write("01\n")
-        with open(cnf_path, "w") as f:
-            f.write(
-                "[ ca ]\ndefault_ca = CA_default\n\n"
-                "[ CA_default ]\n"
-                f"database         = {index_txt}\n"
-                f"crlnumber        = {crlnumber}\n"
-                f"certificate      = {ca_cert}\n"
-                f"private_key      = {ca_key}\n"
-                f"default_crl_days = {days}\n"
-                "default_md       = sha256\n\n"
-                "[ crl_ext ]\n"
-                "authorityKeyIdentifier = keyid:always\n"
-            )
-
-        if subprocess.run([OPENSSL, "ca", "-gencrl", "-config", cnf_path, "-out", crl_path]).returncode != 0:
-            _err("Failed to generate CRL.")
-            return False
-
-    print(f"CRL written to {crl_path}")
-    return True
-
-
-def revoke_cert(cert_path, ca_dir, issuing_ca=None):
-    if not os.path.isfile(cert_path):
-        _err(f"Certificate not found: {cert_path}")
-        return False
-
-    signing_dir = os.path.join(ca_dir, issuing_ca) if issuing_ca else ca_dir
-    if not os.path.isdir(signing_dir):
-        _err(f"CA directory not found: {signing_dir}")
-        return False
-
-    ca_key = issuing_ca or "ca"
-    serial = _read_serial(cert_path)
-    result = _config.revoke_in_history(ca_key, serial, _now())
-
-    if result is None:
-        _err(f"Certificate with serial {serial} not found in history for CA '{ca_key}'.")
-        return False
-    if result is False:
-        print(f"Certificate {cert_path} (serial {serial}) is already revoked.")
-        return True
-
-    print(f"Certificate {cert_path} (serial {serial}) marked as revoked.")
-    return True
-
-
-def _build_parser():
-    parser = argparse.ArgumentParser(
-        description="Simple CA for creating and managing test certificates."
-    )
-    sub = parser.add_subparsers(dest="command", required=True)
-
-    p_ca = sub.add_parser("make-ca", help="Create a root or issuing CA.")
-    p_ca.add_argument("--days", type=int, default=None, help="Validity period in days (default: 3650)")
-    p_ca.add_argument("--issuing-ca", default=None, help="Specify the issuing CA")
-    p_ca.add_argument("--ca-publish-base-url", default=None,
-                      help="Base URL for AIA and CRL distribution point extensions")
-    p_ca.add_argument("--ca-dir", help="Directory to store the CA files")
-    p_ca.add_argument("--openssl", default=None, metavar="PATH",
-                      help=f"Path to the openssl binary (default: {OPENSSL})")
-    p_ca.add_argument("ca_name", help="Name of the CA")
-
-    p_cert = sub.add_parser("make-cert", help="Create a server/client certificate.")
-    p_cert.add_argument("--ca-dir", help="Directory of the CA")
-    p_cert.add_argument("--issuing-ca", default=None, help="Specify the issuing CA")
-    p_cert.add_argument("--days", type=int, default=None, help="Validity period in days (default: 365)")
-    p_cert.add_argument("--cert-dir", help="Directory to store the certificate files")
-    p_cert.add_argument("--openssl", default=None, metavar="PATH",
-                        help=f"Path to the openssl binary (default: {OPENSSL})")
-    p_cert.add_argument("subject_name", help="Subject name for the certificate")
-    p_cert.add_argument("sans", nargs="*", help="Subject Alternative Names (SANs) for the certificate")
-
-    p_pfx = sub.add_parser("make-pfx", help="Create a PKCS#12 (PFX) bundle.")
-    p_pfx.add_argument("--issuing-ca", default=None, help="Specify the issuing CA")
-    p_pfx.add_argument("--ca-dir", help="Directory of the CA")
-    p_pfx.add_argument("--password", help="Password for the PFX file")
-    p_pfx.add_argument("--apple-openssl", action="store_true", default=False,
-                       help="Use Apple's bundled /usr/bin/openssl for PKCS12 generation")
-    p_pfx.add_argument("path", help="Path to the certificate file")
-
-    p_crl = sub.add_parser("make-crl", help="Generate a CRL for a CA.")
-    p_crl.add_argument("--ca-dir", help="Directory of the CA")
-    p_crl.add_argument("--issuing-ca", default=None, help="Generate CRL for this issuing CA")
-    p_crl.add_argument("--days", type=int, default=None, help="CRL validity in days (default: 30)")
-
-    p_rev = sub.add_parser("revoke-cert", help="Revoke a certificate.")
-    p_rev.add_argument("--ca-dir", help="Directory of the CA")
-    p_rev.add_argument("--issuing-ca", default=None, help="Issuing CA that signed the certificate")
-    p_rev.add_argument("cert_path", help="Path to the certificate file to revoke")
-
-    return parser
-
-
-def main(argv=None):
-    global OPENSSL, _config
-
-    parser = _build_parser()
-    args = parser.parse_args(argv)
-    ca_dir = args.ca_dir or os.environ.get("SIMPLE_CA_DIR") or os.getcwd()
-
-    _config = Config(ca_dir)
-
-    OPENSSL             = getattr(args, "openssl", None) or _config.get("openssl", OPENSSL)
-    issuing_ca          = args.issuing_ca or _config.get("issuing_ca")
-    ca_publish_base_url = getattr(args, "ca_publish_base_url", None) or _config.get("ca_publish_base_url")
-
-    days_cfg = _config.get("days", {})
-
-    if args.command == "make-ca":
-        days = args.days or days_cfg.get("ca", 3650)
-        ok = make_ca(
-            ca_dir, args.ca_name,
-            days=days,
-            issuing_ca=issuing_ca,
-            ca_publish_base_url=ca_publish_base_url,
-        )
-    elif args.command == "make-cert":
-        days = args.days or days_cfg.get("cert", 365)
-        ok = make_cert(
-            args.subject_name,
-            sans=args.sans,
-            ca_dir=ca_dir,
-            cert_dir=getattr(args, "cert_dir", None),
-            issuing_ca=issuing_ca,
-            days=days,
-            ca_publish_base_url=ca_publish_base_url,
-        )
-    elif args.command == "make-pfx":
-        ok = make_pfx(
-            args.path, ca_dir,
-            issuing_ca=issuing_ca,
-            password=args.password,
-            apple_openssl=args.apple_openssl,
-        )
-    elif args.command == "make-crl":
-        days = args.days or days_cfg.get("crl", 30)
-        ok = make_crl(ca_dir, issuing_ca=issuing_ca, days=days)
-    elif args.command == "revoke-cert":
-        ok = revoke_cert(args.cert_path, ca_dir, issuing_ca=issuing_ca)
-    else:
-        parser.error(f"Unknown command: {args.command}")
-        ok = False
-
-    return 0 if ok else 1
-
-
-if __name__ == "__main__":
-    sys.exit(main())

debian-package/src/usr/lib/vpn-router/vpnrouter.py

@@ -4,60 +4,182 @@ Everything derivable from the configuration file alone, with no running
 network required. Used by both 'configure' and 'setup'.
 """
 
+import contextlib
+import fcntl
 import ipaddress
 import json
 import os
 import pathlib
 import re
 import shutil
-import subprocess
 import sys
+import tempfile
+import urllib.parse
 
 import jinja2
+import ldap
 
 LIB_DIR = pathlib.Path(__file__).resolve().parent
 if str(LIB_DIR) not in sys.path:
     sys.path.insert(0, str(LIB_DIR))
 
 import vpnrouter_ldap
+import vpnrouter_pki
 import vpnrouter_platforms
-from vpnrouter_net import ConfigError, detect_interfaces, interface_addresses, ip_output
+from vpnrouter_net import DEFAULT_LISTEN, LISTEN_CHOICES, LOOPBACK, ConfigError, detect_interfaces, interface_addresses, ip_output
 
 CONFIG_FILE  = pathlib.Path('/etc/vpn-router/vpn-router.json')
+CONFIG_LOCK  = pathlib.Path('/run/vpn-router/config.lock')
 TEMPLATE_DIR = pathlib.Path('/usr/share/vpn-router/templates')
-PKI_DIR      = pathlib.Path('/etc/vpn-router/pki')
-SIMPLE_CA    = LIB_DIR / 'simple-ca'
 
 SWANCTL_CONF_D = pathlib.Path('/etc/swanctl/conf.d')
-REMOTE_SITE    = SWANCTL_CONF_D / 'remote-site.conf'
+SITE_TO_SITE   = SWANCTL_CONF_D / 'site-to-site.conf'
 ROAD_WARRIOR   = SWANCTL_CONF_D / 'road-warrior.conf'
 P2S_FORWARDER  = pathlib.Path('/etc/systemd/resolved.conf.d/p2s-forwarder.conf')
 WG_CONF        = pathlib.Path('/etc/wireguard/wg0.conf')
 OPENVPN_CONF   = pathlib.Path('/etc/openvpn/server/vpn-router.conf')
 OPENVPN_LDAP   = pathlib.Path('/etc/openvpn/server/vpn-router-ldap.conf')
-USER_CERT_DIR  = PKI_DIR / 'users'
-
-CA_CERT_DEST   = pathlib.Path('/etc/swanctl/x509ca/ca.pem')
-SERVER_CERT    = pathlib.Path('/etc/swanctl/x509/server.pem')
-SERVER_KEY     = pathlib.Path('/etc/swanctl/private/server.key')
-CRL_DEST       = pathlib.Path('/etc/swanctl/x509crl/crl.pem')
-TRUST_ANCHOR   = pathlib.Path('/usr/local/share/ca-certificates/vpn-router-ca.crt')
 
 # section -> option -> default; the type of the default is the required type.
 SCHEMA = {
     'general':    {'platform': 'none', 'mode': 'manual'},
     'interfaces': {'external': '', 'internal': ''},
-    'wan':        {'local_fqdn': '', 'local_id_mode': 'fqdn'},
+    'wan':        {'local_fqdn': ''},
     'local':      {'cidrs': [], 'int_addr': '', 'int_gateway_ip': ''},
-    'remote':     {'addrs': [], 'id': '', 'cidrs': [], 'psk': '', 'psk_file': ''},
-    'p2s':        {'enabled': False, 'address_pool': '', 'ca_name': 'VPN Router CA'},
+    'site_to_site': {},
+    'p2s':        {'enabled': False, 'address_pool': ''},
     'wireguard':  {'enabled': False, 'address': '', 'listen_port': 51820},
     'openvpn':    {'enabled': False, 'address_pool': '', 'port': 1194, 'protocol': 'udp'},
     'docker':     {'mode': 'auto', 'exposed_ports': []},
-    # web.jwt_secret is read and ignored; the tokens are signed with vpnrouter_auth.WEB_KEY.
-    'web':        {'enabled': True, 'port': 7081, 'jwt_secret': '', 'database': '', 'trusted_proxies': []},
+    'web':        {'enabled': True, 'listen': DEFAULT_LISTEN, 'port': 7081, 'database': '', 'trusted_proxies': [],
+                   'status_content': 'basic'},
+    'https':      {'enabled': True, 'email': '', 'web_port': 7444, 'manager_port': 7443, 'caddy_source': 'ubuntu'},
+    'pki':        {'root_name': 'VPN Router Root CA', 'root_days': 7300, 'ca_days': 3650, 'key': 'rsa-4096',
+                   'crl_days': 7, 'root_crl_days': 180, 'publish': ['http'], 'p2s_name': 'VPN Router P2S CA',
+                   's2s_name': 'VPN Router Site-to-Site CA', 'issuing_cas': {}, 'templates': {},
+                   'root_key_warning': True},
+    'metrics':    {'enabled': True, 'listen': 'internal', 'port': 9464, 'https_port': 9465, 'service_account': 'metrics',
+                   'otlp_endpoint': '', 'otlp_username': '', 'otlp_password': '',
+                   'keep_details': '24 hours', 'keep_hourly': '1 month', 'keep_daily': '3 months',
+                   'keep_monthly': '1 year', 'keep_yearly': '10 years'},
+    'backup':     {'enabled': False, 'schedule': 'daily', 'target': 'local', 'path': '/var/backups/vpn-router',
+                   'keep_hourly': 24, 'keep_daily': 7, 'keep_weekly': 4, 'keep_monthly': 12, 'keep_yearly': 0,
+                   'metrics': False, 'rclone_remote': '', 'rclone_mode': 'sync',
+                   'pbs_repository': '', 'pbs_password': '', 'pbs_fingerprint': '', 'pbs_namespace': '',
+                   'pbs_keyfile': ''},
 }
 
+#: site_to_site maps a connection name to its settings: option -> default, the type of the default being the required type.
+CONNECTION = {'addrs': [], 'id': '', 'cidrs': [], 'local_cidrs': [], 'local_id_mode': 'fqdn', 'auth': 'psk', 'psk': '',
+              'psk_file': '', 'remote_ca': ''}
+
+#: How a site_to_site connection authenticates: a pre-shared key, or certificates.
+CONNECTION_AUTH = ('psk', 'certificate')
+
+#: The name of a site_to_site connection, which also names its swanctl connection, child and secret.
+CONNECTION_NAME = re.compile(r'^[a-z][a-z0-9-]{0,30}$')
+
+#: The site_to_site connection the install answers fill.
+FIRST_CONNECTION = 'remote-site'
+
+#: The retention settings of the metrics, from the details to the yearly summaries.
+METRICS_KEEP = ('keep_details', 'keep_hourly', 'keep_daily', 'keep_monthly', 'keep_yearly')
+
+#: A retention period as PostgreSQL reads an interval: a count and a unit.
+METRICS_INTERVAL = re.compile(r'^[1-9]\d{0,4} (hour|day|month|year)s?$')
+
+#: How often backups run, as systemd OnCalendar shorthands, from the most to the least often.
+BACKUP_SCHEDULES = ('hourly', 'daily', 'weekly')
+
+#: Where backups go: an archive in backup.path, or a Proxmox Backup Server.
+BACKUP_TARGETS = ('local', 'pbs')
+
+#: How rclone follows backup.path in the cloud: sync mirrors it, copy deletes nothing remotely.
+RCLONE_MODES = ('sync', 'copy')
+
+#: The retention settings of local backups, from the hourly to the yearly ones.
+BACKUP_KEEP = ('keep_hourly', 'keep_daily', 'keep_weekly', 'keep_monthly', 'keep_yearly')
+
+#: A remote of rclone.conf and an optional path: remote:path.
+RCLONE_REMOTE = re.compile(r'^[\w.][\w. -]*:\S*$')
+
+#: A PBS repository: [[auth-id@]server[:port]:]datastore.
+PBS_REPOSITORY = re.compile(r'^(\S+:)?[A-Za-z0-9_][A-Za-z0-9._-]*$')
+
+#: The SHA-256 fingerprint of a server certificate, as PBS shows it.
+PBS_FINGERPRINT = re.compile(r'^([0-9A-Fa-f]{2}:){31}[0-9A-Fa-f]{2}$')
+
+#: A PBS namespace: up to seven names joined by /.
+PBS_NAMESPACE = re.compile(r'^[A-Za-z0-9_][A-Za-z0-9._-]*(/[A-Za-z0-9_][A-Za-z0-9._-]*){0,6}$')
+
+#: The port of Let's Encrypt's HTTP challenge, which Caddy answers while HTTPS is on.
+ACME_HTTP_PORT = 80
+
+#: The default port of HTTPS, where Caddy publishes the CA files when pki.publish has https-default.
+HTTPS_DEFAULT_PORT = 443
+
+#: Where vpn-router-setup installs Caddy from, https.caddy_source: the Ubuntu archive or Caddy's own repository.
+CADDY_SOURCES = ('ubuntu', 'upstream')
+
+#: Caddy's own repository, which https.caddy_source upstream configures: the source, its signing key and their URLs.
+CADDY_APT_SOURCE = pathlib.Path('/etc/apt/sources.list.d/caddy-stable.sources')
+CADDY_KEYRING    = pathlib.Path('/etc/apt/keyrings/caddy-stable.asc')
+CADDY_KEY_URL    = 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key'
+CADDY_REPOSITORY = 'https://dl.cloudsmith.io/public/caddy/stable/deb/debian'
+
+#: The key types of the CAs, pki.key, and of the templates.
+PKI_KEYS = ('rsa-2048', 'rsa-3072', 'rsa-4096', 'ec-p256', 'ec-p384')
+
+#: Where the CA certificates and CRLs are published, pki.publish, as the URLs in the certificates list them.
+PKI_PUBLISH = ('http', 'http-80', 'https', 'https-default')
+
+#: The key usages a template may give, as the arguments of cryptography's KeyUsage.
+KEY_USAGES = ('digital_signature', 'content_commitment', 'key_encipherment', 'data_encipherment', 'key_agreement')
+
+#: The extended key usages a template may give.
+EXTENDED_KEY_USAGES = ('server_auth', 'client_auth', 'code_signing', 'email_protection', 'time_stamping',
+                       'ocsp_signing', 'ipsec_ike', 'ike_intermediate')
+
+#: What the common name of a template's certificates is: a DNS name, any name, or an e-mail address.
+TEMPLATE_SUBJECTS = ('dns', 'name', 'email')
+
+#: The subject alternative names a template's certificates may have.
+SAN_TYPES = ('dns', 'ip', 'email')
+
+#: A certificate template: option -> default, the type of the default being the required type.
+TEMPLATE = {'key': 'rsa-3072', 'days': 365, 'key_usage': ['digital_signature'], 'extended_key_usage': [],
+            'subject': 'name', 'sans': []}
+
+#: The built-in templates; pki.templates changes their options and adds templates of its own.
+PKI_TEMPLATES = {
+    'vpn-server':   {'key_usage': ['digital_signature', 'key_encipherment'],
+                     'extended_key_usage': ['server_auth', 'ike_intermediate'], 'subject': 'dns', 'sans': ['dns', 'ip']},
+    'vpn-user':     {'extended_key_usage': ['client_auth'], 'subject': 'name', 'sans': ['dns', 'email']},
+    'site-to-site': {'key_usage': ['digital_signature', 'key_encipherment'],
+                     'extended_key_usage': ['server_auth', 'client_auth'], 'subject': 'dns', 'sans': ['dns', 'ip']},
+    'server':       {'key_usage': ['digital_signature', 'key_encipherment'], 'extended_key_usage': ['server_auth'],
+                     'subject': 'dns', 'sans': ['dns', 'ip']},
+    'client':       {'extended_key_usage': ['client_auth'], 'subject': 'name', 'sans': ['dns', 'email']},
+}
+
+#: The templates only the router issues: its VPN server certificate and the users' certificates.
+ROUTER_TEMPLATES = ('vpn-server', 'vpn-user')
+
+#: The built-in issuing CAs: name -> the pki option of the common name and the templates the CA issues.
+BUILTIN_CAS = {'p2s': ('p2s_name', ['vpn-server', 'vpn-user']), 's2s': ('s2s_name', ['site-to-site'])}
+
+#: A general-purpose issuing CA of pki.issuing_cas: option -> default, the type of the default being the required type.
+ISSUING_CA = {'name': '', 'templates': [], 'acme': False}
+
+#: The name of an issuing CA or a template; an issuing CA's name also names its files.
+PKI_NAME = re.compile(r'^[a-z][a-z0-9-]{0,30}$')
+
+#: The longest common name of a CA, in characters.
+CA_NAME_MAX = 64
+
+#: What / and /status of the web service show, web.status_content, from the least to the most.
+STATUS_CONTENTS = ('basic', 'standard', 'full')
+
 DOCKER_MODES = ('auto', 'on', 'off')
 
 OPENVPN_PROTOCOLS = ('udp', 'tcp')
@@ -65,9 +187,6 @@ OPENVPN_PROTOCOLS = ('udp', 'tcp')
 #: The tun device of the OpenVPN server.
 OPENVPN_DEVICE = 'ovpn0'
 
-#: Days a certificate revocation list is valid; the router only reads its own lists.
-CRL_DAYS = 3650
-
 #: A port or a port range as iptables multiport takes it: 443 or 8000:8100.
 DOCKER_PORT = re.compile(r'^\d{1,5}(:\d{1,5})?$')
 
@@ -75,7 +194,7 @@ DOCKER_PORT = re.compile(r'^\d{1,5}(:\d{1,5})?$')
 DOCKER_PORT_LIMIT = 15
 
 TYPE_NAMES = {str: 'a string', list: 'a list of strings', bool: 'a boolean',
-              int: 'an integer'}
+              int: 'an integer', dict: 'an object'}
 
 ID_MODES = ('fqdn', 'public_ip', 'internal_ip')
 
@@ -100,29 +219,76 @@ def fail(message):
     sys.exit(f'vpn-router: error: {message}')
 
 
-def load_config(path=None):
-    """Read the configuration, filling in defaults for anything absent.
+def read_settings(path=None):
+    """The settings in the configuration file, without the defaults; {} when the file does not exist.
 
-    The result maps section -> option -> value. Sections and options not in
-    SCHEMA are kept so that validate() can report them.
+    Raises ConfigError when the file is not JSON or not an object of sections.
+    """
+    path = pathlib.Path(path) if path else CONFIG_FILE
+    try:
+        data = json.loads(path.read_text(encoding='utf-8'))
+    except FileNotFoundError:
+        return {}
+    except ValueError as exc:
+        raise ConfigError(f'cannot parse {path}: {exc}')
+    if not isinstance(data, dict):
+        raise ConfigError(f'cannot parse {path}: the top level is not an object')
+    for section, options in data.items():
+        if not isinstance(options, dict):
+            raise ConfigError(f'cannot parse {path}: {section} is not an object')
+    return data
+
+
+def with_defaults(data):
+    """The configuration of the settings of read_settings, with the defaults for anything absent.
+
+    The result maps section -> option -> value, and site_to_site maps each connection name to
+    its settings filled from CONNECTION. Sections, options and connections that are not
+    objects are kept so that validate() can report them.
+    """
+    cfg = {section: dict(options) for section, options in SCHEMA.items()}
+    for section, options in data.items():
+        cfg.setdefault(section, {}).update(options)
+    cfg['site_to_site'] = {name: {**CONNECTION, **options} if isinstance(options, dict) else options
+                           for name, options in cfg['site_to_site'].items()}
+    return cfg
+
+
+def load_config(path=None):
+    """Read the configuration, filling in defaults for anything absent (see with_defaults).
 
     The path is resolved when called, not when this function is defined, so
     CONFIG_FILE stays overridable.
     """
-    path = pathlib.Path(path) if path else CONFIG_FILE
-    cfg = {section: dict(options) for section, options in SCHEMA.items()}
-    if path.exists():
-        try:
-            data = json.loads(path.read_text(encoding='utf-8'))
-        except ValueError as exc:
-            fail(f'cannot parse {path}: {exc}')
-        if not isinstance(data, dict):
-            fail(f'cannot parse {path}: the top level is not an object')
-        for section, options in data.items():
-            if not isinstance(options, dict):
-                fail(f'cannot parse {path}: {section} is not an object')
-            cfg.setdefault(section, {}).update(options)
-    return cfg
+    try:
+        return with_defaults(read_settings(path))
+    except ConfigError as exc:
+        fail(str(exc))
+
+
+@contextlib.contextmanager
+def config_lock():
+    """Hold the lock of the configuration file from reading it to writing it with save_config."""
+    CONFIG_LOCK.parent.mkdir(parents=True, exist_ok=True)
+    with open(CONFIG_LOCK, 'a', encoding='utf-8') as handle:
+        fcntl.flock(handle, fcntl.LOCK_EX)
+        yield
+
+
+def save_config(data):
+    """Write the settings to the configuration file, indented, with mode 0600, replacing it in one step.
+
+    The caller holds config_lock.
+    """
+    descriptor, name = tempfile.mkstemp(dir=CONFIG_FILE.parent, prefix='.vpn-router.json.')
+    try:
+        with os.fdopen(descriptor, 'w', encoding='utf-8') as handle:
+            handle.write(json.dumps(data, indent=2) + '\n')
+        os.chmod(name, 0o600)
+        os.replace(name, CONFIG_FILE)
+    except OSError:
+        os.unlink(name)
+        raise
 
 
 def _split_list(value):
@@ -134,23 +300,37 @@ def _get(cfg, section, option):
     return value.strip() if isinstance(value, str) else value
 
 
+def _clean(values):
+    return [item.strip() for item in values if item.strip()]
+
+
 def _get_list(cfg, section, option):
-    return [item.strip() for item in cfg[section][option] if item.strip()]
+    return _clean(cfg[section][option])
+
+
+def _check_options(options, defaults, prefix):
+    for option, value in options.items():
+        if option not in defaults:
+            raise ConfigError(f'{prefix}.{option}: unknown setting')
+        expected = type(defaults[option])
+        if type(value) is not expected or (
+                expected is list and
+                not all(isinstance(item, str) for item in value)):
+            raise ConfigError(
+                f'{prefix}.{option} must be {TYPE_NAMES[expected]}')
 
 
 def _check_types(cfg):
     for section, options in cfg.items():
         if section not in SCHEMA:
             raise ConfigError(f'{section}: unknown section')
-        for option, value in options.items():
-            if option not in SCHEMA[section]:
-                raise ConfigError(f'{section}.{option}: unknown setting')
-            expected = type(SCHEMA[section][option])
-            if type(value) is not expected or (
-                    expected is list and
-                    not all(isinstance(item, str) for item in value)):
-                raise ConfigError(
-                    f'{section}.{option} must be {TYPE_NAMES[expected]}')
+        if section != 'site_to_site':
+            _check_options(options, SCHEMA[section], section)
+            continue
+        for name, connection in options.items():
+            if not isinstance(connection, dict):
+                raise ConfigError(f'site_to_site.{name} must be {TYPE_NAMES[dict]}')
+            _check_options(connection, CONNECTION, f'site_to_site.{name}')
 
 
 def _check_networks(values, name):
@@ -202,19 +382,32 @@ def validate(cfg):
                 raise ConfigError(
                     f'{section}.{option} is required in mode {mode or "manual"}')
 
-    for section, option in (('wan', 'local_fqdn'), ('remote', 'id')):
-        if str(_get(cfg, section, option)).startswith('@'):
+    identities = [('wan.local_fqdn', _get(cfg, 'wan', 'local_fqdn'))]
+    for name, connection in cfg['site_to_site'].items():
+        prefix = f'site_to_site.{name}'
+        if not CONNECTION_NAME.match(name):
             raise ConfigError(
-                f'{section}.{option}: remove the leading @; strongSwan reads an '
+                f'{prefix}: a connection name starts with a lowercase letter and has at most 31 '
+                'lowercase letters, digits and hyphens')
+        identities.append((f'{prefix}.id', connection['id'].strip()))
+        id_mode = connection['local_id_mode'].strip()
+        if id_mode and id_mode not in ID_MODES:
+            raise ConfigError(
+                f'{prefix}.local_id_mode: {id_mode!r} is not one of {", ".join(ID_MODES)}')
+        _check_networks(_clean(connection['cidrs']), f'{prefix}.cidrs')
+        _check_networks(_clean(connection['local_cidrs']), f'{prefix}.local_cidrs')
+        if connection['auth'] not in CONNECTION_AUTH:
+            raise ConfigError(f'{prefix}.auth: {connection["auth"]!r} is not one of {", ".join(CONNECTION_AUTH)}')
+        if connection['remote_ca'].strip() and not vpnrouter_pki.is_ca_certificate(connection['remote_ca']):
+            raise ConfigError(f'{prefix}.remote_ca is not the PEM certificate of a CA')
+
+    for option, value in identities:
+        if value.startswith('@'):
+            raise ConfigError(
+                f'{option}: remove the leading @; strongSwan reads an '
                 'address as an IP identity and a name as an FQDN')
 
-    id_mode = _get(cfg, 'wan', 'local_id_mode')
-    if id_mode and id_mode not in ID_MODES:
-        raise ConfigError(
-            f'wan.local_id_mode: {id_mode!r} is not one of {", ".join(ID_MODES)}')
-
     _check_networks(_get_list(cfg, 'local', 'cidrs'), 'local.cidrs')
-    _check_networks(_get_list(cfg, 'remote', 'cidrs'), 'remote.cidrs')
     pool = _get(cfg, 'p2s', 'address_pool')
     _check_networks([pool] if pool else [], 'p2s.address_pool')
     openvpn_pool = _get(cfg, 'openvpn', 'address_pool')
@@ -260,9 +453,178 @@ def validate(cfg):
     if isinstance(web_port, bool) or not isinstance(web_port, int) or not 1 <= web_port <= 65535:
         raise ConfigError(f'web.port: {web_port!r} is not a valid port')
 
+    web_listen = _get(cfg, 'web', 'listen')
+    if web_listen not in LISTEN_CHOICES:
+        raise ConfigError(
+            f'web.listen: {web_listen!r} is not one of {", ".join(LISTEN_CHOICES)}')
+
+    status_content = _get(cfg, 'web', 'status_content')
+    if status_content not in STATUS_CONTENTS:
+        raise ConfigError(
+            f'web.status_content: {status_content!r} is not one of {", ".join(STATUS_CONTENTS)}')
+
+    https_ports = []
+    for option in ('web_port', 'manager_port'):
+        port = _get(cfg, 'https', option)
+        if isinstance(port, bool) or not isinstance(port, int) or not 1 <= port <= 65535:
+            raise ConfigError(f'https.{option}: {port!r} is not a valid port')
+        if port == ACME_HTTP_PORT:
+            raise ConfigError(f'https.{option}: {port} is the port of the HTTP challenge')
+        https_ports.append(port)
+    if https_ports[0] == https_ports[1]:
+        raise ConfigError('https.web_port and https.manager_port must differ')
+
     for item in _get_list(cfg, 'web', 'trusted_proxies'):
         _check_address(item, 'web.trusted_proxies')
 
+    caddy_source = _get(cfg, 'https', 'caddy_source')
+    if caddy_source not in CADDY_SOURCES:
+        raise ConfigError(f'https.caddy_source: {caddy_source!r} is not one of {", ".join(CADDY_SOURCES)}')
+
+    for option in ('root_name', 'p2s_name', 's2s_name'):
+        value = _get(cfg, 'pki', option)
+        if not value or len(value) > CA_NAME_MAX or not value.isprintable():
+            raise ConfigError(f'pki.{option} must have 1 to {CA_NAME_MAX} printable characters')
+    for option in ('root_days', 'ca_days', 'crl_days', 'root_crl_days'):
+        if not 1 <= _get(cfg, 'pki', option) <= 36500:
+            raise ConfigError(f'pki.{option} must be 1 to 36500 days')
+    if _get(cfg, 'pki', 'key') not in PKI_KEYS:
+        raise ConfigError(f'pki.key: {_get(cfg, "pki", "key")!r} is not one of {", ".join(PKI_KEYS)}')
+    publish = _get_list(cfg, 'pki', 'publish')
+    for item in publish:
+        if item not in PKI_PUBLISH:
+            raise ConfigError(f'pki.publish: {item!r} is not one of {", ".join(PKI_PUBLISH)}')
+    if len(set(publish)) != len(publish):
+        raise ConfigError('pki.publish names an endpoint twice')
+    if publish and not _get(cfg, 'wan', 'local_fqdn'):
+        raise ConfigError('pki.publish needs wan.local_fqdn, the name in the published URLs')
+    if 'http' in publish and not _get(cfg, 'web', 'enabled'):
+        raise ConfigError('pki.publish: http is served by the web service, which needs web.enabled')
+    if set(publish) - {'http'} and not _get(cfg, 'https', 'enabled'):
+        raise ConfigError('pki.publish: http-80, https and https-default are served by Caddy, which needs https.enabled')
+    for name, template in cfg['pki']['templates'].items():
+        prefix = f'pki.templates.{name}'
+        if not PKI_NAME.match(name):
+            raise ConfigError(f'{prefix}: a template name starts with a lowercase letter and has at most 31 lowercase '
+                              'letters, digits and hyphens')
+        if not isinstance(template, dict):
+            raise ConfigError(f'{prefix} must be {TYPE_NAMES[dict]}')
+        _check_options(template, TEMPLATE, prefix)
+    templates = pki_templates(cfg)
+    for name, template in templates.items():
+        prefix = f'pki.templates.{name}'
+        if template['key'] not in PKI_KEYS:
+            raise ConfigError(f'{prefix}.key: {template["key"]!r} is not one of {", ".join(PKI_KEYS)}')
+        if not 1 <= template['days'] <= 36500:
+            raise ConfigError(f'{prefix}.days must be 1 to 36500 days')
+        for option, allowed in (('key_usage', KEY_USAGES), ('extended_key_usage', EXTENDED_KEY_USAGES),
+                                ('sans', SAN_TYPES)):
+            for item in template[option]:
+                if item not in allowed:
+                    raise ConfigError(f'{prefix}.{option}: {item!r} is not one of {", ".join(allowed)}')
+        if not template['key_usage']:
+            raise ConfigError(f'{prefix}.key_usage needs at least one usage')
+        if template['subject'] not in TEMPLATE_SUBJECTS:
+            raise ConfigError(f'{prefix}.subject: {template["subject"]!r} is not one of {", ".join(TEMPLATE_SUBJECTS)}')
+    for name, authority in cfg['pki']['issuing_cas'].items():
+        prefix = f'pki.issuing_cas.{name}'
+        if not PKI_NAME.match(name) or name in BUILTIN_CAS or name == 'root':
+            raise ConfigError(f'{prefix}: an issuing CA name starts with a lowercase letter and has at most 31 '
+                              f'lowercase letters, digits and hyphens, and is not root, {" or ".join(BUILTIN_CAS)}')
+        if not isinstance(authority, dict):
+            raise ConfigError(f'{prefix} must be {TYPE_NAMES[dict]}')
+        _check_options(authority, ISSUING_CA, prefix)
+    for name, authority in issuing_cas(cfg).items():
+        prefix = f'pki.issuing_cas.{name}'
+        if authority['builtin']:
+            continue
+        common_name = authority['name'].strip()
+        if not common_name or len(common_name) > CA_NAME_MAX or not common_name.isprintable():
+            raise ConfigError(f'{prefix}.name must have 1 to {CA_NAME_MAX} printable characters')
+        if not authority['templates']:
+            raise ConfigError(f'{prefix}.templates needs at least one template')
+        for template in authority['templates']:
+            if template not in templates:
+                raise ConfigError(f'{prefix}.templates: {template!r} is not a template')
+            if template in ROUTER_TEMPLATES:
+                raise ConfigError(f'{prefix}.templates: {template} is issued by the P2S CA only')
+        if authority['acme'] and not _get(cfg, 'https', 'enabled'):
+            raise ConfigError(f'{prefix}.acme: the ACME server is Caddy\'s, which needs https.enabled')
+
+    taken = {web_port: 'web.port', ACME_HTTP_PORT: 'the HTTP challenge',
+             https_ports[0]: 'https.web_port', https_ports[1]: 'https.manager_port'}
+    if 'https-default' in publish:
+        if HTTPS_DEFAULT_PORT in taken:
+            raise ConfigError(f'pki.publish: https-default takes port {HTTPS_DEFAULT_PORT}, the port of '
+                              f'{taken[HTTPS_DEFAULT_PORT]}')
+        if _get(cfg, 'openvpn', 'protocol') == 'tcp' and _get(cfg, 'openvpn', 'port') == HTTPS_DEFAULT_PORT:
+            raise ConfigError(f'pki.publish: https-default takes port {HTTPS_DEFAULT_PORT}, the port of openvpn.port')
+        taken[HTTPS_DEFAULT_PORT] = 'pki.publish https-default'
+    for option in ('port', 'https_port'):
+        port = _get(cfg, 'metrics', option)
+        if isinstance(port, bool) or not isinstance(port, int) or not 1 <= port <= 65535:
+            raise ConfigError(f'metrics.{option}: {port!r} is not a valid port')
+        if port in taken:
+            raise ConfigError(f'metrics.{option}: {port} is the port of {taken[port]}')
+        taken[port] = f'metrics.{option}'
+
+    metrics_listen = _get(cfg, 'metrics', 'listen')
+    if metrics_listen not in LISTEN_CHOICES:
+        raise ConfigError(
+            f'metrics.listen: {metrics_listen!r} is not one of {", ".join(LISTEN_CHOICES)}')
+
+    endpoint = _get(cfg, 'metrics', 'otlp_endpoint')
+    if endpoint and urllib.parse.urlsplit(endpoint).scheme not in ('http', 'https'):
+        raise ConfigError(f'metrics.otlp_endpoint: {endpoint!r} is not an http or https URL')
+    if bool(_get(cfg, 'metrics', 'otlp_username')) != bool(_get(cfg, 'metrics', 'otlp_password')):
+        raise ConfigError('metrics.otlp_username and metrics.otlp_password are given together')
+
+    for option in METRICS_KEEP:
+        keep = _get(cfg, 'metrics', option)
+        if not METRICS_INTERVAL.match(keep):
+            raise ConfigError(
+                f'metrics.{option}: {keep!r} is not a count and a unit, such as "24 hours" or "3 months"')
+
+    schedule = _get(cfg, 'backup', 'schedule')
+    if schedule not in BACKUP_SCHEDULES:
+        raise ConfigError(
+            f'backup.schedule: {schedule!r} is not one of {", ".join(BACKUP_SCHEDULES)}')
+    target = _get(cfg, 'backup', 'target')
+    if target not in BACKUP_TARGETS:
+        raise ConfigError(f'backup.target: {target!r} is not one of {", ".join(BACKUP_TARGETS)}')
+    for option in ('path', 'pbs_keyfile'):
+        value = _get(cfg, 'backup', option)
+        if value and not value.startswith('/'):
+            raise ConfigError(f'backup.{option}: {value!r} is not an absolute path')
+    if not _get(cfg, 'backup', 'path'):
+        raise ConfigError('backup.path is required')
+    for option in BACKUP_KEEP:
+        if _get(cfg, 'backup', option) < 0:
+            raise ConfigError(f'backup.{option} must not be negative')
+    if target == 'local' and not any(_get(cfg, 'backup', option) for option in backup_keeps(schedule)):
+        raise ConfigError(
+            f'backup: with the {schedule} schedule, at least one of {", ".join(backup_keeps(schedule))} '
+            'is above 0')
+    remote = _get(cfg, 'backup', 'rclone_remote')
+    if remote and not RCLONE_REMOTE.match(remote):
+        raise ConfigError(f'backup.rclone_remote: {remote!r} is not a remote and a path, such as azure:backups/router')
+    rclone_mode = _get(cfg, 'backup', 'rclone_mode')
+    if rclone_mode not in RCLONE_MODES:
+        raise ConfigError(f'backup.rclone_mode: {rclone_mode!r} is not one of {", ".join(RCLONE_MODES)}')
+    if target == 'pbs':
+        repository = _get(cfg, 'backup', 'pbs_repository')
+        if not repository or not _get(cfg, 'backup', 'pbs_password'):
+            raise ConfigError('backup.pbs_repository and backup.pbs_password are required with the pbs target')
+        if not PBS_REPOSITORY.match(repository):
+            raise ConfigError(
+                f'backup.pbs_repository: {repository!r} is not of the form user@realm!token@host:datastore')
+    fingerprint = _get(cfg, 'backup', 'pbs_fingerprint')
+    if fingerprint and not PBS_FINGERPRINT.match(fingerprint):
+        raise ConfigError(f'backup.pbs_fingerprint: {fingerprint!r} is not 32 hexadecimal pairs joined by colons')
+    namespace = _get(cfg, 'backup', 'pbs_namespace')
+    if namespace and not PBS_NAMESPACE.match(namespace):
+        raise ConfigError(f'backup.pbs_namespace: {namespace!r} is not up to seven names joined by /')
+
     docker_mode = _get(cfg, 'docker', 'mode')
     if docker_mode not in DOCKER_MODES:
         raise ConfigError(
@@ -276,21 +638,65 @@ def validate(cfg):
             raise ConfigError(
                 f'docker.exposed_ports: {item!r} is not a port or a port range')
         weight += 2 if ':' in item else 1
+        first, _, last = item.partition(':')
+        if _get(cfg, 'https', 'enabled'):
+            caddy_ports = (ACME_HTTP_PORT, *https_ports, *([HTTPS_DEFAULT_PORT] if 'https-default' in publish else []))
+            for port in caddy_ports:
+                if int(first) <= port <= int(last or first):
+                    raise ConfigError(
+                        f'docker.exposed_ports: {item!r} takes port {port}, which Caddy serves while '
+                        'https.enabled is true')
     if weight > DOCKER_PORT_LIMIT:
         raise ConfigError(
             f'docker.exposed_ports: at most {DOCKER_PORT_LIMIT} ports are allowed, '
             'a range counting as two')
 
 
-def resolve_psk(cfg):
-    """Return the pre-shared key, or '' when none is configured."""
-    psk_file = _get(cfg, 'remote', 'psk_file')
+def backup_keeps(schedule):
+    """The retention settings of local backups that apply to the schedule: none finer than it."""
+    return BACKUP_KEEP[BACKUP_SCHEDULES.index(schedule):]
+
+
+def pki_templates(cfg):
+    """The certificate templates: the built-in ones (PKI_TEMPLATES) and those of pki.templates, each filled from TEMPLATE."""
+    custom = cfg['pki']['templates']
+    return {name: {**TEMPLATE, **PKI_TEMPLATES.get(name, {}),
+                   **(custom.get(name) if isinstance(custom.get(name), dict) else {})}
+            for name in [*PKI_TEMPLATES, *(name for name in custom if name not in PKI_TEMPLATES)]}
+
+
+def issuing_cas(cfg):
+    """The issuing CAs, name -> {name, templates, acme, builtin}: p2s and s2s (BUILTIN_CAS) with their common names from
+    the pki section, then those of pki.issuing_cas, each filled from ISSUING_CA."""
+    found = {name: {'name': _get(cfg, 'pki', option), 'templates': list(templates), 'acme': False, 'builtin': True}
+             for name, (option, templates) in BUILTIN_CAS.items()}
+    for name, options in cfg['pki']['issuing_cas'].items():
+        if isinstance(options, dict):
+            found[name] = {**ISSUING_CA, **options, 'builtin': False}
+    return found
+
+
+def pki_base_urls(cfg):
+    """The base URLs, ending in /, under which the CA certificates and CRLs are published, in the order of pki.publish."""
+    fqdn = _get(cfg, 'wan', 'local_fqdn')
+    bases = {
+        'http': f'http://{fqdn}:{_get(cfg, "web", "port")}/pki/',
+        'http-80': f'http://{fqdn}/pki/',
+        'https': f'https://{fqdn}:{_get(cfg, "https", "web_port")}/pki/',
+        'https-default': f'https://{fqdn}/pki/',
+    }
+    return [bases[item] for item in _get_list(cfg, 'pki', 'publish') if item in bases] if fqdn else []
+
+
+def resolve_psk(connection, name):
+    """Return the pre-shared key of the site_to_site connection: psk_file wins, else psk; '' when none is configured."""
+    psk_file = connection['psk_file'].strip()
     if psk_file:
         path = pathlib.Path(psk_file)
         if not path.is_file():
-            raise ConfigError(f'remote.psk_file: {psk_file} does not exist')
+            raise ConfigError(f'site_to_site.{name}.psk_file: {psk_file} does not exist')
         return path.read_text(encoding='utf-8').strip()
-    return cfg['remote']['psk']
+    return connection['psk']
 
 
 def platform_module(cfg):
@@ -374,29 +780,47 @@ def resolve_interfaces(cfg):
     return ext_iface, int_iface, local_addrs, int_addr, gateway
 
 
-def build_context(cfg, local_id_override=None):
-    """Assemble the template context from the configuration."""
+def build_context(cfg, public_ip=None):
+    """Assemble the template context from the configuration.
+
+    public_ip is the address wan.local_fqdn resolves to, the local identity of the
+    site_to_site connections in mode public_ip; setup resolves it once the network is up and
+    renders again, and without it such a connection is not ready.
+    """
     extras = vpnrouter_platforms.context(platform_module(cfg))
 
     ext_iface, int_iface, local_addrs, int_addr, gateway = resolve_interfaces(cfg)
 
     local_fqdn   = _get(cfg, 'wan', 'local_fqdn')
-    id_mode      = _get(cfg, 'wan', 'local_id_mode') or 'fqdn'
     local_cidrs  = _get_list(cfg, 'local', 'cidrs')
-    remote_cidrs = _get_list(cfg, 'remote', 'cidrs')
     openvpn_pool = _get(cfg, 'openvpn', 'address_pool')
     openvpn_network = ipaddress.ip_network(openvpn_pool) if openvpn_pool else None
+    identities = {'fqdn': local_fqdn, 'public_ip': public_ip or '', 'internal_ip': int_addr}
 
-    if local_id_override:
-        local_id = local_id_override
-    elif id_mode == 'fqdn':
-        local_id = local_fqdn
-    elif id_mode == 'internal_ip':
-        local_id = int_addr
-    else:
-        # public_ip is resolved once the network is up, and the caller then
-        # re-renders with local_id_override set.
-        local_id = ''
+    connections = []
+    for name, connection in cfg['site_to_site'].items():
+        id_mode = connection['local_id_mode'].strip() or 'fqdn'
+        remote_cidrs = _clean(connection['cidrs'])
+        connection_cidrs = _clean(connection['local_cidrs'])
+        auth = connection['auth']
+        entry = {
+            'name':             name,
+            'remote_addrs':     ', '.join(_clean(connection['addrs'])),
+            'remote_id':        connection['id'].strip(),
+            'remote_cidrs':     ', '.join(remote_cidrs),
+            'remote_cidr_list': remote_cidrs,
+            'local_ts':         _join(connection_cidrs, _split_list(extras['EXTRA_LOCAL_TS'])),
+            'local_cidr_list':  connection_cidrs,
+            'local_id_mode':    id_mode,
+            'local_id':         identities.get(id_mode, ''),
+            'auth':             auth,
+            'psk':              resolve_psk(connection, name) if auth == 'psk' else '',
+            'remote_cacert':    vpnrouter_pki.remote_ca_file(name) if connection['remote_ca'].strip()
+                                else vpnrouter_pki.SWANCTL_CA_FILES['s2s'],
+        }
+        required = ('remote_addrs', 'remote_id', 'remote_cidrs', 'local_id', *(('psk',) if auth == 'psk' else ()))
+        entry['ready'] = bool(local_addrs and connection_cidrs and all(entry[key] for key in required))
+        connections.append(entry)
 
     return {
         'platform':        _get(cfg, 'general', 'platform') or PLATFORM_NONE,
@@ -406,21 +830,15 @@ def build_context(cfg, local_id_override=None):
         'local_addrs':     ', '.join(local_addrs),
         'local_addr_list': local_addrs,
         'local_fqdn':      local_fqdn,
-        'local_id_mode':   id_mode,
-        'local_id':        local_id,
         'local_cidrs':     ', '.join(local_cidrs),
         'local_cidr_list': local_cidrs,
         'local_ts':        _join(local_cidrs, _split_list(extras['EXTRA_LOCAL_TS'])),
         'int_addr':        int_addr,
         'int_gateway_ip':  gateway,
-        'remote_addrs':    ', '.join(_get_list(cfg, 'remote', 'addrs')),
-        'remote_id':       _get(cfg, 'remote', 'id'),
-        'remote_cidrs':    ', '.join(remote_cidrs),
-        'remote_cidr_list': remote_cidrs,
-        'psk':             resolve_psk(cfg),
+        'connections':     connections,
         'p2s_enabled':     _get(cfg, 'p2s', 'enabled'),
         'p2s_address_pool': _get(cfg, 'p2s', 'address_pool'),
-        'p2s_ca_name':     _get(cfg, 'p2s', 'ca_name'),
+        'public_ip':       public_ip or '',
         'p2s_dns':         _join(local_addrs, _split_list(extras['EXTRA_P2S_DNS'])),
         'wg_enabled':      _get(cfg, 'wireguard', 'enabled'),
         'wg_address':      _get(cfg, 'wireguard', 'address'),
@@ -436,23 +854,32 @@ def build_context(cfg, local_id_override=None):
         'openvpn_device':  OPENVPN_DEVICE,
         'openvpn_ldap_conf': str(OPENVPN_LDAP),
         'p2s_dns_list':    _split_list(_join(local_addrs, _split_list(extras['EXTRA_P2S_DNS']))),
-        'ca_cert':         str(CA_CERT_DEST),
-        'server_cert':     str(SERVER_CERT),
-        'server_key':      str(SERVER_KEY),
-        'crl':             str(CRL_DEST),
+        'openvpn_ca':      str(vpnrouter_pki.OPENVPN_CA),
+        'openvpn_crl':     str(vpnrouter_pki.OPENVPN_CRL),
+        'server_cert':     str(vpnrouter_pki.VPN_SERVER_CERT),
+        'server_key':      str(vpnrouter_pki.VPN_SERVER_KEY),
+        'server_certificate': False,
         'ldap_uri':        vpnrouter_ldap.URI,
         'ldap_users':      vpnrouter_ldap.USERS,
         'web_enabled':     _get(cfg, 'web', 'enabled'),
         'web_port':        _get(cfg, 'web', 'port'),
+        'https_enabled':   _get(cfg, 'https', 'enabled'),
+        'https_email':     _get(cfg, 'https', 'email'),
+        'https_web_port':  _get(cfg, 'https', 'web_port'),
+        'https_manager_port': _get(cfg, 'https', 'manager_port'),
+        'pki_publish':     _get_list(cfg, 'pki', 'publish'),
+        'pki_acme':        [name for name, authority in issuing_cas(cfg).items() if authority['acme']],
+        'metrics_enabled': _get(cfg, 'metrics', 'enabled'),
+        'metrics_port':    _get(cfg, 'metrics', 'port'),
+        'metrics_https_port': _get(cfg, 'metrics', 'https_port'),
+        'metrics_service_account': _get(cfg, 'metrics', 'service_account'),
+        'loopback':        LOOPBACK,
     }
 
 
 def s2s_ready(ctx):
-    return all([
-        ctx['local_addrs'], ctx['local_fqdn'], ctx['local_cidrs'],
-        ctx['remote_addrs'], ctx['remote_id'], ctx['remote_cidrs'],
-        ctx['psk'], ctx['local_id'],
-    ])
+    """True when any site_to_site connection is ready."""
+    return any(connection['ready'] for connection in ctx['connections'])
 
 
 def routing_ready(ctx):
@@ -462,7 +889,8 @@ def routing_ready(ctx):
 
 
 def p2s_ready(ctx):
-    return bool(ctx['p2s_enabled'] and ctx['p2s_address_pool'] and ctx['local_fqdn'])
+    """Road-warrior access needs its settings and the router's VPN server certificate, which the P2S CA issues."""
+    return bool(ctx['p2s_enabled'] and ctx['p2s_address_pool'] and ctx['local_fqdn'] and ctx['server_certificate'])
 
 
 def wg_ready(ctx):
@@ -470,7 +898,18 @@ def wg_ready(ctx):
 
 
 def openvpn_ready(ctx):
-    return bool(ctx['openvpn_enabled'] and ctx['openvpn_pool'] and ctx['local_fqdn'])
+    """OpenVPN needs its settings and the router's VPN server certificate, which the P2S CA issues."""
+    return bool(ctx['openvpn_enabled'] and ctx['openvpn_pool'] and ctx['local_fqdn'] and ctx['server_certificate'])
+
+
+def https_ready(ctx):
+    """Caddy serves HTTPS once it is enabled and wan.local_fqdn names the certificate."""
+    return bool(ctx['https_enabled'] and ctx['local_fqdn'])
+
+
+def metrics_ready(ctx):
+    """The metrics service collects, keeps and serves the metrics while metrics.enabled is true."""
+    return bool(ctx['metrics_enabled'])
 
 
 def _jinja_env():
@@ -551,63 +990,7 @@ def install_file(src, dest, mode):
     return True
 
 
-def _simple_ca(*args):
-    result = subprocess.run(
-        [sys.executable, str(SIMPLE_CA), *args],
-        capture_output=True, text=True,
-    )
-    if result.returncode != 0:
-        fail(f'simple-ca {" ".join(args)} failed: {result.stderr.strip()}')
-    return result.stdout
-
-
-def provision_pki(ctx):
-    """Create the CA and CRL when the PKI directory is empty, and the server certificate once wan.local_fqdn is set.
-
-    A CA of the router's own (ca_key.pem present) without a CRL gets one, and without a
-    certificate for wan.local_fqdn gets one too; nothing present is overwritten.
-    """
-    PKI_DIR.mkdir(parents=True, exist_ok=True)
-    os.chmod(PKI_DIR, 0o700)
-    if not SIMPLE_CA.is_file():
-        fail(f'{SIMPLE_CA} is missing')
-
-    if not any(PKI_DIR.iterdir()):
-        if not ctx['p2s_ca_name']:
-            warn('p2s.ca_name is empty, not creating a local CA')
-            return
-        _simple_ca('make-ca', '--ca-dir', str(PKI_DIR), ctx['p2s_ca_name'])
-    if not (PKI_DIR / 'ca_key.pem').is_file():
-        return
-    if not (PKI_DIR / 'crl.pem').is_file():
-        _simple_ca('make-crl', '--ca-dir', str(PKI_DIR), '--days', str(CRL_DAYS))
-    label = ctx['local_fqdn'].split('.', 1)[0] if ctx['local_fqdn'] else ''
-    if label and not (PKI_DIR / f'{label}_cert.pem').is_file():
-        _simple_ca('make-cert', '--ca-dir', str(PKI_DIR), ctx['local_fqdn'], ctx['local_fqdn'])
-
-
-def distribute_pki(ctx):
-    """Copy PKI material to each consumer. Returns True when anything changed."""
-    label = ctx['local_fqdn'].split('.', 1)[0] if ctx['local_fqdn'] else ''
-    bundle = PKI_DIR / 'ca_bundle.pem'
-    ca_src = bundle if bundle.is_file() else PKI_DIR / 'ca_cert.pem'
-
-    changed = False
-    changed |= install_file(ca_src, CA_CERT_DEST, 0o644)
-    if label:
-        changed |= install_file(PKI_DIR / f'{label}_cert.pem', SERVER_CERT, 0o644)
-        changed |= install_file(PKI_DIR / f'{label}_key.pem', SERVER_KEY, 0o600)
-    changed |= install_file(PKI_DIR / 'crl.pem', CRL_DEST, 0o644)
-
-    if install_file(PKI_DIR / 'ca_cert.pem', TRUST_ANCHOR, 0o644):
-        changed = True
-        if shutil.which('update-ca-certificates'):
-            subprocess.run(['update-ca-certificates'],
-                           capture_output=True, text=True)
-    return changed
-
-
-def configure(cfg=None, local_id_override=None):
+def configure(cfg=None, public_ip=None):
     """Render configuration files and provision PKI.
 
     Returns (ctx, changed) where changed is a set naming what needs reloading:
@@ -623,22 +1006,36 @@ def configure(cfg=None, local_id_override=None):
 
     try:
         validate(cfg)
-        ctx = build_context(cfg, local_id_override=local_id_override)
+        ctx = build_context(cfg, public_ip=public_ip)
     except ConfigError as exc:
         fail(str(exc))
 
     changed = set()
 
-    provision_pki(ctx)
-    if distribute_pki(ctx):
-        changed.add('swanctl')
+    # Without the directory, which holds the PKI, the copies of the last run stay and what needs none is configured.
+    with vpnrouter_pki.locked():
+        try:
+            notices = vpnrouter_pki.provision(cfg, ctx)
+            if vpnrouter_pki.distribute(cfg):
+                changed.add('swanctl')
+        except vpnrouter_pki.Failure as failure:
+            fail(str(failure))
+        except ldap.LDAPError as exc:
+            notices = [f'the PKI is left as it is: the directory cannot be reached: {exc}']
+    for notice in notices:
+        warn(notice)
+    # What needs the router's own certificates.
+    ctx['server_certificate'] = vpnrouter_pki.VPN_SERVER_CERT.is_file()
+    for connection in ctx['connections']:
+        if connection['auth'] == 'certificate':
+            connection['ready'] = connection['ready'] and vpnrouter_pki.S2S_CERT.is_file()
 
     env = _jinja_env()
 
     if s2s_ready(ctx):
-        if render(env, ctx, 'remote-site.conf.j2', REMOTE_SITE, 0o600):
+        if render(env, ctx, 'site-to-site.conf.j2', SITE_TO_SITE, 0o600):
             changed.add('swanctl')
-    elif remove(REMOTE_SITE):
+    elif remove(SITE_TO_SITE):
         changed.add('swanctl')
 
     if p2s_ready(ctx):

debian-package/src/usr/lib/vpn-router/vpnrouter_auth.py

@@ -5,12 +5,15 @@ router and that never leaves it. /auth issues an access token naming the user in
 a refresh token valid REFRESH_SECONDS; vpn-router-token issues tokens with the admin claim and
 any lifetime. A token without the admin claim must expire within MAX_LIFETIME seconds.
 
+Every failure of the database raises DatabaseError, the only database exception callers see.
+
 Refresh tokens are random strings kept in PostgreSQL as their SHA-256, in the database of
 web.database, by default the vpn-router database on this host that provision() creates on the
 web service's first run, with the password in DATABASE_SECRET. A refresh token is used once: refreshing replaces it.
 """
 
 import base64
+import contextlib
 import hashlib
 import hmac
 import json
@@ -20,7 +23,6 @@ import subprocess
 import time
 
 import psycopg
-from psycopg.conninfo import make_conninfo
 
 WEB_KEY = pathlib.Path('/etc/vpn-router/web.key')
 DATABASE_SECRET = pathlib.Path('/etc/vpn-router/database.secret')
@@ -28,6 +30,9 @@ DATABASE_SECRET = pathlib.Path('/etc/vpn-router/database.secret')
 #: The role and the database provision() creates when web.database is not set.
 DATABASE = 'vpn-router'
 
+#: The user operations of vpnrouter_users after which the user's refresh tokens are revoked.
+REVOKING = ('passwd', 'disable', 'unlock', 'remove')
+
 #: Seconds an access token of /auth and a refresh token are valid.
 ACCESS_SECONDS = 300
 REFRESH_SECONDS = 8 * 3600
@@ -41,6 +46,10 @@ LEEWAY = 60
 SCHEMA = 'CREATE TABLE IF NOT EXISTS refresh_tokens (hash text PRIMARY KEY, name text NOT NULL, expires timestamptz NOT NULL)'
 
 
+class DatabaseError(Exception):
+    """The database of the refresh tokens failed."""
+
+
 def key():
     """The signing key, or '' while the web service has not made it."""
     try:
@@ -112,19 +121,23 @@ def admin_token(seconds):
 
 
 def connection_string(cfg):
-    """web.database, or the database provision() creates."""
+    """web.database, or the database provision() creates; its password, of URL-safe characters, is written unquoted."""
     configured = cfg['web']['database'].strip()
     if configured:
         return configured
     password = DATABASE_SECRET.read_text(encoding='utf-8').strip()
-    return make_conninfo(host='localhost', dbname=DATABASE, user=DATABASE, password=password)
+    return f'host=localhost dbname={DATABASE} user={DATABASE} password={password}'
 
 
-def connect(cfg):
-    """A connection to the database of the refresh tokens, with their table."""
-    connection = psycopg.connect(connection_string(cfg), autocommit=True)
-    connection.execute(SCHEMA)
-    return connection
+@contextlib.contextmanager
+def database(cfg):
+    """A connection to the database of the refresh tokens, with their table, closed at the end; DatabaseError on any failure."""
+    try:
+        with psycopg.connect(connection_string(cfg), autocommit=True) as connection:
+            connection.execute(SCHEMA)
+            yield connection
+    except psycopg.Error as exc:
+        raise DatabaseError(f'the token database failed: {exc}') from exc
 
 
 def digest(token):
@@ -136,7 +149,7 @@ def issue(cfg, name):
     secret = key()
     now = int(time.time())
     refresh = secrets.token_urlsafe(32)
-    with connect(cfg) as connection:
+    with database(cfg) as connection:
         connection.execute('DELETE FROM refresh_tokens WHERE expires < now()')
         connection.execute("INSERT INTO refresh_tokens VALUES (%s, %s, now() + make_interval(secs => %s))",
                            (digest(refresh), name, REFRESH_SECONDS))
@@ -153,7 +166,7 @@ def take(cfg, token):
     """Remove an unexpired refresh token and return the user it was issued to, or None."""
     if not isinstance(token, str) or not token:
         return None
-    with connect(cfg) as connection:
+    with database(cfg) as connection:
         row = connection.execute('DELETE FROM refresh_tokens WHERE hash = %s RETURNING name, expires > now()',
                                  (digest(token),)).fetchone()
     return row[0] if row and row[1] else None
@@ -161,7 +174,7 @@ def take(cfg, token):
 
 def revoke(cfg, name):
     """Remove all refresh tokens of the user."""
-    with connect(cfg) as connection:
+    with database(cfg) as connection:
         connection.execute('DELETE FROM refresh_tokens WHERE name = %s', (name,))
 
 

debian-package/src/usr/lib/vpn-router/vpnrouter_backup.py

@@ -0,0 +1,412 @@
+"""The backups of the router, for vpn-router-backup and the web service.
+
+A backup is a consistent copy of the state that setup cannot make again, staged in STAGING: the
+trees of TREES, the files of site_to_site psk_file outside them under files/, the directory as
+slapcat writes it (ldap/vpn-router.ldif), with backup.metrics the database as pg_dump writes it
+(postgresql/vpn-router.dump), and manifest.json. The trees and the directory are copied while
+the configuration lock and the users lock are held, so the configuration, the PKI and the users
+match; pg_dump reads a snapshot of its own.
+
+backup.target local writes the staging directory as a tar archive compressed with zstd,
+ARCHIVE, to backup.path, removes the archives of this host that the retention settings of the
+schedule no longer keep, and with backup.rclone_remote syncs or copies the archives of this host
+to the remote of RCLONE_CONFIG. backup.target pbs sends the staging directory to a Proxmox Backup
+Server as the archive router.pxar of the snapshot host/<host>/<time>; the server keeps them.
+
+A run holds RUN_LOCK and records itself in STATE: result running, ok, warning (the backup is
+made, the cloud copy failed) or failed, with the times, the target, the archive or snapshot,
+its size, the error and the cloud copy. restore() puts a backup back and runs setup.
+"""
+
+import contextlib
+import datetime
+import fcntl
+import json
+import os
+import pathlib
+import re
+import shutil
+import socket
+import subprocess
+import tempfile
+import time
+
+import vpnrouter
+import vpnrouter_auth
+import vpnrouter_ldap
+import vpnrouter_users
+
+STATE = pathlib.Path('/var/lib/vpn-router/backup.json')
+WORK_DIR = pathlib.Path('/var/lib/vpn-router/backup')
+STAGING = WORK_DIR / 'staging'
+RESTORE_DIR = WORK_DIR / 'restore'
+#: What a restore replaces is kept here, in a directory named after the time of the restore.
+BEFORE_RESTORE = WORK_DIR / 'before-restore'
+RUN_LOCK = pathlib.Path('/run/vpn-router/backup.lock')
+RCLONE_CONFIG = pathlib.Path('/etc/vpn-router/rclone.conf')
+PBS_CLIENT = pathlib.Path('/usr/bin/proxmox-backup-client')
+UNIT = 'vpn-router-backup.service'
+TIMER = 'vpn-router-backup.timer'
+
+#: The state a backup copies: directories and files, each where it is on the system.
+TREES = ('/etc/vpn-router', '/etc/wireguard', '/etc/ufw', '/etc/default/ufw', '/etc/vpn-router-manager')
+
+#: The name of a local archive: the host and the UTC time of the backup.
+ARCHIVE = re.compile(r'^vpn-router-(?P<host>[A-Za-z0-9][A-Za-z0-9.-]*)-(?P<stamp>\d{8}T\d{6}Z)\.tar\.zst$')
+STAMP = '%Y%m%dT%H%M%SZ'
+
+LDIF = 'ldap/vpn-router.ldif'
+DUMP = 'postgresql/vpn-router.dump'
+PBS_ARCHIVE = 'router.pxar'
+
+#: The period of each retention setting, as strftime formats of local time; weeks are ISO weeks.
+PERIODS = {'keep_hourly': '%Y-%m-%d %H', 'keep_daily': '%Y-%m-%d', 'keep_weekly': '%G-%V', 'keep_monthly': '%Y-%m',
+           'keep_yearly': '%Y'}
+
+#: The units a restore stops while it replaces the directory and the database, and the ones it restarts after setup.
+RESTORE_STOPS = ('vpn-router-web.service', 'vpn-router-metrics.service', 'slapd.service')
+RESTORE_RESTARTS = ('wg-quick@wg0.service', 'openvpn-server@vpn-router.service', 'strongswan.service',
+                    'vpn-router-manager.service')
+
+
+class Failure(Exception):
+    """A backup or a restore that cannot be done."""
+
+
+def now():
+    """The local time in ISO 8601, to the second, with its offset."""
+    return datetime.datetime.now().astimezone().isoformat(timespec='seconds')
+
+
+def host():
+    """The name of this host, which names its archives and its PBS backup group."""
+    return socket.gethostname().split('.')[0]
+
+
+def command(args, **kwargs):
+    """Run a command; Failure with its standard error when it fails."""
+    try:
+        result = subprocess.run(args, capture_output=True, text=True, **kwargs)
+    except OSError as exc:
+        raise Failure(f'{args[0]}: {exc.strerror}')
+    if result.returncode != 0:
+        raise Failure(f'{" ".join(args[:2])} failed: {result.stderr.strip() or result.stdout.strip()}')
+    return result.stdout
+
+
+def write_state(state):
+    """Write STATE in one step."""
+    STATE.parent.mkdir(parents=True, exist_ok=True)
+    descriptor, name = tempfile.mkstemp(dir=STATE.parent, prefix='.backup.json.')
+    with os.fdopen(descriptor, 'w', encoding='utf-8') as handle:
+        handle.write(json.dumps(state, indent=2) + '\n')
+    os.chmod(name, 0o600)
+    os.replace(name, STATE)
+
+
+def running():
+    """Whether a run holds RUN_LOCK."""
+    try:
+        with open(RUN_LOCK, encoding='utf-8') as handle:
+            fcntl.flock(handle, fcntl.LOCK_EX | fcntl.LOCK_NB)
+    except FileNotFoundError:
+        return False
+    except BlockingIOError:
+        return True
+    return False
+
+
+def state():
+    """The state of the last run, {} when there was none; a run that ended without recording it is interrupted."""
+    try:
+        recorded = json.loads(STATE.read_text(encoding='utf-8'))
+    except (OSError, ValueError):
+        return {}
+    if recorded.get('result') == 'running' and not running():
+        recorded['result'] = 'interrupted'
+    return recorded
+
+
+@contextlib.contextmanager
+def run_lock():
+    """Hold RUN_LOCK; Failure when another run holds it."""
+    RUN_LOCK.parent.mkdir(parents=True, exist_ok=True)
+    with open(RUN_LOCK, 'a', encoding='utf-8') as handle:
+        try:
+            fcntl.flock(handle, fcntl.LOCK_EX | fcntl.LOCK_NB)
+        except BlockingIOError:
+            raise Failure('another backup or restore is running')
+        yield
+
+
+def archives(path):
+    """The archives of this host in path, newest first, as [{name, time, size}]; time is the UTC time of the backup."""
+    found = []
+    try:
+        entries = list(os.scandir(path))
+    except OSError:
+        return found
+    for entry in entries:
+        match = ARCHIVE.match(entry.name)
+        if match and match['host'] == host() and entry.is_file():
+            when = datetime.datetime.strptime(match['stamp'], STAMP).replace(tzinfo=datetime.timezone.utc)
+            found.append({'name': entry.name, 'time': when.isoformat(), 'size': entry.stat().st_size})
+    return sorted(found, key=lambda item: item['time'], reverse=True)
+
+
+def kept(found, keeps):
+    """The names of the archives the retention settings keep, by the rules of Proxmox Backup Server's prune.
+
+    found is newest first. The settings apply in the order of PERIODS: each keeps the newest archive of each of its
+    last N periods that have archives and no archive a former setting kept, and the older archives of those periods
+    are removed, so no later setting keeps them.
+    """
+    marks = {}
+    for option, period in PERIODS.items():
+        count = keeps.get(option, 0)
+        if not count:
+            continue
+        periods = {moment(item).strftime(period) for item in found if marks.get(item['name']) == 'keep'}
+        included = set()
+        for item in found:
+            if item['name'] in marks:
+                continue
+            key = moment(item).strftime(period)
+            if key in periods:
+                continue
+            if key not in included:
+                if len(included) >= count:
+                    break
+                included.add(key)
+                marks[item['name']] = 'keep'
+            else:
+                marks[item['name']] = 'remove'
+    return {name for name, mark in marks.items() if mark == 'keep'}
+
+
+def moment(item):
+    """The local time of an archive of archives()."""
+    return datetime.datetime.fromisoformat(item['time']).astimezone()
+
+
+def psk_files(cfg):
+    """The psk_file of each site_to_site connection outside the trees of TREES."""
+    files = []
+    for connection in cfg['site_to_site'].values():
+        name = connection['psk_file'].strip()
+        if name and not any(name == tree or name.startswith(tree + '/') for tree in TREES) and os.path.isfile(name):
+            files.append(name)
+    return files
+
+
+def database_args(cfg):
+    """The command prefix and the database argument of pg_dump and pg_restore: postgres and vpn-router, or web.database."""
+    configured = cfg['web']['database'].strip()
+    if configured:
+        return [], [f'--dbname={configured}']
+    return ['runuser', '-u', 'postgres', '--'], [f'--dbname={vpnrouter_auth.DATABASE}']
+
+
+def stage(cfg):
+    """Copy what a backup holds to STAGING, emptied first, and return the manifest."""
+    shutil.rmtree(STAGING, ignore_errors=True)
+    STAGING.mkdir(parents=True, mode=0o700)
+    trees = [tree for tree in TREES if os.path.exists(tree)]
+    files = psk_files(cfg)
+    vpnrouter_users.LOCK_FILE.parent.mkdir(parents=True, exist_ok=True)
+    with vpnrouter.config_lock(), open(vpnrouter_users.LOCK_FILE, 'w') as users:
+        fcntl.flock(users, fcntl.LOCK_EX)
+        command(['cp', '-a', '--parents', *trees, str(STAGING)])
+        if files:
+            (STAGING / 'files').mkdir()
+            command(['cp', '-a', '--parents', *files, str(STAGING / 'files')])
+        (STAGING / LDIF).parent.mkdir()
+        command(['slapcat', '-b', vpnrouter_ldap.BASE, '-l', str(STAGING / LDIF)])
+    items = trees + [f'files{name}' for name in files] + [LDIF]
+    if cfg['backup']['metrics']:
+        (STAGING / DUMP).parent.mkdir()
+        prefix, database = database_args(cfg)
+        with open(STAGING / DUMP, 'wb') as dump:
+            result = subprocess.run([*prefix, 'pg_dump', '-Fc', *database], stdout=dump, stderr=subprocess.PIPE)
+        if result.returncode != 0:
+            raise Failure(f'pg_dump failed: {result.stderr.decode(errors="replace").strip()}')
+        items.append(DUMP)
+    version = subprocess.run(['dpkg-query', '-W', '-f=${Version}', 'vpn-router'], capture_output=True, text=True)
+    manifest = {'host': host(), 'time': now(), 'version': version.stdout.strip() or None,
+                'schedule': cfg['backup']['schedule'], 'items': items}
+    (STAGING / 'manifest.json').write_text(json.dumps(manifest, indent=2) + '\n', encoding='utf-8')
+    return manifest
+
+
+def pbs_environment(cfg):
+    """The environment of proxmox-backup-client: the repository, the token secret and the fingerprint of backup."""
+    if not PBS_CLIENT.exists():
+        raise Failure(f'{PBS_CLIENT} does not exist; install proxmox-backup-client-static')
+    backup = cfg['backup']
+    environment = dict(os.environ, PBS_REPOSITORY=backup['pbs_repository'], PBS_PASSWORD=backup['pbs_password'])
+    if backup['pbs_fingerprint']:
+        environment['PBS_FINGERPRINT'] = backup['pbs_fingerprint']
+    return environment
+
+
+def pbs_options(cfg):
+    """The namespace and key file options of proxmox-backup-client from backup."""
+    options = []
+    if cfg['backup']['pbs_namespace']:
+        options += ['--ns', cfg['backup']['pbs_namespace']]
+    if cfg['backup']['pbs_keyfile']:
+        options += ['--keyfile', cfg['backup']['pbs_keyfile']]
+    return options
+
+
+def cloud(cfg):
+    """Sync or copy the archives of this host in backup.path to backup.rclone_remote; Failure when it fails."""
+    backup = cfg['backup']
+    rclone = shutil.which('rclone')
+    if not rclone:
+        raise Failure('rclone is not installed')
+    if not RCLONE_CONFIG.is_file():
+        raise Failure(f'{RCLONE_CONFIG} does not exist; make the remote with rclone config --config {RCLONE_CONFIG}')
+    command([rclone, backup['rclone_mode'], backup['path'], backup['rclone_remote'], '--config', str(RCLONE_CONFIG),
+             '--include', f'vpn-router-{host()}-*.tar.zst'])
+
+
+def run():
+    """Make a backup to backup.target as the configuration says, recording it in STATE; the recorded state."""
+    cfg = vpnrouter.load_config()
+    backup = cfg['backup']
+    started = time.time()
+    record = {'result': 'running', 'started': now(), 'finished': None, 'target': backup['target'], 'archive': None,
+              'snapshot': None, 'size': None, 'removed': [], 'error': None, 'cloud': None}
+    with run_lock():
+        write_state(record)
+        try:
+            stage(cfg)
+            if backup['target'] == 'pbs':
+                epoch = int(started)
+                command([str(PBS_CLIENT), 'backup', f'{PBS_ARCHIVE}:{STAGING}', '--backup-type', 'host',
+                         '--backup-id', host(), '--backup-time', str(epoch), *pbs_options(cfg)],
+                        env=pbs_environment(cfg))
+                stamp = datetime.datetime.fromtimestamp(epoch, datetime.timezone.utc).strftime('%Y-%m-%dT%H:%M:%SZ')
+                record['snapshot'] = f'host/{host()}/{stamp}'
+                record['size'] = sum(path.stat().st_size for path in STAGING.rglob('*') if path.is_file())
+            else:
+                path = pathlib.Path(backup['path'])
+                path.mkdir(parents=True, mode=0o700, exist_ok=True)
+                name = f'vpn-router-{host()}-{time.strftime(STAMP, time.gmtime(started))}.tar.zst'
+                partial = path / f'{name}.partial'
+                command(['tar', '--zstd', '-cf', str(partial), '-C', str(STAGING), '.'])
+                partial.chmod(0o600)
+                partial.rename(path / name)
+                record['archive'] = name
+                record['size'] = (path / name).stat().st_size
+                found = archives(path)
+                keep = kept(found, {option: backup[option] for option in vpnrouter.backup_keeps(backup['schedule'])})
+                for item in found:
+                    if item['name'] not in keep:
+                        (path / item['name']).unlink()
+                        record['removed'].append(item['name'])
+                if backup['rclone_remote']:
+                    record['cloud'] = {'remote': backup['rclone_remote'], 'mode': backup['rclone_mode'],
+                                       'result': 'ok', 'error': None}
+                    try:
+                        cloud(cfg)
+                    except Failure as exc:
+                        record['cloud'].update(result='failed', error=str(exc))
+            record['result'] = 'warning' if record['cloud'] and record['cloud']['result'] != 'ok' else 'ok'
+        except (Failure, OSError) as exc:
+            record['result'] = 'failed'
+            record['error'] = str(exc)
+        finally:
+            shutil.rmtree(STAGING, ignore_errors=True)
+            record['finished'] = now()
+            write_state(record)
+    return record
+
+
+def snapshots(cfg):
+    """The PBS snapshots of this host, as proxmox-backup-client lists them."""
+    namespace = cfg['backup']['pbs_namespace']
+    return command([str(PBS_CLIENT), 'snapshot', 'list', f'host/{host()}', *(['--ns', namespace] if namespace else [])],
+                   env=pbs_environment(cfg))
+
+
+def extract(cfg, source, pbs):
+    """Unpack a local archive, or with pbs fetch a snapshot, to RESTORE_DIR, emptied first; the manifest."""
+    shutil.rmtree(RESTORE_DIR, ignore_errors=True)
+    RESTORE_DIR.mkdir(parents=True, mode=0o700)
+    if pbs:
+        command([str(PBS_CLIENT), 'restore', source, PBS_ARCHIVE, str(RESTORE_DIR), *pbs_options(cfg)],
+                env=pbs_environment(cfg))
+    else:
+        command(['tar', '--zstd', '-xf', source, '-C', str(RESTORE_DIR)])
+    try:
+        return json.loads((RESTORE_DIR / 'manifest.json').read_text(encoding='utf-8'))
+    except (OSError, ValueError):
+        raise Failure(f'{source} is not a backup of vpn-router-backup: it has no manifest.json')
+
+
+def put_back(relative, target, saved):
+    """Move target to saved, keeping its path, and copy the staged relative path of RESTORE_DIR in its place."""
+    target = pathlib.Path(target)
+    if target.exists() or target.is_symlink():
+        keep = saved / target.relative_to('/')
+        keep.parent.mkdir(parents=True, exist_ok=True)
+        shutil.move(target, keep)
+    target.parent.mkdir(parents=True, exist_ok=True)
+    command(['cp', '-a', str(RESTORE_DIR / relative), str(target)])
+
+
+def restore(source, pbs, confirm):
+    """Put the backup back: a local archive, or with pbs a snapshot; confirm(manifest) is asked before any change.
+
+    What is replaced is kept in BEFORE_RESTORE. The services of the directory and the database are stopped while they
+    are replaced, and setup applies the configuration afterwards. Returns the directory of what was replaced, or None
+    when confirm declined.
+    """
+    cfg = vpnrouter.load_config()
+    with run_lock():
+        try:
+            manifest = extract(cfg, source, pbs)
+            if not confirm(manifest):
+                return None
+            saved = BEFORE_RESTORE / time.strftime(STAMP, time.gmtime())
+            saved.mkdir(parents=True, mode=0o700)
+            for unit in RESTORE_STOPS:
+                subprocess.run(['systemctl', 'stop', unit], capture_output=True)
+            for item in manifest['items']:
+                if item.startswith('/'):
+                    put_back(item.lstrip('/'), item, saved)
+                elif item.startswith('files/'):
+                    put_back(item, item[len('files'):], saved)
+            if LDIF in manifest['items']:
+                directory = vpnrouter_ldap.DB_DIRECTORY
+                if directory.exists():
+                    shutil.move(directory, saved / 'ldap')
+                directory.mkdir(mode=0o700)
+                command(['slapadd', '-b', vpnrouter_ldap.BASE, '-l', str(RESTORE_DIR / LDIF)])
+                command(['chown', '-R', 'openldap:openldap', str(directory)])
+            subprocess.run(['systemctl', 'start', 'slapd.service'], capture_output=True)
+            restored = vpnrouter.load_config()
+            if not restored['web']['database'].strip():
+                secret = vpnrouter_auth.DATABASE_SECRET
+                if secret.is_file() and vpnrouter_auth.psql("SELECT 1 FROM pg_roles WHERE rolname = :'v1'",
+                                                            vpnrouter_auth.DATABASE):
+                    password = secret.read_text(encoding='utf-8').strip()
+                    vpnrouter_auth.psql(f"ALTER ROLE :\"v1\" PASSWORD '{password}'", vpnrouter_auth.DATABASE)
+                vpnrouter_auth.provision(restored)
+            if DUMP in manifest['items']:
+                prefix, database = database_args(restored)
+                with open(RESTORE_DIR / DUMP, 'rb') as dump:
+                    result = subprocess.run([*prefix, 'pg_restore', '--clean', '--if-exists', *database], stdin=dump,
+                                            stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
+                if result.returncode != 0:
+                    raise Failure(f'pg_restore failed: {result.stderr.decode(errors="replace").strip()}')
+            subprocess.run(['systemctl', 'start', 'vpn-router-web.service'], capture_output=True)
+            command(['systemctl', 'restart', 'vpn-router-setup.service'])
+            for unit in RESTORE_RESTARTS:
+                subprocess.run(['systemctl', 'try-restart', unit], capture_output=True)
+            subprocess.run(['ufw', 'reload'], capture_output=True)
+            return saved
+        finally:
+            shutil.rmtree(RESTORE_DIR, ignore_errors=True)

debian-package/src/usr/lib/vpn-router/vpnrouter_ldap.py

@@ -11,6 +11,9 @@ The password policy locks an account for LOCKOUT_SECONDS after LOCKOUT_FAILURES
 a successful bind clears the count. slapd's otp overlay is configured on the database, so a
 bind of a user with a TOTP token takes the password followed by a current code.
 cn=totp,ou=policies holds the TOTP parameters the token of every user points to.
+
+ou=pki holds the router's PKI (vpnrouter_pki): an entry per CA, and under it an entry per
+certificate the CA issued. Private keys are readable by root alone, the database's root DN.
 """
 
 import os
@@ -29,6 +32,7 @@ USERS = f'ou=users,{BASE}'
 GROUPS = f'ou=groups,{BASE}'
 SERVICES = f'ou=services,{BASE}'
 POLICIES = f'ou=policies,{BASE}'
+PKI = f'ou=pki,{BASE}'
 DEFAULT_POLICY = f'cn=default,{POLICIES}'
 TOTP_PARAMS = f'cn=totp,{POLICIES}'
 MANAGER = f'cn=user-manager,{SERVICES}'
@@ -50,6 +54,49 @@ LOCKOUT_SECONDS = 900
 #: The overlays of the database, in the order they are configured.
 OVERLAYS = ('ppolicy', 'otp')
 
+#: The package's own schema, under an OID of the UUID arc 2.25 (ITU-T X.667): attribute types
+#: under .1 and object classes under .2. vpnRouterAccount, auxiliary, adds properties of the
+#: package to a user of either kind: mail, for a Manager-only account, vpnRouterScope,
+#: authorization scopes, of which service marks a service account, and displayName and
+#: description, the labels of a service account. vpnRouterAuthority is a CA of the PKI: its
+#: certificate and CRL (cACertificate and certificateRevocationList of RFC 4523), its key, the
+#: CSR it waits with for the offline root, the number of its last CRL, where the CRL came from,
+#: and when it was retired. vpnRouterCertificate is a certificate a CA issued, named by its
+#: serial number: the certificate, the key the router keeps for its own and the users'
+#: certificates, its kind, template, names and expiry, and its revocation.
+SCHEMA_OID = '2.25.98477352595735834532404256531620013001'
+SCHEMA_NAME = 'vpn-router'
+_STRING = 'EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{256} SINGLE-VALUE'
+_TIME = 'EQUALITY generalizedTimeMatch ORDERING generalizedTimeOrderingMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.24 SINGLE-VALUE'
+_OCTETS = 'EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 SINGLE-VALUE'
+SCHEMA_ATTRIBUTE_TYPES = (
+    f"( {SCHEMA_OID}.1.1 NAME 'vpnRouterScope' DESC 'VPN router: an authorization scope of the account' "
+    "EQUALITY caseIgnoreMatch SUBSTR caseIgnoreSubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{256} )",
+    f"( {SCHEMA_OID}.1.2 NAME 'vpnRouterPrivateKey' DESC 'VPN router: a private key, PEM' {_OCTETS} )",
+    f"( {SCHEMA_OID}.1.3 NAME 'vpnRouterCertificateRequest' DESC 'VPN router: the CSR of a CA, PEM' {_OCTETS} )",
+    f"( {SCHEMA_OID}.1.4 NAME 'vpnRouterCRLNumber' DESC 'VPN router: the number of the last CRL of a CA' "
+    "EQUALITY integerMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.27 SINGLE-VALUE )",
+    f"( {SCHEMA_OID}.1.5 NAME 'vpnRouterCRLSource' DESC 'VPN router: signed or uploaded, where the CRL of a CA came from' {_STRING} )",
+    f"( {SCHEMA_OID}.1.6 NAME 'vpnRouterRetired' DESC 'VPN router: when a CA was retired' {_TIME} )",
+    f"( {SCHEMA_OID}.1.7 NAME 'vpnRouterKind' DESC 'VPN router: ca, router, user or external, the kind of a certificate' {_STRING} )",
+    f"( {SCHEMA_OID}.1.8 NAME 'vpnRouterTemplate' DESC 'VPN router: the template of a certificate' {_STRING} )",
+    f"( {SCHEMA_OID}.1.9 NAME 'vpnRouterSubjectAltName' DESC 'VPN router: a subject alternative name of a certificate' "
+    "EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{256} )",
+    f"( {SCHEMA_OID}.1.10 NAME 'vpnRouterNotAfter' DESC 'VPN router: when a certificate expires' {_TIME} )",
+    f"( {SCHEMA_OID}.1.11 NAME 'vpnRouterRevoked' DESC 'VPN router: when a certificate was revoked' {_TIME} )",
+    f"( {SCHEMA_OID}.1.12 NAME 'vpnRouterRevocationReason' DESC 'VPN router: why a certificate was revoked' {_STRING} )",
+)
+SCHEMA_OBJECT_CLASSES = (
+    f"( {SCHEMA_OID}.2.1 NAME 'vpnRouterAccount' DESC 'VPN router: properties of a user or a Manager-only account' "
+    "SUP top AUXILIARY MAY ( mail $ vpnRouterScope $ displayName $ description ) )",
+    f"( {SCHEMA_OID}.2.2 NAME 'vpnRouterAuthority' DESC 'VPN router: a CA of the PKI' SUP top STRUCTURAL MUST cn "
+    "MAY ( cACertificate $ certificateRevocationList $ vpnRouterPrivateKey $ vpnRouterCertificateRequest $ "
+    "vpnRouterCRLNumber $ vpnRouterCRLSource $ vpnRouterRetired ) )",
+    f"( {SCHEMA_OID}.2.3 NAME 'vpnRouterCertificate' DESC 'VPN router: a certificate a CA of the PKI issued' SUP top "
+    "STRUCTURAL MUST serialNumber MAY ( cn $ userCertificate $ vpnRouterPrivateKey $ vpnRouterKind $ vpnRouterTemplate $ "
+    "vpnRouterSubjectAltName $ vpnRouterNotAfter $ vpnRouterRevoked $ vpnRouterRevocationReason ) )",
+)
+
 #: The TOTP parameters: digits, the OID of HMAC-SHA1, seconds per time step, and the time steps accepted before and after now.
 TOTP_DIGITS = 6
 TOTP_ALGORITHM = '1.2.840.113549.2.7'
@@ -58,6 +105,7 @@ TOTP_WINDOW = 1
 
 #: The ACLs of the database, in order.
 ACCESS = (
+    'to attrs=vpnRouterPrivateKey by * none',
     f'to attrs=userPassword by dn.exact="{MANAGER}" =wx by self =wx by anonymous auth by * none',
     f'to attrs=oathSecret by dn.exact="{MANAGER}" =w by * none',
     f'to dn.subtree="{USERS}" by dn.exact="{MANAGER}" manage by * read',
@@ -109,6 +157,30 @@ def _ensure_modules(connection):
         connection.modify_s(dn, [(ldap.MOD_ADD, 'olcModuleLoad', [name.encode() for name in missing])])
 
 
+def _ensure_schema(connection):
+    """Add the package's schema to cn=config when missing, else add the definitions it lacks; a definition that differs
+    from the one of the same OID replaces the whole list of its kind."""
+    found = connection.search_s('cn=schema,cn=config', ldap.SCOPE_ONELEVEL, f'(cn=*{SCHEMA_NAME})',
+                                ['olcAttributeTypes', 'olcObjectClasses'])
+    attributes = {'olcAttributeTypes': list(SCHEMA_ATTRIBUTE_TYPES), 'olcObjectClasses': list(SCHEMA_OBJECT_CLASSES)}
+    if not found:
+        connection.add_s(f'cn={SCHEMA_NAME},cn=schema,cn=config', ldap.modlist.addModlist(
+            dict({name: [value.encode() for value in values] for name, values in attributes.items()},
+                 objectClass=[b'olcSchemaConfig'], cn=[SCHEMA_NAME.encode()])))
+        return
+    dn, current = found[0]
+    for name, wanted in attributes.items():
+        present = unordered(current.get(name, []))
+        oids = {value.split()[1] for value in present}
+        missing = [value for value in wanted if value not in present]
+        if not missing:
+            continue
+        if any(value.split()[1] in oids for value in missing):
+            connection.modify_s(dn, [(ldap.MOD_REPLACE, name, [value.encode() for value in wanted])])
+        else:
+            connection.modify_s(dn, [(ldap.MOD_ADD, name, [value.encode() for value in missing])])
+
+
 def _ensure_password_hash(connection):
     dn = 'olcDatabase={-1}frontend,cn=config'
     current = connection.search_s(dn, ldap.SCOPE_BASE, attrlist=['olcPasswordHash'])[0][1].get('olcPasswordHash', [])
@@ -188,15 +260,16 @@ def _ensure_manager(connection):
 
 
 def provision():
-    """Create or bring up to date the modules, database, ACLs, password policy, entries and service account."""
+    """Create or bring up to date the modules, the package's schema, database, ACLs, password policy, entries and service account."""
     connection = root()
     try:
         _ensure_modules(connection)
+        _ensure_schema(connection)
         _ensure_password_hash(connection)
         database = _ensure_database(connection)
         _ensure_overlays(connection, database)
         _ensure_entry(connection, BASE, {'objectClass': [b'dcObject', b'organization'], 'dc': [b'vpn-router'], 'o': [b'vpn-router']})
-        for unit in (USERS, GROUPS, SERVICES, POLICIES):
+        for unit in (USERS, GROUPS, SERVICES, POLICIES, PKI):
             _ensure_entry(connection, unit, {'objectClass': [b'organizationalUnit'], 'ou': [unit.split(',')[0][3:].encode()]})
         _ensure_entry(connection, DEFAULT_POLICY, {
             'objectClass': [b'organizationalRole', b'pwdPolicy'],

debian-package/src/usr/lib/vpn-router/vpnrouter_metrics.py

@@ -0,0 +1,207 @@
+"""The metrics of the router in PostgreSQL, for the metrics service and the web service.
+
+The metrics service writes one detail row per minute to the four detail tables, of TABLES:
+metrics_system (CPU, memory, disk and load averages), metrics_network (bytes per second
+received and sent per interface), metrics_vpn (clients per kind) and metrics_site_to_site (per
+site-to-site connection: up, 1 or 0, bytes and packets per second received and sent, and the
+seconds its IKE SA has been up). Each detail table has four
+summary tables, of LEVELS: <table>_hourly from the details, <table>_daily from the hourly,
+<table>_monthly from the daily and <table>_yearly from the monthly rows. A summary row has the
+start of its period, in UTC, and the average and the maximum of each value over the period: of
+the details for the hourly rows, else the average of the averages and the maximum of the maxima
+below. A period is summarized once it is complete; summarize() catches up every complete period
+after the last summarized one, so a stopped service loses no summary of what was written.
+
+collect_garbage() deletes the rows older than the retention of their level, metrics.keep_*;
+the service runs it on start and once a day. history() answers the views of the Metrics tab.
+
+The tables are in the database of the refresh tokens (vpnrouter_auth.connection_string) and are
+created when missing. Every failure of the database raises DatabaseError.
+"""
+
+import contextlib
+
+import psycopg
+
+import vpnrouter_auth
+
+#: The detail tables: their key columns besides time, and their value columns.
+TABLES = {
+    'metrics_system': ((), ('cpu', 'memory_used', 'memory_total', 'disk_free', 'disk_total', 'load1', 'load5', 'load15')),
+    'metrics_network': (('interface',), ('rx', 'tx')),
+    'metrics_vpn': (('kind',), ('clients',)),
+    'metrics_site_to_site': (('connection',), ('up', 'rx', 'tx', 'packets_rx', 'packets_tx', 'uptime')),
+}
+
+#: The summary levels, from the finest: the suffix of the tables, the period of date_trunc and the retention setting.
+LEVELS = (
+    ('hourly', 'hour', 'keep_hourly'),
+    ('daily', 'day', 'keep_daily'),
+    ('monthly', 'month', 'keep_monthly'),
+    ('yearly', 'year', 'keep_yearly'),
+)
+
+#: The kinds of VPN clients of metrics_vpn.
+VPN_KINDS = ('p2s', 'openvpn', 'wireguard')
+
+#: The time slots of the site-to-site availability of each view of history(): the summary level read (None: the
+#: details), the slot as PostgreSQL starts it from time, and its length.
+UP_SLOTS = {
+    'hour': (None, "date_bin('5 minutes', time, timestamptz '2000-01-01 00:00:00+00')", '5 minutes'),
+    'day': (None, "date_trunc('hour', time, 'UTC')", '1 hour'),
+    'month': ('hourly', "date_trunc('day', time, 'UTC')", '1 day'),
+    'year': ('monthly', "date_trunc('month', time, 'UTC')", '1 month'),
+}
+
+#: The views of history(): the summary level read (None: the details), the bucket and the span.
+RANGES = {
+    'hour': (None, '1 minute', '1 hour'),
+    'day': (None, '10 minutes', '1 day'),
+    'month': ('hourly', '6 hours', '1 month'),
+    'year': ('monthly', '1 month', '1 year'),
+}
+
+
+class DatabaseError(Exception):
+    """The metrics database failed."""
+
+
+def schema():
+    """The statements that create the detail and the summary tables."""
+    statements = []
+    for table, (keys, values) in TABLES.items():
+        key_columns = ''.join(f', {key} text NOT NULL' for key in keys)
+        primary = ', '.join(('time', *keys))
+        detail = ''.join(f', {value} double precision' for value in values)
+        statements.append(f'CREATE TABLE IF NOT EXISTS {table} (time timestamptz NOT NULL{key_columns}{detail}, '
+                          f'PRIMARY KEY ({primary}))')
+        summary = ''.join(f', {value}_avg double precision, {value}_max double precision' for value in values)
+        for suffix, _, _ in LEVELS:
+            statements.append(f'CREATE TABLE IF NOT EXISTS {table}_{suffix} (time timestamptz NOT NULL{key_columns}{summary}, '
+                              f'PRIMARY KEY ({primary}))')
+    return statements
+
+
+@contextlib.contextmanager
+def database(cfg):
+    """A connection to the metrics database, with its tables, closed at the end; DatabaseError on any failure."""
+    try:
+        with psycopg.connect(vpnrouter_auth.connection_string(cfg), autocommit=True) as connection:
+            for statement in schema():
+                connection.execute(statement)
+            yield connection
+    except (psycopg.Error, OSError) as exc:
+        raise DatabaseError(f'the metrics database failed: {exc}') from exc
+
+
+def write(connection, system, network, vpn, site_to_site):
+    """Write a sample as the detail rows of the current minute.
+
+    system maps the values of metrics_system to numbers; network maps an interface to its rx and
+    tx; vpn maps a kind of VPN_KINDS to its clients; site_to_site maps a connection to the values
+    of metrics_site_to_site. A value that is None is written as NULL.
+    """
+    with connection.transaction():
+        minute = connection.execute("SELECT date_trunc('minute', now(), 'UTC')").fetchone()[0]
+        names = TABLES['metrics_system'][1]
+        connection.execute(f"INSERT INTO metrics_system (time, {', '.join(names)}) "
+                           f"VALUES (%s{', %s' * len(names)}) ON CONFLICT DO NOTHING",
+                           (minute, *(system.get(name) for name in names)))
+        for interface, (rx, tx) in network.items():
+            connection.execute('INSERT INTO metrics_network (time, interface, rx, tx) VALUES (%s, %s, %s, %s) '
+                               'ON CONFLICT DO NOTHING', (minute, interface, rx, tx))
+        for kind, clients in vpn.items():
+            connection.execute('INSERT INTO metrics_vpn (time, kind, clients) VALUES (%s, %s, %s) ON CONFLICT DO NOTHING',
+                               (minute, kind, clients))
+        names = TABLES['metrics_site_to_site'][1]
+        for name, values in site_to_site.items():
+            connection.execute(f"INSERT INTO metrics_site_to_site (time, connection, {', '.join(names)}) "
+                               f"VALUES (%s, %s{', %s' * len(names)}) ON CONFLICT DO NOTHING",
+                               (minute, name, *(values.get(value) for value in names)))
+
+
+def summarize(connection, level):
+    """Summarize the complete periods of a level of LEVELS that are not summarized yet, for every detail table."""
+    index = [suffix for suffix, _, _ in LEVELS].index(level)
+    _, period, _ = LEVELS[index]
+    for table, (keys, values) in TABLES.items():
+        target = f'{table}_{level}'
+        source = table if index == 0 else f'{table}_{LEVELS[index - 1][0]}'
+        if index == 0:
+            aggregates = ', '.join(f'avg({value}), max({value})' for value in values)
+        else:
+            aggregates = ', '.join(f'avg({value}_avg), max({value}_max)' for value in values)
+        columns = ', '.join(('time', *keys, *(f'{value}_{kind}' for value in values for kind in ('avg', 'max'))))
+        groups = ''.join(f', {key}' for key in keys)
+        connection.execute(
+            f"INSERT INTO {target} ({columns}) "
+            f"SELECT date_trunc('{period}', time, 'UTC') AS period{groups}, {aggregates} FROM {source} "
+            f"WHERE time < date_trunc('{period}', now(), 'UTC') "
+            f"AND time >= coalesce((SELECT max(time) FROM {target}) + interval '1 {period}', '-infinity') "
+            f"GROUP BY period{groups} ON CONFLICT DO NOTHING")
+
+
+def collect_garbage(connection, cfg):
+    """Delete the rows older than the retention of their level: metrics.keep_details, keep_hourly and so on."""
+    metrics = cfg['metrics']
+    for table in TABLES:
+        connection.execute(f'DELETE FROM {table} WHERE time < now() - %s::interval', (metrics['keep_details'],))
+        for suffix, _, keep in LEVELS:
+            connection.execute(f'DELETE FROM {table}_{suffix} WHERE time < now() - %s::interval', (metrics[keep],))
+
+
+def history(connection, view):
+    """The points of a view of RANGES, oldest first, each with the start of its bucket in Unix seconds.
+
+    A point has the average and the maximum of each value of metrics_system, network, which maps
+    each interface to its rx and tx, clients, which maps each kind of VPN client to its clients,
+    and site_to_site, which maps each site-to-site connection to the values of
+    metrics_site_to_site; a value without data is None.
+
+    site_to_site_up is the availability of each site-to-site connection in the time slots of UP_SLOTS, oldest first:
+    for each slot its start in Unix seconds and, per connection, the percentage of its samples in which the connection
+    was up, 0 to 100; the summaries' averages of up give it for the month and the year.
+    """
+    level, bucket, span = RANGES[view]
+    up_level, up_start, up_slot = UP_SLOTS[view]
+
+    def rows(table):
+        keys, values = TABLES[table]
+        source = table if level is None else f'{table}_{level}'
+        if level is None:
+            aggregates = ', '.join(f'avg({value}), max({value})' for value in values)
+        else:
+            aggregates = ', '.join(f'avg({value}_avg), max({value}_max)' for value in values)
+        start = (f"date_bin('{bucket}', time, timestamptz '2000-01-01 00:00:00+00')" if view != 'year'
+                 else "date_trunc('month', time, 'UTC')")
+        groups = ''.join(f', {key}' for key in keys)
+        return connection.execute(
+            f"SELECT extract(epoch FROM {start})::bigint AS bucket{groups}, {aggregates} FROM {source} "
+            f"WHERE time >= now() - interval '{span}' GROUP BY bucket{groups} ORDER BY bucket").fetchall()
+
+    def pairs(row, values):
+        return {value: {'avg': row[2 * index], 'max': row[2 * index + 1]} for index, value in enumerate(values)}
+
+    points = {}
+
+    def point(start):
+        return points.setdefault(start, {'time': start, 'network': {}, 'clients': {}, 'site_to_site': {}})
+
+    for row in rows('metrics_system'):
+        point(row[0]).update(pairs(row[1:], TABLES['metrics_system'][1]))
+    for row in rows('metrics_network'):
+        point(row[0])['network'][row[1]] = pairs(row[2:], TABLES['metrics_network'][1])
+    for row in rows('metrics_vpn'):
+        point(row[0])['clients'][row[1]] = pairs(row[2:], TABLES['metrics_vpn'][1])['clients']
+    for row in rows('metrics_site_to_site'):
+        point(row[0])['site_to_site'][row[1]] = pairs(row[2:], TABLES['metrics_site_to_site'][1])
+    empty = {value: {'avg': None, 'max': None} for value in TABLES['metrics_system'][1]}
+
+    slots = {}
+    for start, name, percent in connection.execute(
+            f"SELECT extract(epoch FROM {up_start})::bigint AS slot, connection, avg({'up' if up_level is None else 'up_avg'}) * 100 "
+            f"FROM {'metrics_site_to_site' if up_level is None else f'metrics_site_to_site_{up_level}'} "
+            f"WHERE time >= now() - interval '{span}' GROUP BY slot, connection ORDER BY slot").fetchall():
+        slots.setdefault(start, {'time': start, 'connections': {}})['connections'][name] = percent
+    return {'range': view, 'bucket': bucket, 'points': [{**empty, **points[start]} for start in sorted(points)],
+            'site_to_site_up': {'slot': up_slot, 'points': [slots[start] for start in sorted(slots)]}}

debian-package/src/usr/lib/vpn-router/vpnrouter_net.py

@@ -1,11 +1,21 @@
 """Interfaces and addresses of the router, read from the system.
 
-Only the Python standard library is used, so the web service can use this module too.
+Only the Python standard library is used, so the web service and its bootstrap server can use
+this module too.
 """
 
 import pathlib
 import subprocess
 
+#: The values of web.listen and of the Router Manager's listen: the interfaces a service listens on.
+LISTEN_CHOICES = ('internal', 'external', 'both')
+
+#: The interfaces a service listens on when it is not set.
+DEFAULT_LISTEN = 'both'
+
+#: The address every service also listens on, where the HTTPS proxy calls it.
+LOOPBACK = '127.0.0.1'
+
 
 class ConfigError(Exception):
     """A setting is present but malformed."""
@@ -107,3 +117,60 @@ def internal_address(mode, internal, int_addr):
     except ConfigError:
         return ''
     return addresses[0] if addresses else ''
+
+
+def external_address(mode, external):
+    """The router's address on the external interface, as general.mode directs, or '' when it is not known yet.
+
+    auto works out which interface that is; the other modes read the address of interfaces.external.
+    """
+    try:
+        if mode == 'auto':
+            external = detect_interfaces()[0]
+        addresses = interface_addresses(external, 'interfaces.external')
+    except ConfigError:
+        return ''
+    return addresses[0] if addresses else ''
+
+
+def _section(configuration, name):
+    """A section of a configuration, or {} when it is missing or not an object."""
+    value = configuration.get(name)
+    return value if isinstance(value, dict) else {}
+
+
+def listen_addresses(configuration, choice):
+    """The addresses a service listens on for choice of LISTEN_CHOICES, from vpn-router.json read as configuration.
+
+    The internal address comes first; an address not known yet is left out. A choice that is not
+    one of LISTEN_CHOICES is DEFAULT_LISTEN. LOOPBACK is not among them; web_listen adds it.
+    """
+    if choice not in LISTEN_CHOICES:
+        choice = DEFAULT_LISTEN
+    general, interfaces, local = (_section(configuration, name) for name in ('general', 'interfaces', 'local'))
+    mode = str(general.get('mode') or 'manual')
+    addresses = []
+    if choice in ('internal', 'both'):
+        addresses.append(internal_address(mode, str(interfaces.get('internal') or ''), str(local.get('int_addr') or '')))
+    if choice in ('external', 'both'):
+        addresses.append(external_address(mode, str(interfaces.get('external') or '')))
+    return [address for address in addresses if address]
+
+
+def web_listen(configuration, default_port):
+    """Where the web service listens, from vpn-router.json read as configuration, as ([(address, port), ...], '').
+
+    The addresses are those of the interfaces web.listen names (listen_addresses), then
+    LOOPBACK, the port web.port, else default_port. While web.enabled is false or no address of
+    an interface is known, the answer is ([], the reason).
+    """
+    web = _section(configuration, 'web')
+    if web.get('enabled', True) is False:
+        return [], 'the web service is disabled in the configuration'
+    port = web.get('port', default_port)
+    if isinstance(port, bool) or not isinstance(port, int) or not 1 <= port <= 65535:
+        port = default_port
+    addresses = listen_addresses(configuration, web.get('listen', DEFAULT_LISTEN))
+    if not addresses:
+        return [], 'the addresses to listen on are not known yet'
+    return [(address, port) for address in [*addresses, LOOPBACK]], ''

debian-package/src/usr/lib/vpn-router/vpnrouter_serve.py

@@ -0,0 +1,46 @@
+"""The listening loop of the web service and of its bootstrap server.
+
+Only the Python standard library is used, so the bootstrap server, which must run whatever else
+is broken, can use it.
+"""
+
+import threading
+import time
+from http.server import ThreadingHTTPServer
+
+
+def serve(wanted, handler, log, interval):
+    """Work out where to listen every interval seconds with wanted, and run an HTTP server of handler at each place.
+
+    wanted answers ([(address, port), ...], '') or ([], the reason not to listen). A server whose
+    place is no longer wanted is stopped, and a new one listens at each new place; a place that
+    cannot be bound is tried again the next round. Every new message is logged with log. Does
+    not return.
+    """
+    servers, reported = {}, None
+    while True:
+        targets, reason = wanted()
+        for target in [target for target in servers if target not in targets]:
+            server, thread = servers.pop(target)
+            server.shutdown()
+            server.server_close()
+            thread.join()
+        messages = [reason] if reason else []
+        for target in targets:
+            if target in servers:
+                continue
+            try:
+                server = ThreadingHTTPServer(target, handler)
+            except OSError as exc:
+                messages.append(f'cannot listen on {target[0]}:{target[1]}: {exc}')
+                continue
+            thread = threading.Thread(target=server.serve_forever, name=f'server {target[0]}', daemon=True)
+            thread.start()
+            servers[target] = (server, thread)
+        if servers:
+            messages.append('serving on ' + ', '.join(f'http://{address}:{port}' for address, port in servers))
+        message = '; '.join(messages)
+        if message and message != reported:
+            log(message)
+            reported = message
+        time.sleep(interval)

debian-package/src/usr/lib/vpn-router/vpnrouter_status.py

@@ -0,0 +1,145 @@
+"""What the router's VPN systems report, for the web service and the metrics service.
+
+Each reader asks the running system once and answers None when it cannot be asked: swanctl for
+the IKE SAs and the state and traffic of the site-to-site connections, wg for the WireGuard
+peers and the OpenVPN server's status file for its clients.
+"""
+
+import subprocess
+import time
+
+COMMAND_TIMEOUT = 5
+
+OPENVPN_STATUS = '/run/openvpn-server/status-vpn-router.log'
+
+#: Seconds since its last handshake within which a WireGuard peer counts as connected.
+WIREGUARD_CONNECTED_SECONDS = 180
+
+
+def run(*args):
+    """The finished process of a command, or None when it cannot be run."""
+    try:
+        return subprocess.run(args, capture_output=True, text=True, timeout=COMMAND_TIMEOUT)
+    except (OSError, subprocess.TimeoutExpired):
+        return None
+
+
+def swanctl(command):
+    """The entries of a swanctl list command, or None when swanctl cannot be asked.
+
+    The --pretty output has one item per line: "name {" opens a section, "key = [" a list
+    whose items follow one per line, "}" and "]" close them, and "key = value" is a pair
+    whose value may contain spaces. Every "... event {" section at the top holds one
+    connection or SA; each becomes an entry with its name under "name". The closing
+    "... reply {" section is left out. Values stay strings, as swanctl prints them.
+    Output with more closing than opening lines gives None.
+    """
+    result = run('swanctl', command, '--pretty')
+    if result is None or result.returncode != 0:
+        return None
+    events = []
+    stack = []
+    for line in result.stdout.splitlines():
+        line = line.strip()
+        if not line:
+            continue
+        if line in ('}', ']'):
+            if not stack:
+                return None
+            stack.pop()
+        elif not stack:
+            section = {}
+            if line.endswith(' event {'):
+                events.append(section)
+            stack.append(section)
+        elif isinstance(stack[-1], list):
+            stack[-1].append(line)
+        elif line.endswith(' {'):
+            stack[-1][line[:-2]] = section = {}
+            stack.append(section)
+        elif line.endswith(' = ['):
+            stack[-1][line[:-4]] = items = []
+            stack.append(items)
+        else:
+            key, _, value = line.partition(' = ')
+            stack[-1][key] = value
+    return [dict(body, name=name) for event in events for name, body in event.items()]
+
+
+def established_each(connections):
+    """The number of established IKE SAs of each of the connections by name, None for each when swanctl cannot be asked."""
+    sas = swanctl('--list-sas')
+    return {name: None if sas is None else sum(1 for sa in sas if sa['name'] == name and sa.get('state') == 'ESTABLISHED')
+            for name in connections}
+
+
+def _count(text):
+    """A counter swanctl prints as a string, as an integer; 0 when it is not a whole number."""
+    return int(text) if isinstance(text, str) and text.isdigit() else 0
+
+
+def site_to_site_tunnels(connections):
+    """The state of each of the connections by name, or None when swanctl cannot be asked.
+
+    Each has up, 1 while it has an established IKE SA and 0 otherwise; uptime, the seconds since
+    its oldest established IKE SA was established, None while it is down; and bytes_in,
+    bytes_out, packets_in and packets_out, the sums over the child SAs of its established IKE
+    SAs. The sums start again from zero when a child SA is rekeyed.
+    """
+    sas = swanctl('--list-sas')
+    if sas is None:
+        return None
+    tunnels = {}
+    for name in connections:
+        up = [sa for sa in sas if sa['name'] == name and sa.get('state') == 'ESTABLISHED']
+        children = [child for sa in up for child in (sa.get('child-sas') or {}).values() if isinstance(child, dict)]
+        tunnels[name] = {
+            'up': 1 if up else 0,
+            'uptime': max((_count(sa.get('established')) for sa in up), default=None),
+            **{key.replace('-', '_'): sum(_count(child.get(key)) for child in children)
+               for key in ('bytes-in', 'bytes-out', 'packets-in', 'packets-out')},
+        }
+    return tunnels
+
+
+def established(connection):
+    """The number of established IKE SAs of the connection, or None when swanctl cannot be asked."""
+    return established_each([connection])[connection]
+
+
+def wireguard_peers():
+    """One entry per WireGuard peer, with its handshake age and traffic, or None when wg0 is down."""
+    result = run('wg', 'show', 'wg0', 'dump')
+    if result is None or result.returncode != 0:
+        return None
+    now = int(time.time())
+    peers = []
+    for line in result.stdout.splitlines()[1:]:
+        fields = line.split('\t')
+        if len(fields) < 7:
+            continue
+        handshake = int(fields[4])
+        peers.append({
+            'handshake_seconds_ago': now - handshake if handshake else None,
+            'received_bytes': int(fields[5]),
+            'sent_bytes': int(fields[6]),
+        })
+    return peers
+
+
+def wireguard_connected():
+    """The number of WireGuard peers with a handshake in the last WIREGUARD_CONNECTED_SECONDS, or None when wg0 is down."""
+    peers = wireguard_peers()
+    if peers is None:
+        return None
+    return sum(1 for peer in peers
+               if peer['handshake_seconds_ago'] is not None and peer['handshake_seconds_ago'] < WIREGUARD_CONNECTED_SECONDS)
+
+
+def openvpn_clients():
+    """The number of connected OpenVPN clients, from the server's status file, or None when it cannot be read."""
+    try:
+        with open(OPENVPN_STATUS, encoding='utf-8') as handle:
+            return sum(1 for line in handle if line.startswith('CLIENT_LIST,'))
+    except OSError:
+        return None

debian-package/src/usr/lib/vpn-router/vpnrouter_users.py

@@ -1,9 +1,19 @@
 """The users and groups of the router's directory, for vpn-router-user, vpn-router-group and the web service.
 
-A user is an entry in the router's directory, with a first name, a last name and an e-mail
-address, whose password OpenVPN checks, and a client certificate of the router CA with the
-user name as its common name, which OpenVPN and IKEv2 accept. Users exist whether or not a
-P2S system is enabled.
+A user is an entry in ou=users of the router's directory, of one of two kinds, fixed when it is
+added. A VPN user is an inetOrgPerson with a first name, a last name and an e-mail address, whose
+password OpenVPN checks, and a client certificate of the P2S issuing CA (vpnrouter_pki) with the
+user name as its common name, which OpenVPN and IKEv2 accept; the certificate is the entry's
+userCertificate, and the PKI keeps it with its key. A Manager-only account is an account,
+simpleSecurityObject and vpnRouterAccount (the package's schema) with a name, a password and
+optionally an e-mail address, no names, no certificate and so no VPN access; it is added when
+neither the first nor the last name is given, and can sign in to the web service. A service
+account is a Manager-only account with the scope SERVICE_SCOPE in vpnRouterScope, which another
+system signs in with to use a router service, and which cannot sign in to the web service; it has
+a display name and a description (labels), in displayName and description of vpnRouterAccount. The
+internal service accounts are those the router creates and manages for its own services, labels
+included (internal_accounts); vpn-router-setup keeps the metrics service's.
+Users exist whether or not a P2S system is enabled.
 
 A group is a groupOfNames in ou=groups with a name, which follows the rules of user names, an
 optional description and at least one member; a group cannot be empty, so the last member
@@ -19,11 +29,11 @@ code of it. slapd's otp overlay then takes the password followed by a code at ev
 A user is disabled when pwdAccountLockedTime holds LOCKED, and locked when the password
 policy has set it after failed binds, for vpnrouter_ldap.LOCKOUT_SECONDS; unlock ends that.
 login() checks a user's password and code with a bind. Members of ADMIN_GROUP manage the
-router through the web service. Changing the password, disabling, unlocking and removing a
-user revoke its refresh tokens.
+router through the web service. The module knows only the directory; the callers revoke a
+user's refresh tokens after the operations of vpnrouter_auth.REVOKING.
 
-Every operation runs in a session(), which holds the lock that serializes changes and the
-connection of the user manager account. A request that cannot be done raises Failure, whose
+Every operation runs in a session(), which holds the lock that serializes changes, the PKI's
+lock too, and the connection of the user manager account. A request that cannot be done raises Failure, whose
 status says why: 2 the request is invalid, 3 the user does not exist, 4 the user exists, 5 a
 login failed, 1 anything else. Answers are dicts with the keys of the JSON answers.
 """
@@ -32,10 +42,7 @@ import base64
 import calendar
 import contextlib
 import fcntl
-import pathlib
 import re
-import subprocess
-import sys
 import time
 
 import ldap
@@ -44,8 +51,8 @@ import ldap.modlist
 import pyotp
 
 import vpnrouter
-import vpnrouter_auth
 import vpnrouter_ldap
+import vpnrouter_pki
 
 #: A user name is one lowercase DNS label, as it is also the common name of the certificate.
 NAME = re.compile(r'^[a-z][a-z0-9-]{0,30}[a-z0-9]$|^[a-z]$')
@@ -58,8 +65,16 @@ EMAIL = re.compile(r'^[^@\s]+@[^@\s]+\.[^@\s]+$')
 EMAIL_MAX = 254
 #: The keys of the person attributes and their LDAP attributes.
 PERSON = (('first_name', 'givenName'), ('last_name', 'sn'), ('email', 'mail'))
+#: The keys of the labels of a service account and their LDAP attributes.
+LABELS = (('display_name', 'displayName'), ('description', 'description'))
 #: The attributes read for a user.
-ATTRIBUTES = ['uid', 'pwdAccountLockedTime', 'givenName', 'sn', 'mail', 'oathTOTPToken']
+ATTRIBUTES = ['uid', 'objectClass', 'pwdAccountLockedTime', 'givenName', 'sn', 'mail', 'oathTOTPToken', 'vpnRouterScope',
+              'displayName', 'description', 'userCertificate;binary']
+#: The vpnRouterScope of a service account.
+SERVICE_SCOPE = 'service'
+#: The object classes of a VPN user and of a Manager-only account.
+VPN_USER_CLASSES = [b'inetOrgPerson']
+MANAGER_ONLY_CLASSES = [b'account', b'simpleSecurityObject', b'vpnRouterAccount']
 #: The readable attributes of a TOTP token; oathSecret is never readable.
 TOKEN_ATTRIBUTES = ['oathTOTPToken', 'oathTOTPParams', 'oathSecretTime', 'oathTOTPLastTimeStep', 'oathTOTPTimeStepDrift',
                     'oathLastLogin', 'oathLastFailure', 'oathFailureCount']
@@ -75,7 +90,7 @@ LOCKED = b'000001010000Z'
 ADMIN_GROUP = 'admins'
 #: The digits of a TOTP code given to login().
 CODE = re.compile(r'^\d{6}$')
-LOCK_FILE = pathlib.Path('/run/vpn-router/users.lock')
+LOCK_FILE = vpnrouter_pki.LOCK_FILE
 
 
 class Failure(Exception):
@@ -135,13 +150,30 @@ def person(values, required):
     return found
 
 
+def labels(values):
+    """The labels of a service account in values ({display_name, description}), checked, as {LDAP attribute: value}.
+
+    A key whose value is None is not given; '' means none. A display name has at most PERSON_NAME_MAX printable
+    characters and a description at most DESCRIPTION_MAX.
+    """
+    found = {}
+    name = values.get('display_name')
+    if name is not None:
+        if not isinstance(name, str):
+            raise Failure('display_name must be a string', 2)
+        name = name.strip()
+        if len(name) > PERSON_NAME_MAX or any(not character.isprintable() for character in name):
+            raise Failure(f'the display name must have at most {PERSON_NAME_MAX} printable characters', 2)
+        found['displayName'] = name
+    if values.get('description') is not None:
+        found['description'] = check_description(values['description'])
+    return found
+
+
 def dn_of(name):
     return f'uid={name},{vpnrouter_ldap.USERS}'
 
 
-def certificate(name):
-    return vpnrouter.USER_CERT_DIR / f'{name}_cert.pem', vpnrouter.USER_CERT_DIR / f'{name}_key.pem'
-
 
 def entry(connection, name):
     """The attributes of the user's entry; Failure with status 3 when there is none."""
@@ -152,18 +184,6 @@ def entry(connection, name):
         raise Failure(f'user {name} does not exist', 3)
 
 
-def simple_ca(*args):
-    result = subprocess.run([sys.executable, str(vpnrouter.SIMPLE_CA), *args], capture_output=True, text=True)
-    if result.returncode != 0:
-        raise Failure(f'simple-ca {args[0]} failed: {result.stderr.strip() or result.stdout.strip()}')
-
-
-def publish_crl():
-    """Give strongSwan the CRL after a revocation; OpenVPN reads it at every connection."""
-    if vpnrouter.install_file(vpnrouter.PKI_DIR / 'crl.pem', vpnrouter.CRL_DEST, 0o644):
-        subprocess.run(['swanctl', '--load-creds'], capture_output=True, text=True)
-
-
 def locked_since(value):
     """The time a pwdAccountLockedTime value names, in seconds since the epoch, or 0 when it names none."""
     try:
@@ -172,18 +192,50 @@ def locked_since(value):
         return 0
 
 
+def manager_only(attributes):
+    """Whether the entry is a Manager-only account."""
+    return b'account' in (value.lower() for value in attributes.get('objectClass', []))
+
+
+def service(attributes):
+    """Whether the entry is a service account."""
+    return SERVICE_SCOPE.encode() in (value.lower() for value in attributes.get('vpnRouterScope', []))
+
+
+def internal_accounts():
+    """The internal service accounts, which the router creates and manages for its own services, by name, each with
+    the labels the router gives it ({display_name, description}): the one metrics.service_account names while metrics
+    are enabled."""
+    try:
+        metrics = vpnrouter.load_config()['metrics']
+    except SystemExit:
+        return {}
+    if not metrics['enabled']:
+        return {}
+    return {metrics['service_account']: {
+        'display_name': 'Metrics',
+        'description': 'The account an OpenTelemetry Collector signs in with to read /metrics of the metrics service.',
+    }}
+
+
 def summary(attributes):
-    """A user as the answers show it; a person attribute the entry does not have is None."""
+    """A user as the answers show it; a person attribute or a label the entry does not have is None.
+
+    internal is whether it is an internal service account (internal_accounts).
+    """
     name = attributes['uid'][0].decode()
     lock = attributes.get('pwdAccountLockedTime', [None])[0]
     user = {
         'name': name,
+        'manager_only': manager_only(attributes),
+        'service': service(attributes),
+        'internal': service(attributes) and name in internal_accounts(),
         'enabled': lock != LOCKED,
         'locked': lock not in (None, LOCKED) and locked_since(lock) > time.time() - vpnrouter_ldap.LOCKOUT_SECONDS,
-        'certificate': certificate(name)[0].is_file(),
+        'certificate': 'userCertificate;binary' in attributes,
         'otp': 'oathTOTPToken' in attributes,
     }
-    for key, attribute in PERSON:
+    for key, attribute in PERSON + LABELS:
         values = attributes.get(attribute)
         user[key] = values[0].decode() if values else None
     return user
@@ -196,38 +248,73 @@ def common_name(attributes, name):
 
 
 def list_users(connection):
-    found = connection.search_s(vpnrouter_ldap.USERS, ldap.SCOPE_ONELEVEL, '(objectClass=inetOrgPerson)', ATTRIBUTES)
+    found = connection.search_s(vpnrouter_ldap.USERS, ldap.SCOPE_ONELEVEL, '(|(objectClass=inetOrgPerson)(objectClass=account))',
+                                ATTRIBUTES)
     return {'users': sorted((summary(attributes) for _, attributes in found), key=lambda user: user['name'])}
 
 
-def add(connection, name, password, values):
-    """Add a user with the person attributes of values, all three required, and issue the certificate."""
+def add(connection, name, password, values, service=False):
+    """Add a user: a VPN user with its certificate when values gives all three person attributes, a Manager-only account
+    with at most the e-mail address when it gives neither name, and, with service, a service account, with at most the
+    e-mail address and its labels (labels)."""
     check_name(name)
-    details = person(values, required=True)
+    details = person(values, required=False)
+    vpn_user = 'givenName' in details or 'sn' in details
+    if service and vpn_user:
+        raise Failure('a service account has no first or last name', 2)
+    if service:
+        details.update({attribute: value for attribute, value in labels(values).items() if value})
+    if vpn_user and len(details) != len(PERSON):
+        raise Failure('give first_name, last_name and email for a VPN user, or at most email for a Manager-only account', 2)
     check_password(password)
-    if not (vpnrouter.PKI_DIR / 'ca_key.pem').is_file():
-        raise Failure('the router CA does not exist; vpn-router-setup creates it once a platform is chosen')
-    record = {'objectClass': [b'inetOrgPerson'], 'uid': [name.encode()],
-              'cn': [common_name(details, name).encode()]}
+    if vpn_user and not vpnrouter_pki.ready('p2s'):
+        raise Failure('the P2S issuing CA has no certificate yet; vpn-router-setup creates it once a platform is chosen, '
+                      'or, while the root CA is offline, once its certificate is imported')
+    if vpn_user:
+        record = {'objectClass': VPN_USER_CLASSES, 'uid': [name.encode()], 'cn': [common_name(details, name).encode()]}
+    else:
+        # simpleSecurityObject requires a userPassword; passwd_s sets the real one at once.
+        record = {'objectClass': MANAGER_ONLY_CLASSES, 'uid': [name.encode()], 'userPassword': [b'{CRYPT}!']}
+        if service:
+            record['vpnRouterScope'] = [SERVICE_SCOPE.encode()]
     record.update({attribute: [value.encode()] for attribute, value in details.items()})
     try:
         connection.add_s(dn_of(name), ldap.modlist.addModlist(record))
     except ldap.ALREADY_EXISTS:
         raise Failure(f'user {name} exists', 4)
     connection.passwd_s(dn_of(name), None, password)
-    vpnrouter.USER_CERT_DIR.mkdir(mode=0o700, parents=True, exist_ok=True)
-    for path in certificate(name):
-        path.unlink(missing_ok=True)
-    simple_ca('make-cert', '--ca-dir', str(vpnrouter.PKI_DIR), '--cert-dir', str(vpnrouter.USER_CERT_DIR), name)
+    if vpn_user:
+        try:
+            vpnrouter_pki.issue_user(vpnrouter.load_config(), name, details['mail'])
+        except vpnrouter_pki.Failure as failure:
+            raise Failure(f'the certificate cannot be issued: {failure}', failure.status)
     return summary(entry(connection, name))
 
 
 def update(connection, name, values):
-    """Change the person attributes values gives; the cn follows the first and the last name."""
+    """Change the person attributes values gives; the cn follows the first and the last name.
+
+    A Manager-only account has only the e-mail address, which '' removes; a service account also its labels
+    (update_service).
+    """
+    current = entry(connection, name)
+    if service(current):
+        return update_service(connection, name, values)
+    if manager_only(current):
+        if values.get('first_name') is not None or values.get('last_name') is not None:
+            raise Failure(f'user {name} is a Manager-only account, which has no first or last name', 2)
+        email = values.get('email')
+        if email is None:
+            raise Failure('give email', 2)
+        if isinstance(email, str) and email.strip() == '':
+            if 'mail' in current:
+                connection.modify_s(dn_of(name), [(ldap.MOD_DELETE, 'mail', None)])
+        else:
+            connection.modify_s(dn_of(name), [(ldap.MOD_REPLACE, 'mail', [person({'email': email}, required=False)['mail'].encode()])])
+        return summary(entry(connection, name))
     details = person(values, required=False)
     if not details:
         raise Failure('give at least one of first_name, last_name and email', 2)
-    current = entry(connection, name)
     names = {attribute: current[attribute][0].decode() for attribute in ('givenName', 'sn') if current.get(attribute)}
     names.update({attribute: details[attribute] for attribute in ('givenName', 'sn') if attribute in details})
     changes = [(ldap.MOD_REPLACE, attribute, [value.encode()]) for attribute, value in details.items()]
@@ -236,18 +323,31 @@ def update(connection, name, values):
     return summary(entry(connection, name))
 
 
+def update_service(connection, name, values):
+    """Change any of the e-mail address and the labels (labels) of a service account that values gives; '' removes one."""
+    if not service(entry(connection, name)):
+        raise Failure(f'{name} is not a service account', 3)
+    changes = [(ldap.MOD_REPLACE, attribute, [value.encode()] if value else []) for attribute, value in labels(values).items()]
+    email = values.get('email')
+    if email is not None:
+        address = '' if isinstance(email, str) and email.strip() == '' else person({'email': email}, required=False)['mail']
+        changes.append((ldap.MOD_REPLACE, 'mail', [address.encode()] if address else []))
+    if not changes:
+        raise Failure('give email, display_name or description', 2)
+    connection.modify_s(dn_of(name), changes)
+    return summary(entry(connection, name))
+
+
 def passwd(connection, name, password):
     check_password(password)
     entry(connection, name)
     connection.passwd_s(dn_of(name), None, password)
-    vpnrouter_auth.revoke(vpnrouter.load_config(), name)
     return summary(entry(connection, name))
 
 
 def disable(connection, name):
     if entry(connection, name).get('pwdAccountLockedTime') != [LOCKED]:
         connection.modify_s(dn_of(name), [(ldap.MOD_REPLACE, 'pwdAccountLockedTime', [LOCKED])])
-    vpnrouter_auth.revoke(vpnrouter.load_config(), name)
     return summary(entry(connection, name))
 
 
@@ -266,7 +366,6 @@ def unlock(connection, name):
         changes.append((ldap.MOD_DELETE, 'pwdAccountLockedTime', None))
     if changes:
         connection.modify_s(dn_of(name), changes)
-    vpnrouter_auth.revoke(vpnrouter.load_config(), name)
     return summary(entry(connection, name))
 
 
@@ -353,27 +452,24 @@ def remove(connection, name):
         raise Failure(f'user {name} is the last member of {", ".join(alone)}; remove the group first', 2)
     for group in groups:
         connection.modify_s(group_dn(group), [(ldap.MOD_DELETE, 'member', [dn_of(name).encode()])])
-    cert, key = certificate(name)
-    if cert.is_file():
-        simple_ca('revoke-cert', '--ca-dir', str(vpnrouter.PKI_DIR), str(cert))
-        simple_ca('make-crl', '--ca-dir', str(vpnrouter.PKI_DIR), '--days', str(vpnrouter.CRL_DAYS))
-        publish_crl()
-    cert.unlink(missing_ok=True)
-    key.unlink(missing_ok=True)
+    try:
+        vpnrouter_pki.revoke_user(vpnrouter.load_config(), name)
+    except vpnrouter_pki.Failure as failure:
+        raise Failure(f'the certificate cannot be revoked: {failure}', failure.status)
     connection.delete_s(dn_of(name))
-    vpnrouter_auth.revoke(vpnrouter.load_config(), name)
     return {'name': name, 'removed': True}
 
 
 def profile(connection, name):
-    """The OpenVPN client profile of the user, with the CA, the certificate and the key inline."""
+    """The OpenVPN client profile of the user, with the root and the P2S CA, the certificate and the key inline."""
     entry(connection, name)
     cfg = vpnrouter.load_config()
     if not cfg['openvpn']['enabled']:
         raise Failure('OpenVPN is not enabled', 2)
-    cert, key = certificate(name)
-    if not cert.is_file() or not key.is_file():
+    credentials = vpnrouter_pki.user_credentials(name)
+    if credentials is None:
         raise Failure(f'user {name} has no certificate')
+    cert, key = (value.decode() for value in credentials)
     fqdn = cfg['wan']['local_fqdn'].strip()
     text = '\n'.join([
         'client',
@@ -388,9 +484,9 @@ def profile(connection, name):
         'auth-user-pass',
         'auth-nocache',
         'verb 3',
-        '<ca>', extract_pem(vpnrouter.CA_CERT_DEST.read_text(encoding='utf-8')), '</ca>',
-        '<cert>', extract_pem(cert.read_text(encoding='utf-8')), '</cert>',
-        '<key>', key.read_text(encoding='utf-8').strip(), '</key>',
+        '<ca>', extract_pem(vpnrouter_pki.OPENVPN_CA.read_text(encoding='utf-8')), '</ca>',
+        '<cert>', extract_pem(cert), '</cert>',
+        '<key>', key.strip(), '</key>',
         '',
     ])
     return {'name': name, 'profile': text}

debian-package/src/usr/lib/vpn-router/wait-for-libpq

@@ -0,0 +1,12 @@
+#!/bin/sh
+# Wait until libpq is in the dynamic linker cache, where psycopg looks it up.
+# ExecStartPre= of vpn-router-web.service; TimeoutStartSec= of the unit bounds the wait.
+
+reported=
+until ldconfig -p | grep -q '^[[:space:]]*libpq\.so\.5 '; do
+    if [ -z "$reported" ]; then
+        echo 'waiting for libpq.so.5 in the linker cache'
+        reported=yes
+    fi
+    sleep 2
+done

debian-package/src/usr/sbin/vpn-config-edit

@@ -0,0 +1,132 @@
+#!/usr/bin/python3
+"""Show and change the router's configuration, /etc/vpn-router/vpn-router.json.
+
+show prints the file as indented JSON; the file is not changed. set takes a setting as
+section.option, such as web.port, or as site_to_site.NAME.option for a site-to-site connection,
+which it creates with the defaults (vpnrouter.CONNECTION) when it does not exist, and a value,
+which it converts to the type the package's schema (vpnrouter.SCHEMA) gives the setting: a whole
+number, true or false, a list of strings separated by commas (an empty value is an empty list),
+or a string. remove takes site_to_site.NAME and removes that connection. The configuration with
+the change is checked as vpn-router-setup checks it, and only then written with
+vpnrouter.save_config, holding vpnrouter.config_lock; vpn-router-setup applies it at its next
+run, which set and remove remind of. A failure is a message on standard error; the exit status
+is 2 for an invalid request and 1 for any other failure. -h or --help shows the options.
+
+  vpn-config-edit show
+  vpn-config-edit set SECTION.OPTION VALUE
+  vpn-config-edit set site_to_site.NAME.OPTION VALUE
+  vpn-config-edit remove site_to_site.NAME
+"""
+
+import argparse
+import copy
+import json
+import sys
+
+sys.path.insert(0, '/usr/lib/vpn-router')
+
+import vpnrouter
+
+
+class Failure(Exception):
+    """A request that cannot be done; status is 2 invalid, 1 other."""
+
+    def __init__(self, message, status=1):
+        super().__init__(message)
+        self.status = status
+
+
+class Parser(argparse.ArgumentParser):
+    """An argument parser whose errors are Failures with status 2."""
+
+    def error(self, message):
+        raise Failure(f'{message}; see {self.prog} --help', 2)
+
+
+def read():
+    """The settings in the configuration file, without the defaults; {} when the file does not exist."""
+    try:
+        return vpnrouter.read_settings()
+    except vpnrouter.ConfigError as exc:
+        raise Failure(str(exc))
+
+
+def converted(key, text):
+    """The path of the setting key, section.option or site_to_site.NAME.option, and text as the type the schema gives it."""
+    path = key.split('.')
+    if path[0] == 'site_to_site' and len(path) == 3 and path[2] in vpnrouter.CONNECTION:
+        kind = type(vpnrouter.CONNECTION[path[2]])
+    elif path[0] != 'site_to_site' and len(path) == 2 and path[0] in vpnrouter.SCHEMA and path[1] in vpnrouter.SCHEMA[path[0]]:
+        kind = type(vpnrouter.SCHEMA[path[0]][path[1]])
+    else:
+        raise Failure(f'{key}: unknown setting; settings are section.option, such as web.port, '
+                      'or site_to_site.NAME.option', 2)
+    if kind is bool:
+        if text not in ('true', 'false'):
+            raise Failure(f'{key} takes true or false', 2)
+        return path, text == 'true'
+    if kind is int:
+        try:
+            return path, int(text)
+        except ValueError:
+            raise Failure(f'{key} takes a whole number', 2)
+    if kind is list:
+        return path, [item.strip() for item in text.split(',') if item.strip()]
+    return path, text
+
+
+def main(argv):
+    try:
+        parser = Parser(prog='vpn-config-edit', description='Show and change the router configuration.')
+        commands = parser.add_subparsers(dest='command', required=True, parser_class=Parser)
+        commands.add_parser('show', help='print the configuration as indented JSON')
+        setter = commands.add_parser('set', help='set a setting and write the configuration')
+        setter.add_argument('key', metavar='SECTION.OPTION',
+                            help='the setting, such as web.port or site_to_site.NAME.option')
+        setter.add_argument('value', metavar='VALUE', help='its value; a list separated by commas')
+        remover = commands.add_parser('remove', help='remove a site-to-site connection and write the configuration')
+        remover.add_argument('key', metavar='site_to_site.NAME', help='the connection')
+        options = parser.parse_args(argv)
+
+        if options.command == 'show':
+            print(json.dumps(read(), indent=2))
+            return 0
+
+        with vpnrouter.config_lock():
+            data = read()
+            if options.command == 'set':
+                path, value = converted(options.key, options.value)
+                if path[0] == 'site_to_site':
+                    connections = data.setdefault('site_to_site', {})
+                    connection = connections.setdefault(path[1], copy.deepcopy(vpnrouter.CONNECTION))
+                    if not isinstance(connection, dict):
+                        raise Failure(f'site_to_site.{path[1]} is not an object')
+                    connection[path[2]] = value
+                else:
+                    data.setdefault(path[0], {})[path[1]] = value
+            else:
+                section, _, name = options.key.partition('.')
+                if section != 'site_to_site' or not name or name not in data.get('site_to_site', {}):
+                    raise Failure(f'{options.key}: no such site-to-site connection', 2)
+                del data['site_to_site'][name]
+            try:
+                vpnrouter.validate(vpnrouter.with_defaults(data))
+            except vpnrouter.ConfigError as exc:
+                raise Failure(str(exc), 2)
+            vpnrouter.save_config(data)
+    except Failure as failure:
+        print(f'vpn-config-edit: {failure}', file=sys.stderr)
+        return failure.status
+    except OSError as exc:
+        print(f'vpn-config-edit: {exc}', file=sys.stderr)
+        return 1
+    if options.command == 'set':
+        print(f'{options.key} set to {json.dumps(value)} in {vpnrouter.CONFIG_FILE}.')
+    else:
+        print(f'{options.key} removed from {vpnrouter.CONFIG_FILE}.')
+    print('Apply it with: systemctl restart vpn-router-setup')
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))

debian-package/src/usr/sbin/vpn-router-backup

@@ -0,0 +1,98 @@
+#!/usr/bin/python3
+"""Make, list and restore backups of the router (vpnrouter_backup).
+
+run makes a backup as backup in /etc/vpn-router/vpn-router.json says, whether or not backups are
+enabled, and prints its result; vpn-router-backup.service runs it on the schedule of its timer.
+list prints the local archives of this host in backup.path, newest first, or with the pbs target
+the snapshots of this host on the Proxmox Backup Server. restore puts back a local archive, or
+with --pbs a snapshot, such as host/router/2026-10-25T03:00:00Z, after showing what it holds and
+asking for yes, which --yes gives; what it replaces is kept in
+/var/lib/vpn-router/backup/before-restore. A failure is a message on standard error; the exit
+status is 2 for an invalid request and 1 for any other failure. -h or --help shows the options.
+
+  vpn-router-backup run
+  vpn-router-backup list
+  vpn-router-backup restore [--yes] ARCHIVE
+  vpn-router-backup restore [--yes] --pbs SNAPSHOT
+"""
+
+import argparse
+import os
+import sys
+
+sys.path.insert(0, '/usr/lib/vpn-router')
+
+import vpnrouter
+import vpnrouter_backup
+
+
+class Parser(argparse.ArgumentParser):
+    """An argument parser whose errors end with status 2."""
+
+    def error(self, message):
+        print(f'{self.prog}: {message}; see {self.prog} --help', file=sys.stderr)
+        sys.exit(2)
+
+
+def confirmed(manifest, yes):
+    """Whether to restore the backup of the manifest: yes, or the answer yes to the question."""
+    print(f'Backup of {manifest.get("host")} made {manifest.get("time")} by vpn-router {manifest.get("version")}:')
+    for item in manifest.get('items', []):
+        print(f'  {item}')
+    if yes:
+        return True
+    return input('Replace the router state with it? Type yes: ').strip() == 'yes'
+
+
+def main(argv):
+    parser = Parser(prog='vpn-router-backup', description='Make, list and restore backups of the router.')
+    commands = parser.add_subparsers(dest='command', required=True, parser_class=Parser)
+    commands.add_parser('run', help='make a backup now')
+    commands.add_parser('list', help='list the backups of this host')
+    restorer = commands.add_parser('restore', help='put a backup back')
+    restorer.add_argument('source', metavar='ARCHIVE|SNAPSHOT', help='a local archive, or with --pbs a snapshot')
+    restorer.add_argument('--pbs', action='store_true', help='restore a snapshot of the Proxmox Backup Server')
+    restorer.add_argument('--yes', action='store_true', help='do not ask before replacing the router state')
+    options = parser.parse_args(argv)
+    os.umask(0o077)
+
+    try:
+        if options.command == 'run':
+            record = vpnrouter_backup.run()
+            if record['result'] == 'failed':
+                print(f'vpn-router-backup: {record["error"]}', file=sys.stderr)
+                return 1
+            print(f'Backup {record["archive"] or record["snapshot"]} made, {record["size"]} bytes.')
+            for name in record['removed']:
+                print(f'Removed {name}.')
+            if record['cloud']:
+                print(f'Cloud copy to {record["cloud"]["remote"]}: {record["cloud"]["error"] or "ok"}.')
+            return 0
+        cfg = vpnrouter.load_config()
+        if options.command == 'list':
+            if cfg['backup']['target'] == 'pbs':
+                print(vpnrouter_backup.snapshots(cfg), end='')
+            else:
+                for item in vpnrouter_backup.archives(cfg['backup']['path']):
+                    print(f'{item["name"]}  {item["size"]}')
+            return 0
+        if options.pbs and options.source.startswith('/'):
+            print('vpn-router-backup: a snapshot is type/id/time, such as host/router/2026-10-25T03:00:00Z',
+                  file=sys.stderr)
+            return 2
+        saved = vpnrouter_backup.restore(options.source, options.pbs, lambda manifest: confirmed(manifest, options.yes))
+    except vpnrouter_backup.Failure as exc:
+        print(f'vpn-router-backup: {exc}', file=sys.stderr)
+        return 1
+    except OSError as exc:
+        print(f'vpn-router-backup: {exc}', file=sys.stderr)
+        return 1
+    if saved is None:
+        print('Nothing restored.')
+        return 1
+    print(f'Restored; what it replaced is in {saved}.')
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))

debian-package/src/usr/sbin/vpn-router-ca

@@ -0,0 +1,212 @@
+#!/usr/bin/python3
+"""Manage the router's PKI, which the router's directory holds, from the command line, with vpnrouter_pki.
+
+status shows the root, the issuing CAs and the warnings; list the certificates the router issued. issue signs a CSR,
+or makes a key and writes a PFX file whose password is read from the first line of standard input. ca csr prints the
+CSR of an issuing CA that waits for the offline root, with the openssl extensions to sign it with, and ca import takes
+the signed certificate. root export-key writes the root key encrypted with the passphrase read from standard input,
+root remove-key removes it from the router, root import-key puts it back, and root import-crl takes a root CRL signed
+offline. maintain signs the stale CRLs again; vpn-router-pki.service runs it daily. A change that needs
+vpn-router-setup restarts it without waiting.
+
+The answer is text for people on standard output, and a failure is a message on standard error. With --json the answer
+is JSON on standard output, and a failure is {"error": ...} there too. The exit status says what failed: 2 the request
+is invalid, 3 something it names does not exist, 4 the state does not allow it, 1 anything else. -h or --help shows the
+commands and their options.
+
+  vpn-router-ca [--json] status
+  vpn-router-ca [--json] list [--ca NAME]
+  vpn-router-ca [--json] issue --ca NAME --template NAME --csr FILE
+  vpn-router-ca [--json] issue --ca NAME --template NAME --subject NAME [--san NAME ...] --pfx FILE   (password on stdin)
+  vpn-router-ca [--json] revoke SERIAL [--reason REASON]
+  vpn-router-ca [--json] maintain
+  vpn-router-ca [--json] ca csr NAME
+  vpn-router-ca [--json] ca import NAME FILE
+  vpn-router-ca [--json] root export-key --out FILE   (passphrase on stdin)
+  vpn-router-ca [--json] root remove-key [--yes]
+  vpn-router-ca [--json] root import-key FILE   (passphrase on stdin, an empty line for none)
+  vpn-router-ca [--json] root import-crl FILE
+"""
+
+import argparse
+import base64
+import json
+import os
+import pathlib
+import subprocess
+import sys
+
+sys.path.insert(0, '/usr/lib/vpn-router')
+
+import ldap
+
+import vpnrouter
+import vpnrouter_pki
+from vpnrouter_pki import Failure
+
+
+class Parser(argparse.ArgumentParser):
+    """An argument parser whose errors end with status 2."""
+
+    def error(self, message):
+        print(f'{self.prog}: {message}; see {self.prog} --help', file=sys.stderr)
+        sys.exit(2)
+
+
+def secret():
+    """The first line of standard input."""
+    return sys.stdin.readline().rstrip('\r\n')
+
+
+def restart_setup():
+    subprocess.run(['systemctl', 'restart', '--no-block', 'vpn-router-setup.service'], capture_output=True, text=True)
+
+
+def show_status(document):
+    root = document['root']
+    if root is None:
+        print('Root CA: none; vpn-router-setup creates it once a platform is chosen')
+    else:
+        print(f'Root CA: {root["subject"]}, expires {root["expires"]}, key {"on the router" if root["key"] else "offline"}'
+              f', CRL until {root["crl_next_update"] or "none"}')
+    for authority in document['authorities']:
+        line = f'Issuing CA {authority["name"]}: {authority["common_name"]}, {authority["state"]}'
+        if authority['expires']:
+            line += f', expires {authority["expires"]}, CRL until {authority["crl_next_update"] or "none"}'
+        line += f', templates {", ".join(authority["templates"])}'
+        if authority['acme_url']:
+            line += f', ACME {authority["acme_url"]}'
+        print(line)
+    for retired in document['retired']:
+        print(f'Retired CA {retired["name"]}: {retired["retired"]}')
+    for warning in document['warnings']:
+        print(f'Warning: {warning}')
+
+
+def run(options):
+    """Do the command of the options; the answer as a document, and a text for people."""
+    cfg = vpnrouter.load_config()
+    if options.command == 'status':
+        document = vpnrouter_pki.status(cfg)
+        return document, None
+    if options.command == 'list':
+        found = [entry for entry in vpnrouter_pki.certificates() if options.ca in (None, entry['ca'])]
+        text = '\n'.join(f'{entry["serial"]}  {entry["ca"]}  {entry["kind"]}  {entry["subject"]}  {entry["state"]}  '
+                         f'{entry["expires"]}' for entry in found)
+        return {'certificates': found}, text
+    with vpnrouter_pki.locked():
+        if options.command == 'issue':
+            if options.csr:
+                document = vpnrouter_pki.issue(cfg, options.ca, options.template,
+                                               csr=pathlib.Path(options.csr).read_text(encoding='ascii'))
+                return document, document['certificate'] + document['chain']
+            if not options.subject or not options.pfx:
+                raise Failure('give --csr, or --subject and --pfx', 2)
+            document = vpnrouter_pki.issue(cfg, options.ca, options.template, subject=options.subject,
+                                           sans=options.san or [], password=secret())
+            out = pathlib.Path(options.pfx)
+            with open(out, 'xb') as handle:
+                handle.write(base64.b64decode(document['pfx']))
+            return {'serial': document['serial'], 'pfx': str(out)}, f'Certificate {document["serial"]} written to {out}.'
+        if options.command == 'revoke':
+            document = vpnrouter_pki.revoke(cfg, options.serial, options.reason)
+            if document['reissue']:
+                restart_setup()
+            return document, f'Certificate {document["serial"]} revoked' + \
+                ('; vpn-router-setup issues a new one.' if document['reissue'] else '.')
+        if options.command == 'maintain':
+            due = vpnrouter_pki.maintain(cfg)
+            if due:
+                restart_setup()
+            return {'renewal': due}, 'CRLs checked' + ('; vpn-router-setup renews the router certificates.' if due else '.')
+        if options.command == 'ca' and options.action == 'csr':
+            document = vpnrouter_pki.csr(cfg, options.name)
+            return document, document['csr'] + '\n' + document['extensions']
+        if options.command == 'ca':
+            document = vpnrouter_pki.import_ca(cfg, options.name, pathlib.Path(options.file).read_text(encoding='ascii'))
+            restart_setup()
+            return document, f'Issuing CA {options.name} has its certificate, which expires {document["expires"]}.'
+        if options.action == 'export-key':
+            data = vpnrouter_pki.export_root_key(secret())
+            out = pathlib.Path(options.out)
+            with open(out, 'xb') as handle:
+                handle.write(data)
+            return {'exported': str(out)}, f'Root key written to {out}, encrypted.'
+        if options.action == 'remove-key':
+            if not options.yes:
+                print('Remove the root key from the router? Keep an exported copy first. Type yes: ', end='',
+                      file=sys.stderr, flush=True)
+                if secret().strip() != 'yes':
+                    return {'removed': False}, 'Nothing removed.'
+            vpnrouter_pki.remove_root_key()
+            return {'removed': True}, 'Root key removed; new issuing CAs now wait for an offline signature.'
+        if options.action == 'import-key':
+            vpnrouter_pki.import_root_key(pathlib.Path(options.file).read_bytes(), secret() or None)
+            restart_setup()
+            return {'imported': True}, 'Root key back on the router.'
+        document = vpnrouter_pki.import_root_crl(pathlib.Path(options.file).read_bytes())
+        return document, f'Root CRL taken, valid until {document["next_update"]}.'
+
+
+def main(argv):
+    parser = Parser(prog='vpn-router-ca', description="Manage the router's PKI.")
+    parser.add_argument('--json', action='store_true', help='answer in JSON')
+    commands = parser.add_subparsers(dest='command', required=True, parser_class=Parser)
+    commands.add_parser('status', help='show the root, the issuing CAs and the warnings')
+    lister = commands.add_parser('list', help='list the certificates the router issued')
+    lister.add_argument('--ca', help='only those of this issuing CA')
+    issuer = commands.add_parser('issue', help='issue a certificate from a CSR, or with a new key into a PFX file')
+    issuer.add_argument('--ca', required=True, help='the issuing CA')
+    issuer.add_argument('--template', required=True, help='the certificate template')
+    issuer.add_argument('--csr', help='the PEM CSR to sign')
+    issuer.add_argument('--subject', help='the common name, without --csr')
+    issuer.add_argument('--san', action='append', help='a subject alternative name, without --csr; repeat for more')
+    issuer.add_argument('--pfx', help='the PFX file to write, without --csr; the password is read from standard input')
+    revoker = commands.add_parser('revoke', help='revoke a certificate the router issued')
+    revoker.add_argument('serial', help='the serial number, hexadecimal')
+    revoker.add_argument('--reason', choices=vpnrouter_pki.REASONS, help='the reason, unspecified by default')
+    commands.add_parser('maintain', help='sign the stale CRLs again')
+    authority = commands.add_parser('ca', help='an issuing CA that waits for the offline root')
+    actions = authority.add_subparsers(dest='action', required=True, parser_class=Parser)
+    actions.add_parser('csr', help='print its CSR and the openssl extensions').add_argument('name')
+    importer = actions.add_parser('import', help='take its certificate signed by the root')
+    importer.add_argument('name')
+    importer.add_argument('file')
+    root = commands.add_parser('root', help='the root CA')
+    root_actions = root.add_subparsers(dest='action', required=True, parser_class=Parser)
+    exporter = root_actions.add_parser('export-key', help='write the root key encrypted with the passphrase from stdin')
+    exporter.add_argument('--out', required=True, help='the file to write')
+    remover = root_actions.add_parser('remove-key', help='remove the root key from the router')
+    remover.add_argument('--yes', action='store_true', help='do not ask')
+    root_actions.add_parser('import-key', help='put the root key back').add_argument('file')
+    root_actions.add_parser('import-crl', help='take a root CRL signed offline').add_argument('file')
+    options = parser.parse_args(argv)
+    os.umask(0o077)
+
+    try:
+        document, text = run(options)
+    except Failure as failure:
+        code, message = failure.status, str(failure)
+    except OSError as exc:
+        code, message = 1, str(exc)
+    except ldap.LDAPError as exc:
+        code, message = 1, f'the directory, which holds the PKI, failed: {exc}'
+    except SystemExit as exc:
+        code, message = 1, str(exc.code)
+    else:
+        if options.json:
+            print(json.dumps(document, indent=2))
+        elif text is None:
+            show_status(document)
+        elif text:
+            print(text.rstrip('\n'))
+        return 0
+    if options.json:
+        print(json.dumps({'error': message}, indent=2))
+    else:
+        print(f'vpn-router-ca: {message}', file=sys.stderr)
+    return code
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))

debian-package/src/usr/sbin/vpn-router-caddy

@@ -0,0 +1,151 @@
+#!/usr/bin/python3
+"""Install Caddy, the router's HTTPS proxy, from the source https.caddy_source names.
+
+status shows the configured source, the source in effect and the installed and candidate versions of Caddy. apply
+installs Caddy when it is missing: from the Ubuntu archive, or with upstream from Caddy's own repository, which it
+configures first (vpnrouter.CADDY_APT_SOURCE, with the key in vpnrouter.CADDY_KEYRING). When upstream is configured
+and the repository is not, it configures the repository and replaces the installed Caddy with upstream's. It then
+restarts vpn-router-setup without waiting, which serves with the new Caddy; vpn-router-caddy.service runs it for
+vpn-router-setup. use-upstream sets https.caddy_source to upstream and applies it, printing each step.
+
+Once the repository is configured Caddy stays upstream's: apt keeps the higher upstream version, and ubuntu has no
+effect. A failure is a message on standard error and the exit status 1. -h or --help shows the commands.
+
+  vpn-router-caddy [--json] status
+  vpn-router-caddy apply
+  vpn-router-caddy use-upstream
+"""
+
+import argparse
+import json
+import os
+import subprocess
+import sys
+import urllib.request
+
+sys.path.insert(0, '/usr/lib/vpn-router')
+
+import vpnrouter
+
+#: apt-get, waiting up to ten minutes for dpkg's lock, which a package installation that runs setup holds.
+APT = ['apt-get', '-o', 'DPkg::Lock::Timeout=600']
+
+#: The options of apt-get install: no questions, and the configuration files of the installed Caddy kept.
+INSTALL = ['-y', '-o', 'Dpkg::Options::=--force-confdef', '-o', 'Dpkg::Options::=--force-confold', 'install']
+
+#: Seconds the key of Caddy's repository may take to download.
+KEY_TIMEOUT = 60
+
+
+class Parser(argparse.ArgumentParser):
+    """An argument parser whose errors end with status 2."""
+
+    def error(self, message):
+        print(f'{self.prog}: {message}; see {self.prog} --help', file=sys.stderr)
+        sys.exit(2)
+
+
+def step(message):
+    print(message, flush=True)
+
+
+def installed():
+    """The installed version of caddy, or None when it is not installed."""
+    result = subprocess.run(['dpkg-query', '-W', '-f=${Status}\t${Version}', 'caddy'], capture_output=True, text=True)
+    status, _, version = result.stdout.partition('\t')
+    return version if result.returncode == 0 and status == 'install ok installed' else None
+
+
+def candidate():
+    """The version of caddy apt would install, or None when it knows none."""
+    result = subprocess.run(['apt-cache', 'policy', 'caddy'], capture_output=True, text=True)
+    for line in result.stdout.splitlines():
+        if line.strip().startswith('Candidate:'):
+            value = line.split(':', 1)[1].strip()
+            return None if value == '(none)' else value
+    return None
+
+
+def effective():
+    """The source Caddy comes from: upstream once its repository is configured, ubuntu otherwise."""
+    return 'upstream' if vpnrouter.CADDY_APT_SOURCE.is_file() else 'ubuntu'
+
+
+def command(args):
+    """Run a command with its output shown; SystemExit when it fails."""
+    step(f'$ {" ".join(args)}')
+    result = subprocess.run(args, env=dict(os.environ, DEBIAN_FRONTEND='noninteractive'))
+    if result.returncode != 0:
+        sys.exit(f'vpn-router-caddy: {args[0]} failed with status {result.returncode}')
+
+
+def apply():
+    """Install Caddy, or replace it with upstream's, as https.caddy_source asks (see the module)."""
+    source = vpnrouter.load_config()['https']['caddy_source']
+    repository = source == 'upstream' and not vpnrouter.CADDY_APT_SOURCE.is_file()
+    if installed() and not repository:
+        if source == 'ubuntu' and effective() == 'upstream':
+            step("https.caddy_source is ubuntu, but Caddy's repository is configured, so Caddy stays upstream's.")
+        else:
+            step(f'Caddy {installed()} is installed from the {source} source.')
+        return
+    if repository:
+        step(f"Downloading the key of Caddy's repository from {vpnrouter.CADDY_KEY_URL}")
+        with urllib.request.urlopen(vpnrouter.CADDY_KEY_URL, timeout=KEY_TIMEOUT) as response:
+            key = response.read()
+        if not key.lstrip().startswith(b'-----BEGIN PGP PUBLIC KEY BLOCK-----'):
+            sys.exit(f'vpn-router-caddy: {vpnrouter.CADDY_KEY_URL} did not answer with an OpenPGP public key')
+        vpnrouter.CADDY_KEYRING.parent.mkdir(mode=0o755, parents=True, exist_ok=True)
+        vpnrouter.CADDY_KEYRING.write_bytes(key)
+        os.chmod(vpnrouter.CADDY_KEYRING, 0o644)
+        vpnrouter.CADDY_APT_SOURCE.write_text(
+            f'Types: deb\nURIs: {vpnrouter.CADDY_REPOSITORY}\nSuites: any-version\nComponents: main\n'
+            f'Signed-By: {vpnrouter.CADDY_KEYRING}\n', encoding='utf-8')
+        os.chmod(vpnrouter.CADDY_APT_SOURCE, 0o644)
+        step(f"Caddy's repository is configured in {vpnrouter.CADDY_APT_SOURCE}.")
+    command([*APT, 'update'])
+    command([*APT, *INSTALL, 'caddy'])
+    step(f'Caddy {installed()} is installed from the {effective()} source.')
+    subprocess.run(['systemctl', 'restart', '--no-block', 'vpn-router-setup.service'], capture_output=True, text=True)
+    step('vpn-router-setup restarted.')
+
+
+def main(argv):
+    parser = Parser(prog='vpn-router-caddy', description='Install Caddy from the source https.caddy_source names.')
+    parser.add_argument('--json', action='store_true', help='answer status in JSON')
+    commands = parser.add_subparsers(dest='command', required=True, parser_class=Parser)
+    commands.add_parser('status', help='show the sources and the versions of Caddy')
+    commands.add_parser('apply', help='install Caddy, or replace it with upstream\'s, as https.caddy_source asks')
+    commands.add_parser('use-upstream', help="switch to Caddy's own repository and replace Caddy with upstream's")
+    options = parser.parse_args(argv)
+
+    try:
+        if options.command == 'status':
+            document = {'configured': vpnrouter.load_config()['https']['caddy_source'], 'source': effective(),
+                        'installed': installed(), 'candidate': candidate()}
+            if options.json:
+                print(json.dumps(document, indent=2))
+            else:
+                print(f'Configured source: {document["configured"]}\nSource in effect: {document["source"]}\n'
+                      f'Installed: {document["installed"] or "no"}\nCandidate: {document["candidate"] or "none"}')
+            return 0
+        if options.command == 'use-upstream':
+            with vpnrouter.config_lock():
+                data = vpnrouter.read_settings()
+                current = data.get('https')
+                data['https'] = {**(current if isinstance(current, dict) else {}), 'caddy_source': 'upstream'}
+                vpnrouter.validate(vpnrouter.with_defaults(data))
+                vpnrouter.save_config(data)
+            step('https.caddy_source is upstream.')
+        apply()
+    except vpnrouter.ConfigError as exc:
+        print(f'vpn-router-caddy: {exc}', file=sys.stderr)
+        return 1
+    except OSError as exc:
+        print(f'vpn-router-caddy: {exc}', file=sys.stderr)
+        return 1
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))

debian-package/src/usr/sbin/vpn-router-user

@@ -10,7 +10,11 @@ user exists, 1 anything else. -h or --help shows the commands and their options.
 
   vpn-router-user [--json] list
   vpn-router-user [--json] add <name> --first-name F --last-name L --email E   (password on stdin)
+  vpn-router-user [--json] add <name> [--email E]   (a Manager-only account; password on stdin)
+  vpn-router-user [--json] add <name> --service [--display-name N] [--description D] [--email E]
+                                                  (a service account; password on stdin)
   vpn-router-user [--json] update <name> [--first-name F] [--last-name L] [--email E]
+  vpn-router-user [--json] update <name> [--display-name N] [--description D] [--email E]   (a service account)
   vpn-router-user [--json] passwd <name>       (password on stdin)
   vpn-router-user [--json] disable <name>
   vpn-router-user [--json] enable <name>
@@ -29,16 +33,20 @@ import sys
 sys.path.insert(0, '/usr/lib/vpn-router')
 
 import ldap
-import psycopg
 
+import vpnrouter
+import vpnrouter_auth
 import vpnrouter_users
 from vpnrouter_users import Failure
 
 #: The help line of each command.
 HELP = {
     'list': 'list the users',
-    'add': 'add a user and issue the certificate; the password is read from standard input',
-    'update': 'change the first name, the last name or the e-mail address',
+    'add': 'add a VPN user with the three person options and issue the certificate, or without the names a '
+           'Manager-only account, or with --service a service account, each with an optional e-mail address; the '
+           'password is read from standard input',
+    'update': "change the first name, the last name or the e-mail address; a Manager-only account has only the "
+              "e-mail address, which '' removes",
     'passwd': 'set the password, read from standard input',
     'disable': 'lock the user out of OpenVPN',
     'enable': 'let the user in again',
@@ -51,7 +59,9 @@ HELP = {
 }
 
 #: The help of the person options.
-PERSON_HELP = {'first_name': 'the first name', 'last_name': 'the last name', 'email': 'the e-mail address'}
+PERSON_HELP = {'first_name': 'the first name', 'last_name': 'the last name', 'email': 'the e-mail address',
+               'display_name': "a service account's display name; '' removes it",
+               'description': "a service account's description; '' removes it"}
 
 
 class Parser(argparse.ArgumentParser):
@@ -70,9 +80,10 @@ def parse(argv):
         if command != 'list':
             sub.add_argument('name', help='the user name')
         if command in ('add', 'update'):
-            for key, _ in vpnrouter_users.PERSON:
-                sub.add_argument(f'--{key.replace("_", "-")}', dest=key, metavar='TEXT', required=command == 'add',
-                                 help=PERSON_HELP[key])
+            for key, _ in vpnrouter_users.PERSON + vpnrouter_users.LABELS:
+                sub.add_argument(f'--{key.replace("_", "-")}', dest=key, metavar='TEXT', help=PERSON_HELP[key])
+        if command == 'add':
+            sub.add_argument('--service', action='store_true', help='add a service account, which cannot sign in to the web service')
         if command == 'otp-enrol':
             sub.add_argument('code', help='a code of the secret from the authenticator')
     return parser.parse_args(argv)
@@ -88,9 +99,9 @@ def run(connection, options, secret):
     command = options.command
     if command == 'list':
         return vpnrouter_users.list_users(connection)
-    values = {key: getattr(options, key, None) for key, _ in vpnrouter_users.PERSON}
+    values = {key: getattr(options, key, None) for key, _ in vpnrouter_users.PERSON + vpnrouter_users.LABELS}
     if command == 'add':
-        return vpnrouter_users.add(connection, options.name, secret, values)
+        return vpnrouter_users.add(connection, options.name, secret, values, service=options.service)
     if command == 'update':
         return vpnrouter_users.update(connection, options.name, values)
     if command == 'passwd':
@@ -111,8 +122,9 @@ def text_of(answer):
     users = answer['users'] if 'users' in answer else [answer]
     if not users:
         return 'no users'
-    head = ('USER', 'STATE', 'CERTIFICATE', 'OTP', 'FIRST NAME', 'LAST NAME', 'E-MAIL')
-    rows = [(user['name'], 'disabled' if not user['enabled'] else 'locked' if user['locked'] else 'enabled',
+    head = ('USER', 'KIND', 'STATE', 'CERTIFICATE', 'OTP', 'FIRST NAME', 'LAST NAME', 'E-MAIL')
+    rows = [(user['name'], 'internal' if user['internal'] else 'service' if user['service'] else 'manager' if user['manager_only'] else 'vpn',
+             'disabled' if not user['enabled'] else 'locked' if user['locked'] else 'enabled',
              'issued' if user['certificate'] else 'missing',
              'enrolled' if user['otp'] else '-', user['first_name'] or '-', user['last_name'] or '-', user['email'] or '-')
             for user in users]
@@ -131,12 +143,14 @@ def main(argv):
         secret = password() if options.command in ('add', 'passwd', 'otp-enrol') else None
         with vpnrouter_users.session() as connection:
             answer = run(connection, options, secret)
+        if options.command in vpnrouter_auth.REVOKING:
+            vpnrouter_auth.revoke(vpnrouter.load_config(), options.name)
     except Failure as failure:
         message, status = str(failure), failure.status
     except (ldap.LDAPError, OSError) as exc:
         message, status = f'the user directory failed: {exc}', 1
-    except psycopg.Error as exc:
-        message, status = f'the token database failed: {exc}', 1
+    except vpnrouter_auth.DatabaseError as exc:
+        message, status = str(exc), 1
     else:
         print(json.dumps(answer) if as_json else text_of(answer))
         return 0

debian-package/src/usr/share/doc/vpn-router/vpn-router.json.example

@@ -8,25 +8,37 @@
     "internal": "eth1"
   },
   "wan": {
-    "local_fqdn": "router.example.com",
-    "local_id_mode": "fqdn"
+    "local_fqdn": "router.example.com"
   },
   "local": {
     "cidrs": ["10.0.0.0/24"],
     "int_addr": "10.1.1.4",
     "int_gateway_ip": "10.1.1.1"
   },
-  "remote": {
-    "addrs": ["peer.example.net"],
-    "id": "peer.example.net",
-    "cidrs": ["192.168.0.0/24"],
-    "psk": "change-me",
-    "psk_file": ""
+  "site_to_site": {
+    "remote-site": {
+      "addrs": ["peer.example.net"],
+      "id": "peer.example.net",
+      "cidrs": ["192.168.0.0/24"],
+      "local_cidrs": ["10.0.0.0/24"],
+      "local_id_mode": "fqdn",
+      "auth": "psk",
+      "psk": "change-me",
+      "psk_file": "",
+      "remote_ca": ""
+    },
+    "branch": {
+      "addrs": ["branch.example.org"],
+      "id": "branch.example.org",
+      "cidrs": ["192.168.10.0/24"],
+      "local_cidrs": ["10.0.0.0/24"],
+      "local_id_mode": "fqdn",
+      "auth": "certificate"
+    }
   },
   "p2s": {
     "enabled": false,
-    "address_pool": "172.16.0.0/24",
-    "ca_name": "VPN Router CA"
+    "address_pool": "172.16.0.0/24"
   },
   "wireguard": {
     "enabled": false,
@@ -45,8 +57,75 @@
   },
   "web": {
     "enabled": true,
+    "listen": "both",
     "port": 7081,
     "database": "",
-    "trusted_proxies": []
+    "trusted_proxies": [],
+    "status_content": "basic"
+  },
+  "https": {
+    "enabled": true,
+    "email": "",
+    "web_port": 7444,
+    "manager_port": 7443,
+    "caddy_source": "ubuntu"
+  },
+  "pki": {
+    "root_name": "VPN Router Root CA",
+    "root_days": 7300,
+    "ca_days": 3650,
+    "key": "rsa-4096",
+    "crl_days": 7,
+    "root_crl_days": 180,
+    "publish": ["http"],
+    "p2s_name": "VPN Router P2S CA",
+    "s2s_name": "VPN Router Site-to-Site CA",
+    "issuing_cas": {
+      "servers": {
+        "name": "VPN Router Server CA",
+        "templates": ["server"],
+        "acme": false
+      }
+    },
+    "templates": {
+      "server": {
+        "days": 397
+      }
+    },
+    "root_key_warning": true
+  },
+  "metrics": {
+    "enabled": true,
+    "listen": "internal",
+    "port": 9464,
+    "https_port": 9465,
+    "service_account": "metrics",
+    "otlp_endpoint": "",
+    "otlp_username": "",
+    "otlp_password": "",
+    "keep_details": "24 hours",
+    "keep_hourly": "1 month",
+    "keep_daily": "3 months",
+    "keep_monthly": "1 year",
+    "keep_yearly": "10 years"
+  },
+  "backup": {
+    "enabled": false,
+    "schedule": "daily",
+    "target": "local",
+    "path": "/var/backups/vpn-router",
+    "keep_hourly": 24,
+    "keep_daily": 7,
+    "keep_weekly": 4,
+    "keep_monthly": 12,
+    "keep_yearly": 0,
+    "metrics": false,
+    "rclone_remote": "",
+    "rclone_mode": "sync",
+    "pbs_repository": "",
+    "pbs_password": "",
+    "pbs_fingerprint": "",
+    "pbs_namespace": "",
+    "pbs_keyfile": ""
   }
 }

debian-package/src/usr/share/vpn-router/templates/Caddyfile.j2

@@ -0,0 +1,81 @@
+# Written by vpn-router-setup from /etc/vpn-router/vpn-router.json; edits are overwritten.
+{
+	auto_https disable_redirects
+{%- if https_email %}
+	email {{ https_email }}
+{%- endif %}
+{%- if acme_cas %}
+	skip_install_trust
+	pki {
+{%- for ca in acme_cas %}
+		ca {{ ca.id }} {
+			name "{{ ca.name }}"
+			root {
+				cert {{ ca.cert }}
+				key {{ ca.key }}
+			}
+			intermediate {
+				cert {{ ca.cert }}
+				key {{ ca.key }}
+			}
+		}
+{%- endfor %}
+	}
+{%- endif %}
+}
+{% if web_enabled or 'https' in pki_publish or acme_cas %}
+{{ local_fqdn }}:{{ https_web_port }} {
+{%- if 'https' in pki_publish %}
+	handle_path /pki/* {
+		root * {{ pki_publish_dir }}
+		file_server
+	}
+{%- endif %}
+{%- for ca in acme_cas %}
+	handle /acme/{{ ca.id }}/* {
+		acme_server {
+			ca {{ ca.id }}
+		}
+	}
+{%- endfor %}
+	handle {
+{%- if web_enabled %}
+		reverse_proxy {{ loopback }}:{{ web_port }}
+{%- else %}
+		respond 404
+{%- endif %}
+	}
+}
+{% endif %}
+{%- if 'http-80' in pki_publish %}
+http://{{ local_fqdn }} {
+	handle_path /pki/* {
+		root * {{ pki_publish_dir }}
+		file_server
+	}
+	handle {
+		respond 404
+	}
+}
+{% endif %}
+{%- if 'https-default' in pki_publish %}
+{{ local_fqdn }}:443 {
+	handle_path /pki/* {
+		root * {{ pki_publish_dir }}
+		file_server
+	}
+	handle {
+		respond 404
+	}
+}
+{% endif %}
+{%- if manager_port %}
+{{ local_fqdn }}:{{ https_manager_port }} {
+	reverse_proxy {{ loopback }}:{{ manager_port }}
+}
+{% endif %}
+{%- if metrics_enabled %}
+{{ local_fqdn }}:{{ metrics_https_port }} {
+	reverse_proxy {{ loopback }}:{{ metrics_port }}
+}
+{% endif %}

debian-package/src/usr/share/vpn-router/templates/openvpn-server.conf.j2

@@ -5,15 +5,16 @@ dev-type tun
 topology subnet
 server {{ openvpn_network }} {{ openvpn_netmask }}
 
-ca {{ ca_cert }}
+ca {{ openvpn_ca }}
 cert {{ server_cert }}
 key {{ server_key }}
-crl-verify {{ crl }}
+crl-verify {{ openvpn_crl }}
 dh none
 remote-cert-tls client
 
 plugin /usr/lib/openvpn/openvpn-auth-ldap.so {{ openvpn_ldap_conf }}
 script-security 2
+tls-verify /usr/lib/vpn-router/openvpn-issuer
 auth-user-pass-verify /usr/lib/vpn-router/openvpn-same-user via-env
 
 {% for network, netmask in openvpn_routes -%}

debian-package/src/usr/share/vpn-router/templates/remote-site.conf.j2

@@ -1,46 +0,0 @@
-connections {
-  remote-site {
-    version = 2
-    proposals = aes256-sha256-modp2048,aes256-sha1-modp1024
-    reauth_time = 28800
-    unique = replace
-
-    local_addrs  = {{ local_addrs }}
-    remote_addrs = {{ remote_addrs }}
-
-    local {
-      id = {{ local_id }}
-      auth = psk
-    }
-
-    remote {
-      id = {{ remote_id }}
-      auth = psk
-    }
-
-    children {
-      site2site {
-        mode = tunnel
-        local_ts = {{ local_ts }}
-        remote_ts = {{ remote_cidrs }}
-        esp_proposals = aes256-sha256,aes256-sha1
-        life_time = 3600
-        dpd_action = restart
-        start_action = start
-        close_action = none
-      }
-    }
-
-    rekey_time = 10800
-    dpd_delay = 10
-    dpd_timeout = 120
-  }
-}
-
-secrets {
-  ike-psk {
-    id-1 = {{ local_id }}
-    id-2 = {{ remote_id }}
-    secret = "{{ psk }}"
-  }
-}

debian-package/src/usr/share/vpn-router/templates/road-warrior.conf.j2

@@ -8,13 +8,13 @@ connections {
 
     local {
       auth = pubkey
-      certs = server.pem
+      certs = vpn-server.pem
       id = @{{ local_fqdn }}
     }
 
     remote {
       auth = eap-tls
-      cacerts = ca.pem
+      cacerts = p2s.pem
       eap_id = %any
     }
 

debian-package/src/usr/share/vpn-router/templates/site-to-site.conf.j2

@@ -0,0 +1,60 @@
+connections {
+{%- for connection in connections if connection.ready %}
+  {{ connection.name }} {
+    version = 2
+    proposals = aes256-sha256-modp2048,aes256-sha1-modp1024
+    reauth_time = 28800
+    unique = replace
+
+    local_addrs  = {{ local_addrs }}
+    remote_addrs = {{ connection.remote_addrs }}
+
+    local {
+      id = {{ connection.local_id }}
+{%- if connection.auth == 'certificate' %}
+      auth = pubkey
+      certs = site-to-site.pem
+{%- else %}
+      auth = psk
+{%- endif %}
+    }
+
+    remote {
+      id = {{ connection.remote_id }}
+{%- if connection.auth == 'certificate' %}
+      auth = pubkey
+      cacerts = {{ connection.remote_cacert }}
+{%- else %}
+      auth = psk
+{%- endif %}
+    }
+
+    children {
+      {{ connection.name }} {
+        mode = tunnel
+        local_ts = {{ connection.local_ts }}
+        remote_ts = {{ connection.remote_cidrs }}
+        esp_proposals = aes256-sha256,aes256-sha1
+        life_time = 3600
+        dpd_action = restart
+        start_action = start
+        close_action = none
+      }
+    }
+
+    rekey_time = 10800
+    dpd_delay = 10
+    dpd_timeout = 120
+  }
+{%- endfor %}
+}
+
+secrets {
+{%- for connection in connections if connection.ready and connection.auth == 'psk' %}
+  ike-psk-{{ connection.name }} {
+    id-1 = {{ connection.local_id }}
+    id-2 = {{ connection.remote_id }}
+    secret = "{{ connection.psk }}"
+  }
+{%- endfor %}
+}

examples/azure/router.tf

@@ -3,7 +3,7 @@ data "http" "repo_gpg_key" {
 }
 
 locals {
-  supply_pki = var.ca_cert_file != ""
+  supply_pki = var.root_cert_file != ""
 
   cloud_init_vars = {
     hostname        = var.name
@@ -15,15 +15,19 @@ locals {
     # Contents of /etc/vpn-router/vpn-router.json.
     config = {
       general = { platform = "azure", mode = "auto" }
-      wan     = { local_fqdn = var.local_fqdn, local_id_mode = var.local_id_mode }
-      local   = { cidrs = [for c in split(",", var.local_cidrs) : trimspace(c) if trimspace(c) != ""] }
-      remote = {
-        addrs = [for a in split(",", var.remote_addrs) : trimspace(a) if trimspace(a) != ""]
-        id    = var.remote_id
-        cidrs = local.remote_cidr_list
-        psk   = var.psk
+      wan     = { local_fqdn = var.local_fqdn }
+      local   = { cidrs = local.local_cidr_list }
+      site_to_site = {
+        "remote-site" = {
+          addrs         = [for a in split(",", var.remote_addrs) : trimspace(a) if trimspace(a) != ""]
+          id            = var.remote_id
+          cidrs         = local.remote_cidr_list
+          local_cidrs   = local.local_cidr_list
+          local_id_mode = var.local_id_mode
+          psk           = var.psk
+        }
       }
-      p2s       = { enabled = var.p2s_enabled, address_pool = var.p2s_address_pool, ca_name = "VPN Router CA" }
+      p2s       = { enabled = var.p2s_enabled, address_pool = var.p2s_address_pool }
       wireguard = { enabled = var.wireguard_enabled, address = var.wireguard_address, listen_port = var.wireguard_listen_port }
     }
   }
@@ -98,10 +102,8 @@ resource "azurerm_linux_virtual_machine" "router" {
   custom_data = base64encode(
     local.supply_pki
     ? templatefile("${path.module}/../cloud-init-with-pki.yaml.tpl", merge(local.cloud_init_vars, {
-      label       = split(".", var.local_fqdn)[0]
-      ca_cert     = file(var.ca_cert_file)
-      server_cert = file(var.server_cert_file)
-      server_key  = file(var.server_key_file)
+      root_cert = file(var.root_cert_file)
+      root_key  = var.root_key_file != "" ? file(var.root_key_file) : ""
     }))
     : templatefile("${path.module}/../cloud-init.yaml.tpl", local.cloud_init_vars)
   )

examples/azure/routes.tf

@@ -1,5 +1,6 @@
 locals {
   remote_cidr_list = [for c in split(",", var.remote_cidrs) : trimspace(c) if trimspace(c) != ""]
+  local_cidr_list  = [for c in split(",", var.local_cidrs) : trimspace(c) if trimspace(c) != ""]
 }
 
 resource "azurerm_route_table" "router" {

examples/azure/variables.tf

@@ -100,20 +100,14 @@ variable "p2s_address_pool" {
   default     = ""
 }
 
-variable "ca_cert_file" {
-  description = "Path to an existing CA certificate PEM to supply instead of letting the package generate one. Leave empty to auto-generate."
+variable "root_cert_file" {
+  description = "Path to an existing root CA certificate PEM to supply instead of letting the package create one. Leave empty to create one."
   type        = string
   default     = ""
 }
 
-variable "server_cert_file" {
-  description = "Path to an existing server certificate PEM, paired with ca_cert_file."
-  type        = string
-  default     = ""
-}
-
-variable "server_key_file" {
-  description = "Path to an existing server private key PEM, paired with ca_cert_file."
+variable "root_key_file" {
+  description = "Path to the private key PEM of root_cert_file. Leave empty to keep the root offline: the issuing CAs then wait for certificates signed elsewhere."
   type        = string
   default     = ""
   sensitive   = true

examples/cloud-init-with-pki.yaml.tpl

@@ -1,12 +1,13 @@
 #cloud-config
 #
 # Example: install and configure vpn-router during first boot, supplying an
-# existing CA and server certificate instead of letting the package generate
-# one. See cloud-init.yaml.tpl for the variant that generates its own CA.
+# existing root CA instead of letting the package create one. See
+# cloud-init.yaml.tpl for the variant that creates its own.
 #
 # Template variables are Terraform templatefile() placeholders. Adapt or drop
-# them for whatever renders this file. <label> is the first component of the
-# FQDN, for example "router" in router.example.com.
+# them for whatever renders this file. With root_key empty the root stays
+# offline: the router's issuing CAs wait for their certificates, signed with
+# the root elsewhere and imported with vpn-router-ca ca import.
 
 hostname: ${hostname}
 fqdn: ${fqdn}
@@ -28,21 +29,18 @@ write_files:
     content: |
       ${jsonencode(config)}
 
-  - path: /etc/vpn-router/pki/ca_cert.pem
+  - path: /etc/vpn-router/pki/root/cert.pem
     permissions: '0644'
     owner: root:root
     content: |
-      ${indent(6, trimspace(ca_cert))}
-  - path: /etc/vpn-router/pki/${label}_cert.pem
-    permissions: '0644'
-    owner: root:root
-    content: |
-      ${indent(6, trimspace(server_cert))}
-  - path: /etc/vpn-router/pki/${label}_key.pem
+      ${indent(6, trimspace(root_cert))}
+%{ if root_key != "" ~}
+  - path: /etc/vpn-router/pki/root/key.pem
     permissions: '0600'
     owner: root:root
     content: |
-      ${indent(6, trimspace(server_key))}
+      ${indent(6, trimspace(root_key))}
+%{ endif ~}
 
 package_update: true
 

examples/cloud-init.yaml.tpl

@@ -10,9 +10,9 @@
 # Template variables are Terraform templatefile() placeholders. Adapt or drop
 # them for whatever renders this file.
 #
-# This variant lets the package generate its own CA on first boot. See
-# cloud-init-with-pki.yaml.tpl for the variant that supplies existing PKI
-# material instead.
+# This variant lets the package create its own CAs on first boot. See
+# cloud-init-with-pki.yaml.tpl for the variant that supplies an existing root
+# CA instead.
 
 hostname: ${hostname}
 fqdn: ${fqdn}

examples/gcp/router.tf

@@ -3,7 +3,7 @@ data "http" "repo_gpg_key" {
 }
 
 locals {
-  supply_pki = var.ca_cert_file != ""
+  supply_pki = var.root_cert_file != ""
 
   cloud_init_vars = {
     hostname        = var.name
@@ -15,24 +15,26 @@ locals {
     # Contents of /etc/vpn-router/vpn-router.json.
     config = {
       general = { platform = "gcp", mode = "auto" }
-      wan     = { local_fqdn = var.local_fqdn, local_id_mode = var.local_id_mode }
-      local   = { cidrs = [for c in split(",", var.local_cidrs) : trimspace(c) if trimspace(c) != ""] }
-      remote = {
-        addrs = [for a in split(",", var.remote_addrs) : trimspace(a) if trimspace(a) != ""]
-        id    = var.remote_id
-        cidrs = local.remote_cidr_list
-        psk   = var.psk
+      wan     = { local_fqdn = var.local_fqdn }
+      local   = { cidrs = local.local_cidr_list }
+      site_to_site = {
+        "remote-site" = {
+          addrs         = [for a in split(",", var.remote_addrs) : trimspace(a) if trimspace(a) != ""]
+          id            = var.remote_id
+          cidrs         = local.remote_cidr_list
+          local_cidrs   = local.local_cidr_list
+          local_id_mode = var.local_id_mode
+          psk           = var.psk
+        }
       }
-      p2s       = { enabled = var.p2s_enabled, address_pool = var.p2s_address_pool, ca_name = "VPN Router CA" }
+      p2s       = { enabled = var.p2s_enabled, address_pool = var.p2s_address_pool }
       wireguard = { enabled = var.wireguard_enabled, address = var.wireguard_address, listen_port = var.wireguard_listen_port }
     }
   }
 
   cloud_init_rendered = local.supply_pki ? templatefile("${path.module}/../cloud-init-with-pki.yaml.tpl", merge(local.cloud_init_vars, {
-    label       = split(".", var.local_fqdn)[0]
-    ca_cert     = file(var.ca_cert_file)
-    server_cert = file(var.server_cert_file)
-    server_key  = file(var.server_key_file)
+    root_cert = file(var.root_cert_file)
+    root_key  = var.root_key_file != "" ? file(var.root_key_file) : ""
   })) : templatefile("${path.module}/../cloud-init.yaml.tpl", local.cloud_init_vars)
 }
 

examples/gcp/routes.tf

@@ -1,5 +1,6 @@
 locals {
   remote_cidr_list = [for c in split(",", var.remote_cidrs) : trimspace(c) if trimspace(c) != ""]
+  local_cidr_list  = [for c in split(",", var.local_cidrs) : trimspace(c) if trimspace(c) != ""]
 }
 
 resource "google_compute_route" "to_remote" {

examples/gcp/variables.tf

@@ -97,20 +97,14 @@ variable "p2s_address_pool" {
   default     = ""
 }
 
-variable "ca_cert_file" {
-  description = "Path to an existing CA certificate PEM to supply instead of letting the package generate one. Leave empty to auto-generate."
+variable "root_cert_file" {
+  description = "Path to an existing root CA certificate PEM to supply instead of letting the package create one. Leave empty to create one."
   type        = string
   default     = ""
 }
 
-variable "server_cert_file" {
-  description = "Path to an existing server certificate PEM, paired with ca_cert_file."
-  type        = string
-  default     = ""
-}
-
-variable "server_key_file" {
-  description = "Path to an existing server private key PEM, paired with ca_cert_file."
+variable "root_key_file" {
+  description = "Path to the private key PEM of root_cert_file. Leave empty to keep the root offline: the issuing CAs then wait for certificates signed elsewhere."
   type        = string
   default     = ""
   sensitive   = true

examples/install.sh

@@ -26,7 +26,7 @@ REMOTE_CIDRS="${REMOTE_CIDRS:-[\"192.168.0.0/24\"]}"
 PSK="${PSK:-change-me}"
 P2S_ENABLED="${P2S_ENABLED:-false}"
 P2S_ADDRESS_POOL="${P2S_ADDRESS_POOL:-172.16.0.0/24}"
-P2S_CA_NAME="${P2S_CA_NAME:-VPN Router CA}"
+PKI_ROOT_NAME="${PKI_ROOT_NAME:-VPN Router Root CA}"
 WG_ENABLED="${WG_ENABLED:-false}"
 WG_ADDRESS="${WG_ADDRESS:-}"
 WG_LISTEN_PORT="${WG_LISTEN_PORT:-51820}"
@@ -47,24 +47,29 @@ cat > /etc/vpn-router/vpn-router.json <<EOF
     "internal": "${INTERNAL_INTERFACE}"
   },
   "wan": {
-    "local_fqdn": "${LOCAL_FQDN}",
-    "local_id_mode": "${LOCAL_ID_MODE}"
+    "local_fqdn": "${LOCAL_FQDN}"
   },
   "local": {
     "cidrs": ${LOCAL_CIDRS},
     "int_addr": "${INT_ADDR}",
     "int_gateway_ip": "${INT_GATEWAY_IP}"
   },
-  "remote": {
-    "addrs": ${REMOTE_ADDRS},
-    "id": "${REMOTE_ID}",
-    "cidrs": ${REMOTE_CIDRS},
-    "psk_file": "/etc/vpn-router/psk"
+  "site_to_site": {
+    "remote-site": {
+      "addrs": ${REMOTE_ADDRS},
+      "id": "${REMOTE_ID}",
+      "cidrs": ${REMOTE_CIDRS},
+      "local_cidrs": ${LOCAL_CIDRS},
+      "local_id_mode": "${LOCAL_ID_MODE}",
+      "psk_file": "/etc/vpn-router/psk"
+    }
   },
   "p2s": {
     "enabled": ${P2S_ENABLED},
-    "address_pool": "${P2S_ADDRESS_POOL}",
-    "ca_name": "${P2S_CA_NAME}"
+    "address_pool": "${P2S_ADDRESS_POOL}"
+  },
+  "pki": {
+    "root_name": "${PKI_ROOT_NAME}"
   },
   "wireguard": {
     "enabled": ${WG_ENABLED},

release.py

@@ -1,9 +1,9 @@
 #!/usr/bin/env python3
 """Commit everything, tag the version of the newest changelog entry and push main with the tags.
 
-The tag is the upstream version of the first entry of debian-package/debian/changelog,
-without the Debian revision: vpn-router (0.5.1-1) is tagged 0.5.1, which starts the
-release workflow.
+The tag is the version of the first entry of debian-package/debian/changelog, without the
+Debian revision 1: vpn-router (0.5.1-1) is tagged 0.5.1 and (0.5.1-2) 0.5.1-2. The tag
+starts the release workflow.
 """
 
 import argparse
@@ -48,13 +48,14 @@ def summary():
 
 
 def version():
-    """The upstream version of the first changelog entry."""
+    """The tag of the first changelog entry: its version, without the Debian revision when that is 1."""
     first = CHANGELOG.read_text(encoding='utf-8').splitlines()[0]
     start, end = first.find('('), first.find(')')
     if start == -1 or end < start:
         sys.exit(f'{CHANGELOG}: the first line has no version: {first}')
     full = first[start + 1:end]
-    return full.rsplit('-', 1)[0] if '-' in full else full
+    upstream, _, revision = full.rpartition('-')
+    return upstream if upstream and revision == '1' else full
 
 
 def main():