update: ADR-24 names Ubuntu's apt-daily timer, not a cron job, as what updates the package lists.

ADR.md

@@ -26,3 +26,656 @@ because `vpn-router-docker` is already active. A published
 port is reachable from the external interface only when it is in `exposed_ports`. Other rules
 in `DOCKER-USER` are not touched, because the package owns only its own chain. Restarting Docker
 restores the rules. The rules cover IPv4 only.
+
+## ADR-2: A small status web service, on the interfaces web.listen chooses
+
+Status: Accepted
+
+Context: The router has no way to see its state without a shell. A check from the protected
+network, by a person or by a monitor, needs a page or an endpoint, and so does the platform that
+deploys the router, which reaches it on its public address. The router is an internet-facing
+host, so what the service shows and where it listens matter.
+
+Decision: The `vpn-router-web` unit runs one Python script on the standard library's
+`http.server`, with the package's own modules and the Python packages of the Ubuntu archive that
+the package depends on, and no web framework. It listens on `web.port`, 7081 by default, on the
+address of the internal interface, of the external interface or of both, as `web.listen` sets
+(`internal`, `external` or `both`, the default), with a server for each address, and is on
+unless `web.enabled` is `false` (ADR-7). `/` is for people, in plain text for `curl` and `wget`
+and as a Tailwind CSS page for a browser, and `/status` is the same status as JSON. Both show as
+much as `web.status_content` says, since the router is reachable from the internet and the
+operator decides how much it tells strangers: `basic`, the default because it tells the least,
+the overall status and the Client IP, `standard` also the package versions and the number of
+active and other services, and `full` also each service and the state of the site-to-site, road-warrior, OpenVPN and
+WireGuard features. The Client IP is the address the request came from, through the trusted
+proxies, so a person can see the address the router sees. Nothing comes from the configuration.
+`/status` names its content, so clients such as manage-ui and the Router Manager show what it
+has. `vpn-router-setup` opens the
+port on both interfaces in UFW, so the choice of `web.listen` is made by the service alone, and
+writes the features and the facts it resolves to `/run/vpn-router/web.json`.
+
+Consequences: By default the service is reachable from the internet on the external address;
+`web.listen` set to `internal` keeps it to the internal network. It serves plain HTTP itself;
+HTTPS is Caddy's (ADR-21), so on the plain HTTP port the passwords of `/auth` and the tokens
+cross the internet unencrypted. A change of `web.listen`
+needs no run of setup, since the firewall does not depend on it; the package's upgrade runs
+setup, which writes the rules. `/` and `/status` have no authentication, which is acceptable because they are
+read only and show no configuration. It runs as root, because `swanctl` and `wg` need it, with `NoNewPrivileges`,
+`ProtectSystem=strict`, `ProtectHome` and `PrivateTmp`. The Tailwind style comes from a CDN, so a
+browser on a network without internet shows the unstyled page, and `curl` is unaffected.
+
+## ADR-3: The protected paths take JWTs that only the router signs
+
+Status: Accepted
+
+Context: Clients such as the Router Manager, and people with `curl`, need the configuration,
+the users and the groups of the router, which `/` and `/status` must not show. Both people and
+programs sign in with a user of the directory. A secret shared with the platform that deploys
+the router would let whoever holds it, and every copy of the deployment's state, make tokens.
+The service listens on plain HTTP.
+
+Decision: `/config`, `/users` and `/groups` need a JWT in an `Authorization: Bearer` header,
+HS256, verified with `hmac` and `hashlib`; no other algorithm is accepted. The key is
+`/etc/vpn-router/web.key`, random, made by the web service on its first run and readable by root
+only; it is never configured or output. The router itself issues
+the tokens: `POST /auth` checks a user's password and TOTP code with a bind (ADR-14) and answers
+an access token naming the user in `sub`, valid for 5 minutes, and a refresh token valid for 8
+hours (ADR-15); `vpn-router-token` makes tokens with the `admin` claim (ADR-13). A token without
+`admin` must expire within ten minutes, so a leaked one is of little use. Without the key the
+paths and `/auth` are not served. The configuration shown has no secret options (ADR-8).
+
+Consequences: Nothing outside the router can make a token, so a token is either issued to a
+user who signed in or made by root on the router. A client keeps the refresh token to stay
+signed in. A token of `vpn-router-token` can be revoked only by replacing the key, which ends
+every token. Tokens, passwords and answers cross the network unencrypted, so the API is for a
+trusted internal network; HTTPS would need a certificate that the package does not have.
+
+## ADR-4: Users are kept in a local OpenLDAP directory
+
+Status: Accepted
+
+Context: OpenVPN needs user accounts with passwords, and the users are meant to cover every P2S
+system of the router, including IKEv2 EAP-TLS and WireGuard, which an OIDC provider cannot
+authenticate. Everything the router runs comes from the Ubuntu archive. slapd, its Python module
+`python3-ldap` and the OpenVPN plugin `openvpn-auth-ldap` are all in the archive; the plugin
+checks a password by binding as the user, so the hash scheme is slapd's alone. An OIDC provider
+can read the directory through its LDAP connector.
+
+Decision: The users are entries in slapd. The package keeps its own database,
+`dc=vpn-router,dc=local` in `/var/lib/ldap/vpn-router` (the path the slapd AppArmor profile
+allows), next to the database the slapd package creates from the host's domain, so the package
+does not depend on the debconf answers of slapd. Its root DN is root's peer-credential DN on the
+local socket, so `vpn-router-setup` manages it with SASL EXTERNAL and no administrator password
+exists. Users are changed by `cn=user-manager,ou=services`, whose ACLs allow writing `ou=users`
+and setting passwords (`=wx`) but not reading them. Passwords are hashed with ARGON2, and the
+ppolicy overlay locks a disabled user (`pwdAccountLockedTime`, effective with `pwdLockout`).
+slapd listens on the local socket only. Provisioning checks every step before changing it, so it
+runs at every start of `vpn-router-setup`.
+
+Consequences: The package depends on `slapd`, `python3-ldap`, `openvpn` and
+`openvpn-auth-ldap`, and `vpn-router-setup` and `vpn-router-user` import `python3-ldap`. The
+service account's password is a root-only file. The directory is user data: it is kept on purge,
+and it is backed up with `slapcat`, not by copying files.
+
+## ADR-5: OpenVPN clients need a certificate and the password of the user it names
+
+Status: Accepted
+
+Context: The router CA issues client certificates for IKEv2. OpenVPN can check a certificate, a
+password, or both. With both, a certificate alone and a password alone are not enough, but
+`openvpn-auth-ldap` checks only the password of the user name given at login, which need not be
+the user of the certificate.
+
+Decision: OpenVPN checks the certificate against the CA and the CRL, the password with
+`openvpn-auth-ldap`, and runs `openvpn-same-user` as `auth-user-pass-verify` with `via-env`,
+which accepts the login only when the user name equals the certificate's common name. OpenVPN
+calls both and needs both to pass. A user's certificate has the user name as its common name, is
+issued when the user is added and revoked when the user is removed. The CRL is created with the
+CA, so that `crl-verify` has a file from the start, and is valid for ten years, because the router
+renews it only when it revokes a certificate. The server uses the router's own certificate, whose
+common name is `wan.local_fqdn`; the client profile checks that name (`verify-x509-name`), since
+the CA's certificates carry both the server and the client purpose.
+
+Consequences: Disabling a user locks the password, which stops OpenVPN but not IKEv2, which
+checks certificates only; removing a user revokes the certificate for both. A user's
+certificate expires after the CA's default of a year and is renewed by removing and adding the
+user again.
+
+## ADR-6: The web service and vpn-router-user share one module for users
+
+Status: Accepted
+
+Context: Users are managed from the command line, by an operator at a terminal, and through
+`/users` of the web service, by the Router Manager. Both must apply the same rules to names,
+passwords and person attributes, and both change the directory and the CA. `simple-ca` keeps
+the CA's state in one file, so two changes must not run at the same time. The web unit is
+sandboxed with `ProtectSystem=strict`.
+
+Decision: `vpnrouter_users.py` holds the user operations, the checks and a session that takes a
+lock and the connection of the user manager account. The web service imports it and maps a
+`Failure` to the HTTP status; `vpn-router-user` imports it and adds the argument parsing and
+the output, text for people and JSON with `--json`. The web unit allows writing the paths the
+operations need: `/run/vpn-router` for the lock, its `RuntimeDirectory`, which systemd creates
+before the service starts, also before setup has run, and keeps when it stops, as setup's state
+files are there too; `/etc/vpn-router/pki` for the certificates and the CA's state, and
+`/etc/swanctl/x509crl` for the published CRL.
+
+Consequences: The rules exist once. A request starts no process, and a password never leaves
+the process that received it. The web service imports `python3-ldap` and the package's
+configuration module. The API is protected by the tokens of ADR-3, so only members of `admins`
+and the holders of an admin token manage users.
+
+## ADR-7: The web service starts on its own and reads its inputs at each request
+
+Status: Accepted
+
+Context: The status page is the tool to see a problem of `vpn-router-setup`, so it must run
+whatever state setup is in, also before setup has run and while it fails. The addresses,
+`web.listen`, `web.port` and `web.enabled` can change at any time.
+
+Decision: The service depends on nothing to start and does not end on its own. A discovery
+thread works out, every 30 seconds, the addresses of the interfaces `web.listen` names from
+`vpn-router.json` as `general.mode` directs, with the discovery code in `vpnrouter_net.py`,
+which `vpnrouter.py` uses too, and `web.port` and `web.enabled` from the same file. It runs an
+HTTP server in a thread of its own at each address and the port, stops one and starts a new one
+when they change, and runs none while the service is disabled or no address is known; an address
+that cannot be bound is tried again the next round. Each request
+reads the signing key and `vpn-router.json`, and the features and resolved facts from
+`web.json`, which setup writes when it applies the configuration, so the service shows what setup
+applied; while that file is missing the router shows no features. The unit is not
+ordered after `vpn-router-setup`, and setup does not restart it.
+
+Consequences: The status page answers as soon as an address exists, whatever state setup is in,
+and follows a changed address, `web.listen`, port or `web.enabled` within 30 seconds without a
+restart. A new signing key applies to the next request. The interfaces are read every 30
+seconds, which costs a few `ip` commands.
+
+## ADR-8: /config leaves secret options out
+
+Status: Accepted
+
+Context: A secret has no use outside the router, so a client never needs it, and showing even
+that a secret is set tells a reader more than it needs.
+
+Decision: The configuration that `vpn-router-setup` writes for the web service has no secret
+options at all. They are recognised by name: `psk`, `database`, whose connection string may hold
+a password, and options with `secret`, `password`, `token` or a `_key` ending in the name. The
+names are checked at every level, so the settings of each site-to-site connection lose their
+`psk` too; an object is kept whatever its name, so a connection named like a secret is still
+shown.
+
+Consequences: No secret reaches the web service's state file or any answer of the API. Through
+`/config` a client cannot tell whether a secret is set; `/site-to-site` says whether each
+connection has a key (ADR-25), as an admin managing connections needs to know. A new secret
+option needs a name that matches.
+
+## ADR-9: The Router Manager is a separate package with a small Python server
+
+Status: Accepted
+
+Context: The router needs a web application to manage it, which may run on the router or on
+another Linux system. The router's web service issues tokens through `/auth` (ADR-3) and sends
+no CORS headers, so a browser cannot call it directly, and tokens kept in the page could be read
+by its scripts.
+
+Decision: `vpn-router-manager` is a Debian package of its own, built in the repository
+`linux-cloud-router-manager`. A React and MUI application in the style of manage-ui is built
+with Vite when the package is built and shipped as static files. A Python server, using the
+Ubuntu packages `python3-requests` and `python3-psycopg`, serves them, signs users in through the
+router's `/auth`, keeps their tokens in a server-side session in PostgreSQL behind an
+`HttpOnly` cookie, and passes every other `/api/<path>` call to the router's `/<path>` with the
+session's access token, refreshing it as needed; it holds no logic of the router's own and no
+key of it. It reads its settings from `/etc/vpn-router-manager/manager.json` at each request; on
+the router the settings not given come from `vpn-router.json`, and it listens on the internal
+address, the external address or both, as its own `listen` sets, like the status web service,
+with the same discovery thread. `vpn-router-setup` opens the manager's port on the internal and
+the external interface while the manager is installed, so the firewall stays in one place.
+
+Consequences: The router needs no Node.js; the package needs only `python3` and packages of the
+Ubuntu archive. Tokens never reach the browser, and sessions survive a restart of the manager.
+The router decides what a user may do; the manager only signs in. The application checks the
+shape of every answer of the router. The router's firewall depends on whether the manager is
+installed, and a changed manager port needs `vpn-router-setup` to run again.
+
+## ADR-10: Users and their certificates do not depend on a P2S system being enabled
+
+Status: Accepted
+
+Context: Users are prepared before a P2S system is switched on, and the Router Manager manages
+them on any configured router, with or without one.
+
+Decision: Once a platform is chosen, `vpn-router-setup` always provisions the user directory in
+slapd and the CA with its CRL; the server certificate follows once `wan.local_fqdn` is set.
+`/users` and `vpn-router-user` rely only on the directory and the CA. Only the OpenVPN client
+profile needs OpenVPN to be enabled.
+
+Consequences: slapd always holds the directory and the CA always exists on a configured router,
+even when no P2S system uses them. An operator who wants a CA of their own must place it before
+choosing the platform.
+
+## ADR-11: Groups are groupOfNames entries that never become empty
+
+Status: Accepted
+
+Context: Users are to be organised in groups. Groups record membership only; nothing on the
+router reads them. LDAP clients that read groups, such as `openvpn-auth-ldap` and the LDAP
+connectors of OIDC providers, expect `groupOfNames` with member DNs, and OpenLDAP's core schema
+requires a `groupOfNames` to have at least one `member`.
+
+Decision: A group is `cn=<name>,ou=groups,dc=vpn-router,dc=local`, a `groupOfNames` with the
+DNs of its members and an optional `description`. A group is created with at least one member,
+and taking out its last member is refused; such a group is removed instead. Removing a user
+takes it out of its groups and is refused while the user is the last member of one. The group
+operations are in `vpnrouter_users.py` with the user operations, so they share the session and
+its lock; `vpn-router-group` is a tool of its own beside `vpn-router-user`, and the web service
+serves them under `/groups`. Group names follow the rules of user names.
+
+Consequences: A group always satisfies the schema without a placeholder member, and an empty
+group cannot exist. Removing a user or a member can need another step first. A later use of
+groups, such as allowing OpenVPN only to some, can read them as they are.
+
+## ADR-12: TOTP secrets are kept with the OATH-LDAP schema of slapd's otp module
+
+Status: Accepted
+
+Context: Users are enrolled for TOTP one-time passwords, and enrolled users must give a code
+whenever they log in, to OpenVPN and to `/auth`. Ubuntu's slapd ships the `otp` module, which
+holds the OATH-LDAP schema and an overlay that checks a code appended to the password at bind
+time; both OpenVPN (`openvpn-auth-ldap`) and `/auth` check passwords with a bind. Ubuntu
+packages `python3-pyotp`. An authenticator app is enrolled from a QR code of an `otpauth://` URI.
+
+Decision: The `otp` module is loaded and its overlay is configured on the package's database,
+after ppolicy, so every bind of an enrolled user takes the password followed by a current code
+and slapd records the time step used. The TOTP token is the user entry itself: enrolment adds `oathTOTPUser` and `oathTOTPToken` to it, with
+`oathTOTPToken` naming the entry, the secret in `oathSecret` and `oathTOTPParams` naming one
+shared parameter entry, `cn=totp,ou=policies`, with the parameters authenticator apps assume
+(6 digits, HMAC-SHA1, 30 seconds). An ACL lets the user manager account set `oathSecret` and
+no one read it. Enrolment is confirmed: `otp_new` makes a secret with `pyotp` and returns it
+with its URI, storing nothing, and `otp_enrol` stores the secret it is given back only when a
+code of it verifies, so a secret is never stored that the user's app does not produce. The
+router answers the URI; the client renders the QR code.
+
+Consequences: Removing a user removes its token. One check covers OpenVPN and `/auth`, and
+an enrolled user types the code after the password in the OpenVPN client. The secret passes
+through the client between `otp_new` and `otp_enrol`, so the router keeps no pending state;
+enrolling again replaces the secret only once a code of the new one verifies.
+
+## ADR-13: Members of admins manage the router; only CLI tokens change admins
+
+Status: Accepted
+
+Context: Signing in through `/auth` (ADR-3) is open to every user of the directory, most of
+whom only use OpenVPN. Finer permissions per operation are not needed. Somebody must be able to
+manage the router before any administrator exists and after every administrator lost access,
+and administrators must not be able to grant themselves or others the administrator role over
+the API.
+
+Decision: A token naming a user in `sub` may call the protected paths only while the user is a
+member of the group `admins`, read at each request; other users get 403. `vpn-router-token`, run
+as root on the router, makes tokens with the claim `admin: true` and a lifetime given at
+creation, without a maximum, signed with the router's key; `/auth` never issues the claim. An
+admin token may call everything, and only an admin token may change `admins`: create it, rename a
+group to it, change or remove it, add or take out its members, or remove a user who is a member
+of it. The first member is added with `vpn-router-group` on the router, which acts as root.
+
+Consequences: Removing a user from `admins` takes effect at the next request, without waiting
+for the user's tokens to expire. Membership of `admins` changes only through root on the router.
+An admin token is as powerful as root over the API and is revoked only by replacing the key, so
+it is given a short lifetime unless it is meant for automation.
+
+## ADR-14: Failed logins lock the account in slapd; /auth is rate limited per client
+
+Status: Accepted
+
+Context: `/auth` is open to the internal network and checks passwords, so it invites guessing;
+OpenVPN checks passwords from the internet. Both check with a bind, so slapd sees every attempt.
+ppolicy locks an account after a number of failed binds for a fixed time; it has no growing
+lockout. A Router Manager signs all its users in from one address.
+
+Decision: The default password policy locks an account for 15 minutes after 5 failed binds
+within 15 minutes; a successful bind clears the count; `vpn-router-user unlock` and
+`POST /users/<name>/unlock` end a lockout and clear the count, and leave a disabled user disabled.
+`/auth` answers the same 401 for a wrong name, password or code and for a locked or disabled
+user. `/auth` and `/auth/refresh` allow 10 calls a minute per client address, counted in memory,
+and answer 429 with `Retry-After` beyond that. The client address is the peer, or the last
+`X-Forwarded-For` entry for a peer in `web.trusted_proxies`, by default the address the call came
+in on, which is the address a manager on the router calls from.
+
+Consequences: One lockout covers OpenVPN and `/auth`, kept by slapd alone. Someone who knows a
+user name can lock the user out for 15 minutes at a time. The rate limit restarts with the
+service. A manager elsewhere than on the router has its address added to `web.trusted_proxies`,
+or its users share one limit.
+
+## ADR-15: Refresh tokens are kept in PostgreSQL
+
+Status: Accepted
+
+Context: An access token lives 5 minutes, and a client stays signed in with a refresh token for
+8 hours; refresh tokens must be revocable when a user signs out, is disabled, changes the
+password or is removed, and must survive a restart of the service. The Router Manager keeps its
+sessions in PostgreSQL too.
+
+Decision: Refresh tokens are random strings, stored as their SHA-256 with the user name and the
+expiry in the table `refresh_tokens`, through psycopg 3 (`python3-psycopg`). The database is
+`web.database`, or by default the database and role `vpn-router` on the router, which the web
+service creates on its first run (ADR-16) with `runuser -u postgres -- psql` and a random password in
+`/etc/vpn-router/database.secret`, reached over TCP on `localhost`, as Ubuntu's `pg_hba.conf`
+takes peer authentication on the local socket. A refresh token is used once: refreshing deletes
+it and issues a new one, after checking that the user is neither disabled nor locked.
+
+Consequences: The package depends on `postgresql` and `python3-psycopg`. A stolen database holds
+no usable token. Changing a user's password, disabling, unlocking or removing the user, and
+signing out, end the user's sessions at once; an access token already issued lives out its
+5 minutes.
+
+## ADR-16: Each service creates what it needs on its first run
+
+Status: Accepted
+
+Context: The package manager configures packages in an order of its own; on a new VM the Router
+Manager was configured before the PostgreSQL server, so a database created at installation was
+missing. PostgreSQL may also start after the services at boot.
+
+Decision: The web service makes its signing key and creates the refresh tokens' database itself,
+and the Router Manager its sessions database; neither `vpn-router-setup` nor a maintainer script
+creates them. Each service does it before it listens, retrying every 15 seconds and logging each
+failure, then listens. `vpnrouter_auth` turns every failure of the database into
+`DatabaseError`, the only database exception the web service and `vpn-router-user` handle.
+
+psycopg (Ubuntu ships only its pure-Python implementation) finds libpq through the dynamic linker
+cache, which dpkg refreshes with its deferred `ldconfig` only at the end of an installation, so a
+service started during the installation cannot import it yet; a failed import cannot be repeated
+in the same process either, as psycopg caches the lookup. The services therefore import psycopg
+plainly, and their units wait instead: `ExecStartPre=wait-for-libpq`, a shell script of each
+package, waits until `ldconfig -p` lists `libpq.so.5`, within `TimeoutStartSec=5min`. The
+maintainer scripts start the units with `systemctl --no-block` (debhelper's start is turned off),
+because a start that waited would hold dpkg before its `ldconfig`.
+
+Consequences: The order of installation and of the start of PostgreSQL and of the linker cache
+does not matter; a database that cannot be created is retried until the cause is gone, and the
+reason is in the service's log. Outside an installation the wait ends at once. The web unit may
+write `/etc/vpn-router` for the key and the database password.
+
+## ADR-17: Bootstrap diagnostics run only until the first setup succeeds
+
+Status: Accepted
+
+Context: A new VM is reachable only once its setup has worked, and the reason of a failure must
+be readable from the internal network before an SSH session is possible. A service that reports
+the setup cannot be one of the services being set up: those import packages that may not load yet
+(psycopg could not find libpq before `ldconfig` ran), and they would fail before reporting
+anything. Once the router has been set up, SSH is the way to diagnose it. The more a diagnostic
+page does, the more ways it has to fail itself.
+
+Decision: Each package ships a bootstrap service of the Python standard library alone, a script
+that answers every request with the plain-text output of a fixed list of commands: the setup
+state, the services, the failed units, the addresses and the end of the logs. The router's
+listens on 17081 and the Router Manager's on 17080, beside the ports of the services, 7081 and
+7080, on every address. The units are never enabled and not started at installation;
+`vpn-router-setup` starts them while `/var/lib/vpn-router/setup.done` does not exist, and opens
+their ports on the internal interface. The first successful run writes that file, closes the ports
+and stops them; a failed run leaves them running, so the reason stays readable. Setup records its
+state, step and failure in `/run/vpn-router/setup.json` at every step; `vpnrouter.fail` ends with
+a `SystemExit` carrying its message, so setup records it.
+
+Consequences: The diagnostics need nothing the setup provides and cannot collide with the real
+services. After the first success they never run again, also not after an upgrade or a reboot.
+The page shows log lines to whoever reaches the internal network during the first setup.
+
+## ADR-18: Manager-only accounts are a second kind of user; the package can create the first admin
+
+Status: Accepted
+
+Context: The Router Manager needs a first administrator before anyone has added users, as other
+packages create a default user from their debconf answers. Such an account, and others that only
+manage the router, have no person behind them and need no VPN access. A VPN user is an
+`inetOrgPerson`, which requires a last name and a common name, and its certificate needs the
+names. The Manager is configured before the router's directory exists during an installation.
+
+Decision: A user is one of two kinds in `ou=users`, fixed when it is added: a VPN user, an
+`inetOrgPerson` with the three person attributes and a certificate, or a Manager-only account,
+an `account`, `simpleSecurityObject` and `vpnRouterAccount` (ADR-19) with the name, the password,
+an optional e-mail address for notifications, and no certificate. `add` with all three person
+attributes adds a VPN user and without the first and the last name a Manager-only account; one
+name without the rest is refused. The CLI and the API add both kinds; the Manager's dialog
+requires the names unless Manager access only is chosen, which offers the e-mail address only.
+Answers carry `manager_only`, and `update` changes only the e-mail address of a Manager-only
+account. The Manager's package asks `admin_username` and `admin_password`; with both
+it writes them to a root-only file, clears the answers, and `vpn-router-setup` creates the account
+after it provisions the directory, adds it to `admins` (creating the group when missing) and
+removes the file. Without a name nothing is created. `vpnrouter_users` knows only the directory,
+so setup can use it before PostgreSQL's driver can be loaded; `web` and `vpn-router-user` revoke
+refresh tokens after the operations of `vpnrouter_auth.REVOKING`.
+
+Consequences: A Manager-only account cannot become a VPN user, as LDAP cannot change an entry's
+structural class and its password hash cannot be copied; it is removed and added again instead.
+It has no certificate, so OpenVPN and road-warrior access refuse it. The administrator's password
+is on disk only until setup has run.
+
+## ADR-19: The package has an LDAP schema of its own under a UUID OID
+
+Status: Accepted
+
+Context: A Manager-only account (ADR-18) is an `account`, which allows no `mail`, yet notifications
+need an address for it; no standard auxiliary class allows `mail` except `extensibleObject`, which
+allows every attribute. Authorization scopes for users of both kinds may follow. An attribute or
+class of its own needs an OID, and the package has no registered arc.
+
+Decision: The package defines its schema, `cn=vpn-router,cn=schema,cn=config`, which
+`vpnrouter_ldap.provision` adds when missing and keeps as `SCHEMA_ATTRIBUTE_TYPES` and
+`SCHEMA_OBJECT_CLASSES`. Its OID is `2.25.98477352595735834532404256531620013001`, made from a
+random UUID under the arc 2.25 (ITU-T X.667), which needs no registration; attribute types are
+under `.1` and object classes under `.2`. It has the attribute `vpnRouterScope` (multi-valued,
+case-insensitive strings) and the auxiliary class `vpnRouterAccount`,
+`MAY ( mail $ vpnRouterScope $ displayName $ description )`; `displayName` and `description`, the
+standard attributes, are the display name and the description of a service account, which
+`account` cannot hold. Manager-only accounts have the class from their creation, so an e-mail
+address can be added later without changing their classes; VPN users get it once scopes are used.
+`vpnrouter_ldap.provision` replaces the definitions in `cn=config` that differ, so an extended class
+reaches an existing directory at the next setup run.
+
+Consequences: The schema check stays strict. New attributes extend `vpnRouterAccount`, and their
+OIDs continue the arcs. The scope `service` marks a service account (ADR-23). The OID must never
+be reused for anything else.
+
+## ADR-20: Users manage their own account through /account, proving who they are again
+
+Status: Accepted
+
+Context: Users who are not members of `admins` can sign in but get 403 on every path, so they
+cannot change their password or enrol an authenticator without an administrator. A session
+token alone is a weak proof for such changes: whoever holds an open session could lock the user
+out. A wrong current password must not look like an ended session to the Manager, which signs a
+user out on 401.
+
+Decision: `/account` serves the user named in the token's `sub`, admin or not: `GET` the user
+and whether they are a member of `admins`, `PUT /account/password`, and `POST` and
+`PUT /account/otp` to enrol or replace the TOTP enrolment. The routes do not take a user name.
+Each change sends the current password and, for an enrolled user, a current code, checked with a
+bind as the user like `/auth`, so it counts toward the lockout (ADR-14); a wrong one is 403. A
+new password is checked before the bind, so an invalid one does not use up a code. A user cannot
+remove their own enrolment; `/users` and `vpn-router-user` do. Tokens of `vpn-router-token` name
+no user and get 403 there.
+
+Consequences: slapd's `otp` overlay accepts each code once, so a change right after signing in
+needs the next code. Changing the password revokes the user's refresh tokens, so the user's
+Manager session ends with its access token. An enrolled user who has lost the authenticator
+needs an administrator.
+
+## ADR-21: Caddy from the Ubuntu archive serves HTTPS with a certificate from Let's Encrypt
+
+Status: Accepted
+
+Context: The status web service and the Router Manager are reachable from the internet (ADR-2),
+and `/auth` and the Manager's sign-in carry passwords. Both are small servers on the standard
+library's `http.server`, which would need certificates, their renewal and a TLS setup of their
+own. Browsers trust a public CA, not the router's.
+
+Decision: Caddy, the `caddy` package of the Ubuntu archive, is the HTTPS proxy. While
+`https.enabled` is `true` (the default) and `wan.local_fqdn` is set, `vpn-router-setup` writes
+`/etc/caddy/vpn-router.Caddyfile`: a site for `wan.local_fqdn` on `https.web_port` (7444) for the
+web service and, while the Router Manager is installed, one on `https.manager_port` (7443) for
+the manager, each passing calls to the service on `127.0.0.1`. Caddy obtains and renews the
+certificate from Let's Encrypt with the HTTP challenge on port 80, which the firewall opens on
+the external interface; `auto_https disable_redirects` keeps port 80 to the challenge. The
+package's drop-in for `caddy.service` points it at that file and keeps it from starting before
+the file exists, so `/etc/caddy/Caddyfile`, a configuration file of the `caddy` package, is left
+unchanged. Both services always listen on `127.0.0.1` as well as on the interfaces their
+`listen` settings name, and trust `X-Forwarded-For` from it, which Caddy sets from the peer and
+never takes from the client.
+
+Consequences: HTTPS needs `wan.local_fqdn` in public DNS pointing at the router and port 80
+reachable from the internet; until then Caddy retries and the HTTPS ports do not answer. The
+certificate and the ACME account are in `/var/lib/caddy`. Plain HTTP stays as the `listen`
+settings set it, so `internal` keeps passwords off the internet. Caddy's admin endpoint listens
+on `localhost:2019`, which `caddy reload` uses. Port 80 and the HTTPS ports are not available
+to Docker containers while HTTPS is on. The Manager's HTTPS port is a setting of the router, not
+of the manager.
+
+## ADR-22: Metrics are kept in PostgreSQL and served in the Prometheus format, pushed with OTLP
+
+Status: Accepted
+
+Context: The router should keep its CPU, memory, disk, network, load and VPN client metrics for
+hour, day, month and year views, as Proxmox VE does, and expose them to OpenTelemetry. Ubuntu
+noble has no OpenTelemetry Python SDK and no OpenTelemetry Collector package. OTLP itself is
+push-only; a collector reads a scraped endpoint in the Prometheus format with its `prometheus`
+receiver. The router already runs PostgreSQL for the refresh tokens (ADR-15).
+
+Decision: A service of its own, `vpn-router-metrics`, takes a sample every minute with `psutil`
+(`python3-psutil`) and the VPN readers it shares with the web service (`vpnrouter_status`). It
+keeps the samples in the router's database, in four detail tables (`metrics_system`,
+`metrics_network`, `metrics_vpn`, `metrics_site_to_site`, see ADR-25). Each detail table has hourly, daily, monthly and yearly summary
+tables, with the average and the maximum of each value per period, written when the clock passes a
+period from the level below with `INSERT ... ON CONFLICT DO NOTHING`, so a stopped service catches
+up. Garbage collection runs on start and once a day, not with every write, and deletes the rows
+older than the retention of their level, `metrics.keep_*`, PostgreSQL intervals. The service serves
+the last sample as `/metrics` in the Prometheus text format with `prometheus_client`
+(`python3-prometheus-client`) and pushes it as OTLP/HTTP JSON, built with the standard library, to
+`metrics.otlp_endpoint`. The names follow the OpenTelemetry semantic conventions. The web service
+answers the views from the tables at `/metrics/history`, protected like `/config`, for the Router
+Manager.
+
+Consequences: The metrics live in the database of the refresh tokens and are dropped with it on
+purge. A view shows only what its level keeps, so a shorter retention empties the older part of a
+view. A month view needs the hourly rows and a year view the monthly rows. `/metrics` has only the
+last sample; history is not scraped.
+
+## ADR-23: /metrics takes HTTP Basic authentication of an internal service account the router manages
+
+Status: Accepted
+
+Context: An OpenTelemetry Collector's `prometheus` receiver authenticates with HTTP Basic, not with
+the router's tokens, which last minutes (ADR-3). The scraper should not be able to sign in to the
+API or the Manager, and its password should be checked and locked out like any other. The account
+belongs to the package's metrics, not to the people using the router.
+
+Decision: A service account is a Manager-only account (ADR-18) with `vpnRouterScope` `service`,
+which another system signs in with to use a router service. Administrators add service accounts
+with `vpn-router-user add --service` and the web service's `/service-accounts`.
+`/service-accounts` is a path of its own, apart from `/users`, which does not list or serve them:
+a service account has no names, certificate, profile or TOTP, and is not a person an
+administrator manages with the users. A service account has a display name and a description
+besides its name, so an administrator can tell what each is for. Internal service accounts are
+those the router creates and manages for its own services (`vpnrouter_users.internal_accounts`),
+with every property, display name and description included, so the system, not an
+administrator, owns them; at present that is the metrics service's, which setup names Metrics. `/service-accounts` marks
+them as `internal` and only lists them: every other request on one, and putting one in or taking
+one out of a group at `/groups`, gets 403, whatever the token. The router changes them through the
+directory's user-manager account, not the API, and root may still with `vpn-router-user`. For the
+metrics service, `vpn-router-setup` adds the account `metrics.service_account` names, `metrics` by
+default, while metrics are enabled, with a random password it writes to
+`/etc/vpn-router/metrics.secret` (root only), and sets a new random password, ending a lockout,
+when the file is missing or no longer holds the password. `/auth` refuses every service account with the message of a wrong password. `/metrics` accepts HTTP Basic only for that account, and
+only when it is a service account; the password is checked with a bind as the account, so the
+password policy's lockout applies (ADR-14). TLS is Caddy's, on `metrics.https_port` (ADR-21);
+plain HTTP is served on the interfaces `metrics.listen` names, `internal` by default.
+
+Consequences: Every scrape binds to slapd once. The collector's operator copies the password from
+the router; deleting the file and running setup changes it. A user who already has the name is not
+taken over; setup fails until `metrics.service_account` names another. An account root disabled
+stays disabled. A scraper on the external interface over plain HTTP sends the password in the clear, so
+`metrics.listen` stays `internal` unless HTTPS is used.
+
+## ADR-24: The Router Manager upgrades the router's packages in a transient unit outside its own
+
+Status: Accepted
+
+Context: An admin should upgrade `vpn-router` and `vpn-router-manager` from the Router Manager. The
+upgrade restarts the process that starts it: the manager's postinst restarts
+`vpn-router-manager.service`, and the router's restarts `vpn-router-web` and `vpn-router-setup`. A
+child of the manager would be stopped with the manager's cgroup in the middle of dpkg, leaving
+packages half configured, and the manager's unit cannot run apt at all, since `ProtectSystem=strict`
+makes `/usr` and `/var/lib/dpkg` read-only. Ubuntu's `apt-daily` timer updates the package lists
+twice a day, so a release newer than its last run is not seen until the next one.
+
+Decision: The manager starts the helper `/usr/lib/vpn-router-manager/upgrade` with `systemd-run` as
+the transient unit `vpn-router-manager-upgrade`, of its own cgroup and without a sandbox, and does
+not wait for it. The helper:
+
+- finishes an interrupted dpkg run with `dpkg --configure -a`;
+- runs `apt-get install --only-upgrade` of the two packages, non-interactive, keeping changed configuration files, with `needrestart` suspended, and waiting for the dpkg lock that `apt-daily` or `unattended-upgrades` may hold;
+- records the state and the result in `/var/lib/vpn-router-manager/upgrade.json`, replaced by a rename, and the output in `upgrade.log`.
+
+A restarted manager reads them again. A state of running without an active unit is shown as interrupted. The manager reboots the router the same way, as a transient timer `vpn-router-manager-reboot` 5 seconds later, so the answer reaches the browser first. It checks for updates the same way too: Check update runs `apt-get update` in the transient unit `vpn-router-manager-check`, which touches apt's update stamp when it succeeds, so an admin need not wait for the `apt-daily` timer. Only members of `admins`, as the router's `/account` answers, may start any of them.
+
+Consequences: The fixed unit name allows one upgrade at a time. The page loses the manager and the
+router's web service for a while during an upgrade and reads again until they answer. Sessions
+survive in PostgreSQL. The manager can upgrade only the system it runs on, so a manager on another
+host upgrades that host's packages, not the router's. The operating system's other packages are
+left to unattended-upgrades.
+
+## ADR-25: Site-to-site connections are a map of named connections, edited through the API and applied by setup
+
+Status: Accepted
+
+Context: The router had one IPsec site-to-site tunnel, the `remote` section, run as the swanctl
+connection `remote-site`, with the local identity mode in `wan` and the local subnets in `local`.
+A router often links more than one site, each with its own peer, subnets and key, and an admin
+should add them without a shell. The install answers and the Cloud Platform describe one
+connection, and the platform rewrites `vpn-router.json` when it rebuilds the VM.
+
+Decision: `site_to_site` maps a connection name to its settings: `addrs`, `id`, `cidrs`,
+`local_cidrs`, `local_id_mode`, `psk` and `psk_file`. `remote` and `wan.local_id_mode` are gone,
+without a fallback for the old format. The name, a lowercase letter followed by at most 30
+lowercase letters, digits and hyphens, names the swanctl connection, its child and its secret
+`ike-psk-<name>`, so it is a valid swanctl section name and stays the same while the connection
+lives. The install answers and the Cloud Platform write the connection `remote-site`, the first
+one, so its name is the one the tunnel always had.
+
+- **Local settings per connection:** each connection has its own `local_cidrs` and
+  `local_id_mode`, as each peer expects its own traffic selectors and identity. `wan.local_fqdn`
+  stays the router's one name, used by the `fqdn` mode, resolved once for the `public_ip` mode,
+  and used by HTTPS, road-warrior and OpenVPN. `local.cidrs` stays the subnets the router
+  routes, masquerades and offers to road-warrior and OpenVPN clients.
+- **One file:** `vpn-router-setup` writes every connection that is ready to
+  `/etc/swanctl/conf.d/site-to-site.conf`, with the proposals and timers the one tunnel had, and
+  removes the file when none is. The NAT exemptions pair each connection's `local_cidrs` with its
+  `cidrs`.
+- **Editing:** the status web service serves `/site-to-site` to members of `admins` and admin
+  tokens, like `/users`. It never returns a key, only whether one is set, and takes a new key only
+  when one is given. `vpn-config-edit` edits the same map. Both read, check with the validation
+  of `vpn-router-setup` and write `vpn-router.json` holding one lock, replacing the file in one
+  step, so neither loses the other's change.
+- **Applying:** the web service restarts `vpn-router-setup` without waiting for it, the one
+  place that renders, loads and opens the firewall, so a change through the API takes the same
+  path as a change by hand.
+- **Metrics per connection:** the metrics service keeps, per connection, whether it is up, the
+  bytes and packets per second through its child SAs and the seconds its IKE SA has been up, in
+  a detail table of its own, `metrics_site_to_site`, keyed by the connection's name, with the
+  summaries and retention of the other tables (ADR-22). One row per connection keeps the values
+  of a tunnel together and lets the set of connections change without changing the schema;
+  `metrics_vpn` counts only clients. swanctl's child SA counters start again at a rekey, so the
+  service keeps rates computed between two samples, treating a lower counter as counted from
+  zero, and `/metrics` and the OTLP push serve them as gauges rather than counters that would
+  appear to reset.
+
+Consequences: A configuration with `remote` is refused, so the package and a configuration in
+the new format go together. The status shows each connection. A rate across a rekey misses the
+traffic of the old child SA since the previous sample. A renamed connection starts a new series.
+A connection added through the API lives in
+`vpn-router.json` only; the Cloud Platform's rebuild of the VM writes the file again with its
+one connection, dropping the others. A change through the API is answered before setup has
+applied it, so a configuration that setup cannot apply shows in the setup state, not in the
+answer. A connection with a key file keeps it until a key is given through the API, which then
+replaces the file.

README.md

@@ -4,9 +4,13 @@ A Debian package that configures a Linux host or virtual machine as a VPN router
 
 - site-to-site IKEv2 IPSec with pre-shared key authentication (strongSwan swanctl)
 - road-warrior (P2S) access using IKEv2 EAP-TLS, with a bundled certificate authority
+- OpenVPN (P2S) access with a certificate and a password per user, the users kept in a local
+  LDAP directory and managed with `vpn-router-user` or the web service
 - optional WireGuard endpoint
 - routing and NAT for a protected subnet, TCP MSS clamping, and UFW firewall rules
 - firewall rules for the ports Docker publishes, when Docker is installed
+- metrics of the router kept in PostgreSQL, served for an OpenTelemetry Collector and pushed
+  over OTLP
 
 Nothing in the package is specific to a cloud provider. It is built on the external and
 internal NIC names, and a `mode` setting says how much of the rest you are supplying and
@@ -83,8 +87,10 @@ that deploy a router on Azure or GCP.
 ## Configuration
 
 `/etc/vpn-router/vpn-router.json` is JSON, mode 0600, and is created on first install. It
-is managed with `ucf`, so it is yours to edit: a later `dpkg-reconfigure` applies what
+is managed with `ucf`, so it is yours to edit: a later `dpkg-reconfigure` applies the answers
-changed and asks before touching anything you altered by hand. A complete example is
+given and asks before touching anything you altered by hand, and an upgrade asks no questions
+and leaves the file as it is. Settings the file leaves out take their defaults when it is read, so a
+file written by cloud-init or by hand needs only the settings that differ. A complete example is
 installed at `/usr/share/doc/vpn-router/vpn-router.json.example`.
 
 The file is an object of sections, each an object of settings. A section or setting that
@@ -97,24 +103,69 @@ must have the type shown below; an unknown section or setting is an error.
 | `general` | `mode` | string | `manual` | How much you supply: `manual`, `interfaces` or `auto` |
 | `interfaces` | `external` | string | | Name of the NIC facing the untrusted network |
 | `interfaces` | `internal` | string | | Name of the NIC facing the protected network |
-| `wan` | `local_fqdn` | string | | This router's name or IP address, used as its IKE identity in `fqdn` mode |
+| `wan` | `local_fqdn` | string | | This router's name or IP address: its IKE identity in `fqdn` mode, the name resolved in `public_ip` mode, and the name of HTTPS, road-warrior and OpenVPN |
-| `wan` | `local_id_mode` | string | `fqdn` | IKE identity source: `fqdn`, `public_ip` or `internal_ip` |
+| `local` | `cidrs` | list | | Local subnets routed via the internal side, masqueraded to the external one, and offered to road-warrior and OpenVPN clients |
-| `local` | `cidrs` | list | | Local subnets advertised into the tunnel |
 | `local` | `int_addr` | string | | This host's address on the internal network |
 | `local` | `int_gateway_ip` | string | | Next hop on the internal side |
-| `remote` | `addrs` | list | | Remote gateway addresses or FQDNs |
+| `site_to_site` | | object | | The site-to-site connections, each a name and an object of the settings below |
-| `remote` | `id` | string | | Remote IKE identity, an FQDN or an IPv4 or IPv6 address |
-| `remote` | `cidrs` | list | | Remote subnets reachable through the tunnel |
-| `remote` | `psk` | string | | Pre-shared key |
-| `remote` | `psk_file` | string | | Path to a file holding the key; wins over `psk` |
 | `p2s` | `enabled` | boolean | `false` | Road-warrior access |
 | `p2s` | `address_pool` | string | | CIDR assigned to road-warrior clients |
 | `p2s` | `ca_name` | string | `VPN Router CA` | Common name for a locally created CA |
 | `wireguard` | `enabled` | boolean | `false` | WireGuard endpoint |
 | `wireguard` | `address` | string | | Address and prefix for `wg0`, for example `192.168.200.1/24` |
 | `wireguard` | `listen_port` | integer | `51820` | UDP port |
+| `openvpn` | `enabled` | boolean | `false` | OpenVPN access for users |
+| `openvpn` | `address_pool` | string | | IPv4 CIDR assigned to OpenVPN clients |
+| `openvpn` | `port` | integer | `1194` | Port of the OpenVPN server on the external interface |
+| `openvpn` | `protocol` | string | `udp` | `udp` or `tcp` |
 | `docker` | `mode` | string | `auto` | `auto` secures Docker when its service is installed, `on` requires it, `off` removes the rules |
 | `docker` | `exposed_ports` | list | | TCP ports and port ranges (`8000:8100`) reachable from the external interface; at most 15, a range counting as two |
+| `web` | `enabled` | boolean | `true` | The status web service |
+| `web` | `listen` | string | `both` | The interfaces the status web service listens on: `internal`, `external` or `both` |
+| `web` | `port` | integer | `7081` | TCP port of the status web service |
+| `web` | `database` | string | | A PostgreSQL connection string for the refresh tokens; without it, the `vpn-router` database on this host that the web service creates |
+| `web` | `trusted_proxies` | list | the address the call came in on | Addresses whose `X-Forwarded-For` header names the client for the `/auth` rate limit and the Client IP of the status |
+| `web` | `status_content` | string | `basic` | What `/` and `/status` show: `basic`, `standard` or `full` (see Status web service) |
+| `https` | `enabled` | boolean | `true` | Caddy serves the status web service and the Router Manager over HTTPS for `wan.local_fqdn` |
+| `https` | `email` | string | | The e-mail address of the Let's Encrypt account |
+| `https` | `web_port` | integer | `7444` | HTTPS port of the status web service |
+| `https` | `manager_port` | integer | `7443` | HTTPS port of the Router Manager |
+| `metrics` | `enabled` | boolean | `true` | The metrics service: collection, history and `/metrics` |
+| `metrics` | `listen` | string | `internal` | The interfaces `/metrics` is served on: `internal`, `external` or `both` |
+| `metrics` | `port` | integer | `9464` | TCP port of `/metrics` |
+| `metrics` | `https_port` | integer | `9465` | HTTPS port of `/metrics`, served by Caddy while `https.enabled` is true |
+| `metrics` | `service_account` | string | `metrics` | The service account `/metrics` takes, which `vpn-router-setup` adds and keeps |
+| `metrics` | `otlp_endpoint` | string | | An OTLP/HTTP URL the metrics are pushed to every minute, for example `https://collector.example.com:4318/v1/metrics` |
+| `metrics` | `otlp_username` | string | | The user name of HTTP Basic authentication at the OTLP endpoint |
+| `metrics` | `otlp_password` | string | | Its password |
+| `metrics` | `keep_details` | string | `24 hours` | How long the per-minute metrics are kept |
+| `metrics` | `keep_hourly` | string | `1 month` | How long the hourly summaries are kept |
+| `metrics` | `keep_daily` | string | `3 months` | How long the daily summaries are kept |
+| `metrics` | `keep_monthly` | string | `1 year` | How long the monthly summaries are kept |
+| `metrics` | `keep_yearly` | string | `10 years` | How long the yearly summaries are kept |
+
+### Site-to-site connections
+
+`site_to_site` maps a connection name to its settings. A name starts with a lowercase letter
+and has at most 31 lowercase letters, digits and hyphens; it names the strongSwan connection,
+its child and its secret (`ike-psk-<name>`). The install answers fill the connection
+`remote-site`; more are added in the file, with `vpn-config-edit`, or through `/site-to-site`
+of the status web service, which the Router Manager uses.
+
+| Setting | Type | Default | Meaning |
+|---|---|---|---|
+| `addrs` | list | | Remote gateway addresses or FQDNs |
+| `id` | string | | Remote IKE identity, an FQDN or an IPv4 or IPv6 address |
+| `cidrs` | list | | Remote subnets reachable through the tunnel |
+| `local_cidrs` | list | | Local subnets advertised into the tunnel |
+| `local_id_mode` | string | `fqdn` | The router's IKE identity: `fqdn` is `wan.local_fqdn`, `public_ip` the address it resolves to, `internal_ip` the internal address |
+| `psk` | string | | Pre-shared key |
+| `psk_file` | string | | Path to a file holding the key; wins over `psk` |
+
+A connection is set up once all of `addrs`, `id`, `cidrs`, `local_cidrs`, the key and its
+local identity are known; all such connections are written to
+`/etc/swanctl/conf.d/site-to-site.conf`. Traffic from a connection's `local_cidrs` to its
+`cidrs` is not masqueraded.
 
 A minimal site-to-site configuration in `auto` mode:
 
@@ -123,19 +174,22 @@ A minimal site-to-site configuration in `auto` mode:
   "general": { "platform": "generic", "mode": "auto" },
   "wan": { "local_fqdn": "router.example.com" },
   "local": { "cidrs": ["10.0.0.0/24"] },
-  "remote": {
+  "site_to_site": {
-    "addrs": ["peer.example.net"],
+    "remote-site": {
-    "id": "peer.example.net",
+      "addrs": ["peer.example.net"],
-    "cidrs": ["192.168.0.0/24"],
+      "id": "peer.example.net",
-    "psk": "s3cr3t"
+      "cidrs": ["192.168.0.0/24"],
+      "local_cidrs": ["10.0.0.0/24"],
+      "psk": "s3cr3t"
+    }
   }
 }
 ```
 
-The router's IKE identity (`wan.local_fqdn` in `fqdn` mode) and the remote one (`remote.id`) are
+The router's IKE identity (`wan.local_fqdn` in `fqdn` mode) and the remote one (`id`) are
 an FQDN or an IPv4 or IPv6 address. They are written to the strongSwan configuration as given;
 strongSwan reads an address as an IP identity and a name as an FQDN, so a leading `@` is not needed
-and is refused. Road-warrior access needs a name in `wan.local_fqdn`.
+and is refused. Road-warrior and OpenVPN access need a name in `wan.local_fqdn`.
 
 WireGuard peers are added by hand in `/etc/wireguard/wg0.conf`. The package owns the
 `[Interface]` section of that file and rewrites it when the settings above change, but it
@@ -214,12 +268,37 @@ external interface, in this order:
 Traffic from every other interface, including the internal one and the Docker bridges, is
 not affected. The rules cover IPv4 only.
 
-The `vpn-router-docker` service applies the rules after `docker.service` starts and removes
+`vpn-router-setup` starts after `docker.service`, so Docker is up when it looks for it; the
+ordering has no effect on a host without Docker. The `vpn-router-docker` service applies the
+rules after `docker.service` starts and removes
 them when Docker stops, so restarting Docker restores them. `systemctl restart
 vpn-router-setup` applies a changed configuration while Docker runs. With `docker.mode` set to
 `on`, the service fails when Docker is not installed or its daemon is not running; with
 `off` it removes the rules.
 
+## Editing the configuration
+
+`vpn-config-edit`, run as root, shows and changes `/etc/vpn-router/vpn-router.json`:
+
+```sh
+vpn-config-edit show
+vpn-config-edit set web.status_content standard
+vpn-config-edit set web.port 7082
+vpn-config-edit set web.trusted_proxies 127.0.0.1,10.0.0.4
+vpn-config-edit set site_to_site.branch.addrs branch.example.net
+vpn-config-edit remove site_to_site.branch
+```
+
+`show` prints the file as indented JSON and does not change it. `set` takes a setting as
+`section.option`, or as `site_to_site.NAME.option` for a site-to-site connection, which it
+creates with the defaults when it does not exist, and converts the value to the setting's type:
+a whole number, `true` or `false`, a list separated by commas (an empty value is an empty list),
+or a string; an unknown setting is refused. `remove site_to_site.NAME` removes a connection. It
+checks the whole configuration as `vpn-router-setup` does, and only then writes the file,
+indented, with mode 0600, and reminds that `vpn-router-setup` applies it. The status web service
+writes the file the same way, and a lock keeps the two from overwriting each other's change. A
+refused request exits with status 2, any other failure with 1.
+
 ## Applying changes
 
 ```sh
@@ -229,8 +308,10 @@ systemctl restart vpn-router-setup
 The service also runs at boot. With platform `none` it does nothing and says so.
 Otherwise it regenerates the configuration files, resolves the interfaces and their
 addresses as `mode` directs, applies routes, generates WireGuard keys and injects
-firewall rules. It then starts `strongswan` and `wg-quick@wg0` if the corresponding
+firewall rules. It provisions the user directory in `slapd` while OpenVPN or road-warrior
-feature is configured, reloading or restarting them only when their inputs changed, and
+access is configured. It then starts `strongswan`, `wg-quick@wg0` and
+`openvpn-server@vpn-router` if the corresponding feature is configured, reloading or
+restarting them only when their inputs changed, and
 restarts `systemd-resolved` only when the road-warrior DNS drop-in changed. Turning a
 feature off in the configuration stops its service and removes its files. Every step is
 idempotent, so restarting with no changes does nothing.
@@ -259,10 +340,12 @@ ca_cert.pem  ca_key.pem  ca_bundle.pem  <label>_cert.pem  <label>_key.pem  crl.p
 `<label>` is the first component of the router FQDN, so `router.example.com` gives
 `router_cert.pem`.
 
-If road-warrior access is enabled and this directory is empty, a certificate authority and
+Once a platform is chosen and while this directory is empty, a certificate authority and a CRL
-a server certificate are created there automatically. To use your own instead, place the
+valid for ten years are created there automatically, whether or not road-warrior or OpenVPN
-files in that layout before enabling the feature; nothing is generated or overwritten when
+access is enabled. A CA of the router's own (with `ca_key.pem`) without a CRL gets one, and
-material is already present.
+without a server certificate gets one once `wan.local_fqdn` is set. User certificates are issued
+by `vpn-router-user` into `users/`. To use your own CA instead, place the files in that layout
+before choosing the platform; nothing present is overwritten.
 
 Either way, the package copies what each consumer needs into place: the CA into
 `/etc/swanctl/x509ca/`, the server certificate and key into `/etc/swanctl/x509/` and
@@ -280,6 +363,172 @@ Issue a client certificate with the bundled tool:
 `simple-ca` comes from a separate project, <https://gitea.koszewscy.waw.pl/slawek/simple-ca>,
 and also supports `make-crl` and `revoke-cert`.
 
+## OpenVPN and users
+
+With `openvpn.enabled`, an `address_pool` and `wan.local_fqdn` set, `vpn-router-setup` runs
+`openvpn-server@vpn-router` with `/etc/openvpn/server/vpn-router.conf`. The server uses the
+router's CA, certificate and CRL (the copies under `/etc/swanctl`), pushes routes to
+`local.cidrs` and the router as DNS server, and opens `openvpn.port` on the external interface.
+
+A client needs both a certificate of the router CA and the password of the user the
+certificate names:
+
+- the certificate is checked against the CA and the CRL (`crl-verify`, read at every
+  connection);
+- the password is checked by `openvpn-auth-ldap`, which binds to the user directory as the
+  user; a user enrolled for TOTP types the password followed by the current 6-digit code;
+- `/usr/lib/vpn-router/openvpn-same-user` refuses a login whose user name is not the common
+  name of the certificate.
+
+### The user directory
+
+Once a platform is chosen, `vpn-router-setup` keeps a directory of users in the local slapd,
+whether or not OpenVPN or road-warrior access is enabled, so users can be managed before either
+is switched on:
+
+- its own database, `dc=vpn-router,dc=local` in `/var/lib/ldap/vpn-router`, next to the one the
+  slapd package creates; root manages it through the local socket, with no administrator
+  password;
+- `ou=users` holds one entry per user, named by `uid`: an `inetOrgPerson` for a VPN user, an
+  `account` for a Manager-only account (see Managing users);
+- the package's own schema, `cn=vpn-router,cn=schema,cn=config`, under the OID
+  `2.25.98477352595735834532404256531620013001` of the UUID arc: the auxiliary class
+  `vpnRouterAccount` with `MAY ( mail $ vpnRouterScope $ displayName $ description )`, which
+  Manager-only and service accounts have, and the attribute `vpnRouterScope`, authorization
+  scopes, of which `service` marks a service account; `displayName` and `description` are the
+  display name and the description of a service account;
+- `ou=groups` holds one `groupOfNames` per group, named by `cn`, with its members' DNs in
+  `member` and an optional `description`;
+- `cn=user-manager,ou=services` is the account that changes users and groups; it may write
+  `ou=users` and `ou=groups` and set passwords, never read them, and its password is in
+  `/etc/vpn-router/ldap/user-manager.secret` (root only);
+- passwords are stored as ARGON2 hashes, and a disabled user is locked with the password policy
+  (`pwdAccountLockedTime`);
+- the password policy locks an account for 15 minutes after 5 failed binds within 15 minutes,
+  whether they come from OpenVPN or from `/auth`; a successful bind clears the count, and
+  `vpn-router-user unlock` ends a lockout;
+- slapd's `otp` overlay is configured on the database; a user enrolled for TOTP is also an
+  `oathTOTPUser` and `oathTOTPToken` whose `oathTOTPToken` names the entry itself, with the
+  secret in `oathSecret`, which the user manager account may set and no one can read, and
+  `oathTOTPParams` naming `cn=totp,ou=policies` (6 digits, HMAC-SHA1, 30 second steps, one step
+  of tolerance). A bind of an enrolled user takes the password followed by a current code;
+- slapd listens on its local socket only (`SLAPD_SERVICES` in `/etc/default/slapd`).
+
+The directory is kept on purge.
+
+### Managing users
+
+`vpn-router-user` manages the users. A user is a directory entry of one of three kinds, fixed when
+it is added:
+
+- a VPN user, an `inetOrgPerson` with a first name, a last name and an e-mail address
+  (`givenName`, `sn` and `mail`; `cn` is the first and the last name), and a client certificate
+  with the user name as its common name, in `/etc/vpn-router/pki/users`;
+- a Manager-only account, an `account`, `simpleSecurityObject` and `vpnRouterAccount` with the
+  name, the password and an optional e-mail address, for notifications: no names and no
+  certificate, so no VPN access. It can sign in to `/auth` and, as a member of `admins`, manage
+  the router. `add` without the first and the last name adds one; its names cannot be set later;
+- a service account, a Manager-only account whose `vpnRouterScope` is `service`, which another
+  system signs in with to use a router service, such as an OpenTelemetry Collector reading
+  `/metrics`; it cannot sign in to `/auth`. Besides its name, it has a display name and a
+  description. `add --service` adds one, and the web service manages them at `/service-accounts`,
+  not at `/users`. Internal service accounts are those the router creates and manages for its own
+  services, display name and description included; at present that is the metrics service's, the
+  account `metrics.service_account` names while metrics are enabled (see Metrics), which
+  `vpn-router-setup` names Metrics. The API only lists
+  them: `/service-accounts` refuses every other request on one, and `/groups` refuses to put one
+  in or take one out of a group, with 403.
+
+Passwords and TOTP secrets are read from the first line of standard input. The answer is text: a
+table of the users, the client profile, a TOTP secret or a short message, and a failure is a
+message on standard error.
+`--json`, before the command, gives the answer as JSON on standard output, and a failure as
+`{"error": ...}` there too; the web service uses it. `-h` or `--help` shows the commands, and
+after a command its options.
+
+```sh
+echo 's3cr3t-pass' | vpn-router-user add alice --first-name Alice --last-name Smith --email alice@example.com
+echo 'm4nager-pass' | vpn-router-user add ops
+echo 's3rvice-pass' | vpn-router-user add backup --service
+vpn-router-user update alice --email a.smith@example.com
+vpn-router-user list
+vpn-router-user profile alice > alice.ovpn
+vpn-router-user --json list
+echo 'n3w-pass' | vpn-router-user passwd alice
+vpn-router-user disable alice
+vpn-router-user enable alice
+vpn-router-user unlock alice
+vpn-router-user remove alice
+vpn-router-user otp-new alice
+echo 'JBSWY3DPEHPK3PXPJBSWY3DPEHPK3PXP' | vpn-router-user otp-enrol alice 123456
+vpn-router-user otp-remove alice
+```
+
+| Command | Effect |
+| --- | --- |
+| `list` | The users with their kind, names and e-mail addresses, whether each is enabled or locked, has a certificate and is enrolled for TOTP |
+| `add <name> --first-name F --last-name L --email E` | Adds a VPN user and issues the certificate |
+| `add <name> [--email E]` | Adds a Manager-only account, with the e-mail address when given; one name without the other two options is refused |
+| `add <name> --service [--display-name N] [--description D] [--email E]` | Adds a service account, with its display name, description and e-mail address when given; the names are refused |
+| `update <name> [--first-name F] [--last-name L] [--email E]` | Changes the names or the e-mail address |
+| `update <name> [--display-name N] [--description D] [--email E]` | Changes the display name, the description or the e-mail address of a service account; `''` removes one |
+| `passwd <name>` | Sets the password |
+| `disable <name>`, `enable <name>` | Locks the user out of OpenVPN and `/auth`, and lets the user in again |
+| `unlock <name>` | Ends a lockout after failed logins and clears the failure count; a disabled user stays disabled |
+| `remove <name>` | Takes the user out of its groups, revokes the certificate, renews the CRL and removes the user; refused while the user is the last member of a group |
+| `profile <name>` | The OpenVPN client profile, with the CA, the certificate and the key inline |
+| `otp-new <name>` | A new TOTP secret and its `otpauth://` URI, issued by `wan.local_fqdn`, for an authenticator app; nothing is stored |
+| `otp-enrol <name> <code>` | Stores the secret read from standard input when the code is a current code of it, replacing a secret the user has |
+| `otp-remove <name>` | Removes the user's TOTP secret |
+
+A user name is 1 to 32 lowercase letters, digits and hyphens, starting with a letter; a
+password has 8 to 128 characters; a first or last name has 1 to 64 printable characters; an
+e-mail address has one `@`, no spaces and a dot in its domain; a TOTP secret is 32 base32
+characters, as `otp-new` makes it, and a code 6 digits. Once enrolled, a user logs in to OpenVPN
+and `/auth` with the password followed by a current code. In the answers `manager_only`, `service`
+and `internal` tell the kind, and a Manager-only account has its names as `null`; `update` changes only its e-mail
+address, and `--email ''` removes it. Changing a user's password, disabling, unlocking or removing the user also revokes its
+refresh tokens of `/auth`. The exit status is 0 on success, 2 for an invalid request, 3 when the
+user does not exist, 4 when the user exists and 1 for any other failure.
+
+When the Router Manager is installed with an administrator (its debconf questions
+`vpn-router-manager/admin_username` and `admin_password`), its package leaves the answers in
+`/etc/vpn-router-manager/admin.json` (root only). `vpn-router-setup` then creates that user as a
+Manager-only account, adds it to the group `admins`, which it creates when missing, and removes
+the file.
+
+The certificate is also accepted by road-warrior (IKEv2 EAP-TLS) access, which checks
+certificates only, so `disable` does not affect it; `remove` revokes the certificate for both.
+
+### Managing groups
+
+`vpn-router-group` manages the groups. A group has a name, with the rules of a user name, an
+optional description of at most 256 printable characters, and at least one member, each a user
+of the directory. Groups record membership only; nothing on the router reads them. The output,
+`--json`, `-h` and `--help` work as for `vpn-router-user`.
+
+```sh
+vpn-router-group add ops --member alice --member bob --description 'Operations team'
+vpn-router-group list
+vpn-router-group update ops --name netops
+vpn-router-group update netops --description ''
+vpn-router-group member-add netops carol
+vpn-router-group member-remove netops bob
+vpn-router-group remove netops
+```
+
+| Command | Effect |
+| --- | --- |
+| `list` | The groups with their members and descriptions |
+| `add <name> --member U [--member U ...] [--description D]` | Adds the group with its members |
+| `update <name> [--name N] [--description D]` | Renames the group or sets its description; `''` removes it |
+| `remove <name>` | Removes the group; its members are kept |
+| `member-add <name> <user>` | Adds the user to the group |
+| `member-remove <name> <user>` | Takes the user out of the group; refused for the last member |
+
+The exit status is 0 on success, 2 for an invalid request, 3 when the group or the user does
+not exist, 4 when the group exists and 1 for any other failure.
+
 ## Platform modules
 
 Platform-specific values do not belong in the core, so they are Python modules in
@@ -321,18 +570,430 @@ EXTRA_LOCAL_TS = '192.0.2.53/32'
 Set `general.platform` to `example` and restart the service. The debconf question offers only the
 modules shipped with the package; the configuration file accepts any module installed.
 
+## Status web service
+
+The `vpn-router-web` service shows the state of the router on the addresses of the interfaces
+`web.listen` names, the internal, the external or both (the default), and on `127.0.0.1`, where
+Caddy calls it (see HTTPS), with `web.port` (7081 by default). Its pages `/` and `/status` need no authentication and show no
+configuration: no key, identity, subnet, or address but the client's. They show as much as
+`web.status_content` says, read at each request:
+
+| `status_content` | `/` and `/status` show |
+|---|---|
+| `basic` (default) | The overall status (`ok` or `degraded`), the time and the Client IP |
+| `standard` | Also the `vpn-router` version, the `vpn-router-manager` version or that it is not installed, and the number of active services and of services in any other state, without their names |
+| `full` | Also each service and its state, Docker, and the state of each site-to-site connection, road-warrior, OpenVPN and WireGuard |
+
+The Client IP is the address the request came from: the last entry of `X-Forwarded-For` for a
+request from one of `web.trusted_proxies`, by default the address the request came in on, so
+behind Caddy or a Router Manager on the router it is the browser's. `/status` names the content in
+`content` and the Client IP in `client_ip`; `/` keeps the time and the link to `/status`.
+
+The service first makes its signing key and the refresh tokens' database, retrying every 15
+seconds, for example while PostgreSQL is not running yet, and logs each failure; then it listens.
+
+### Bootstrap diagnostics
+
+Until a run of `vpn-router-setup` has succeeded, every run starts `vpn-router-bootstrap`, which
+serves diagnostics as plain text on port 17081 of every address, and the firewall opens 17081 and
+17080 (the Router Manager's bootstrap) on the internal interface. The first successful run stops
+the bootstrap services and closes their ports, and writes `/var/lib/vpn-router/setup.done`, after
+which they never run again; a failed run leaves them running, so the reason stays readable until
+a later run succeeds. The unit is never enabled.
+
+The page, read again every 10 seconds, shows the state, the step and the failure of the last
+setup run (`/run/vpn-router/setup.json`, which setup writes at every step), the state of the
+router's services, the failed units, the addresses and routes, `cloud-init status`, and the end of
+the logs of `vpn-router-setup` and `vpn-router-web`:
+
+```bash
+curl http://10.0.0.4:17081/
+```
+
+| Page | Content |
+| --- | --- |
+| `/` | The status for people: plain text for `curl` and `wget`, a page styled with Tailwind CSS for a browser, in the browser's light or dark scheme, that reloads itself every minute (the style sheet is loaded from a CDN, so the browser needs internet access; the text is readable without it) |
+| `/status` | The same status as JSON |
+| `/auth` | Signing in with a user's password and TOTP code, refreshing and signing out |
+| `/config` | The configuration without its secret options, and what the router resolved and runs, as JSON; needs a token |
+| `/users` | The users of OpenVPN and road-warrior access; needs a token |
+| `/service-accounts` | The service accounts; needs a token |
+| `/groups` | The groups of users; needs a token |
+| `/site-to-site` | The site-to-site connections: list, add, change and remove; needs a token |
+| `/account` | The account of the signed-in user: its details, its password and its TOTP enrolment; needs a user's token |
+
+The status holds the state of `vpn-router-setup`, `slapd` and each PostgreSQL cluster unit
+(`postgresql@<version>-<cluster>`, or `postgresql` as `no cluster` without one), of
+`vpn-router-manager` when that package is installed, of `strongswan` when site-to-site or
+road-warrior access is configured, of `wg-quick@wg0` when WireGuard is, and of
+`openvpn-server@vpn-router` when OpenVPN is; the number of connected
+OpenVPN clients, from the server's status file, which OpenVPN rewrites every minute; the package version and,
+as `manager_version`, the installed version of `vpn-router-manager` (`null`, shown as not installed, without it); whether
+Docker is installed, and when it is, the state of `docker.service` and of `vpn-router-docker`
+(the Docker rules), which do not change `status`; as `site_to_site.connections`, whether each
+site-to-site connection that is set up is established (`null` when swanctl cannot be asked); the number of connected road-warrior (IPsec P2S) clients; and
+the WireGuard peers with the age of their last handshake and their traffic. `status` is `ok` when
+all these services are active, and `degraded` otherwise.
+
+```bash
+curl http://10.0.0.4:7081/
+curl http://10.0.0.4:7081/status
+```
+
+### Tokens and signing in
+
+`/config`, `/users`, `/service-accounts`, `/groups`, `/site-to-site` and `/account` need a JWT in an `Authorization: Bearer` header, signed with
+HS256 by the key the web service makes on its first run, `/etc/vpn-router/web.key` (root only).
+The key never leaves the router; while it does not exist these paths and `/auth` are not served
+(404). A missing, expired or wrongly signed token, or one of another algorithm, gets 401 with
+`WWW-Authenticate: Bearer`. There are two kinds of tokens:
+
+- an access token of `/auth` names a user in `sub` and is valid for 5 minutes; every user may
+  call `/account`, and for the other paths the user must be a member of the group `admins`,
+  otherwise the request gets 403;
+- a token of `vpn-router-token`, run as root on the router, has the claim `admin` and is valid
+  for the lifetime given, without a maximum. It names no user, so `/account` refuses it with 403;
+  it may call everything else, and only such a token may
+  change the group `admins`: create it, rename a group to it, change or remove it, add or take
+  out its members, or remove a user who is a member of it.
+
+The first member of `admins` is added on the router:
+
+```sh
+vpn-router-group add admins --member alice --description 'Router administrators'
+vpn-router-token --hours 1
+vpn-router-token --json --days 30
+```
+
+`POST /auth` signs a user in. Request and answer bodies are JSON:
+
+| Request | Body | Answer |
+| --- | --- | --- |
+| `POST /auth` | `{"name", "password", "code"}`; `code` only for a user enrolled for TOTP | the tokens |
+| `POST /auth/refresh` | `{"refresh_token"}` | new tokens; the refresh token given is used up |
+| `POST /auth/logout` | `{"refresh_token"}` | `{"revoked": true}`, or `false` for a token not known |
+
+The tokens are `{"access_token", "token_type": "Bearer", "expires_in": 300, "refresh_token",
+"refresh_expires_in": 28800}`. A wrong name, password or code, and a locked or disabled user,
+get 401 with the same message; a refresh token is refused once its user is disabled, locked or
+removed. Changing a user's password, disabling, unlocking or removing the user revokes the
+user's refresh tokens. Five failed logins lock the account for 15 minutes (see the user
+directory). `/auth` and `/auth/refresh` take at most 10 calls a minute from one client address,
+and answer 429 with `Retry-After` beyond that; for a call from one of `web.trusted_proxies`, by
+default the address the call came in on, the client address is the last entry of
+`X-Forwarded-For`.
+
+Refresh tokens are kept, as their SHA-256, in PostgreSQL: by default in the database and role
+`vpn-router` on the router, which the web service creates on its first run with a password in
+`/etc/vpn-router/database.secret` (root only), reached over TCP on `localhost`; `web.database`
+names another database. Tokens and passwords travel over plain HTTP on the internal network.
+
+```bash
+curl -H 'Content-Type: application/json' -d '{"name": "alice", "password": "s3cr3t-pass", "code": "123456"}' \
+     http://10.0.0.4:7081/auth
+curl -H "Authorization: Bearer $TOKEN" http://10.0.0.4:7081/config
+```
+
+### The /config page
+
+`/config` shows what `/` and `/status` leave out, for a client such as manage-ui:
+
+| Key | Content |
+| --- | --- |
+| `configuration` | `vpn-router.json` with the defaults applied, without its secret options at any level (`psk`, `database`, and any option named like a secret, password, token or key) |
+| `network` | The external and internal interface, their addresses, the internal gateway, `local.cidrs`, `site_to_site`: each connection with `name`, `local_identity`, `remote_identity`, `local_cidrs` and `remote_cidrs`, and the IPv4 routes as `ip -j -4 route` prints them |
+| `ipsec` | `connections` and `sas`: one object per connection and per IKE SA, read from `swanctl --list-conns --pretty` and `swanctl --list-sas --pretty`, with the connection's name under `name` and swanctl's keys and string values |
+| `wireguard` | The public key and port of `wg0`, and each peer's public key, endpoint, allowed IPs, handshake age and traffic; no private or preshared key |
+| `docker` | `docker.exposed_ports` and the rules of the `VPN-ROUTER-DOCKER` chain |
+| `postgresql` | `clusters`: each PostgreSQL cluster of `pg_lsclusters` with its version, name, port, status and data directory, and for one that is online its `settings`: `server_version`, `listen_addresses`, `port`, `max_connections`, `shared_buffers`, `effective_cache_size`, `work_mem`, `password_encryption`, `ssl` and `timezone`; `null` when `pg_lsclusters` fails |
+
+### The /postgresql page
+
+`/postgresql`, with the same token as `/config`, shows the databases of the PostgreSQL clusters on the
+router:
+
+- **Content:** `clusters` lists each cluster with its version, name, port and status, its `databases` (name, owner, encoding and size in bytes, `pg_database_size`) and their total `size`.
+- **A cluster that is not online:** `databases` and `size` are `null`.
+- **Collection:** the web service reads it with `psql` as the `postgres` user at each request.
+
+### The /users API
+
+`/users` manages users with the same token as `/config`, with the module `vpn-router-user` uses
+too, `/usr/lib/vpn-router/vpnrouter_users.py`, so both apply the same rules. Request
+and answer bodies are JSON; a request body is at most 4096 bytes.
+
+| Request | Body | Answer |
+| --- | --- | --- |
+| `GET /users` | | `{"users": [{"name", "manager_only", "service", "internal", "enabled", "locked", "certificate", "otp", "first_name", "last_name", "email"}, ...]}` |
+| `POST /users` | `{"name", "password", "first_name", "last_name", "email"}` for a VPN user; `{"name", "password", "email"?}` for a Manager-only account | 201 and the user |
+| `PUT /users/<name>` | any of `{"first_name", "last_name", "email"}`; for a Manager-only account only `{"email"}`, `""` removing it | the user |
+| `PUT /users/<name>/password` | `{"password": ...}` | the user |
+| `POST /users/<name>/disable` | | the user |
+| `POST /users/<name>/enable` | | the user |
+| `POST /users/<name>/unlock` | | the user |
+| `DELETE /users/<name>` | | `{"name", "removed": true}` |
+| `GET /users/<name>/profile` | | `{"name", "profile"}`, the OpenVPN client profile |
+| `POST /users/<name>/otp` | | `{"name", "secret", "uri"}`, a new TOTP secret; nothing is stored |
+| `PUT /users/<name>/otp` | `{"secret", "code"}` | the user, once the code matches the secret |
+| `DELETE /users/<name>/otp` | | the user |
+
+A failure is `{"error": ...}` with 400 for an invalid request, 404 for a user that does not
+exist or is a service account, 409 for a user that exists and 500 otherwise. `/users` does not
+list service accounts; `/service-accounts` serves them. `/users` works whether or not OpenVPN or
+road-warrior access is enabled; the profile needs OpenVPN. Passwords, private keys and TOTP
+secrets cross the internal network in plain HTTP, like the token.
+
+```bash
+curl -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
+     -d '{"name": "alice", "password": "s3cr3t-pass", "first_name": "Alice", "last_name": "Smith", "email": "alice@example.com"}' \
+     http://10.0.0.4:7081/users
+```
+
+### The /service-accounts API
+
+`/service-accounts` manages the service accounts with the same token and the same module as
+`/users`.
+
+| Request | Body | Answer |
+| --- | --- | --- |
+| `GET /service-accounts` | | `{"service_accounts": [{"name", "display_name", "description", "internal", "enabled", "locked", "email", ...}, ...]}`, each as `/users` shows a user |
+| `POST /service-accounts` | `{"name", "password", "display_name"?, "description"?, "email"?}` | 201 and the account |
+| `PUT /service-accounts/<name>` | any of `{"display_name", "description", "email"}`; `""` removes one | the account |
+| `PUT /service-accounts/<name>/password` | `{"password": ...}` | the account |
+| `POST /service-accounts/<name>/disable` | | the account |
+| `POST /service-accounts/<name>/enable` | | the account |
+| `POST /service-accounts/<name>/unlock` | | the account |
+| `DELETE /service-accounts/<name>` | | `{"name", "removed": true}` |
+
+A display name has at most 64 printable characters and a description at most 256. The internal
+service accounts, `internal` true, are only listed: every other request on one gets 403, as the
+router manages them, their display name and description included. A failure is otherwise as for
+`/users`, with 404 also for a name that is not a service account.
+
+### The /account API
+
+`/account` serves the user the access token names, whether or not the user is a member of
+`admins`, with the same module as `/users`:
+
+| Request | Body | Answer |
+| --- | --- | --- |
+| `GET /account` | | the user as `/users` shows it, and `"admin"`, whether the user is a member of `admins` |
+| `PUT /account/password` | `{"current_password", "current_code", "password"}` | the user |
+| `POST /account/otp` | | `{"name", "secret", "uri"}`, a new TOTP secret; nothing is stored |
+| `PUT /account/otp` | `{"current_password", "current_code", "secret", "code"}` | the user, once the code matches the secret |
+
+`current_code` is needed only for a user enrolled for TOTP. A wrong current password or code gets
+403 and counts as a failed login; slapd accepts each code once, so a code used to sign in is not
+accepted again, and a change right after signing in waits for the next code. A user enrols or
+replaces the enrolment; removing it is left to `/users` and `vpn-router-user`. Changing the
+password revokes the user's refresh tokens. Other failures are those of `/users`.
+
+### The /groups API
+
+`/groups` manages groups with the same token and the same module as `/users`.
+
+| Request | Body | Answer |
+| --- | --- | --- |
+| `GET /groups` | | `{"groups": [{"name", "description", "members": [...]}, ...]}` |
+| `POST /groups` | `{"name", "members": [...]}`, and `"description"` | 201 and the group |
+| `PUT /groups/<name>` | any of `{"name", "description"}`; `""` removes the description | the group |
+| `DELETE /groups/<name>` | | `{"name", "removed": true}` |
+| `POST /groups/<name>/members` | `{"user": ...}` | the group |
+| `DELETE /groups/<name>/members/<user>` | | the group |
+
+A failure is `{"error": ...}` with 400 for an invalid request, among them taking out the last
+member, 404 for a group or user that does not exist, 409 for a group that exists and 500
+otherwise. `DELETE /users/<name>` is refused with 400 while the user is the last member of a
+group.
+
+### The /site-to-site API
+
+`/site-to-site` manages the site-to-site connections of `vpn-router.json` with the same token as
+`/users`.
+
+| Request | Body | Answer |
+| --- | --- | --- |
+| `GET /site-to-site` | | `{"connections": [{"name", "addrs", "id", "cidrs", "local_cidrs", "local_id_mode", "psk_file", "psk_set", "established"}, ...]}` |
+| `POST /site-to-site` | `{"name", "addrs", "id", "cidrs", "local_cidrs", "local_id_mode", "psk"}`; `psk` is required | 201 and the connection |
+| `PUT /site-to-site/<name>` | any of the settings of `POST` but `name`; an empty or absent `psk` keeps the key, a `psk` given replaces `psk_file` | the connection |
+| `DELETE /site-to-site/<name>` | | `{"name", "removed": true}` |
+
+A connection is shown without its key: `psk_set` says whether `psk` holds one, and `established`
+whether its tunnel is up, `null` when swanctl cannot be asked. Each change is made under the lock
+`vpn-config-edit` takes, checked as `vpn-router-setup` checks the configuration, written to
+`vpn-router.json`, and applied by restarting `vpn-router-setup` without waiting for it. A failure
+is `{"error": ...}` with 400 for an invalid request or configuration, with the reason, 404 for a
+connection that does not exist, 409 for a name that exists and 500 otherwise. Keys cross the
+internal network in plain HTTP, like the token.
+
+The service starts on its own, without waiting for `vpn-router-setup`, and keeps running. Every
+30 seconds it reads `web.listen`, `web.port` and `web.enabled` from `vpn-router.json` and works
+out the addresses of the interfaces `web.listen` names as `general.mode` directs; it listens on
+each, moves when an address or the port changes, and stops listening while `web.enabled` is
+`false` or no address is known. A change of `web.listen` takes effect within 30 seconds, or when
+the service restarts. Each request reads the signing key and `vpn-router.json`, and the features
+and the facts for `/config` from `/run/vpn-router/web.json` (readable by root only), which
+`vpn-router-setup` writes; before that file exists the page shows no features.
+`vpn-router-setup` opens `web.port` on the internal and the external interface in the firewall.
+
+## HTTPS
+
+Caddy, from Ubuntu's `caddy` package, serves the status web service on `https.web_port` (7444 by
+default), while the Router Manager is installed the manager on `https.manager_port` (7443), and
+while metrics are enabled the metrics service on `metrics.https_port` (9465), all for `wan.local_fqdn`, while `https.enabled` is `true` and `wan.local_fqdn` is set. It obtains
+and renews the certificate from Let's Encrypt with the HTTP challenge, so `wan.local_fqdn` must
+resolve to the router's public address and port 80 must be reachable from the internet; the
+certificate and the ACME account are kept in `/var/lib/caddy`. Port 80 answers the challenge only;
+it does not redirect to HTTPS.
+
+`vpn-router-setup` writes Caddy's configuration to `/etc/caddy/vpn-router.Caddyfile` (root, group
+`caddy`, the group Caddy runs as, mode 0640), which the
+package's drop-in `/usr/lib/systemd/system/caddy.service.d/vpn-router.conf` makes `caddy.service`
+read; `caddy.service` does not start before the file exists, and `/etc/caddy/Caddyfile` is not
+used. Setup starts Caddy, reloads it when the file changes, and stops it and removes the file when
+HTTPS is off. In the firewall it opens port 80 on the external interface and the HTTPS ports on the
+internal and the external interface. Caddy calls both services on `127.0.0.1`, where they listen
+whatever `web.listen` and the manager's `listen` say, and passes the client's address in
+`X-Forwarded-For`, which the manager trusts from `127.0.0.1` and the web service by default, as
+the address the call came in on; a `web.trusted_proxies` that is set must list `127.0.0.1`. The plain HTTP ports stay as
+`web.listen` and the manager's `listen` set them. `docker.exposed_ports` may not take port 80 or
+the HTTPS ports while `https.enabled` is `true`.
+
+```bash
+curl https://router.example.com:7444/status
+```
+
+## Metrics
+
+While `metrics.enabled` is `true`, `vpn-router-setup` runs `vpn-router-metrics`, which every minute
+takes a sample of the router with `psutil`:
+
+- the CPU utilization;
+- the memory used and in total;
+- the free and the total space of `/`;
+- the bytes received and sent per second on the external and the internal interface;
+- the 1, 5 and 15 minute load averages;
+- the VPN clients of each enabled VPN system: road-warrior (IPsec P2S) and OpenVPN clients, and
+  WireGuard peers with a handshake in the last 3 minutes;
+- for each site-to-site connection that is set up: whether it is up (1 or 0), the bytes and the
+  packets per second received and sent through its child SAs since the previous sample, and the
+  seconds its IKE SA has been up. The child SAs' counters start again at a rekey; a counter lower
+  than in the previous sample counts from zero.
+
+### History
+
+The samples are kept in the router's PostgreSQL database, the one of the refresh tokens
+(`web.database`), in four detail tables, one row per minute, and per interface, kind of client or
+site-to-site connection, and each is summarized hourly, daily, monthly and yearly. A summary row
+has the start of its period, in UTC, and the average and the maximum of each value; the average
+of a connection's `up` is the part of the period it was up:
+
+| Level | Tables | Kept |
+|---|---|---|
+| Details | `metrics_system`, `metrics_network`, `metrics_vpn`, `metrics_site_to_site` | `metrics.keep_details` |
+| Hourly | `metrics_system_hourly`, `metrics_network_hourly`, `metrics_vpn_hourly`, `metrics_site_to_site_hourly` | `metrics.keep_hourly` |
+| Daily | `metrics_system_daily`, `metrics_network_daily`, `metrics_vpn_daily`, `metrics_site_to_site_daily` | `metrics.keep_daily` |
+| Monthly | `metrics_system_monthly`, `metrics_network_monthly`, `metrics_vpn_monthly`, `metrics_site_to_site_monthly` | `metrics.keep_monthly` |
+| Yearly | `metrics_system_yearly`, `metrics_network_yearly`, `metrics_vpn_yearly`, `metrics_site_to_site_yearly` | `metrics.keep_yearly` |
+
+`metrics_site_to_site` has the columns `connection`, `up`, `rx`, `tx`, `packets_rx`, `packets_tx`
+and `uptime`.
+
+How it works:
+
+- **Summaries:** a period is summarized once it is complete, from the level below. A stopped service catches up on the periods it missed, as long as their rows are still kept.
+- **Garbage collection:** runs on start and once a day. It deletes the rows older than their retention, a count and a unit such as `24 hours`, `3 months` or `10 years`.
+- **History API:** `GET /metrics/history?range=hour|day|month|year` of the status web service, protected like `/config`, answers the views of the Router Manager's Metrics tab. Each point has the system values, `network` by interface, `clients` by kind and `site_to_site` by connection, each value as its `avg` and `max`:
+
+  | Range | Bucket | Source |
+  |---|---|---|
+  | `hour` | 1 minute | details |
+  | `day` | 10 minutes | details |
+  | `month` | 6 hours | hourly |
+  | `year` | 1 month | monthly |
+
+- **Site-to-site availability:** `site_to_site_up` of the same answer divides the view into slots of its own, `slot`, and gives for each slot and connection the percentage of the samples in which the connection was up, 0 to 100, as `points` of `{"time", "connections": {name: percent}}`:
+
+  | Range | Slot | Source |
+  |---|---|---|
+  | `hour` | 5 minutes | details |
+  | `day` | 1 hour | details |
+  | `month` | 1 day | hourly |
+  | `year` | 1 month | monthly |
+
+  A sample is taken every minute, so an interruption shorter than a minute may be missed; a slot without samples has no value, and the year view, like the others of its level, has the complete months only.
+
+### Scraping
+
+`GET /metrics` of the metrics service serves the last sample in the Prometheus text format, written by `prometheus_client`.
+
+- **Metric names:** the OpenTelemetry semantic convention names in their Prometheus form:
+  - `system_cpu_utilization`;
+  - `system_memory_usage_bytes` and `system_memory_limit_bytes`;
+  - `system_filesystem_usage_bytes` and `system_filesystem_utilization`;
+  - `system_network_io_bytes_total`;
+  - `system_cpu_load_average_1m`, `_5m` and `_15m`;
+  - `vpn_router_clients`, by `kind`: `p2s`, `openvpn` and `wireguard`;
+  - by `connection`, for each site-to-site connection: `vpn_router_site_to_site_up`, `vpn_router_site_to_site_uptime_seconds`, and, by `direction` too, `vpn_router_site_to_site_io_bytes_per_second` and `vpn_router_site_to_site_packets_per_second`. The rates are gauges, since the child SAs' counters start again at each rekey.
+- **Listening:** on the interfaces `metrics.listen` names, `internal` by default, at `metrics.port`, and on `127.0.0.1`. While HTTPS is on, Caddy serves it at `metrics.https_port` for `wan.local_fqdn`. `vpn-router-setup` opens both ports on the internal and the external interface.
+- **Authentication:** HTTP Basic as the service account `metrics.service_account` names, `metrics` by default. The password is checked with a bind to the user directory, so 5 failed attempts lock the account as for any user.
+- **The account:** `vpn-router-setup` owns it while metrics are enabled.
+  - It adds the account with a random password, which it writes to `/etc/vpn-router/metrics.secret` (root only). Give that password to the collector.
+  - When the file is missing or no longer holds the account's password, setup sets a new random password and ends any lockout. Delete the file and restart `vpn-router-setup` to change the password.
+  - A disabled account is left as it is.
+  - A user of that name who is not a service account is a setup failure.
+
+An OpenTelemetry Collector scrapes it with its `prometheus` receiver:
+
+```yaml
+receivers:
+  prometheus:
+    config:
+      scrape_configs:
+        - job_name: vpn-router
+          scrape_interval: 60s
+          scheme: https
+          basic_auth:
+            username: metrics
+            password_file: /etc/otelcol/vpn-router-metrics.secret   # a copy of /etc/vpn-router/metrics.secret
+          static_configs:
+            - targets: ['router.example.com:9465']
+```
+
+### Pushing
+
+When `metrics.otlp_endpoint` is set, every sample is also pushed there as an OTLP/HTTP JSON request.
+
+- **Content:** the semantic convention names (`system.cpu.utilization`, `system.network.io` and so on), `vpn_router.clients`, the gauges `vpn_router.site_to_site.up`, `.uptime`, `.io` and `.packets` with the attribute `connection`, and the resource attributes `service.name` `vpn-router` and `host.name`.
+- **Authentication:** HTTP Basic with `metrics.otlp_username` and `metrics.otlp_password` when they are set.
+- **Failures:** a failed push is logged and not repeated; the next sample is pushed a minute later.
+
+## Router Manager
+
+The `vpn-router-manager` package, built in the repository `linux-cloud-router-manager`, is a web
+application that shows the state of the router and manages its users and site-to-site
+connections through the status web service. It can be installed on the router or on another Linux system. While it is installed on
+the router, `vpn-router-setup` opens its port (`port` in `/etc/vpn-router-manager/manager.json`,
+7080 by default, HTTP; HTTPS is Caddy's, on `https.manager_port`) on the internal and the external
+interface; restart `vpn-router-setup` after changing it. The manager's `listen` chooses the
+interfaces it listens on.
+
 ## Removal
 
 `apt-get remove` stops `vpn-router-docker`, which removes the Docker rules. It terminates the
-tunnels and takes `wg-quick@wg0` down, but leaves the
+tunnels and takes `wg-quick@wg0`, `openvpn-server@vpn-router`, `caddy` and `vpn-router-metrics`
+down, but leaves the
 configuration in place. `strongswan` is left enabled, since it is a shared service that
 may be in use by something else.
 
 `apt-get purge` additionally removes the generated files and the firewall rules, restoring
 `/etc/ufw/before.rules` to its original content, and hands `vpn-router.json` back to `ucf`
-before deleting it so a later reinstall starts clean. Key material is never deleted:
+before deleting it so a later reinstall starts clean. It also removes the web service's
-`/etc/vpn-router/pki`, the WireGuard key pair and everything under `/etc/swanctl` are left
+signing key, the password file of the metrics service account, and the database of the refresh
-alone.
+tokens and the metrics, its role and their password
+file when the package created them, and Caddy's configuration `/etc/caddy/vpn-router.Caddyfile`; the certificate in
+`/var/lib/caddy` belongs to the `caddy` package. Key material and users are never deleted: `/etc/vpn-router/pki`, `/etc/vpn-router/ldap`, the user directory in slapd, the
+WireGuard key pair and everything under `/etc/swanctl` are left alone.
 
 ## Building
 

debian-package/debian/changelog

@@ -1,3 +1,629 @@
+vpn-router (0.19.1-3) unstable; urgency=medium
+
+  update: ADR-24 names Ubuntu's apt-daily timer, not a cron job, as what updates the package lists.
+
+  * ADR-24 says that Ubuntu's apt-daily timer updates the package lists
+    twice a day, and that apt-daily or unattended-upgrades may hold the
+    dpkg lock the upgrade waits for.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Tue, 27 Oct 2026 10:00:00 +0100
+
+vpn-router (0.19.1-2) unstable; urgency=medium
+
+  update: ADR-24 says that the Router Manager checks for updates on demand.
+
+  * ADR-24 describes Check update of the Router Manager, which updates
+    apt's package lists in the transient unit vpn-router-manager-check.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Tue, 27 Oct 2026 09:00:00 +0100
+
+vpn-router (0.19.1-1) unstable; urgency=medium
+
+  fix: /metrics/history gives the availability of each site-to-site connection as the percentage of time it was up in slots of the view.
+
+  * site_to_site_up of /metrics/history divides the view into slots of 5
+    minutes for an hour, 1 hour for a day, 1 day for a month and 1 month
+    for a year, and gives for each slot and connection the percentage of
+    the samples in which the connection was up.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Mon, 26 Oct 2026 21:00:00 +0100
+
+vpn-router (0.19.0-1) unstable; urgency=medium
+
+  feature: service accounts have a display name and a description, which the router sets for its internal ones.
+
+  * vpnRouterAccount of the package's schema also takes displayName and
+    description; vpn-router-setup updates the schema of an existing
+    directory.
+  * /service-accounts shows display_name and description, takes them when
+    adding a service account and changes them with PUT; "" removes one.
+    /users shows them as null.
+  * vpn-router-user add --service and update take --display-name and
+    --description.
+  * vpn-router-setup gives the metrics service's internal account the
+    display name Metrics and its description, and sets them again when
+    they differ.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Mon, 26 Oct 2026 20:00:00 +0100
+
+vpn-router (0.18.3-1) unstable; urgency=medium
+
+  fix: internal service accounts are the router's own, of which the metrics service's is the one at present, and the documentation says what a service account is.
+
+  * /service-accounts, /groups and the internal field of the answers
+    take every internal service account, the accounts the router creates
+    and manages for its own services, instead of only the metrics
+    service's; the 403 answers say that the router manages them.
+  * The README, the ADR and the users module describe a service account
+    as the account another system, such as an OpenTelemetry Collector
+    reading /metrics, signs in with, and the internal ones as managed by
+    the router.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Mon, 26 Oct 2026 19:00:00 +0100
+
+vpn-router (0.18.2-1) unstable; urgency=medium
+
+  fix: service accounts are managed at /service-accounts, not /users, and the internal one of the metrics service is only listed.
+
+  * /service-accounts, for members of admins and admin tokens, lists the
+    service accounts, adds one with a password and an optional e-mail
+    address, changes the e-mail address, sets the password, disables,
+    enables, unlocks and removes them.
+  * /users no longer lists service accounts or takes "service" on POST,
+    and answers 404 to a request naming one.
+  * /service-accounts answers 403 to every request but the list on the
+    account metrics.service_account names while metrics are enabled, and
+    /groups to putting it in or taking it out of a group;
+    vpn-router-setup and root's vpn-router-user still change it.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Mon, 26 Oct 2026 18:00:00 +0100
+
+vpn-router (0.18.1-1) unstable; urgency=medium
+
+  fix: vpn-router-setup owns only the creation of the metrics service's account, which is marked internal, and administrators add service accounts with vpn-router-user add --service and /users.
+
+  * POST /users takes "service": true and adds a service account, which
+    cannot sign in to /auth; vpn-router-user add takes --service.
+  * The users of /users and vpn-router-user have internal, true for the
+    account metrics.service_account names while metrics are enabled;
+    vpn-router-user list shows its kind as internal.
+  * PUT /users/<name>/password refuses the internal service account,
+    whose password vpn-router-setup keeps.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Mon, 26 Oct 2026 17:00:00 +0100
+
+vpn-router (0.18.0-2) unstable; urgency=medium
+
+  update: release.py tags a Debian revision other than 1 as x.y.z-n.
+
+  * release.py tags the newest entry x.y.z for the Debian revision 1 and
+    x.y.z-n for a later revision, so a packaging change gets a release
+    of its own.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Mon, 26 Oct 2026 14:00:00 +0100
+
+vpn-router (0.18.0-1) unstable; urgency=medium
+
+  feature: site_to_site holds any number of named site-to-site connections, which /site-to-site lists, adds, changes and removes.
+
+  * site_to_site maps a connection name to addrs, id, cidrs, local_cidrs,
+    local_id_mode, psk and psk_file; it replaces remote and
+    wan.local_id_mode, and a configuration with them is refused.
+  * The install answers fill the connection remote-site; the local subnets
+    fill both its local_cidrs and local.cidrs.
+  * Every ready connection is written to
+    /etc/swanctl/conf.d/site-to-site.conf as the swanctl connection and
+    child of its name with the secret ike-psk-<name>; remote-site.conf is
+    no longer written.
+  * Traffic from each connection's local_cidrs to its cidrs is not
+    masqueraded.
+  * /site-to-site, for members of admins and admin tokens, lists the
+    connections without their keys, adds, changes and removes them, and
+    restarts vpn-router-setup to apply the change.
+  * vpn-config-edit sets site_to_site.NAME.option, creating the
+    connection, and removes site_to_site.NAME; it and the web service
+    write vpn-router.json under one lock.
+  * The status shows site_to_site.connections, whether each connection is
+    established.
+  * The metrics service keeps, per site-to-site connection, whether it is
+    up, the bytes and packets per second received and sent, and the
+    seconds its IKE SA has been up, in metrics_site_to_site and its
+    summaries; /metrics/history has them in site_to_site of each point.
+  * /metrics serves vpn_router_site_to_site_up,
+    vpn_router_site_to_site_uptime_seconds,
+    vpn_router_site_to_site_io_bytes_per_second and
+    vpn_router_site_to_site_packets_per_second by connection, and the
+    OTLP push the vpn_router.site_to_site gauges; vpn_router_clients and
+    metrics_vpn no longer have the kind site_to_site.
+  * /config shows each connection in network.site_to_site, and no psk at
+    any level of the configuration.
+  * Removing the package terminates every site-to-site connection; purging
+    removes site-to-site.conf.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 23:00:00 +0100
+
+vpn-router (0.17.0-1) unstable; urgency=medium
+
+  feature: /config shows the router's PostgreSQL clusters and their settings, and /postgresql their databases with their sizes.
+
+  * /config has a postgresql section: each cluster of pg_lsclusters with
+    its version, name, port, status and data directory, and the main
+    settings of each online cluster.
+  * /postgresql, protected like /config, lists the databases of each
+    online cluster with owner, encoding and size, and the total size.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 21:00:00 +0200
+
+vpn-router (0.16.0-1) unstable; urgency=medium
+
+  feature: the router keeps its metrics in PostgreSQL, serves them for an OpenTelemetry Collector and pushes them over OTLP.
+
+  * vpn-router-metrics takes a sample every minute: CPU, memory, free
+    space of /, bytes per second on the external and the internal
+    interface, the load averages, and the VPN clients of each enabled
+    system.
+  * The samples are kept in metrics_system, metrics_network and
+    metrics_vpn, and summarized hourly, daily, monthly and yearly in
+    their _hourly, _daily, _monthly and _yearly tables; once a day the
+    rows older than metrics.keep_details, keep_hourly, keep_daily,
+    keep_monthly and keep_yearly are deleted.
+  * /metrics serves the last sample in the Prometheus format on the
+    interfaces metrics.listen names (internal by default) at
+    metrics.port, and over HTTPS by Caddy at metrics.https_port, with
+    HTTP Basic authentication of the service account
+    metrics.service_account names (metrics by default).
+  * metrics.otlp_endpoint pushes every sample as OTLP/HTTP JSON, with
+    metrics.otlp_username and metrics.otlp_password.
+  * /metrics/history of the web service answers the hour, day, month
+    and year views for the Router Manager.
+  * vpn-router-setup adds and keeps that service account, which cannot
+    sign in to /auth, with a random password in
+    /etc/vpn-router/metrics.secret, set anew when the file is missing.
+  * Depends on python3-psutil and python3-prometheus-client.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 19:00:00 +0200
+
+vpn-router (0.15.0-1) unstable; urgency=medium
+
+  feature: vpn-config-edit shows and changes the router's configuration.
+
+  * vpn-config-edit show prints /etc/vpn-router/vpn-router.json as
+    indented JSON.
+  * vpn-config-edit set SECTION.OPTION VALUE converts the value to the
+    setting's type, checks the whole configuration as vpn-router-setup
+    does, writes the file indented with mode 0600, and reminds that
+    vpn-router-setup applies it.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 18:00:00 +0200
+
+vpn-router (0.14.1-1) unstable; urgency=medium
+
+  fix: web.status_content is basic by default.
+
+  * Without web.status_content, or with a value that is not allowed, /
+    and /status show only the overall status, the time and the Client
+    IP; set it to standard or full to show more.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 17:00:00 +0200
+
+vpn-router (0.14.0-1) unstable; urgency=medium
+
+  feature: the status page shows the Client IP, and web.status_content sets how much it shows.
+
+  * / and /status show the Client IP, the address the request came from:
+    the last X-Forwarded-For entry for a request from a trusted proxy, so
+    behind Caddy or the Router Manager the browser's; /status has it as
+    client_ip.
+  * web.status_content sets what / and /status show: basic the overall
+    status, the time and the Client IP; standard also the package
+    versions and the number of active and failed services; full, the
+    default, everything as before. /status names it in content.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 16:00:00 +0200
+
+vpn-router (0.13.0-1) unstable; urgency=medium
+
+  feature: Caddy serves the status page and the Router Manager over HTTPS with a certificate from Let's Encrypt.
+
+  * The package depends on caddy. While https.enabled is true, the
+    default, and wan.local_fqdn is set, vpn-router-setup writes
+    /etc/caddy/vpn-router.Caddyfile and runs Caddy, which serves the
+    status page on https.web_port (7444) and the Router Manager, when it
+    is installed, on https.manager_port (7443) for wan.local_fqdn, and
+    obtains the certificate from Let's Encrypt with the HTTP challenge.
+  * The package's drop-in for caddy.service makes Caddy read that file
+    and keeps it from starting before the file exists. The file belongs
+    to root and the group caddy, which Caddy runs as, with mode 0640.
+  * The firewall opens port 80 on the external interface and the HTTPS
+    ports on both interfaces while HTTPS is on.
+  * The web service also listens on 127.0.0.1, where Caddy calls it.
+  * docker.exposed_ports may not take port 80 or the HTTPS ports while
+    HTTPS is on.
+  * Removing the package stops and disables caddy; purging it removes
+    /etc/caddy/vpn-router.Caddyfile.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 15:00:00 +0200
+
+vpn-router (0.12.3-1) unstable; urgency=medium
+
+  fix: the web service listens on the internal address, the external address or both.
+
+  * web.listen in vpn-router.json sets the interfaces the web service
+    listens on: internal, external or both, the default. It is read
+    every 30 seconds, so a change takes effect without running setup,
+    at the latest when vpn-router-web restarts.
+  * The firewall opens the ports of the web service and of the Router
+    Manager on the internal and the external interface.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 13:00:00 +0200
+
+vpn-router (0.12.2-1) unstable; urgency=medium
+
+  fix: the web service manages users also when it started before setup.
+
+  * vpn-router-web creates /run/vpn-router when it starts, so the user
+    operations take their lock there; started before vpn-router-setup had
+    made the directory, they failed with "Read-only file system".
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 12:00:00 +0200
+
+vpn-router (0.12.1-1) unstable; urgency=medium
+
+  fix: the status page shows whether the Router Manager runs.
+
+  * When vpn-router-manager is installed, the services show the state of
+    vpn-router-manager, and status is degraded while it is not active.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 11:00:00 +0200
+
+vpn-router (0.12.0-1) unstable; urgency=medium
+
+  feature: every user manages their own account through /account.
+
+  * /account serves the signed-in user, admin or not: GET shows the user
+    and whether they are a member of admins; PUT /account/password sets
+    the password and POST and PUT /account/otp enrol or replace the TOTP
+    enrolment, given the current password and, when enrolled, a current
+    code. A wrong one is 403 and counts as a failed login. Tokens of
+    vpn-router-token get 403 there.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 10:00:00 +0200
+
+vpn-router (0.11.0-1) unstable; urgency=medium
+
+  feature: the status page shows the Router Manager's version.
+
+  * / and /status show the installed version of vpn-router-manager, as
+    manager_version in JSON, or that it is not installed.
+  * The services show the state of each PostgreSQL cluster, as
+    postgresql@<version>-<cluster>, or postgresql: no cluster without one.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 09:00:00 +0200
+
+vpn-router (0.10.0-1) unstable; urgency=medium
+
+  feature: Manager-only accounts, and the Router Manager's first administrator.
+
+  * vpn-router-user add without the person options, and POST /users without
+    them, add a Manager-only account: a name and a password, no names, no
+    e-mail address and no certificate, so no VPN access. Giving only some of
+    the three is refused. The answers show the kind as manager_only, list
+    shows a KIND column, and update refuses a Manager-only account.
+  * vpn-router-setup creates the administrator the Router Manager's package
+    leaves in /etc/vpn-router-manager/admin.json as a Manager-only account in
+    the group admins, which it creates when missing, and removes the file.
+  * The directory has the package's own schema, with the auxiliary class
+    vpnRouterAccount and the attribute vpnRouterScope. A Manager-only account
+    can have an e-mail address: add --email, update --email ('' removes it),
+    and email in POST and PUT /users.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 08:00:00 +0200
+
+vpn-router (0.9.2-1) unstable; urgency=medium
+
+  fix: the web service starts once libpq can be found.
+
+  * The web service unit waits, with wait-for-libpq, until libpq is in the
+    linker cache before it starts the service, for at most 5 minutes; the
+    service imports psycopg at start again.
+  * The package starts the web service without waiting for it.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 06:00:00 +0200
+
+vpn-router (0.9.1-1) unstable; urgency=medium
+
+  fix: the web service waits for the PostgreSQL driver instead of failing at start.
+
+  * The web service imports psycopg as the first step of its
+    initialization, retrying every 15 seconds and logging the failure while
+    libpq cannot be found yet during an installation.
+  * vpn-router-user and vpn-router-token load without psycopg; a command
+    that needs the token database then reports that the driver cannot be
+    loaded, as it reports every failure of that database.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 05:00:00 +0200
+
+vpn-router (0.9.0-1) unstable; urgency=medium
+
+  feature: bootstrap diagnostics on port 17081 during the first setup.
+
+  * vpn-router-bootstrap serves, on port 17081, the setup state, the
+    services, the failed units, the addresses, cloud-init status and the end
+    of the logs of setup and the web service, as plain text.
+  * vpn-router-setup starts it, and the Router Manager's bootstrap on port
+    17080, until a run has succeeded, and opens both ports on the internal
+    interface; the first successful run writes
+    /var/lib/vpn-router/setup.done, closes the ports and stops them. A
+    failed run leaves them running.
+  * The web service no longer has a starting phase: it creates its key and
+    database, then listens.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 04:00:00 +0200
+
+vpn-router (0.8.0-1) unstable; urgency=medium
+
+  feature: the web service creates its key and database itself and shows its starting phase.
+
+  * vpn-router-web makes its signing key and creates the refresh tokens'
+    database on its first run, retrying every 15 seconds until PostgreSQL
+    answers; vpn-router-setup no longer creates them.
+  * Until then / and /status show the web service's phase and its failure,
+    with the state, the step and the failure of the last run of
+    vpn-router-setup, and every page answers 503.
+  * vpn-router-setup records its state, current step and failure in
+    /run/vpn-router/setup.json.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 03:00:00 +0200
+
+vpn-router (0.7.1-1) unstable; urgency=medium
+
+  fix: a failing user directory or token database no longer leaves the tunnels down.
+
+  * vpn-router-setup brings up strongSwan and WireGuard before it provisions
+    the user directory and the token database. A failure of either no
+    longer stops the other steps; setup reports it and fails at the end.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 02:00:00 +0200
+
+vpn-router (0.7.0-1) unstable; urgency=medium
+
+  feature: users sign in to the web service, and only members of admins manage the router.
+
+  * POST /auth signs a user in with the password, and the TOTP code for an
+    enrolled user, and answers a 5 minute access token and an 8 hour refresh
+    token; /auth/refresh renews them and /auth/logout revokes the refresh
+    token.
+  * /config, /users and /groups take tokens of users in the group admins,
+    and tokens of the new vpn-router-token, which has an admin claim and a
+    chosen lifetime. Only these may change the group admins.
+  * The tokens are signed with /etc/vpn-router/web.key, which
+    vpn-router-setup makes on the router; web.jwt_secret is ignored.
+  * Five failed logins lock an account for 15 minutes, for OpenVPN and
+    /auth. vpn-router-user unlock and POST /users/<name>/unlock end the
+    lockout, and the users show locked.
+  * /auth takes at most 10 calls a minute per client address; for calls from
+    web.trusted_proxies the address is read from X-Forwarded-For.
+  * Users enrolled for TOTP log in to OpenVPN with the password followed by
+    the code.
+  * Refresh tokens are kept in PostgreSQL, in the database web.database
+    names or in the vpn-router database vpn-router-setup creates. The
+    package depends on postgresql and python3-psycopg.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 01:00:00 +0200
+
+vpn-router (0.6.0-1) unstable; urgency=medium
+
+  feature: users can be enrolled for TOTP one-time passwords.
+
+  * vpn-router-user otp-new makes a TOTP secret and its otpauth URI,
+    otp-enrol stores the secret, read from standard input, given a code of
+    it, and otp-remove removes it. list and every answer show whether a user
+    is enrolled, as otp.
+  * The web service enrols with POST, PUT and DELETE /users/<name>/otp.
+  * The secret is kept in the user's entry with the OATH-LDAP schema of
+    slapd's otp module; only the user manager account may set it, and no
+    one can read it. Logins do not ask for the codes.
+  * The package depends on python3-pyotp.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 24 Oct 2026 23:30:00 +0200
+
+vpn-router (0.5.9-1) unstable; urgency=medium
+
+  fix: package upgrades no longer ask the configuration questions.
+
+  * The configuration questions are asked only on a fresh installation and
+    by dpkg-reconfigure.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 24 Oct 2026 21:00:00 +0200
+
+vpn-router (0.5.8-1) unstable; urgency=medium
+
+  fix: an upgrade keeps the configuration and asks nothing.
+
+  * postinst passes on only the debconf answers that were given. The
+    defaults of questions never asked, such as platform none, no longer
+    replace settings of vpn-router.json.
+  * Without an answer that changes it, the configuration ucf is given is
+    vpn-router.json byte for byte, so an upgrade has no conflict to ask
+    about; settings the file leaves out keep their defaults when it is read.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 24 Oct 2026 20:00:00 +0200
+
+vpn-router (0.5.7-1) unstable; urgency=medium
+
+  feature: users have a first name, a last name and an e-mail address.
+
+  * vpn-router-user add needs --first-name, --last-name and --email, kept as
+    givenName, sn and mail, with cn set to the first and the last name.
+    vpn-router-user update changes them, and list and every answer show
+    them as first_name, last_name and email.
+  * POST /users needs first_name, last_name and email, and PUT
+    /users/<name> changes them.
+  * vpn-router-user answers in text: a table of the users, the client
+    profile or a short message, with failures on standard error. --json
+    gives the JSON answers. -h and --help show the commands and their
+    options.
+  * The web service manages users with vpnrouter_users, the module
+    vpn-router-user uses, instead of running vpn-router-user for each
+    request.
+  * Groups of users, kept as groupOfNames entries in ou=groups with a
+    description and at least one member. vpn-router-group lists, adds,
+    renames, describes and removes them and adds and takes out members, and
+    the web service serves the same under /groups. Removing a user takes it
+    out of its groups and is refused while it is the last member of one.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 24 Oct 2026 19:00:00 +0200
+
+vpn-router (0.5.6-1) unstable; urgency=medium
+
+  update: the status web service listens on port 7081 by default.
+
+  * web.port defaults to 7081.
+  * vpn-router-setup opens the Router Manager's port, 7080 by default, on
+    the internal interface.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 24 Oct 2026 18:00:00 +0200
+
+vpn-router (0.5.5-1) unstable; urgency=medium
+
+  feature: users and their certificates no longer need a P2S system enabled.
+
+  * vpn-router-setup provisions the user directory, the CA and its CRL once
+    a platform is chosen, whether or not OpenVPN or road-warrior access is
+    enabled. The server certificate is issued once wan.local_fqdn is set,
+    also for an existing CA that has none.
+  * /users and vpn-router-user manage users and issue and revoke their
+    certificates without a P2S system; only the OpenVPN client profile
+    needs OpenVPN.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 24 Oct 2026 16:00:00 +0200
+
+vpn-router (0.5.4-1) unstable; urgency=medium
+
+  feature: the firewall lets the Router Manager in on the internal interface.
+
+  * While vpn-router-manager is installed, vpn-router-setup opens its port,
+    port in /etc/vpn-router-manager/manager.json or 8090, on the internal
+    interface. Purging the package removes the rule.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 24 Oct 2026 13:00:00 +0200
+
+vpn-router (0.5.3-1) unstable; urgency=medium
+
+  fix: the status always shows the state of slapd.
+
+  * /status and the status page show the state of slapd whether or not
+    OpenVPN or road-warrior access is configured, since the package always
+    runs it.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 24 Oct 2026 12:00:00 +0200
+
+vpn-router (0.5.2-1) unstable; urgency=medium
+
+  fix: /config no longer shows secret options.
+
+  * The configuration shown by /config leaves out the secret options (psk,
+    jwt_secret, and options named like a password, token or key) instead of
+    showing them masked.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 24 Oct 2026 11:00:00 +0200
+
+vpn-router (0.5.1-1) unstable; urgency=medium
+
+  fix: the web service works in every state of the router.
+
+  * vpn-router-web works whether /etc/vpn-router/pki and
+    /etc/swanctl/x509crl exist or not; they exist only while OpenVPN or
+    road-warrior access is configured. The paths the unit may write are
+    used when they exist and ignored when they do not.
+  * vpn-router-web starts on its own and keeps running. Every 30 seconds a
+    discovery thread works out the internal address from vpn-router.json as
+    general.mode directs, with web.port and web.enabled, and starts, moves
+    or stops the server thread to match. Each request reads the token
+    secret, the features and the facts it shows. The service is no longer
+    ordered after vpn-router-setup, which no longer restarts it, and
+    web.json no longer holds the address, the port or the secret.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 24 Oct 2026 10:00:00 +0200
+
+vpn-router (0.5.0-1) unstable; urgency=medium
+
+  * OpenVPN P2S access, set in the openvpn section (enabled, address_pool,
+    port, protocol). Clients need a certificate of the router CA and the
+    password of the user named in it; the password is checked against a
+    local LDAP directory by openvpn-auth-ldap.
+  * The users live in the package's own slapd database, dc=vpn-router,
+    dc=local, which vpn-router-setup provisions while OpenVPN or road-warrior
+    access is configured. slapd listens on its local socket only. Passwords
+    are stored as ARGON2 hashes, and a disabled user is locked with the
+    password policy.
+  * vpn-router-user lists, adds, disables, enables and removes users, sets
+    passwords and gives OpenVPN client profiles. Adding a user also issues
+    the user's client certificate; removing one revokes it.
+  * The web service manages users under /users, with the same token as
+    /config, and /status shows the OpenVPN clients.
+  * The CA gets a certificate revocation list when it is created, and an
+    existing CA without one gets one.
+  * Purging the package removes the firewall rules of the web service, which
+    it left behind before.
+  * The status page follows the browser's light or dark colour scheme,
+    uses a proportional font and reloads itself every minute.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Fri, 23 Oct 2026 23:00:00 +0200
+
+vpn-router (0.4.3-1) unstable; urgency=medium
+
+  * swanctl output with more closing than opening lines is read as not
+    available instead of failing the request.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Fri, 23 Oct 2026 22:00:00 +0200
+
+vpn-router (0.4.2-1) unstable; urgency=medium
+
+  * /config gives the IPsec connections and SAs as objects, read from
+    swanctl --pretty, and the routes as objects from ip -j, instead of
+    lines of text. The site-to-site and road-warrior state of /status is
+    counted from the same swanctl objects.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Fri, 23 Oct 2026 21:30:00 +0200
+
+vpn-router (0.4.1-1) unstable; urgency=medium
+
+  * Fix the start of vpn-router-setup hanging on a host that runs Docker.
+    vpn-router-setup restarted vpn-router-docker and waited for it, but that
+    unit was ordered after vpn-router-setup, so each waited for the other,
+    the installation of the package never finished and vpn-router-web was
+    never enabled or started. vpn-router-docker is no longer ordered after
+    vpn-router-setup, so vpn-router-setup still waits for it and the Docker
+    rules are in place when it ends. The restart of vpn-router-web is queued
+    without waiting.
+  * Start vpn-router-setup after docker.service, so that Docker is up when
+    the setup looks for it and does not start in parallel with it. The
+    ordering applies only when Docker is installed.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Fri, 23 Oct 2026 20:10:00 +0200
+
+vpn-router (0.4.0-1) unstable; urgency=medium
+
+  * Add the vpn-router-web service, a small status web service that uses only
+    the Python standard library. It listens on the internal address and on
+    web.port, 8080 by default. / shows the state of the router for people,
+    as plain text for curl and wget and as a Tailwind CSS page for a
+    browser, and /status shows the same as JSON: the state of the services,
+    the package version, whether Docker is installed, and the state of
+    site-to-site, road-warrior and WireGuard, with no configuration data.
+  * Add the web configuration section. web.enabled (true by default) and
+    web.port. vpn-router-setup opens the port on the internal interface in
+    UFW and restarts the service.
+  * Add /config to the status web service, for a client such as manage-ui. It
+    shows the configuration with its secrets masked, the interfaces,
+    addresses, identities and routes the router resolved, the IPsec
+    connections and SAs, the WireGuard peers (no keys) and the Docker rules.
+    It needs a JWT signed with HS256 by web.jwt_secret, with an exp claim
+    within ten minutes, and is not served while web.jwt_secret is not set.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Fri, 23 Oct 2026 19:00:00 +0200
+
 vpn-router (0.3.2-1) unstable; urgency=medium
 
   * Accept an IPv4 or IPv6 address as remote.id and as wan.local_fqdn. The

debian-package/debian/config

@@ -2,6 +2,12 @@
 set -e
 . /usr/share/debconf/confmodule
 
+# An upgrade (configure with the old version) asks nothing; vpn-router.json holds the
+# configuration, and dpkg-reconfigure asks the questions again.
+if [ "$1" = "configure" ] && [ -n "$2" ]; then
+    exit 0
+fi
+
 # Every question has a default, so a non-interactive install with nothing
 # preseeded completes without prompting. The defaults are platform "none" and
 # mode "manual", which together mean: install the files, configure nothing.

debian-package/debian/control

@@ -20,13 +20,26 @@ Depends: ${misc:Depends},
          openssl,
          python3,
          python3-jinja2,
+         python3-ldap,
+         python3-pyotp,
+         python3-psycopg,
+         python3-psutil,
+         python3-prometheus-client,
+         postgresql,
+         slapd,
+         openvpn,
+         openvpn-auth-ldap,
          iproute2,
-         systemd-resolved
+         systemd-resolved,
-Description: Linux VPN router with IPSec, road-warrior and WireGuard support
+         caddy
+Description: Linux VPN router with IPSec, road-warrior, OpenVPN and WireGuard support
  Configures a Linux host or virtual machine as a VPN router: site-to-site
  IKEv2 IPSec (strongSwan swanctl), optional road-warrior access using EAP-TLS,
- optional WireGuard, and the routing, NAT, MSS clamping and firewall rules a
+ optional OpenVPN with a certificate and a password per user, kept in a local
- router needs. Nothing in the package is specific to a cloud provider.
+ LDAP directory, optional WireGuard, and the routing, NAT, MSS clamping and
+ firewall rules a router needs. It keeps metrics of the router in PostgreSQL
+ and serves them for an OpenTelemetry Collector. Nothing in the package is
+ specific to a cloud provider.
  .
  Settings live in /etc/vpn-router/vpn-router.json, which is created on first
  install and owned by the administrator afterwards. Editing it and restarting

debian-package/debian/install

@@ -1,10 +1,27 @@
 src/etc/sysctl.d/99-vpn-router.conf                   etc/sysctl.d/
 src/usr/lib/vpn-router/vpnrouter.py                   usr/lib/vpn-router/
+src/usr/lib/vpn-router/vpnrouter_ldap.py              usr/lib/vpn-router/
+src/usr/lib/vpn-router/vpnrouter_net.py               usr/lib/vpn-router/
+src/usr/lib/vpn-router/vpnrouter_serve.py             usr/lib/vpn-router/
+src/usr/lib/vpn-router/vpnrouter_users.py             usr/lib/vpn-router/
+src/usr/lib/vpn-router/vpnrouter_auth.py              usr/lib/vpn-router/
+src/usr/lib/vpn-router/vpnrouter_metrics.py           usr/lib/vpn-router/
+src/usr/lib/vpn-router/vpnrouter_status.py            usr/lib/vpn-router/
+src/usr/lib/vpn-router/openvpn-same-user              usr/lib/vpn-router/
+src/usr/sbin/vpn-router-user                          usr/sbin/
+src/usr/sbin/vpn-router-group                         usr/sbin/
+src/usr/sbin/vpn-router-token                         usr/sbin/
+src/usr/sbin/vpn-config-edit                          usr/sbin/
 src/usr/lib/vpn-router/simple-ca                      usr/lib/vpn-router/
 src/usr/lib/vpn-router/configure                      usr/lib/vpn-router/
 src/usr/lib/vpn-router/docker-rules                   usr/lib/vpn-router/
 src/usr/lib/vpn-router/setup                          usr/lib/vpn-router/
+src/usr/lib/vpn-router/web                            usr/lib/vpn-router/
+src/usr/lib/vpn-router/metrics                        usr/lib/vpn-router/
+src/usr/lib/vpn-router/bootstrap                      usr/lib/vpn-router/
+src/usr/lib/vpn-router/wait-for-libpq                 usr/lib/vpn-router/
 src/usr/lib/vpn-router/generate-config                usr/lib/vpn-router/
 src/usr/lib/vpn-router/vpnrouter_platforms/*.py       usr/lib/vpn-router/vpnrouter_platforms/
 src/usr/share/vpn-router/templates/*                  usr/share/vpn-router/templates/
+src/usr/lib/systemd/system/caddy.service.d/vpn-router.conf usr/lib/systemd/system/caddy.service.d/
 src/usr/share/doc/vpn-router/vpn-router.json.example  usr/share/doc/vpn-router/

debian-package/debian/postinst

@@ -2,28 +2,39 @@
 set -e
 . /usr/share/debconf/confmodule
 
+# The answer to a question in ANSWER, or "" when the question has not been seen; the
+# default debconf holds for an unseen question is not an answer.
+answer() {
+    ANSWER=""
+    db_fget "$1" seen
+    if [ "$RET" = "true" ]; then
+        db_get "$1"
+        ANSWER="$RET"
+    fi
+}
+
 case "$1" in
     configure)
         # --- Read the installer's answers ---
-        db_get vpn-router/platform;         VPN_ROUTER_PLATFORM="$RET"
+        answer vpn-router/platform;         VPN_ROUTER_PLATFORM="$ANSWER"
-        db_get vpn-router/mode;             VPN_ROUTER_MODE="$RET"
+        answer vpn-router/mode;             VPN_ROUTER_MODE="$ANSWER"
-        db_get vpn-router/int_addr;         VPN_ROUTER_INT_ADDR="$RET"
+        answer vpn-router/int_addr;         VPN_ROUTER_INT_ADDR="$ANSWER"
-        db_get vpn-router/external_interface; VPN_ROUTER_EXTERNAL_INTERFACE="$RET"
+        answer vpn-router/external_interface; VPN_ROUTER_EXTERNAL_INTERFACE="$ANSWER"
-        db_get vpn-router/internal_interface; VPN_ROUTER_INTERNAL_INTERFACE="$RET"
+        answer vpn-router/internal_interface; VPN_ROUTER_INTERNAL_INTERFACE="$ANSWER"
-        db_get vpn-router/local_fqdn;       VPN_ROUTER_LOCAL_FQDN="$RET"
+        answer vpn-router/local_fqdn;       VPN_ROUTER_LOCAL_FQDN="$ANSWER"
-        db_get vpn-router/local_id_mode;    VPN_ROUTER_LOCAL_ID_MODE="$RET"
+        answer vpn-router/local_id_mode;    VPN_ROUTER_LOCAL_ID_MODE="$ANSWER"
-        db_get vpn-router/local_cidrs;      VPN_ROUTER_LOCAL_CIDRS="$RET"
+        answer vpn-router/local_cidrs;      VPN_ROUTER_LOCAL_CIDRS="$ANSWER"
-        db_get vpn-router/int_gateway_ip;   VPN_ROUTER_INT_GATEWAY_IP="$RET"
+        answer vpn-router/int_gateway_ip;   VPN_ROUTER_INT_GATEWAY_IP="$ANSWER"
-        db_get vpn-router/remote_addrs;     VPN_ROUTER_REMOTE_ADDRS="$RET"
+        answer vpn-router/remote_addrs;     VPN_ROUTER_REMOTE_ADDRS="$ANSWER"
-        db_get vpn-router/remote_id;        VPN_ROUTER_REMOTE_ID="$RET"
+        answer vpn-router/remote_id;        VPN_ROUTER_REMOTE_ID="$ANSWER"
-        db_get vpn-router/remote_cidrs;     VPN_ROUTER_REMOTE_CIDRS="$RET"
+        answer vpn-router/remote_cidrs;     VPN_ROUTER_REMOTE_CIDRS="$ANSWER"
-        db_get vpn-router/psk;              VPN_ROUTER_PSK="$RET"
+        answer vpn-router/psk;              VPN_ROUTER_PSK="$ANSWER"
-        db_get vpn-router/p2s_enabled;      VPN_ROUTER_P2S_ENABLED="$RET"
+        answer vpn-router/p2s_enabled;      VPN_ROUTER_P2S_ENABLED="$ANSWER"
-        db_get vpn-router/p2s_address_pool; VPN_ROUTER_P2S_ADDRESS_POOL="$RET"
+        answer vpn-router/p2s_address_pool; VPN_ROUTER_P2S_ADDRESS_POOL="$ANSWER"
-        db_get vpn-router/p2s_ca_name;      VPN_ROUTER_P2S_CA_NAME="$RET"
+        answer vpn-router/p2s_ca_name;      VPN_ROUTER_P2S_CA_NAME="$ANSWER"
-        db_get vpn-router/wg_enabled;       VPN_ROUTER_WG_ENABLED="$RET"
+        answer vpn-router/wg_enabled;       VPN_ROUTER_WG_ENABLED="$ANSWER"
-        db_get vpn-router/wg_address;       VPN_ROUTER_WG_ADDRESS="$RET"
+        answer vpn-router/wg_address;       VPN_ROUTER_WG_ADDRESS="$ANSWER"
-        db_get vpn-router/wg_listen_port;   VPN_ROUTER_WG_LISTEN_PORT="$RET"
+        answer vpn-router/wg_listen_port;   VPN_ROUTER_WG_LISTEN_PORT="$ANSWER"
 
         export VPN_ROUTER_PLATFORM VPN_ROUTER_MODE VPN_ROUTER_INT_ADDR \
                VPN_ROUTER_EXTERNAL_INTERFACE \
@@ -59,6 +70,12 @@ case "$1" in
         # The firewall is not touched here. Enabling it, allowing SSH and
         # setting the forward policy all belong to vpn-router-setup, which does
         # them only once the configuration says this host is a router.
+
+        # The web service waits for libpq in the linker cache, which dpkg
+        # refreshes after this script, so it is started without waiting for it.
+        if [ -d /run/systemd/system ]; then
+            systemctl --no-block restart vpn-router-web.service >/dev/null 2>&1 || true
+        fi
         ;;
 esac
 

debian-package/debian/postrm

@@ -11,10 +11,10 @@ strip_ufw_blocks() {
 
     tmp="$(mktemp)"
     awk '
-        /^# (IPSEC|WIREGUARD|P2S DNS|ROUTER FORWARD|ROUTER NAT|ROUTER MSS) RULES START$/ {
+        /^# (IPSEC|WIREGUARD|OPENVPN|P2S DNS|WEB|BOOTSTRAP|MANAGER|HTTPS|METRICS|ROUTER FORWARD|ROUTER NAT|ROUTER MSS) RULES START$/ {
             skip = 1; pending = 0; next
         }
-        /^# (IPSEC|WIREGUARD|P2S DNS|ROUTER FORWARD|ROUTER NAT|ROUTER MSS) RULES END$/ {
+        /^# (IPSEC|WIREGUARD|OPENVPN|P2S DNS|WEB|BOOTSTRAP|MANAGER|HTTPS|METRICS|ROUTER FORWARD|ROUTER NAT|ROUTER MSS) RULES END$/ {
             skip = 0; next
         }
         skip { next }
@@ -43,14 +43,29 @@ case "$1" in
             ucfr --purge vpn-router /etc/vpn-router/vpn-router.json
         fi
 
-        rm -f /etc/swanctl/conf.d/remote-site.conf \
+        rm -f /etc/swanctl/conf.d/site-to-site.conf \
               /etc/swanctl/conf.d/road-warrior.conf \
               /etc/systemd/resolved.conf.d/p2s-forwarder.conf \
               /etc/wireguard/wg0.conf \
-              /etc/vpn-router/vpn-router.json
+              /etc/openvpn/server/vpn-router.conf \
+              /etc/openvpn/server/vpn-router-ldap.conf \
+              /etc/caddy/vpn-router.Caddyfile \
+              /etc/vpn-router/vpn-router.json \
+              /etc/vpn-router/web.key \
+              /etc/vpn-router/metrics.secret \
+              /var/lib/vpn-router/setup.done
+        rmdir --ignore-fail-on-non-empty /var/lib/vpn-router 2>/dev/null || true
 
-        # Key material is not package state: /etc/vpn-router/pki, the WireGuard
+        # The refresh tokens' database and role, when the package created them.
-        # key pair and everything under /etc/swanctl are left in place.
+        if [ -f /etc/vpn-router/database.secret ] && command -v psql >/dev/null 2>&1; then
+            runuser -u postgres -- psql -X -q -c 'DROP DATABASE IF EXISTS "vpn-router"' || true
+            runuser -u postgres -- psql -X -q -c 'DROP ROLE IF EXISTS "vpn-router"' || true
+        fi
+        rm -f /etc/vpn-router/database.secret
+
+        # Key material and users are not package state: /etc/vpn-router/pki,
+        # /etc/vpn-router/ldap, the user directory in slapd, the WireGuard key
+        # pair and everything under /etc/swanctl are left in place.
         rmdir --ignore-fail-on-non-empty /etc/vpn-router 2>/dev/null || true
         ;;
 esac

debian-package/debian/prerm

@@ -6,20 +6,28 @@ case "$1" in
         # Stop offering the tunnels, but leave strongSwan itself enabled: it is
         # a shared service and may be in use by something other than this
         # package. The connection files are removed on purge.
+        # The site-to-site connections are the names of site_to_site in vpn-router.json.
         if [ -d /run/systemd/system ]; then
-            swanctl --terminate --ike remote-site >/dev/null 2>&1 || true
+            connections="$(python3 -c 'import json, sys; print(" ".join(json.load(open(sys.argv[1])).get("site_to_site", {})))' \
+                /etc/vpn-router/vpn-router.json 2>/dev/null || true)"
+            for connection in $connections; do
+                swanctl --terminate --ike "$connection" >/dev/null 2>&1 || true
+            done
             swanctl --terminate --ike road-warrior >/dev/null 2>&1 || true
         fi
 
-        # This package enabled wg-quick@wg0, so it takes it down again. Use the
+        # vpn-router-setup enabled wg-quick@wg0, openvpn-server@vpn-router,
-        # debhelper wrappers rather than systemctl, so the script behaves on a
+        # caddy and vpn-router-metrics, so the package takes them down again.
-        # machine without systemd.
+        # Use the debhelper wrappers rather than systemctl, so the script
-        if command -v deb-systemd-invoke >/dev/null 2>&1; then
+        # behaves on a machine without systemd.
-            deb-systemd-invoke stop wg-quick@wg0.service >/dev/null 2>&1 || true
+        for unit in wg-quick@wg0.service openvpn-server@vpn-router.service caddy.service vpn-router-metrics.service; do
-        fi
+            if command -v deb-systemd-invoke >/dev/null 2>&1; then
-        if command -v deb-systemd-helper >/dev/null 2>&1; then
+                deb-systemd-invoke stop "$unit" >/dev/null 2>&1 || true
-            deb-systemd-helper disable wg-quick@wg0.service >/dev/null 2>&1 || true
+            fi
-        fi
+            if command -v deb-systemd-helper >/dev/null 2>&1; then
+                deb-systemd-helper disable "$unit" >/dev/null 2>&1 || true
+            fi
+        done
         ;;
 esac
 

debian-package/debian/rules

@@ -5,3 +5,6 @@
 override_dh_installsystemd:
 	dh_installsystemd --name=vpn-router-setup
 	dh_installsystemd --name=vpn-router-docker
+	dh_installsystemd --name=vpn-router-web --no-start
+	dh_installsystemd --name=vpn-router-metrics --no-enable --no-start
+	dh_installsystemd --name=vpn-router-bootstrap --no-enable --no-start

debian-package/debian/templates

@@ -66,7 +66,8 @@ Type: select
 Choices: fqdn, public_ip, internal_ip
 Default: fqdn
 Description: IKE local identity mode:
- How to derive the IKE identity advertised to the remote site:
+ How to derive the IKE identity the first site-to-site connection
+ advertises to the remote site:
  .
  fqdn - use the FQDN, which must match what the peer expects.
  .
@@ -78,8 +79,9 @@ Template: vpn-router/local_cidrs
 Type: string
 Default:
 Description: Local subnet CIDR(s):
- Comma-separated list of local subnet CIDRs to advertise into the
+ Comma-separated list of local subnet CIDRs routed by this router and
- site-to-site tunnel (for example 10.0.0.0/24 or 10.0.0.0/24,10.0.1.0/24).
+ advertised into the first site-to-site connection (for example
+ 10.0.0.0/24 or 10.0.0.0/24,10.0.1.0/24).
 
 Template: vpn-router/int_gateway_ip
 Type: string
@@ -94,25 +96,26 @@ Type: string
 Default:
 Description: Remote site WAN IP address(es):
  Comma-separated list of remote gateway addresses or FQDNs for the
- site-to-site IPSec tunnel.
+ first site-to-site connection.
 
 Template: vpn-router/remote_id
 Type: string
 Default:
 Description: Remote site IKE identity:
- IKE identity of the remote peer, without a leading @.
+ IKE identity of the remote peer of the first site-to-site connection,
+ without a leading @.
 
 Template: vpn-router/remote_cidrs
 Type: string
 Default:
 Description: Remote subnet CIDR(s):
  Comma-separated list of remote subnet CIDRs reachable through the
- site-to-site tunnel (for example 192.168.0.0/24).
+ first site-to-site connection (for example 192.168.0.0/24).
 
 Template: vpn-router/psk
 Type: password
 Description: Pre-shared key (PSK):
- Pre-shared key for the site-to-site IKEv2 tunnel. Must match the value
+ Pre-shared key of the first site-to-site connection. Must match the value
  configured on the remote peer. Stored in the configuration file and cleared
  from the debconf database after installation.
 

debian-package/debian/vpn-router.vpn-router-bootstrap.service

@@ -0,0 +1,14 @@
+[Unit]
+Description=VPN Router bootstrap diagnostics page
+Documentation=file:/usr/share/doc/vpn-router/README.md
+
+# Never enabled: vpn-router-setup starts it until a setup has succeeded, and stops it after that.
+[Service]
+Type=simple
+ExecStart=/usr/bin/python3 /usr/lib/vpn-router/bootstrap
+Restart=on-failure
+RestartSec=5
+NoNewPrivileges=yes
+ProtectSystem=strict
+ProtectHome=yes
+PrivateTmp=yes

debian-package/debian/vpn-router.vpn-router-docker.service

@@ -1,7 +1,7 @@
 [Unit]
 Description=VPN Router rules for Docker published ports
 Documentation=file:/usr/share/doc/vpn-router/README.md
-After=docker.service vpn-router-setup.service
+After=docker.service
 PartOf=docker.service
 
 [Service]

debian-package/debian/vpn-router.vpn-router-metrics.service

@@ -0,0 +1,22 @@
+[Unit]
+Description=VPN Router metrics service
+Documentation=file:/usr/share/doc/vpn-router/README.md
+After=network-online.target postgresql.service vpn-router-web.service
+Wants=network-online.target
+
+[Service]
+Type=simple
+ExecStartPre=/usr/lib/vpn-router/wait-for-libpq
+ExecStart=/usr/bin/python3 /usr/lib/vpn-router/metrics
+TimeoutStartSec=5min
+Restart=on-failure
+RestartSec=15
+NoNewPrivileges=yes
+ProtectSystem=strict
+RuntimeDirectory=vpn-router
+RuntimeDirectoryPreserve=yes
+ProtectHome=yes
+PrivateTmp=yes
+
+[Install]
+WantedBy=multi-user.target

debian-package/debian/vpn-router.vpn-router-setup.service

@@ -1,8 +1,8 @@
 [Unit]
 Description=VPN Router Setup
 Documentation=file:/usr/share/doc/vpn-router/README.md
-Wants=network-online.target
+Wants=network-online.target postgresql.service
-After=network-online.target
+After=network-online.target docker.service postgresql.service
 
 [Service]
 Type=oneshot

debian-package/debian/vpn-router.vpn-router-web.service

@@ -0,0 +1,23 @@
+[Unit]
+Description=VPN Router status web service
+Documentation=file:/usr/share/doc/vpn-router/README.md
+After=network-online.target postgresql.service
+Wants=network-online.target
+
+[Service]
+Type=simple
+ExecStartPre=/usr/lib/vpn-router/wait-for-libpq
+ExecStart=/usr/bin/python3 /usr/lib/vpn-router/web
+TimeoutStartSec=5min
+Restart=on-failure
+RestartSec=15
+NoNewPrivileges=yes
+ProtectSystem=strict
+RuntimeDirectory=vpn-router
+RuntimeDirectoryPreserve=yes
+ReadWritePaths=-/etc/vpn-router -/etc/swanctl/x509crl
+ProtectHome=yes
+PrivateTmp=yes
+
+[Install]
+WantedBy=multi-user.target

debian-package/src/usr/lib/systemd/system/caddy.service.d/vpn-router.conf

@@ -0,0 +1,10 @@
+# Caddy is the HTTPS proxy of vpn-router: it reads the configuration vpn-router-setup writes, and
+# does not start until that exists.
+[Unit]
+ConditionPathExists=/etc/caddy/vpn-router.Caddyfile
+
+[Service]
+ExecStart=
+ExecStart=/usr/bin/caddy run --environ --adapter caddyfile --config /etc/caddy/vpn-router.Caddyfile
+ExecReload=
+ExecReload=/usr/bin/caddy reload --adapter caddyfile --config /etc/caddy/vpn-router.Caddyfile --force

debian-package/src/usr/lib/vpn-router/bootstrap

@@ -0,0 +1,55 @@
+#!/usr/bin/python3
+"""Serve diagnostics of the router's first setup on PORT, with the Python standard library alone.
+
+Run by vpn-router-bootstrap.service, which vpn-router-setup starts until a setup has succeeded and
+then stops. Every request on any address answers, as plain text, the output of COMMANDS run at that
+moment: the state of the last setup run, the state of the services, the addresses, and the end of
+the logs.
+"""
+
+import subprocess
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+
+PORT = 17081
+
+#: The title of each section and its command.
+COMMANDS = (
+    ('Setup state', ['cat', '/run/vpn-router/setup.json']),
+    ('Services', ['systemctl', 'is-active', 'vpn-router-setup', 'vpn-router-web', 'slapd', 'postgresql',
+                  'strongswan', 'openvpn-server@vpn-router', 'wg-quick@wg0']),
+    ('Failed units', ['systemctl', '--failed', '--no-legend', '--plain']),
+    ('Addresses', ['ip', '-brief', 'address']),
+    ('Routes', ['ip', 'route']),
+    ('cloud-init', ['cloud-init', 'status', '--long']),
+    ('vpn-router-setup log', ['journalctl', '--unit', 'vpn-router-setup', '--lines', '60', '--no-pager', '--output', 'short-iso']),
+    ('vpn-router-web log', ['journalctl', '--unit', 'vpn-router-web', '--lines', '30', '--no-pager', '--output', 'short-iso']),
+)
+
+
+def report():
+    """The output of every command of COMMANDS under its title."""
+    sections = []
+    for title, command in COMMANDS:
+        try:
+            result = subprocess.run(command, capture_output=True, text=True, timeout=10)
+            text = (result.stdout + result.stderr).rstrip()
+        except (OSError, subprocess.TimeoutExpired) as exc:
+            text = f'cannot run {command[0]}: {exc}'
+        sections.append(f'== {title} ==\n{text}\n')
+    return '\n'.join(sections)
+
+
+class Handler(BaseHTTPRequestHandler):
+    def do_GET(self):
+        body = report().encode()
+        self.send_response(200)
+        self.send_header('Content-Type', 'text/plain; charset=utf-8')
+        self.send_header('Content-Length', str(len(body)))
+        self.send_header('Cache-Control', 'no-store')
+        self.send_header('Refresh', '10')
+        self.end_headers()
+        self.wfile.write(body)
+
+
+if __name__ == '__main__':
+    ThreadingHTTPServer(('', PORT), Handler).serve_forever()

debian-package/src/usr/lib/vpn-router/generate-config

@@ -6,22 +6,21 @@ Called from postinst, which hands the result to ucf(1). ucf compares it with
 this script never writes to /etc and never has to worry about destroying an
 administrator's edits.
 
-The candidate is built in three layers, each overriding the one before:
+The candidate is the current /etc/vpn-router/vpn-router.json as it is, or the
-
+shipped defaults when there is none, with every debconf answer that was given
-  1. the shipped defaults
+written over it. Without an answer that changes it, the candidate is the current
-  2. the current contents of /etc/vpn-router/vpn-router.json, if it exists
+file byte for byte, so
-  3. any debconf answer that was actually given
+an upgrade or a reconfiguration that changes nothing gives ucf nothing to ask;
-
+defaults the file leaves out are applied by vpnrouter.load_config() when the
-Layer 2 is what makes reconfiguration safe: a setting the administrator edited
+file is read. An empty answer means "leave alone", which is why clearing the
-by hand, and that debconf has nothing to say about, is carried through
+pre-shared key from the debconf database after installation does not blank it
-unchanged, so ucf only sees a diff where something really changed. An empty
+on the next dpkg-reconfigure.
-answer means "leave alone", which is why clearing the pre-shared key from the
-debconf database after installation does not blank it on the next
-dpkg-reconfigure.
 
 Answers arrive as VPN_ROUTER_* environment variables and are converted to the
 type of the setting they fill: comma-separated lists become arrays, "true" and
-"false" become booleans and digits become integers.
+"false" become booleans and digits become integers. The site-to-site answers
+fill the connection vpnrouter.FIRST_CONNECTION of site_to_site; the local
+subnets fill both its local_cidrs and local.cidrs.
 """
 
 import json
@@ -32,27 +31,29 @@ sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
 
 import vpnrouter
 
-# environment variable -> (section, option)
+FIRST = ('site_to_site', vpnrouter.FIRST_CONNECTION)
+
+# environment variable -> the settings it fills, each (section, option) or (site_to_site, name, option)
 ANSWERS = {
-    'VPN_ROUTER_PLATFORM':        ('general', 'platform'),
+    'VPN_ROUTER_PLATFORM':        (('general', 'platform'),),
-    'VPN_ROUTER_MODE':            ('general', 'mode'),
+    'VPN_ROUTER_MODE':            (('general', 'mode'),),
-    'VPN_ROUTER_INT_ADDR':        ('local', 'int_addr'),
+    'VPN_ROUTER_INT_ADDR':        (('local', 'int_addr'),),
-    'VPN_ROUTER_EXTERNAL_INTERFACE': ('interfaces', 'external'),
+    'VPN_ROUTER_EXTERNAL_INTERFACE': (('interfaces', 'external'),),
-    'VPN_ROUTER_INTERNAL_INTERFACE': ('interfaces', 'internal'),
+    'VPN_ROUTER_INTERNAL_INTERFACE': (('interfaces', 'internal'),),
-    'VPN_ROUTER_LOCAL_FQDN':      ('wan', 'local_fqdn'),
+    'VPN_ROUTER_LOCAL_FQDN':      (('wan', 'local_fqdn'),),
-    'VPN_ROUTER_LOCAL_ID_MODE':   ('wan', 'local_id_mode'),
+    'VPN_ROUTER_LOCAL_ID_MODE':   ((*FIRST, 'local_id_mode'),),
-    'VPN_ROUTER_LOCAL_CIDRS':     ('local', 'cidrs'),
+    'VPN_ROUTER_LOCAL_CIDRS':     (('local', 'cidrs'), (*FIRST, 'local_cidrs')),
-    'VPN_ROUTER_INT_GATEWAY_IP':  ('local', 'int_gateway_ip'),
+    'VPN_ROUTER_INT_GATEWAY_IP':  (('local', 'int_gateway_ip'),),
-    'VPN_ROUTER_REMOTE_ADDRS':    ('remote', 'addrs'),
+    'VPN_ROUTER_REMOTE_ADDRS':    ((*FIRST, 'addrs'),),
-    'VPN_ROUTER_REMOTE_ID':       ('remote', 'id'),
+    'VPN_ROUTER_REMOTE_ID':       ((*FIRST, 'id'),),
-    'VPN_ROUTER_REMOTE_CIDRS':    ('remote', 'cidrs'),
+    'VPN_ROUTER_REMOTE_CIDRS':    ((*FIRST, 'cidrs'),),
-    'VPN_ROUTER_PSK':             ('remote', 'psk'),
+    'VPN_ROUTER_PSK':             ((*FIRST, 'psk'),),
-    'VPN_ROUTER_P2S_ENABLED':     ('p2s', 'enabled'),
+    'VPN_ROUTER_P2S_ENABLED':     (('p2s', 'enabled'),),
-    'VPN_ROUTER_P2S_ADDRESS_POOL': ('p2s', 'address_pool'),
+    'VPN_ROUTER_P2S_ADDRESS_POOL': (('p2s', 'address_pool'),),
-    'VPN_ROUTER_P2S_CA_NAME':     ('p2s', 'ca_name'),
+    'VPN_ROUTER_P2S_CA_NAME':     (('p2s', 'ca_name'),),
-    'VPN_ROUTER_WG_ENABLED':      ('wireguard', 'enabled'),
+    'VPN_ROUTER_WG_ENABLED':      (('wireguard', 'enabled'),),
-    'VPN_ROUTER_WG_ADDRESS':      ('wireguard', 'address'),
+    'VPN_ROUTER_WG_ADDRESS':      (('wireguard', 'address'),),
-    'VPN_ROUTER_WG_LISTEN_PORT':  ('wireguard', 'listen_port'),
+    'VPN_ROUTER_WG_LISTEN_PORT':  (('wireguard', 'listen_port'),),
 }
 
 
@@ -62,25 +63,51 @@ def main():
         return 2
     output = sys.argv[1]
 
-    cfg = vpnrouter.load_config()
+    try:
+        current = vpnrouter.CONFIG_FILE.read_text(encoding='utf-8')
+    except FileNotFoundError:
+        current = None
+    if current is None:
+        cfg = vpnrouter.load_config()
+    else:
+        try:
+            cfg = json.loads(current)
+        except ValueError as exc:
+            vpnrouter.fail(f'cannot parse {vpnrouter.CONFIG_FILE}: {exc}')
+        if not isinstance(cfg, dict):
+            vpnrouter.fail(f'cannot parse {vpnrouter.CONFIG_FILE}: the top level is not an object')
 
-    for variable, (section, option) in ANSWERS.items():
+    answered = False
-        value = os.environ.get(variable)
+    for variable, paths in ANSWERS.items():
-        if not value:
+        answer = os.environ.get(variable)
+        if not answer:
             continue
-        # A value that does not convert is kept as a string for validate() to report.
+        for path in paths:
-        default = vpnrouter.SCHEMA[section][option]
+            # A value that does not convert is kept as a string for validate() to report.
-        if isinstance(default, bool):
+            default = vpnrouter.CONNECTION[path[2]] if path[0] == 'site_to_site' else vpnrouter.SCHEMA[path[0]][path[1]]
-            value = value == 'true'
+            value = answer
-        elif isinstance(default, int):
+            if isinstance(default, bool):
-            value = int(value) if value.isdigit() else value
+                value = value == 'true'
-        elif isinstance(default, list):
+            elif isinstance(default, int):
-            value = [item.strip() for item in value.split(',') if item.strip()]
+                value = int(value) if value.isdigit() else value
-        cfg[section][option] = value
+            elif isinstance(default, list):
+                value = [item.strip() for item in value.split(',') if item.strip()]
+            target = cfg
+            for key in path[:-1]:
+                if not isinstance(target.setdefault(key, {}), dict):
+                    vpnrouter.fail(f'cannot parse {vpnrouter.CONFIG_FILE}: {".".join(path[:-1])} is not an object')
+                target = target[key]
+            if target.get(path[-1]) != value:
+                target[path[-1]] = value
+                answered = True
 
+    # Without a change the candidate is the current file byte for byte, as ucf compares checksums.
     with open(output, 'w', encoding='utf-8') as handle:
-        json.dump(cfg, handle, indent=2)
+        if current is not None and not answered:
-        handle.write('\n')
+            handle.write(current)
+        else:
+            json.dump(cfg, handle, indent=2)
+            handle.write('\n')
     os.chmod(output, 0o600)
     return 0
 

debian-package/src/usr/lib/vpn-router/metrics

@@ -0,0 +1,456 @@
+#!/usr/bin/python3
+"""Collect the metrics of the router, keep them in PostgreSQL, serve them for scraping and push them with OTLP.
+
+Run by vpn-router-metrics.service while metrics.enabled is true; vpn-router-setup enables and
+disables the unit.
+
+Every SAMPLE_SECONDS, on the minute, a sample is taken with psutil: the CPU utilization since the
+previous sample, the memory used and in total, the free and the total space of /, the bytes
+received and sent per second on the external and the internal interface, and the load averages;
+and the VPN clients of each enabled VPN system (vpnrouter_status): road-warrior (p2s) and
+OpenVPN clients and WireGuard peers with a recent handshake; and of each site-to-site connection
+that is set up, whether it is up, the bytes and packets per second through its child SAs since
+the previous sample, and how long its IKE SA has been up. The interfaces, the enabled systems and
+the connections are those vpn-router-setup resolved, read from /run/vpn-router/web.json.
+
+The sample is written to the detail tables of vpnrouter_metrics. When the clock has passed into
+a new hour, day, month or year, the complete periods are summarized, and once a day, after the
+daily summary, the rows older than their retention are deleted; both also run on start. A
+failure of the database is logged and tried again with the next sample.
+
+GET /metrics answers the last sample in the Prometheus text format, written by
+prometheus_client, with OpenTelemetry semantic convention names in their Prometheus form, for an
+OpenTelemetry Collector's prometheus receiver. It needs HTTP Basic authentication as the service
+account metrics.service_account names, which vpn-router-setup keeps with the password in
+/etc/vpn-router/metrics.secret, checked with a bind to the router's directory, so failed attempts
+lock the account as for any user. The service listens on the interfaces metrics.listen
+names and on the loopback address, where Caddy serves it over HTTPS at metrics.https_port; the
+address of a call through Caddy is the last entry of its X-Forwarded-For header.
+
+When metrics.otlp_endpoint is set, every sample is also pushed there as an OTLP/HTTP JSON
+ExportMetricsServiceRequest, with HTTP Basic authentication when metrics.otlp_username is set.
+"""
+
+import base64
+import datetime
+import json
+import os
+import signal
+import socket
+import sys
+import threading
+import time
+import urllib.error
+import urllib.request
+from http.server import BaseHTTPRequestHandler
+
+sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
+
+import ldap
+import psutil
+from prometheus_client import CONTENT_TYPE_LATEST, generate_latest
+from prometheus_client.core import CollectorRegistry, CounterMetricFamily, GaugeMetricFamily
+
+import vpnrouter
+import vpnrouter_metrics
+import vpnrouter_net
+import vpnrouter_serve
+import vpnrouter_status
+import vpnrouter_users
+
+STATE_FILE = '/run/vpn-router/web.json'
+CONFIG_FILE = '/etc/vpn-router/vpn-router.json'
+
+SAMPLE_SECONDS = 60
+
+#: Seconds between two discoveries of where the service should listen.
+DISCOVERY_SECONDS = 30
+
+#: Seconds an OTLP push may take.
+OTLP_TIMEOUT = 10
+
+#: The mount point whose space is measured.
+MOUNTPOINT = '/'
+
+#: The last sample, as sample() makes it, or None before the first.
+_latest = {'sample': None}
+_latest_lock = threading.Lock()
+
+
+def log(message):
+    print(f'vpn-router-metrics: {message}', file=sys.stderr, flush=True)
+
+
+def read_json(path):
+    """The object in a JSON file, or {} when the file is missing or holds no object."""
+    try:
+        with open(path, encoding='utf-8') as handle:
+            data = json.load(handle)
+    except (OSError, ValueError):
+        return {}
+    return data if isinstance(data, dict) else {}
+
+
+def section(configuration, name):
+    """A section of the configuration, or {} when it is missing or not an object."""
+    value = configuration.get(name)
+    return value if isinstance(value, dict) else {}
+
+
+def clients(state):
+    """The VPN clients of each enabled VPN system, None where it cannot be asked."""
+    counts = {}
+    if state.get('p2s'):
+        counts['p2s'] = vpnrouter_status.established('road-warrior')
+    if state.get('openvpn'):
+        counts['openvpn'] = vpnrouter_status.openvpn_clients()
+    if state.get('wireguard'):
+        counts['wireguard'] = vpnrouter_status.wireguard_connected()
+    return counts
+
+
+def sample(previous):
+    """A sample of the router, with the rates since the previous sample, None for the first."""
+    state = read_json(STATE_FILE)
+    network = section(state, 'network')
+    names = [name for name in dict.fromkeys((network.get('external_interface'), network.get('internal_interface')))
+             if isinstance(name, str) and name]
+    counters = psutil.net_io_counters(pernic=True)
+    memory = psutil.virtual_memory()
+    disk = psutil.disk_usage(MOUNTPOINT)
+    load1, load5, load15 = os.getloadavg()
+    connections = state.get('site_to_site')
+    tunnels = vpnrouter_status.site_to_site_tunnels(connections) if isinstance(connections, list) and connections else {}
+    now = time.time()
+    current = {
+        'time': now,
+        'monotonic': time.monotonic(),
+        'system': {
+            'cpu': psutil.cpu_percent(interval=None),
+            'memory_used': memory.total - memory.available,
+            'memory_total': memory.total,
+            'disk_free': disk.free,
+            'disk_total': disk.total,
+            'load1': load1,
+            'load5': load5,
+            'load15': load15,
+        },
+        'counters': {name: (counters[name].bytes_recv, counters[name].bytes_sent) for name in names if name in counters},
+        'clients': clients(state),
+        'rates': {},
+        'tunnels': tunnels or {},
+        'site_to_site': {},
+    }
+    elapsed = current['monotonic'] - previous['monotonic'] if previous is not None else 0
+    if elapsed > 0:
+        for name, (received, sent) in current['counters'].items():
+            if name in previous['counters']:
+                before_received, before_sent = previous['counters'][name]
+                current['rates'][name] = (max(received - before_received, 0) / elapsed, max(sent - before_sent, 0) / elapsed)
+    # A connection's rates need its counters of the previous sample; a counter that went down was rekeyed and counts from zero.
+    for name, tunnel in current['tunnels'].items():
+        before = previous['tunnels'].get(name) if elapsed > 0 else None
+        values = {'up': tunnel['up'], 'uptime': tunnel['uptime']}
+        for value, counter in (('rx', 'bytes_in'), ('tx', 'bytes_out'), ('packets_rx', 'packets_in'), ('packets_tx', 'packets_out')):
+            if before is None:
+                values[value] = None
+            else:
+                count = tunnel[counter]
+                values[value] = (count - before[counter] if count >= before[counter] else count) / elapsed
+        current['site_to_site'][name] = values
+    return current
+
+
+def periods(moment):
+    """The hour, the day, the month and the year of a UTC moment, keyed by the levels of vpnrouter_metrics.LEVELS."""
+    return {'hourly': moment.strftime('%Y-%m-%dT%H'), 'daily': moment.strftime('%Y-%m-%d'),
+            'monthly': moment.strftime('%Y-%m'), 'yearly': moment.strftime('%Y')}
+
+
+def store(cfg, current, done):
+    """Write the sample, summarize the periods the clock has passed and collect the garbage once a day.
+
+    done holds the periods summarized so far, and garbage the day of the last garbage collection;
+    it is updated only for what succeeded.
+    """
+    now = periods(datetime.datetime.now(datetime.timezone.utc))
+    with vpnrouter_metrics.database(cfg) as connection:
+        vpnrouter_metrics.write(connection, current['system'], current['rates'], current['clients'], current['site_to_site'])
+        for level, _, _ in vpnrouter_metrics.LEVELS:
+            if done.get(level) != now[level]:
+                vpnrouter_metrics.summarize(connection, level)
+                done[level] = now[level]
+        if done.get('garbage') != now['daily']:
+            vpnrouter_metrics.collect_garbage(connection, cfg)
+            done['garbage'] = now['daily']
+
+
+def attributes(**values):
+    """OTLP attributes of string values."""
+    return [{'key': key, 'value': {'stringValue': value}} for key, value in values.items()]
+
+
+def otlp_request(current, started):
+    """The OTLP/HTTP JSON ExportMetricsServiceRequest of a sample; started is when the counters were first read, in Unix seconds."""
+    stamp = str(int(current['time'] * 1e9))
+    start = str(int(started * 1e9))
+    system = current['system']
+
+    def gauge(name, unit, points):
+        return {'name': name, 'unit': unit, 'gauge': {'dataPoints': [
+            {'timeUnixNano': stamp, 'asDouble': float(value), 'attributes': attributes(**labels)}
+            for value, labels in points if value is not None]}}
+
+    metrics = [
+        gauge('system.cpu.utilization', '1', [(system['cpu'] / 100, {})]),
+        gauge('system.memory.usage', 'By', [(system['memory_used'], {'state': 'used'})]),
+        gauge('system.memory.limit', 'By', [(system['memory_total'], {})]),
+        gauge('system.filesystem.usage', 'By', [(system['disk_free'], {'mountpoint': MOUNTPOINT, 'state': 'free'}),
+                                               (system['disk_total'] - system['disk_free'], {'mountpoint': MOUNTPOINT, 'state': 'used'})]),
+        gauge('system.filesystem.utilization', '1',
+              [(1 - system['disk_free'] / system['disk_total'] if system['disk_total'] else None, {'mountpoint': MOUNTPOINT})]),
+        gauge('system.cpu.load_average.1m', '{thread}', [(system['load1'], {})]),
+        gauge('system.cpu.load_average.5m', '{thread}', [(system['load5'], {})]),
+        gauge('system.cpu.load_average.15m', '{thread}', [(system['load15'], {})]),
+        gauge('vpn_router.clients', '{client}', [(count, {'kind': kind}) for kind, count in current['clients'].items()]),
+        gauge('vpn_router.site_to_site.up', '1', [(values['up'], {'connection': name})
+                                                 for name, values in current['site_to_site'].items()]),
+        gauge('vpn_router.site_to_site.uptime', 's', [(values['uptime'], {'connection': name})
+                                                     for name, values in current['site_to_site'].items()]),
+        gauge('vpn_router.site_to_site.io', 'By/s', [(values[value], {'connection': name, 'direction': direction})
+                                                    for name, values in current['site_to_site'].items()
+                                                    for value, direction in (('rx', 'receive'), ('tx', 'transmit'))]),
+        gauge('vpn_router.site_to_site.packets', '{packet}/s',
+              [(values[value], {'connection': name, 'direction': direction})
+               for name, values in current['site_to_site'].items()
+               for value, direction in (('packets_rx', 'receive'), ('packets_tx', 'transmit'))]),
+        {'name': 'system.network.io', 'unit': 'By', 'sum': {
+            'aggregationTemporality': 2, 'isMonotonic': True, 'dataPoints': [
+                {'startTimeUnixNano': start, 'timeUnixNano': stamp, 'asInt': str(value),
+                 'attributes': attributes(device=name, direction=direction)}
+                for name, counters in current['counters'].items()
+                for value, direction in zip(counters, ('receive', 'transmit'))]}},
+    ]
+    return {'resourceMetrics': [{
+        'resource': {'attributes': attributes(**{'service.name': 'vpn-router', 'host.name': socket.gethostname()})},
+        'scopeMetrics': [{'scope': {'name': 'vpn-router-metrics'}, 'metrics': metrics}],
+    }]}
+
+
+def push(metrics, current, started):
+    """Push the sample to metrics.otlp_endpoint; raises OSError when the collector does not take it."""
+    request = urllib.request.Request(metrics['otlp_endpoint'], data=json.dumps(otlp_request(current, started)).encode(),
+                                     headers={'Content-Type': 'application/json'}, method='POST')
+    if metrics['otlp_username']:
+        credentials = f"{metrics['otlp_username']}:{metrics['otlp_password']}".encode()
+        request.add_header('Authorization', 'Basic ' + base64.b64encode(credentials).decode())
+    with urllib.request.urlopen(request, timeout=OTLP_TIMEOUT) as response:
+        response.read()
+
+
+def collector_loop():
+    """Take a sample on every minute, store it and push it; every new failure is logged once."""
+    previous, done, reported = None, {}, {}
+    started = time.time()
+    psutil.cpu_percent(interval=None)
+
+    def report(kind, message):
+        if message != reported.get(kind):
+            if message:
+                log(message)
+            elif reported.get(kind):
+                log(f'{kind} works again')
+            reported[kind] = message
+
+    while True:
+        time.sleep(SAMPLE_SECONDS - time.time() % SAMPLE_SECONDS)
+        try:
+            cfg = vpnrouter.load_config()
+        except SystemExit as exc:
+            report('configuration', str(exc.code))
+            continue
+        report('configuration', '')
+        current = sample(previous)
+        previous = current
+        with _latest_lock:
+            _latest['sample'] = current
+        try:
+            store(cfg, current, done)
+            report('database', '')
+        except vpnrouter_metrics.DatabaseError as exc:
+            report('database', str(exc))
+        metrics = cfg['metrics']
+        if metrics['otlp_endpoint']:
+            try:
+                push(metrics, current, started)
+                report('OTLP push', '')
+            except (OSError, ValueError) as exc:
+                report('OTLP push', f"pushing to {metrics['otlp_endpoint']} failed: {exc}")
+
+
+class Sample:
+    """A prometheus_client collector of a sample."""
+
+    def __init__(self, current):
+        self.current = current
+
+    def collect(self):
+        system = self.current['system']
+        yield GaugeMetricFamily('system_cpu_utilization', 'CPU utilization since the previous sample, 0 to 1',
+                                value=system['cpu'] / 100)
+        memory = GaugeMetricFamily('system_memory_usage_bytes', 'Memory in use', labels=['state'])
+        memory.add_metric(['used'], system['memory_used'])
+        yield memory
+        yield GaugeMetricFamily('system_memory_limit_bytes', 'Total memory', value=system['memory_total'])
+        usage = GaugeMetricFamily('system_filesystem_usage_bytes', 'Space of the file system', labels=['mountpoint', 'state'])
+        usage.add_metric([MOUNTPOINT, 'free'], system['disk_free'])
+        usage.add_metric([MOUNTPOINT, 'used'], system['disk_total'] - system['disk_free'])
+        yield usage
+        if system['disk_total']:
+            utilization = GaugeMetricFamily('system_filesystem_utilization', 'Used part of the file system, 0 to 1',
+                                            labels=['mountpoint'])
+            utilization.add_metric([MOUNTPOINT], 1 - system['disk_free'] / system['disk_total'])
+            yield utilization
+        for period in ('1m', '5m', '15m'):
+            yield GaugeMetricFamily(f'system_cpu_load_average_{period}', f'Load average over {period}',
+                                    value=system[f'load{period[:-1]}'])
+        network = CounterMetricFamily('system_network_io_bytes', 'Bytes received and sent', labels=['device', 'direction'])
+        for name, (received, sent) in self.current['counters'].items():
+            network.add_metric([name, 'receive'], received)
+            network.add_metric([name, 'transmit'], sent)
+        yield network
+        vpn = GaugeMetricFamily('vpn_router_clients', 'Connected VPN clients', labels=['kind'])
+        for kind, count in self.current['clients'].items():
+            if count is not None:
+                vpn.add_metric([kind], count)
+        yield vpn
+        tunnels = self.current['site_to_site']
+        up = GaugeMetricFamily('vpn_router_site_to_site_up', 'Whether the site-to-site connection is established, 1 or 0',
+                               labels=['connection'])
+        uptime = GaugeMetricFamily('vpn_router_site_to_site_uptime_seconds', 'Seconds the IKE SA of the site-to-site connection has been up',
+                                   labels=['connection'])
+        io = GaugeMetricFamily('vpn_router_site_to_site_io_bytes_per_second',
+                               'Bytes per second received and sent through the site-to-site connection', labels=['connection', 'direction'])
+        packets = GaugeMetricFamily('vpn_router_site_to_site_packets_per_second',
+                                    'Packets per second received and sent through the site-to-site connection',
+                                    labels=['connection', 'direction'])
+        for name, values in tunnels.items():
+            up.add_metric([name], values['up'])
+            if values['uptime'] is not None:
+                uptime.add_metric([name], values['uptime'])
+            for family, received, sent in ((io, 'rx', 'tx'), (packets, 'packets_rx', 'packets_tx')):
+                for value, direction in ((received, 'receive'), (sent, 'transmit')):
+                    if values[value] is not None:
+                        family.add_metric([name, direction], values[value])
+        yield from (up, uptime, io, packets)
+
+
+def authorized(header, account):
+    """Whether an Authorization header holds the password of the service account; LDAP errors propagate."""
+    if not header.startswith('Basic '):
+        return False
+    try:
+        name, _, password = base64.b64decode(header[6:].strip(), validate=True).decode().partition(':')
+    except ValueError:
+        return False
+    if name != account:
+        return False
+    try:
+        with vpnrouter_users.session() as connection:
+            if not vpnrouter_users.service(vpnrouter_users.entry(connection, name)):
+                return False
+        vpnrouter_users.login(name, password, None)
+    except vpnrouter_users.Failure:
+        return False
+    return True
+
+
+class Handler(BaseHTTPRequestHandler):
+    protocol_version = 'HTTP/1.1'
+
+    def client(self):
+        """The client address: the last X-Forwarded-For entry of a call through Caddy on the loopback address, else the peer."""
+        peer = self.client_address[0]
+        forwarded = self.headers.get('X-Forwarded-For', '')
+        if peer == vpnrouter_net.LOOPBACK and forwarded.strip():
+            return forwarded.split(',')[-1].strip()
+        return peer
+
+    def answer(self, send_body):
+        self.principal = None
+        extra = []
+        path = self.path.split('?', 1)[0]
+        account = str(section(read_json(CONFIG_FILE), 'metrics').get('service_account',
+                                                                       vpnrouter.SCHEMA['metrics']['service_account'])).strip()
+        with _latest_lock:
+            current = _latest['sample']
+        if path != '/metrics':
+            body, kind, code = b'Not found\n', 'text/plain; charset=utf-8', 404
+        elif not account:
+            body, kind, code = b'metrics.service_account is not set\n', 'text/plain; charset=utf-8', 503
+        else:
+            try:
+                allowed = authorized(self.headers.get('Authorization', ''), account)
+            except (ldap.LDAPError, OSError) as exc:
+                allowed = None
+                body, kind, code = f'the user directory failed: {exc}\n'.encode(), 'text/plain; charset=utf-8', 500
+            if allowed is False:
+                body, kind, code = b'Unauthorized\n', 'text/plain; charset=utf-8', 401
+                extra.append(('WWW-Authenticate', 'Basic realm="vpn-router metrics", charset="UTF-8"'))
+            elif allowed:
+                self.principal = account
+                if current is None:
+                    body, kind, code = b'no sample yet\n', 'text/plain; charset=utf-8', 503
+                else:
+                    registry = CollectorRegistry()
+                    registry.register(Sample(current))
+                    body, kind, code = generate_latest(registry), CONTENT_TYPE_LATEST, 200
+        self.send_response(code)
+        self.send_header('Content-Type', kind)
+        self.send_header('Content-Length', str(len(body)))
+        self.send_header('Cache-Control', 'no-store')
+        for name, value in extra:
+            self.send_header(name, value)
+        self.end_headers()
+        if send_body:
+            self.wfile.write(body)
+
+    def do_GET(self):
+        self.answer(True)
+
+    def do_HEAD(self):
+        self.answer(False)
+
+    def log_message(self, format, *args):
+        principal = getattr(self, 'principal', None) or '-'
+        sys.stderr.write('%s - %s [%s] %s\n' % (self.client(), principal, self.log_date_time_string(), format % args))
+
+
+def wanted():
+    """Where the service listens: the addresses of the interfaces metrics.listen names and LOOPBACK, at metrics.port."""
+    configuration = read_json(CONFIG_FILE)
+    metrics = section(configuration, 'metrics')
+    defaults = vpnrouter.SCHEMA['metrics']
+    if metrics.get('enabled', defaults['enabled']) is False:
+        return [], 'metrics are disabled in the configuration'
+    port = metrics.get('port', defaults['port'])
+    if isinstance(port, bool) or not isinstance(port, int) or not 1 <= port <= 65535:
+        port = defaults['port']
+    addresses = vpnrouter_net.listen_addresses(configuration, metrics.get('listen', defaults['listen']))
+    if not addresses:
+        return [], 'the addresses to listen on are not known yet'
+    return [(address, port) for address in [*addresses, vpnrouter_net.LOOPBACK]], ''
+
+
+def main():
+    signal.signal(signal.SIGTERM, lambda *_: sys.exit(0))
+    threading.Thread(target=collector_loop, name='collector', daemon=True).start()
+    try:
+        vpnrouter_serve.serve(wanted, Handler, log, DISCOVERY_SECONDS)
+    except KeyboardInterrupt:
+        pass
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main())

debian-package/src/usr/lib/vpn-router/openvpn-same-user

@@ -0,0 +1,12 @@
+#!/usr/bin/python3
+"""Accept an OpenVPN login only when the user name is the common name of the client certificate.
+
+Run by the OpenVPN server as auth-user-pass-verify with via-env, next to the LDAP plugin that
+checks the password, so a user's certificate cannot be used with another user's password.
+"""
+
+import os
+import sys
+
+username = os.environ.get('username', '')
+sys.exit(0 if username and username == os.environ.get('common_name') else 1)

debian-package/src/usr/lib/vpn-router/setup

@@ -5,32 +5,85 @@ Regenerates the configuration files first, then does everything that needs
 the live system: routes, DNS-derived identity, WireGuard keys, firewall rules
 and service reloads. Idempotent - a second run changes nothing.
 
+Until a run has succeeded (SETUP_DONE), every run starts the bootstrap
+services, BOOTSTRAP_UNITS, which show the progress and the failure recorded in
+SETUP_STATE on BOOTSTRAP_PORTS, and the firewall opens those ports on the
+internal interface. The first successful run writes SETUP_DONE, closes the
+ports and stops the bootstrap services; a failed run leaves them running.
+
 Interface names come from the configuration and are never guessed. The
 firewall and NAT rules are written against them, so choosing the wrong one
 would be a silent and security-relevant mistake.
 """
 
+import grp
+import json
 import os
 import pathlib
+import re
+import secrets
 import socket
 import subprocess
 import sys
+import time
 
 sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
 
+import ldap
+
 import vpnrouter
+import vpnrouter_ldap
+import vpnrouter_users
 from vpnrouter import fail
 
 BEFORE_RULES = pathlib.Path('/etc/ufw/before.rules')
 UFW_DEFAULTS = pathlib.Path('/etc/default/ufw')
 WG_DIR       = pathlib.Path('/etc/wireguard')
+WEB_STATE    = pathlib.Path('/run/vpn-router/web.json')
+
+#: The state, the current step and the failure of the last run, which the bootstrap services show.
+SETUP_STATE  = pathlib.Path('/run/vpn-router/setup.json')
+
+#: Written once a run has succeeded; until then setup runs the bootstrap services and opens their ports.
+SETUP_DONE   = pathlib.Path('/var/lib/vpn-router/setup.done')
+
+#: The units of the bootstrap services, and their ports: the Router Manager's and the router's.
+BOOTSTRAP_UNITS = ('vpn-router-bootstrap.service', 'vpn-router-manager-bootstrap.service')
+BOOTSTRAP_PORTS = (17080, 17081)
+
+#: The Router Manager, vpn-router-manager, when it is installed, and its settings.
+MANAGER        = pathlib.Path('/usr/lib/vpn-router-manager/manager')
+MANAGER_CONFIG = pathlib.Path('/etc/vpn-router-manager/manager.json')
+
+#: The default admin of the Router Manager, {"username", "password"}, which its package writes from
+#: its debconf answers; setup creates the account and removes the file.
+MANAGER_ADMIN  = pathlib.Path('/etc/vpn-router-manager/admin.json')
+MANAGER_PORT   = 7080
+
+#: The password of the metrics service account, which setup makes and keeps (root only).
+METRICS_SECRET = pathlib.Path('/etc/vpn-router/metrics.secret')
+
+#: The configuration of Caddy, the HTTPS proxy, which the package's drop-in for caddy.service names;
+#: Caddy runs as the user and group CADDY_GROUP, which the caddy package creates.
+CADDYFILE   = pathlib.Path('/etc/caddy/vpn-router.Caddyfile')
+CADDY_UNIT  = 'caddy.service'
+CADDY_GROUP = 'caddy'
 
 FILTER_ANCHOR = '# End required lines'
 
+#: Configuration options left out wherever the configuration is shown.
+SECRET_OPTION = re.compile(r'(^psk$)|secret|password|token|_key$|^database$')
+
 BLOCKS = (
     ('# IPSEC RULES START',         '# IPSEC RULES END'),
     ('# WIREGUARD RULES START',     '# WIREGUARD RULES END'),
+    ('# OPENVPN RULES START',       '# OPENVPN RULES END'),
     ('# P2S DNS RULES START',       '# P2S DNS RULES END'),
+    ('# WEB RULES START',           '# WEB RULES END'),
+    ('# BOOTSTRAP RULES START',     '# BOOTSTRAP RULES END'),
+    ('# MANAGER RULES START',       '# MANAGER RULES END'),
+    ('# HTTPS RULES START',         '# HTTPS RULES END'),
+    ('# METRICS RULES START',       '# METRICS RULES END'),
     ('# ROUTER FORWARD RULES START', '# ROUTER FORWARD RULES END'),
     ('# ROUTER NAT RULES START',    '# ROUTER NAT RULES END'),
     ('# ROUTER MSS RULES START',    '# ROUTER MSS RULES END'),
@@ -114,7 +167,17 @@ def is_configured(ctx):
     a host that was working before.
     """
     return any((vpnrouter.s2s_ready(ctx), vpnrouter.routing_ready(ctx),
-                vpnrouter.p2s_ready(ctx), vpnrouter.wg_ready(ctx)))
+                vpnrouter.p2s_ready(ctx), vpnrouter.wg_ready(ctx),
+                vpnrouter.openvpn_ready(ctx)))
+
+
+def manager_port():
+    """The port of the Router Manager: port in its settings, or MANAGER_PORT when it is not set or not a port."""
+    try:
+        port = json.loads(MANAGER_CONFIG.read_text(encoding='utf-8')).get('port')
+    except (OSError, ValueError, AttributeError):
+        return MANAGER_PORT
+    return port if isinstance(port, int) and not isinstance(port, bool) and 1 <= port <= 65535 else MANAGER_PORT
 
 
 def filter_block(ctx):
@@ -139,16 +202,81 @@ def filter_block(ctx):
             '# WIREGUARD RULES END',
         ]
 
-    if vpnrouter.p2s_ready(ctx) and ctx['local_addr_list']:
+    if vpnrouter.openvpn_ready(ctx):
-        pool = ctx['p2s_address_pool']
+        device = ctx['openvpn_device']
-        lines += ['', '# P2S DNS RULES START']
+        lines += [
-        for address in ctx['local_addr_list']:
+            '',
+            '# OPENVPN RULES START',
+            f'-A ufw-before-input -i {ctx["wan_iface"]} -p {ctx["openvpn_protocol"]} --dport {ctx["openvpn_port"]} -j ACCEPT',
+        ]
+        for subnet in ctx['local_cidr_list']:
             lines += [
-                f'-A ufw-before-input -s {pool} -d {address} -p udp --dport 53 -j ACCEPT',
+                f'-A ufw-before-forward -i {device} -d {subnet} -j ACCEPT',
-                f'-A ufw-before-input -s {pool} -d {address} -p tcp --dport 53 -j ACCEPT',
+                f'-A ufw-before-forward -o {device} -s {subnet} -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT',
             ]
+        lines += ['# OPENVPN RULES END']
+
+    pools = [pool for ready, pool in ((vpnrouter.p2s_ready(ctx), ctx['p2s_address_pool']),
+                                      (vpnrouter.openvpn_ready(ctx), ctx['openvpn_pool'])) if ready]
+    if pools and ctx['local_addr_list']:
+        lines += ['', '# P2S DNS RULES START']
+        for pool in pools:
+            for address in ctx['local_addr_list']:
+                lines += [
+                    f'-A ufw-before-input -s {pool} -d {address} -p udp --dport 53 -j ACCEPT',
+                    f'-A ufw-before-input -s {pool} -d {address} -p tcp --dport 53 -j ACCEPT',
+                ]
         lines += ['# P2S DNS RULES END']
 
+    # The web service, the Router Manager and the metrics service choose their interfaces themselves, so their ports are open on both.
+    interfaces = [name for name in (ctx['int_iface'], ctx['wan_iface']) if name]
+    if ctx['web_enabled'] and interfaces:
+        lines += [
+            '',
+            '# WEB RULES START',
+            *(f'-A ufw-before-input -i {name} -p tcp --dport {ctx["web_port"]} -j ACCEPT' for name in interfaces),
+            '# WEB RULES END',
+        ]
+
+    if not SETUP_DONE.exists() and ctx['int_iface']:
+        lines += [
+            '',
+            '# BOOTSTRAP RULES START',
+            *(f'-A ufw-before-input -i {ctx["int_iface"]} -p tcp --dport {port} -j ACCEPT' for port in BOOTSTRAP_PORTS),
+            '# BOOTSTRAP RULES END',
+        ]
+
+    if MANAGER.exists() and interfaces:
+        port = manager_port()
+        lines += [
+            '',
+            '# MANAGER RULES START',
+            *(f'-A ufw-before-input -i {name} -p tcp --dport {port} -j ACCEPT' for name in interfaces),
+            '# MANAGER RULES END',
+        ]
+
+    if vpnrouter.metrics_ready(ctx) and interfaces:
+        lines += [
+            '',
+            '# METRICS RULES START',
+            *(f'-A ufw-before-input -i {name} -p tcp --dport {ctx["metrics_port"]} -j ACCEPT' for name in interfaces),
+            '# METRICS RULES END',
+        ]
+
+    # Let's Encrypt reaches the HTTP challenge on the external interface; the HTTPS ports are open on both.
+    if vpnrouter.https_ready(ctx) and interfaces:
+        ports = [port for wanted, port in ((ctx['web_enabled'], ctx['https_web_port']),
+                                           (MANAGER.exists(), ctx['https_manager_port']),
+                                           (vpnrouter.metrics_ready(ctx), ctx['metrics_https_port'])) if wanted]
+        lines += [
+            '',
+            '# HTTPS RULES START',
+            *([f'-A ufw-before-input -i {ctx["wan_iface"]} -p tcp --dport {vpnrouter.ACME_HTTP_PORT} -j ACCEPT']
+              if ctx['wan_iface'] else []),
+            *(f'-A ufw-before-input -i {name} -p tcp --dport {port} -j ACCEPT' for name in interfaces for port in ports),
+            '# HTTPS RULES END',
+        ]
+
     if vpnrouter.routing_ready(ctx):
         lines += ['', '# ROUTER FORWARD RULES START']
         for subnet in ctx['local_cidr_list']:
@@ -166,7 +294,6 @@ def nat_and_mangle_block(ctx):
 
     if vpnrouter.routing_ready(ctx):
         subnets = ctx['local_cidr_list']
-        remote = ctx['remote_cidr_list']
         lines += [
             '',
             '# ROUTER NAT RULES START',
@@ -174,11 +301,14 @@ def nat_and_mangle_block(ctx):
             ':POSTROUTING ACCEPT [0:0]',
             '-F POSTROUTING',
         ]
-        # Every RETURN precedes every MASQUERADE, so tunnel-bound traffic
+        # Every RETURN precedes every MASQUERADE, so traffic bound for a tunnel
-        # escapes NAT whichever local subnet it came from.
+        # escapes NAT; each ready connection pairs its local and remote subnets.
-        for subnet in subnets:
+        for connection in ctx['connections']:
-            for cidr in remote:
+            if not connection['ready']:
-                lines.append(f'-A POSTROUTING -s {subnet} -d {cidr} -j RETURN')
+                continue
+            for subnet in connection['local_cidr_list']:
+                for cidr in connection['remote_cidr_list']:
+                    lines.append(f'-A POSTROUTING -s {subnet} -d {cidr} -j RETURN')
         for subnet in subnets:
             lines.append(
                 f'-A POSTROUTING -s {subnet} -o {ctx["wan_iface"]} -j MASQUERADE')
@@ -287,6 +417,155 @@ def manage_wireguard(ctx, changed):
         run(['systemctl', 'restart', unit], check=False)
 
 
+def manage_directory():
+    """Provision the user directory in slapd, whether or not a P2S system is configured; the problem, or ''."""
+    if vpnrouter_ldap.ensure_services() or not unit_is_active('slapd'):
+        result = run(['systemctl', 'restart', 'slapd'], check=False)
+        if result.returncode != 0:
+            return f'slapd cannot be restarted: {result.stderr.strip()}'
+    try:
+        vpnrouter_ldap.provision()
+    except Exception as exc:
+        return f'the user directory cannot be provisioned: {exc!r}'
+    return ''
+
+
+def manage_admin():
+    """Create the Router Manager's default admin from MANAGER_ADMIN, then remove the file; the problem, or ''.
+
+    The admin is a Manager-only account and a member of the admins group, which is created with it
+    when it does not exist.
+    """
+    if not MANAGER_ADMIN.exists():
+        return ''
+    try:
+        answers = json.loads(MANAGER_ADMIN.read_text(encoding='utf-8'))
+        name, password = answers['username'], answers['password']
+        with vpnrouter_users.session() as connection:
+            vpnrouter_users.add(connection, name, password, {})
+            try:
+                vpnrouter_users.group_member_add(connection, vpnrouter_users.ADMIN_GROUP, name)
+            except vpnrouter_users.Failure as failure:
+                if failure.status != 3:
+                    raise
+                vpnrouter_users.group_add(connection, vpnrouter_users.ADMIN_GROUP, 'Router administrators', [name])
+        MANAGER_ADMIN.unlink()
+    except (OSError, ValueError, KeyError, TypeError, ldap.LDAPError, vpnrouter_users.Failure) as exc:
+        return f'the Router Manager admin cannot be created from {MANAGER_ADMIN}: {exc}'
+    return ''
+
+
+def manage_metrics_account(ctx):
+    """Keep the service account metrics.service_account names, whose password is in METRICS_SECRET; the problem, or ''.
+
+    While metrics are enabled, setup owns the account, an internal service account: it adds it as a service account
+    with a random password and the labels of vpnrouter_users.internal_accounts when it is missing, sets those labels
+    when they differ, and gives it a new random password, ending any lockout, when METRICS_SECRET is missing or does
+    not hold its password. The password of a disabled account is left as it is. An account of that name that is not a
+    service account is left alone and is a problem.
+    """
+    if not vpnrouter.metrics_ready(ctx):
+        return ''
+    name = ctx['metrics_service_account']
+    wanted = vpnrouter_users.internal_accounts().get(name, {})
+    try:
+        vpnrouter_users.check_name(name)
+        try:
+            password = METRICS_SECRET.read_text(encoding='utf-8').strip()
+        except OSError:
+            password = ''
+        with vpnrouter_users.session() as connection:
+            try:
+                current = vpnrouter_users.entry(connection, name)
+            except vpnrouter_users.Failure as failure:
+                if failure.status != 3:
+                    raise
+                current = None
+            if current is not None and not vpnrouter_users.service(current):
+                return f'metrics.service_account: {name} is a user, not a service account'
+            if current is not None and any(vpnrouter_users.summary(current)[key] != value for key, value in wanted.items()):
+                vpnrouter_users.update_service(connection, name, wanted)
+            # An administrator who disabled the account meant it; its password is left as it is.
+            if current is not None and not vpnrouter_users.summary(current)['enabled']:
+                return ''
+            valid = False
+            if current is not None and password:
+                try:
+                    vpnrouter_users.login(name, password, None)
+                    valid = True
+                except vpnrouter_users.Failure:
+                    pass
+            if not valid:
+                password = secrets.token_urlsafe(32)
+                METRICS_SECRET.touch(mode=0o600)
+                METRICS_SECRET.chmod(0o600)
+                METRICS_SECRET.write_text(password + '\n', encoding='utf-8')
+                if current is None:
+                    vpnrouter_users.add(connection, name, password, wanted, service=True)
+                else:
+                    vpnrouter_users.passwd(connection, name, password)
+                    vpnrouter_users.unlock(connection, name)
+    except (OSError, ldap.LDAPError, vpnrouter_users.Failure) as exc:
+        return f'the metrics service account cannot be kept: {exc}'
+    return ''
+
+
+def manage_openvpn(ctx, changed):
+    """Run the OpenVPN server when it is configured, and stop it when it is not."""
+    unit = 'openvpn-server@vpn-router'
+
+    if not vpnrouter.openvpn_ready(ctx):
+        if unit_is_active(unit):
+            run(['systemctl', 'disable', '--now', unit], check=False)
+        return
+
+    if not unit_is_active(unit):
+        run(['systemctl', 'enable', '--now', unit], check=False)
+    elif 'openvpn' in changed or 'swanctl' in changed:
+        run(['systemctl', 'restart', unit], check=False)
+
+
+def manage_caddy(ctx):
+    """Run Caddy, the HTTPS proxy of the web service, the Router Manager and the metrics service, while HTTPS is ready, and stop it when not.
+
+    CADDYFILE serves the web service on https.web_port, while the manager is installed the
+    manager on https.manager_port, and while metrics are enabled the metrics service on
+    metrics.https_port, all for wan.local_fqdn with a certificate from Let's Encrypt, and all
+    calling the service on its loopback address.
+    """
+    if not vpnrouter.https_ready(ctx):
+        vpnrouter.remove(CADDYFILE)
+        if unit_is_active(CADDY_UNIT):
+            run(['systemctl', 'disable', '--now', CADDY_UNIT], check=False)
+        return
+
+    try:
+        group = grp.getgrnam(CADDY_GROUP).gr_gid
+    except KeyError:
+        fail(f'the group {CADDY_GROUP} does not exist; is the caddy package installed?')
+    caddy_ctx = dict(ctx, manager_port=manager_port() if MANAGER.exists() else 0)
+    changed = vpnrouter.render(vpnrouter._jinja_env(), caddy_ctx, 'Caddyfile.j2', CADDYFILE, 0o640)
+    os.chown(CADDYFILE, 0, group)
+    if not unit_is_active(CADDY_UNIT):
+        run(['systemctl', 'enable', '--now', CADDY_UNIT], check=False)
+    elif changed:
+        run(['systemctl', 'reload', CADDY_UNIT], check=False)
+
+
+def manage_metrics(ctx):
+    """Run the metrics service while metrics are enabled, and stop it when not.
+
+    The service reads its settings itself, so it is restarted only to run the code of a new package version, which
+    every run of setup after an upgrade does.
+    """
+    unit = 'vpn-router-metrics.service'
+    if not vpnrouter.metrics_ready(ctx):
+        run(['systemctl', 'disable', '--now', unit], check=False)
+        return
+    run(['systemctl', 'enable', unit], check=False)
+    run(['systemctl', 'restart', unit], check=False)
+
+
 def run_platform_setup(cfg, ctx):
     """Let the platform module apply its own state, if it defines any."""
     module = vpnrouter.platform_module(cfg)
@@ -303,31 +582,112 @@ def refresh_docker_rules():
     """Apply the new settings to the Docker rules while Docker is running.
 
     When Docker is not running, vpn-router-docker applies them after
-    docker.service starts.
+    docker.service starts. The restart is waited for, so the rules are in place
+    when this setup ends; vpn-router-docker is therefore not ordered after
+    vpn-router-setup.
     """
     if unit_is_active('docker.service'):
         run(['systemctl', 'restart', 'vpn-router-docker.service'], check=False)
 
 
+def refresh_web(cfg, ctx):
+    """Hand the web service the features and facts it shows.
+
+    The service reads the file at each request: the features of the router,
+    the configuration without its secret options and the facts resolved from
+    the system, which it cannot work out with the standard library alone. The
+    file is readable by root only. The service follows a changed address or
+    port itself. Secret options are left out at every level, so also from each
+    site-to-site connection.
+    """
+    def public(options):
+        return {option: public(value) if isinstance(value, dict) else value
+                for option, value in options.items()
+                if isinstance(value, dict) or not SECRET_OPTION.search(option)}
+
+    WEB_STATE.parent.mkdir(parents=True, exist_ok=True)
+    WEB_STATE.touch(mode=0o600)
+    WEB_STATE.chmod(0o600)
+    WEB_STATE.write_text(json.dumps({
+        'site_to_site': [connection['name'] for connection in ctx['connections'] if connection['ready']],
+        'p2s': bool(vpnrouter.p2s_ready(ctx)),
+        'wireguard': bool(vpnrouter.wg_ready(ctx)),
+        'openvpn': bool(vpnrouter.openvpn_ready(ctx)),
+        'configuration': public(cfg),
+        'network': {
+            'external_interface': ctx['wan_iface'],
+            'internal_interface': ctx['int_iface'],
+            'external_addresses': ctx['local_addr_list'],
+            'internal_address': ctx['int_addr'],
+            'internal_gateway': ctx['int_gateway_ip'],
+            'local_cidrs': ctx['local_cidr_list'],
+            'site_to_site': [{
+                'name': connection['name'],
+                'local_identity': connection['local_id'],
+                'remote_identity': connection['remote_id'],
+                'local_cidrs': connection['local_cidr_list'],
+                'remote_cidrs': connection['remote_cidr_list'],
+            } for connection in ctx['connections']],
+        },
+    }) + '\n', encoding='utf-8')
+
+
+def progress(state, step, error=None):
+    """Record where setup is in SETUP_STATE: running, deferred, done or failed, the step, and the error of a failure."""
+    SETUP_STATE.parent.mkdir(parents=True, exist_ok=True)
+    SETUP_STATE.write_text(json.dumps({
+        'state': state,
+        'step': step,
+        'error': error,
+        'time': time.strftime('%Y-%m-%dT%H:%M:%S%z'),
+    }) + '\n', encoding='utf-8')
+
+
+def start_bootstrap():
+    """Start the bootstrap services while no run has succeeded; a unit that is not installed is skipped."""
+    if not SETUP_DONE.exists():
+        for unit in BOOTSTRAP_UNITS:
+            run(['systemctl', 'start', '--no-block', unit], check=False)
+
+
+def end_bootstrap(ctx):
+    """After the first successful run: write SETUP_DONE, close the bootstrap ports and stop the bootstrap services."""
+    if SETUP_DONE.exists():
+        return
+    SETUP_DONE.parent.mkdir(parents=True, exist_ok=True)
+    SETUP_DONE.touch()
+    if setup_ufw(ctx):
+        run(['ufw', 'reload'], check=False)
+    for unit in BOOTSTRAP_UNITS:
+        run(['systemctl', 'stop', unit], check=False)
+
+
 def main():
+    start_bootstrap()
+    progress('running', 'configuration')
     cfg = vpnrouter.load_config()
     ctx, changed = vpnrouter.configure(cfg)
 
     if ctx is None:
         print('vpn-router: platform is none, configuration deferred; '
               f'set general.platform in {vpnrouter.CONFIG_FILE}')
+        progress('deferred', 'configuration', f'general.platform is none; set it in {vpnrouter.CONFIG_FILE}')
         return 0
 
+    progress('running', 'routes')
     if vpnrouter.routing_ready(ctx):
         apply_routes(ctx)
 
-    if ctx['local_id_mode'] == 'public_ip' and ctx['local_fqdn']:
+    if ctx['local_fqdn'] and any(connection['local_id_mode'] == 'public_ip' for connection in ctx['connections']):
+        progress('running', 'public address')
         address = resolve_public_ip(ctx['local_fqdn'])
-        ctx, more = vpnrouter.configure(cfg, local_id_override=address)
+        ctx, more = vpnrouter.configure(cfg, public_ip=address)
         changed |= more
 
+    progress('running', 'WireGuard keys')
     setup_wireguard(ctx)
 
+    progress('running', 'firewall')
     rules_changed = setup_ufw(ctx)
     if is_configured(ctx):
         status = run(['ufw', 'status'], check=False)
@@ -340,16 +700,66 @@ def main():
     elif rules_changed:
         run(['ufw', 'reload'], check=False)
 
+    progress('running', 'strongSwan')
     manage_strongswan(ctx, changed)
+    progress('running', 'WireGuard')
     manage_wireguard(ctx, changed)
 
+    # The tunnels are up before the directory and the admin, whose problem ends setup only after every other step.
+    failed_step = 'user directory'
+    progress('running', failed_step)
+    problem = manage_directory()
+    if not problem:
+        failed_step = 'Router Manager admin'
+        progress('running', failed_step)
+        problem = manage_admin()
+    if not problem:
+        failed_step = 'metrics service account'
+        progress('running', failed_step)
+        problem = manage_metrics_account(ctx)
+    progress('running', 'OpenVPN')
+    manage_openvpn(ctx, changed)
+    progress('running', 'HTTPS proxy')
+    manage_caddy(ctx)
+
     if 'resolved' in changed:
+        progress('running', 'resolver')
         run(['systemctl', 'restart', 'systemd-resolved'], check=False)
 
+    progress('running', 'platform module')
     run_platform_setup(cfg, ctx)
+    progress('running', 'Docker rules')
     refresh_docker_rules()
+    progress('running', 'web service state')
+    refresh_web(cfg, ctx)
+    progress('running', 'metrics service')
+    manage_metrics(ctx)
+    if problem:
+        progress('failed', failed_step, problem)
+        print(f'vpn-router: error: {problem}', file=sys.stderr)
+        return 1
+    progress('done', 'done')
+    end_bootstrap(ctx)
     return 0
 
 
+def record_failure(error):
+    """Mark the run failed in the step SETUP_STATE holds."""
+    try:
+        step = json.loads(SETUP_STATE.read_text(encoding='utf-8')).get('step')
+    except (OSError, ValueError):
+        step = None
+    progress('failed', step, error)
+
+
 if __name__ == '__main__':
-    sys.exit(main())
+    # A failing step ends with SystemExit carrying its message (vpnrouter.fail).
+    try:
+        code = main()
+    except SystemExit as exc:
+        record_failure(str(exc.code))
+        raise
+    except Exception as exc:
+        record_failure(repr(exc))
+        raise
+    sys.exit(code)

debian-package/src/usr/lib/vpn-router/vpnrouter.py

@@ -4,6 +4,8 @@ Everything derivable from the configuration file alone, with no running
 network required. Used by both 'configure' and 'setup'.
 """
 
+import contextlib
+import fcntl
 import ipaddress
 import json
 import os
@@ -12,6 +14,8 @@ import re
 import shutil
 import subprocess
 import sys
+import tempfile
+import urllib.parse
 
 import jinja2
 
@@ -19,18 +23,24 @@ LIB_DIR = pathlib.Path(__file__).resolve().parent
 if str(LIB_DIR) not in sys.path:
     sys.path.insert(0, str(LIB_DIR))
 
+import vpnrouter_ldap
 import vpnrouter_platforms
+from vpnrouter_net import DEFAULT_LISTEN, LISTEN_CHOICES, LOOPBACK, ConfigError, detect_interfaces, interface_addresses, ip_output
 
 CONFIG_FILE  = pathlib.Path('/etc/vpn-router/vpn-router.json')
+CONFIG_LOCK  = pathlib.Path('/run/vpn-router/config.lock')
 TEMPLATE_DIR = pathlib.Path('/usr/share/vpn-router/templates')
 PKI_DIR      = pathlib.Path('/etc/vpn-router/pki')
 SIMPLE_CA    = LIB_DIR / 'simple-ca'
 
 SWANCTL_CONF_D = pathlib.Path('/etc/swanctl/conf.d')
-REMOTE_SITE    = SWANCTL_CONF_D / 'remote-site.conf'
+SITE_TO_SITE   = SWANCTL_CONF_D / 'site-to-site.conf'
 ROAD_WARRIOR   = SWANCTL_CONF_D / 'road-warrior.conf'
 P2S_FORWARDER  = pathlib.Path('/etc/systemd/resolved.conf.d/p2s-forwarder.conf')
 WG_CONF        = pathlib.Path('/etc/wireguard/wg0.conf')
+OPENVPN_CONF   = pathlib.Path('/etc/openvpn/server/vpn-router.conf')
+OPENVPN_LDAP   = pathlib.Path('/etc/openvpn/server/vpn-router-ldap.conf')
+USER_CERT_DIR  = PKI_DIR / 'users'
 
 CA_CERT_DEST   = pathlib.Path('/etc/swanctl/x509ca/ca.pem')
 SERVER_CERT    = pathlib.Path('/etc/swanctl/x509/server.pem')
@@ -42,16 +52,53 @@ TRUST_ANCHOR   = pathlib.Path('/usr/local/share/ca-certificates/vpn-router-ca.cr
 SCHEMA = {
     'general':    {'platform': 'none', 'mode': 'manual'},
     'interfaces': {'external': '', 'internal': ''},
-    'wan':        {'local_fqdn': '', 'local_id_mode': 'fqdn'},
+    'wan':        {'local_fqdn': ''},
     'local':      {'cidrs': [], 'int_addr': '', 'int_gateway_ip': ''},
-    'remote':     {'addrs': [], 'id': '', 'cidrs': [], 'psk': '', 'psk_file': ''},
+    'site_to_site': {},
     'p2s':        {'enabled': False, 'address_pool': '', 'ca_name': 'VPN Router CA'},
     'wireguard':  {'enabled': False, 'address': '', 'listen_port': 51820},
+    'openvpn':    {'enabled': False, 'address_pool': '', 'port': 1194, 'protocol': 'udp'},
     'docker':     {'mode': 'auto', 'exposed_ports': []},
+    'web':        {'enabled': True, 'listen': DEFAULT_LISTEN, 'port': 7081, 'database': '', 'trusted_proxies': [],
+                   'status_content': 'basic'},
+    'https':      {'enabled': True, 'email': '', 'web_port': 7444, 'manager_port': 7443},
+    'metrics':    {'enabled': True, 'listen': 'internal', 'port': 9464, 'https_port': 9465, 'service_account': 'metrics',
+                   'otlp_endpoint': '', 'otlp_username': '', 'otlp_password': '',
+                   'keep_details': '24 hours', 'keep_hourly': '1 month', 'keep_daily': '3 months',
+                   'keep_monthly': '1 year', 'keep_yearly': '10 years'},
 }
 
+#: site_to_site maps a connection name to its settings: option -> default, the type of the default being the required type.
+CONNECTION = {'addrs': [], 'id': '', 'cidrs': [], 'local_cidrs': [], 'local_id_mode': 'fqdn', 'psk': '', 'psk_file': ''}
+
+#: The name of a site_to_site connection, which also names its swanctl connection, child and secret.
+CONNECTION_NAME = re.compile(r'^[a-z][a-z0-9-]{0,30}$')
+
+#: The site_to_site connection the install answers fill.
+FIRST_CONNECTION = 'remote-site'
+
+#: The retention settings of the metrics, from the details to the yearly summaries.
+METRICS_KEEP = ('keep_details', 'keep_hourly', 'keep_daily', 'keep_monthly', 'keep_yearly')
+
+#: A retention period as PostgreSQL reads an interval: a count and a unit.
+METRICS_INTERVAL = re.compile(r'^[1-9]\d{0,4} (hour|day|month|year)s?$')
+
+#: The port of Let's Encrypt's HTTP challenge, which Caddy answers while HTTPS is on.
+ACME_HTTP_PORT = 80
+
+#: What / and /status of the web service show, web.status_content, from the least to the most.
+STATUS_CONTENTS = ('basic', 'standard', 'full')
+
 DOCKER_MODES = ('auto', 'on', 'off')
 
+OPENVPN_PROTOCOLS = ('udp', 'tcp')
+
+#: The tun device of the OpenVPN server.
+OPENVPN_DEVICE = 'ovpn0'
+
+#: Days a certificate revocation list is valid; the router only reads its own lists.
+CRL_DAYS = 3650
+
 #: A port or a port range as iptables multiport takes it: 443 or 8000:8100.
 DOCKER_PORT = re.compile(r'^\d{1,5}(:\d{1,5})?$')
 
@@ -59,7 +106,7 @@ DOCKER_PORT = re.compile(r'^\d{1,5}(:\d{1,5})?$')
 DOCKER_PORT_LIMIT = 15
 
 TYPE_NAMES = {str: 'a string', list: 'a list of strings', bool: 'a boolean',
-              int: 'an integer'}
+              int: 'an integer', dict: 'an object'}
 
 ID_MODES = ('fqdn', 'public_ip', 'internal_ip')
 
@@ -75,42 +122,85 @@ MODES = ('manual', 'interfaces', 'auto')
 PLATFORM_NONE = 'none'
 
 
-class ConfigError(Exception):
-    """A setting is present but malformed."""
-
-
 def warn(message):
     print(f'vpn-router: warning: {message}', file=sys.stderr)
 
 
 def fail(message):
-    print(f'vpn-router: error: {message}', file=sys.stderr)
+    """End with status 1; SystemExit carries the message, which Python prints to standard error."""
-    sys.exit(1)
+    sys.exit(f'vpn-router: error: {message}')
+
+
+def read_settings(path=None):
+    """The settings in the configuration file, without the defaults; {} when the file does not exist.
+
+    Raises ConfigError when the file is not JSON or not an object of sections.
+    """
+    path = pathlib.Path(path) if path else CONFIG_FILE
+    try:
+        data = json.loads(path.read_text(encoding='utf-8'))
+    except FileNotFoundError:
+        return {}
+    except ValueError as exc:
+        raise ConfigError(f'cannot parse {path}: {exc}')
+    if not isinstance(data, dict):
+        raise ConfigError(f'cannot parse {path}: the top level is not an object')
+    for section, options in data.items():
+        if not isinstance(options, dict):
+            raise ConfigError(f'cannot parse {path}: {section} is not an object')
+    return data
+
+
+def with_defaults(data):
+    """The configuration of the settings of read_settings, with the defaults for anything absent.
+
+    The result maps section -> option -> value, and site_to_site maps each connection name to
+    its settings filled from CONNECTION. Sections, options and connections that are not
+    objects are kept so that validate() can report them.
+    """
+    cfg = {section: dict(options) for section, options in SCHEMA.items()}
+    for section, options in data.items():
+        cfg.setdefault(section, {}).update(options)
+    cfg['site_to_site'] = {name: {**CONNECTION, **options} if isinstance(options, dict) else options
+                           for name, options in cfg['site_to_site'].items()}
+    return cfg
 
 
 def load_config(path=None):
-    """Read the configuration, filling in defaults for anything absent.
+    """Read the configuration, filling in defaults for anything absent (see with_defaults).
-
-    The result maps section -> option -> value. Sections and options not in
-    SCHEMA are kept so that validate() can report them.
 
     The path is resolved when called, not when this function is defined, so
     CONFIG_FILE stays overridable.
     """
-    path = pathlib.Path(path) if path else CONFIG_FILE
+    try:
-    cfg = {section: dict(options) for section, options in SCHEMA.items()}
+        return with_defaults(read_settings(path))
-    if path.exists():
+    except ConfigError as exc:
-        try:
+        fail(str(exc))
-            data = json.loads(path.read_text(encoding='utf-8'))
+
-        except ValueError as exc:
+
-            fail(f'cannot parse {path}: {exc}')
+@contextlib.contextmanager
-        if not isinstance(data, dict):
+def config_lock():
-            fail(f'cannot parse {path}: the top level is not an object')
+    """Hold the lock of the configuration file from reading it to writing it with save_config."""
-        for section, options in data.items():
+    CONFIG_LOCK.parent.mkdir(parents=True, exist_ok=True)
-            if not isinstance(options, dict):
+    with open(CONFIG_LOCK, 'a', encoding='utf-8') as handle:
-                fail(f'cannot parse {path}: {section} is not an object')
+        fcntl.flock(handle, fcntl.LOCK_EX)
-            cfg.setdefault(section, {}).update(options)
+        yield
-    return cfg
+
+
+def save_config(data):
+    """Write the settings to the configuration file, indented, with mode 0600, replacing it in one step.
+
+    The caller holds config_lock.
+    """
+    descriptor, name = tempfile.mkstemp(dir=CONFIG_FILE.parent, prefix='.vpn-router.json.')
+    try:
+        with os.fdopen(descriptor, 'w', encoding='utf-8') as handle:
+            handle.write(json.dumps(data, indent=2) + '\n')
+        os.chmod(name, 0o600)
+        os.replace(name, CONFIG_FILE)
+    except OSError:
+        os.unlink(name)
+        raise
 
 
 def _split_list(value):
@@ -122,23 +212,37 @@ def _get(cfg, section, option):
     return value.strip() if isinstance(value, str) else value
 
 
+def _clean(values):
+    return [item.strip() for item in values if item.strip()]
+
+
 def _get_list(cfg, section, option):
-    return [item.strip() for item in cfg[section][option] if item.strip()]
+    return _clean(cfg[section][option])
+
+
+def _check_options(options, defaults, prefix):
+    for option, value in options.items():
+        if option not in defaults:
+            raise ConfigError(f'{prefix}.{option}: unknown setting')
+        expected = type(defaults[option])
+        if type(value) is not expected or (
+                expected is list and
+                not all(isinstance(item, str) for item in value)):
+            raise ConfigError(
+                f'{prefix}.{option} must be {TYPE_NAMES[expected]}')
 
 
 def _check_types(cfg):
     for section, options in cfg.items():
         if section not in SCHEMA:
             raise ConfigError(f'{section}: unknown section')
-        for option, value in options.items():
+        if section != 'site_to_site':
-            if option not in SCHEMA[section]:
+            _check_options(options, SCHEMA[section], section)
-                raise ConfigError(f'{section}.{option}: unknown setting')
+            continue
-            expected = type(SCHEMA[section][option])
+        for name, connection in options.items():
-            if type(value) is not expected or (
+            if not isinstance(connection, dict):
-                    expected is list and
+                raise ConfigError(f'site_to_site.{name} must be {TYPE_NAMES[dict]}')
-                    not all(isinstance(item, str) for item in value)):
+            _check_options(connection, CONNECTION, f'site_to_site.{name}')
-                raise ConfigError(
-                    f'{section}.{option} must be {TYPE_NAMES[expected]}')
 
 
 def _check_networks(values, name):
@@ -190,21 +294,37 @@ def validate(cfg):
                 raise ConfigError(
                     f'{section}.{option} is required in mode {mode or "manual"}')
 
-    for section, option in (('wan', 'local_fqdn'), ('remote', 'id')):
+    identities = [('wan.local_fqdn', _get(cfg, 'wan', 'local_fqdn'))]
-        if str(_get(cfg, section, option)).startswith('@'):
+    for name, connection in cfg['site_to_site'].items():
+        prefix = f'site_to_site.{name}'
+        if not CONNECTION_NAME.match(name):
             raise ConfigError(
-                f'{section}.{option}: remove the leading @; strongSwan reads an '
+                f'{prefix}: a connection name starts with a lowercase letter and has at most 31 '
+                'lowercase letters, digits and hyphens')
+        identities.append((f'{prefix}.id', connection['id'].strip()))
+        id_mode = connection['local_id_mode'].strip()
+        if id_mode and id_mode not in ID_MODES:
+            raise ConfigError(
+                f'{prefix}.local_id_mode: {id_mode!r} is not one of {", ".join(ID_MODES)}')
+        _check_networks(_clean(connection['cidrs']), f'{prefix}.cidrs')
+        _check_networks(_clean(connection['local_cidrs']), f'{prefix}.local_cidrs')
+
+    for option, value in identities:
+        if value.startswith('@'):
+            raise ConfigError(
+                f'{option}: remove the leading @; strongSwan reads an '
                 'address as an IP identity and a name as an FQDN')
 
-    id_mode = _get(cfg, 'wan', 'local_id_mode')
-    if id_mode and id_mode not in ID_MODES:
-        raise ConfigError(
-            f'wan.local_id_mode: {id_mode!r} is not one of {", ".join(ID_MODES)}')
-
     _check_networks(_get_list(cfg, 'local', 'cidrs'), 'local.cidrs')
-    _check_networks(_get_list(cfg, 'remote', 'cidrs'), 'remote.cidrs')
     pool = _get(cfg, 'p2s', 'address_pool')
     _check_networks([pool] if pool else [], 'p2s.address_pool')
+    openvpn_pool = _get(cfg, 'openvpn', 'address_pool')
+    if openvpn_pool:
+        try:
+            if ipaddress.ip_network(openvpn_pool).version != 4:
+                raise ValueError
+        except ValueError:
+            raise ConfigError(f'openvpn.address_pool: {openvpn_pool!r} is not an IPv4 network')
     _check_address(_get(cfg, 'local', 'int_addr'), 'local.int_addr')
     _check_address(_get(cfg, 'local', 'int_gateway_ip'), 'local.int_gateway_ip')
 
@@ -228,6 +348,70 @@ def validate(cfg):
     if not 1 <= port <= 65535:
         raise ConfigError(f'wireguard.listen_port: {port} is not a valid port')
 
+    openvpn_port = _get(cfg, 'openvpn', 'port')
+    if not 1 <= openvpn_port <= 65535:
+        raise ConfigError(f'openvpn.port: {openvpn_port} is not a valid port')
+
+    protocol = _get(cfg, 'openvpn', 'protocol')
+    if protocol not in OPENVPN_PROTOCOLS:
+        raise ConfigError(
+            f'openvpn.protocol: {protocol!r} is not one of {", ".join(OPENVPN_PROTOCOLS)}')
+
+    web_port = _get(cfg, 'web', 'port')
+    if isinstance(web_port, bool) or not isinstance(web_port, int) or not 1 <= web_port <= 65535:
+        raise ConfigError(f'web.port: {web_port!r} is not a valid port')
+
+    web_listen = _get(cfg, 'web', 'listen')
+    if web_listen not in LISTEN_CHOICES:
+        raise ConfigError(
+            f'web.listen: {web_listen!r} is not one of {", ".join(LISTEN_CHOICES)}')
+
+    status_content = _get(cfg, 'web', 'status_content')
+    if status_content not in STATUS_CONTENTS:
+        raise ConfigError(
+            f'web.status_content: {status_content!r} is not one of {", ".join(STATUS_CONTENTS)}')
+
+    https_ports = []
+    for option in ('web_port', 'manager_port'):
+        port = _get(cfg, 'https', option)
+        if isinstance(port, bool) or not isinstance(port, int) or not 1 <= port <= 65535:
+            raise ConfigError(f'https.{option}: {port!r} is not a valid port')
+        if port == ACME_HTTP_PORT:
+            raise ConfigError(f'https.{option}: {port} is the port of the HTTP challenge')
+        https_ports.append(port)
+    if https_ports[0] == https_ports[1]:
+        raise ConfigError('https.web_port and https.manager_port must differ')
+
+    for item in _get_list(cfg, 'web', 'trusted_proxies'):
+        _check_address(item, 'web.trusted_proxies')
+
+    taken = {web_port: 'web.port', ACME_HTTP_PORT: 'the HTTP challenge',
+             https_ports[0]: 'https.web_port', https_ports[1]: 'https.manager_port'}
+    for option in ('port', 'https_port'):
+        port = _get(cfg, 'metrics', option)
+        if isinstance(port, bool) or not isinstance(port, int) or not 1 <= port <= 65535:
+            raise ConfigError(f'metrics.{option}: {port!r} is not a valid port')
+        if port in taken:
+            raise ConfigError(f'metrics.{option}: {port} is the port of {taken[port]}')
+        taken[port] = f'metrics.{option}'
+
+    metrics_listen = _get(cfg, 'metrics', 'listen')
+    if metrics_listen not in LISTEN_CHOICES:
+        raise ConfigError(
+            f'metrics.listen: {metrics_listen!r} is not one of {", ".join(LISTEN_CHOICES)}')
+
+    endpoint = _get(cfg, 'metrics', 'otlp_endpoint')
+    if endpoint and urllib.parse.urlsplit(endpoint).scheme not in ('http', 'https'):
+        raise ConfigError(f'metrics.otlp_endpoint: {endpoint!r} is not an http or https URL')
+    if bool(_get(cfg, 'metrics', 'otlp_username')) != bool(_get(cfg, 'metrics', 'otlp_password')):
+        raise ConfigError('metrics.otlp_username and metrics.otlp_password are given together')
+
+    for option in METRICS_KEEP:
+        keep = _get(cfg, 'metrics', option)
+        if not METRICS_INTERVAL.match(keep):
+            raise ConfigError(
+                f'metrics.{option}: {keep!r} is not a count and a unit, such as "24 hours" or "3 months"')
+
     docker_mode = _get(cfg, 'docker', 'mode')
     if docker_mode not in DOCKER_MODES:
         raise ConfigError(
@@ -241,21 +425,28 @@ def validate(cfg):
             raise ConfigError(
                 f'docker.exposed_ports: {item!r} is not a port or a port range')
         weight += 2 if ':' in item else 1
+        first, _, last = item.partition(':')
+        if _get(cfg, 'https', 'enabled'):
+            for port in (ACME_HTTP_PORT, *https_ports):
+                if int(first) <= port <= int(last or first):
+                    raise ConfigError(
+                        f'docker.exposed_ports: {item!r} takes port {port}, which Caddy serves while '
+                        'https.enabled is true')
     if weight > DOCKER_PORT_LIMIT:
         raise ConfigError(
             f'docker.exposed_ports: at most {DOCKER_PORT_LIMIT} ports are allowed, '
             'a range counting as two')
 
 
-def resolve_psk(cfg):
+def resolve_psk(connection, name):
-    """Return the pre-shared key, or '' when none is configured."""
+    """Return the pre-shared key of the site_to_site connection: psk_file wins, else psk; '' when none is configured."""
-    psk_file = _get(cfg, 'remote', 'psk_file')
+    psk_file = connection['psk_file'].strip()
     if psk_file:
         path = pathlib.Path(psk_file)
         if not path.is_file():
-            raise ConfigError(f'remote.psk_file: {psk_file} does not exist')
+            raise ConfigError(f'site_to_site.{name}.psk_file: {psk_file} does not exist')
         return path.read_text(encoding='utf-8').strip()
-    return cfg['remote']['psk']
+    return connection['psk']
 
 
 def platform_module(cfg):
@@ -286,55 +477,6 @@ def deferred(cfg):
     return (_get(cfg, 'general', 'platform') or PLATFORM_NONE) == PLATFORM_NONE
 
 
-def _ip_route(*args):
-    result = subprocess.run(['ip', *args], capture_output=True, text=True)
-    return result.stdout if result.returncode == 0 else ''
-
-
-def detect_interfaces():
-    """Work out the external and internal interface names.
-
-    Only reached in auto mode, where the operator has accepted that a wrong
-    guess is possible. The external interface is whichever one carries the
-    default route; the internal one is the only other addressed interface.
-    """
-    tokens = _ip_route('route', 'get', '1.1.1.1').split()
-    external = ''
-    for index, token in enumerate(tokens):
-        if token == 'dev' and index + 1 < len(tokens):
-            external = tokens[index + 1]
-            break
-    if not external:
-        raise ConfigError('auto mode: cannot determine the external interface')
-
-    candidates = [name for name in _addressed_interfaces() if name != external]
-    if len(candidates) != 1:
-        raise ConfigError(
-            f'auto mode: expected one internal interface, found {len(candidates)}; '
-            'use mode = interfaces and name them')
-    return external, candidates[0]
-
-
-def _is_virtual(name):
-    """True for an interface the kernel creates without a device behind it.
-
-    Loopback, bridges, veth pairs and tunnels live under /sys/devices/virtual,
-    which is where the Docker bridge is; a NIC of the virtual machine does not.
-    """
-    link = pathlib.Path('/sys/class/net') / name
-    return '/devices/virtual/' in str(link.resolve())
-
-
-def _addressed_interfaces():
-    """Names of the NICs holding at least one IPv4 address; virtual interfaces are left out."""
-    names = []
-    for line in _ip_route('-o', '-4', 'addr', 'show').splitlines():
-        fields = line.split()
-        if len(fields) >= 4 and fields[1] not in names and not _is_virtual(fields[1]):
-            names.append(fields[1])
-    return names
-
-
 def detect_gateway(iface, address):
     """Derive the next hop on the internal side.
 
@@ -342,12 +484,12 @@ def detect_gateway(iface, address):
     back to the first host address of the connected subnet, which is a
     convention rather than a fact - so say so.
     """
-    for line in _ip_route('-o', 'route', 'show', 'dev', iface).splitlines():
+    for line in ip_output('-o', 'route', 'show', 'dev', iface).splitlines():
         fields = line.split()
         if 'via' in fields:
             return fields[fields.index('via') + 1]
 
-    for line in _ip_route('-o', '-4', 'addr', 'show', 'dev', iface).splitlines():
+    for line in ip_output('-o', '-4', 'addr', 'show', 'dev', iface).splitlines():
         fields = line.split()
         if len(fields) >= 4 and fields[3].split('/')[0] == address:
             network = ipaddress.ip_interface(fields[3]).network
@@ -358,34 +500,6 @@ def detect_gateway(iface, address):
     return ''
 
 
-def interface_addresses(name, setting):
-    """IPv4 addresses configured on an interface.
-
-    The interface name is configuration; its addresses are read from the
-    system. Looking up the addresses of a named interface is deterministic -
-    unlike working out which interface to use, which is a guess the package
-    deliberately does not make.
-    """
-    if not name:
-        return []
-    if not (pathlib.Path('/sys/class/net') / name).exists():
-        raise ConfigError(f'{setting}: interface {name!r} does not exist')
-
-    result = subprocess.run(
-        ['ip', '-o', '-4', 'addr', 'show', 'dev', name],
-        capture_output=True, text=True,
-    )
-    if result.returncode != 0:
-        raise ConfigError(f'{setting}: cannot read addresses of {name!r}')
-
-    addresses = []
-    for line in result.stdout.splitlines():
-        fields = line.split()
-        if len(fields) >= 4:
-            addresses.append(fields[3].split('/')[0])
-    return addresses
-
-
 def resolve_interfaces(cfg):
     """Return (ext_iface, int_iface, local_addrs, int_addr, int_gateway_ip).
 
@@ -416,27 +530,43 @@ def resolve_interfaces(cfg):
     return ext_iface, int_iface, local_addrs, int_addr, gateway
 
 
-def build_context(cfg, local_id_override=None):
+def build_context(cfg, public_ip=None):
-    """Assemble the template context from the configuration."""
+    """Assemble the template context from the configuration.
+
+    public_ip is the address wan.local_fqdn resolves to, the local identity of the
+    site_to_site connections in mode public_ip; setup resolves it once the network is up and
+    renders again, and without it such a connection is not ready.
+    """
     extras = vpnrouter_platforms.context(platform_module(cfg))
 
     ext_iface, int_iface, local_addrs, int_addr, gateway = resolve_interfaces(cfg)
 
     local_fqdn   = _get(cfg, 'wan', 'local_fqdn')
-    id_mode      = _get(cfg, 'wan', 'local_id_mode') or 'fqdn'
     local_cidrs  = _get_list(cfg, 'local', 'cidrs')
-    remote_cidrs = _get_list(cfg, 'remote', 'cidrs')
+    openvpn_pool = _get(cfg, 'openvpn', 'address_pool')
+    openvpn_network = ipaddress.ip_network(openvpn_pool) if openvpn_pool else None
+    identities = {'fqdn': local_fqdn, 'public_ip': public_ip or '', 'internal_ip': int_addr}
 
-    if local_id_override:
+    connections = []
-        local_id = local_id_override
+    for name, connection in cfg['site_to_site'].items():
-    elif id_mode == 'fqdn':
+        id_mode = connection['local_id_mode'].strip() or 'fqdn'
-        local_id = local_fqdn
+        remote_cidrs = _clean(connection['cidrs'])
-    elif id_mode == 'internal_ip':
+        connection_cidrs = _clean(connection['local_cidrs'])
-        local_id = int_addr
+        entry = {
-    else:
+            'name':             name,
-        # public_ip is resolved once the network is up, and the caller then
+            'remote_addrs':     ', '.join(_clean(connection['addrs'])),
-        # re-renders with local_id_override set.
+            'remote_id':        connection['id'].strip(),
-        local_id = ''
+            'remote_cidrs':     ', '.join(remote_cidrs),
+            'remote_cidr_list': remote_cidrs,
+            'local_ts':         _join(connection_cidrs, _split_list(extras['EXTRA_LOCAL_TS'])),
+            'local_cidr_list':  connection_cidrs,
+            'local_id_mode':    id_mode,
+            'local_id':         identities.get(id_mode, ''),
+            'psk':              resolve_psk(connection, name),
+        }
+        entry['ready'] = bool(local_addrs and connection_cidrs and all(
+            entry[key] for key in ('remote_addrs', 'remote_id', 'remote_cidrs', 'local_id', 'psk')))
+        connections.append(entry)
 
     return {
         'platform':        _get(cfg, 'general', 'platform') or PLATFORM_NONE,
@@ -446,18 +576,12 @@ def build_context(cfg, local_id_override=None):
         'local_addrs':     ', '.join(local_addrs),
         'local_addr_list': local_addrs,
         'local_fqdn':      local_fqdn,
-        'local_id_mode':   id_mode,
-        'local_id':        local_id,
         'local_cidrs':     ', '.join(local_cidrs),
         'local_cidr_list': local_cidrs,
         'local_ts':        _join(local_cidrs, _split_list(extras['EXTRA_LOCAL_TS'])),
         'int_addr':        int_addr,
         'int_gateway_ip':  gateway,
-        'remote_addrs':    ', '.join(_get_list(cfg, 'remote', 'addrs')),
+        'connections':     connections,
-        'remote_id':       _get(cfg, 'remote', 'id'),
-        'remote_cidrs':    ', '.join(remote_cidrs),
-        'remote_cidr_list': remote_cidrs,
-        'psk':             resolve_psk(cfg),
         'p2s_enabled':     _get(cfg, 'p2s', 'enabled'),
         'p2s_address_pool': _get(cfg, 'p2s', 'address_pool'),
         'p2s_ca_name':     _get(cfg, 'p2s', 'ca_name'),
@@ -465,15 +589,40 @@ def build_context(cfg, local_id_override=None):
         'wg_enabled':      _get(cfg, 'wireguard', 'enabled'),
         'wg_address':      _get(cfg, 'wireguard', 'address'),
         'wg_listen_port':  _get(cfg, 'wireguard', 'listen_port'),
+        'openvpn_enabled': _get(cfg, 'openvpn', 'enabled'),
+        'openvpn_pool':    openvpn_pool,
+        'openvpn_network': str(openvpn_network.network_address) if openvpn_network else '',
+        'openvpn_netmask': str(openvpn_network.netmask) if openvpn_network else '',
+        'openvpn_routes':  [(str(network.network_address), str(network.netmask))
+                            for network in (ipaddress.ip_network(cidr, strict=False) for cidr in local_cidrs)],
+        'openvpn_port':    _get(cfg, 'openvpn', 'port'),
+        'openvpn_protocol': _get(cfg, 'openvpn', 'protocol'),
+        'openvpn_device':  OPENVPN_DEVICE,
+        'openvpn_ldap_conf': str(OPENVPN_LDAP),
+        'p2s_dns_list':    _split_list(_join(local_addrs, _split_list(extras['EXTRA_P2S_DNS']))),
+        'ca_cert':         str(CA_CERT_DEST),
+        'server_cert':     str(SERVER_CERT),
+        'server_key':      str(SERVER_KEY),
+        'crl':             str(CRL_DEST),
+        'ldap_uri':        vpnrouter_ldap.URI,
+        'ldap_users':      vpnrouter_ldap.USERS,
+        'web_enabled':     _get(cfg, 'web', 'enabled'),
+        'web_port':        _get(cfg, 'web', 'port'),
+        'https_enabled':   _get(cfg, 'https', 'enabled'),
+        'https_email':     _get(cfg, 'https', 'email'),
+        'https_web_port':  _get(cfg, 'https', 'web_port'),
+        'https_manager_port': _get(cfg, 'https', 'manager_port'),
+        'metrics_enabled': _get(cfg, 'metrics', 'enabled'),
+        'metrics_port':    _get(cfg, 'metrics', 'port'),
+        'metrics_https_port': _get(cfg, 'metrics', 'https_port'),
+        'metrics_service_account': _get(cfg, 'metrics', 'service_account'),
+        'loopback':        LOOPBACK,
     }
 
 
 def s2s_ready(ctx):
-    return all([
+    """True when any site_to_site connection is ready."""
-        ctx['local_addrs'], ctx['local_fqdn'], ctx['local_cidrs'],
+    return any(connection['ready'] for connection in ctx['connections'])
-        ctx['remote_addrs'], ctx['remote_id'], ctx['remote_cidrs'],
-        ctx['psk'], ctx['local_id'],
-    ])
 
 
 def routing_ready(ctx):
@@ -490,6 +639,20 @@ def wg_ready(ctx):
     return bool(ctx['wg_enabled'] and ctx['wg_address'])
 
 
+def openvpn_ready(ctx):
+    return bool(ctx['openvpn_enabled'] and ctx['openvpn_pool'] and ctx['local_fqdn'])
+
+
+def https_ready(ctx):
+    """Caddy serves HTTPS once it is enabled and wan.local_fqdn names the certificate."""
+    return bool(ctx['https_enabled'] and ctx['local_fqdn'])
+
+
+def metrics_ready(ctx):
+    """The metrics service collects, keeps and serves the metrics while metrics.enabled is true."""
+    return bool(ctx['metrics_enabled'])
+
+
 def _jinja_env():
     return jinja2.Environment(
         loader=jinja2.FileSystemLoader(str(TEMPLATE_DIR)),
@@ -579,21 +742,28 @@ def _simple_ca(*args):
 
 
 def provision_pki(ctx):
-    """Create the CA and server certificate when the PKI directory is empty."""
+    """Create the CA and CRL when the PKI directory is empty, and the server certificate once wan.local_fqdn is set.
+
+    A CA of the router's own (ca_key.pem present) without a CRL gets one, and without a
+    certificate for wan.local_fqdn gets one too; nothing present is overwritten.
+    """
     PKI_DIR.mkdir(parents=True, exist_ok=True)
     os.chmod(PKI_DIR, 0o700)
-
-    if any(PKI_DIR.iterdir()):
-        return
-    if not ctx['p2s_ca_name']:
-        warn('p2s.ca_name is empty, not creating a local CA')
-        return
     if not SIMPLE_CA.is_file():
         fail(f'{SIMPLE_CA} is missing')
 
-    _simple_ca('make-ca', '--ca-dir', str(PKI_DIR), ctx['p2s_ca_name'])
+    if not any(PKI_DIR.iterdir()):
-    _simple_ca('make-cert', '--ca-dir', str(PKI_DIR),
+        if not ctx['p2s_ca_name']:
-               ctx['local_fqdn'], ctx['local_fqdn'])
+            warn('p2s.ca_name is empty, not creating a local CA')
+            return
+        _simple_ca('make-ca', '--ca-dir', str(PKI_DIR), ctx['p2s_ca_name'])
+    if not (PKI_DIR / 'ca_key.pem').is_file():
+        return
+    if not (PKI_DIR / 'crl.pem').is_file():
+        _simple_ca('make-crl', '--ca-dir', str(PKI_DIR), '--days', str(CRL_DAYS))
+    label = ctx['local_fqdn'].split('.', 1)[0] if ctx['local_fqdn'] else ''
+    if label and not (PKI_DIR / f'{label}_cert.pem').is_file():
+        _simple_ca('make-cert', '--ca-dir', str(PKI_DIR), ctx['local_fqdn'], ctx['local_fqdn'])
 
 
 def distribute_pki(ctx):
@@ -617,11 +787,11 @@ def distribute_pki(ctx):
     return changed
 
 
-def configure(cfg=None, local_id_override=None):
+def configure(cfg=None, public_ip=None):
     """Render configuration files and provision PKI.
 
     Returns (ctx, changed) where changed is a set naming what needs reloading:
-    'swanctl', 'resolved', 'wireguard'.
+    'swanctl', 'resolved', 'wireguard', 'openvpn'.
     """
     if cfg is None:
         cfg = load_config()
@@ -633,35 +803,35 @@ def configure(cfg=None, local_id_override=None):
 
     try:
         validate(cfg)
-        ctx = build_context(cfg, local_id_override=local_id_override)
+        ctx = build_context(cfg, public_ip=public_ip)
     except ConfigError as exc:
         fail(str(exc))
 
     changed = set()
 
-    if p2s_ready(ctx):
+    provision_pki(ctx)
-        provision_pki(ctx)
     if distribute_pki(ctx):
         changed.add('swanctl')
 
     env = _jinja_env()
 
     if s2s_ready(ctx):
-        if render(env, ctx, 'remote-site.conf.j2', REMOTE_SITE, 0o600):
+        if render(env, ctx, 'site-to-site.conf.j2', SITE_TO_SITE, 0o600):
             changed.add('swanctl')
-    elif remove(REMOTE_SITE):
+    elif remove(SITE_TO_SITE):
         changed.add('swanctl')
 
     if p2s_ready(ctx):
         if render(env, ctx, 'road-warrior.conf.j2', ROAD_WARRIOR, 0o600):
             changed.add('swanctl')
+    elif remove(ROAD_WARRIOR):
+        changed.add('swanctl')
+
+    if p2s_ready(ctx) or openvpn_ready(ctx):
         if render(env, ctx, 'p2s-forwarder.conf.j2', P2S_FORWARDER, 0o644):
             changed.add('resolved')
-    else:
+    elif remove(P2S_FORWARDER):
-        if remove(ROAD_WARRIOR):
+        changed.add('resolved')
-            changed.add('swanctl')
-        if remove(P2S_FORWARDER):
-            changed.add('resolved')
 
     if wg_ready(ctx):
         if render_wg_conf(env, ctx):
@@ -669,4 +839,14 @@ def configure(cfg=None, local_id_override=None):
     elif remove(WG_CONF):
         changed.add('wireguard')
 
+    if openvpn_ready(ctx):
+        if render(env, ctx, 'openvpn-server.conf.j2', OPENVPN_CONF, 0o600):
+            changed.add('openvpn')
+        if render(env, ctx, 'openvpn-ldap.conf.j2', OPENVPN_LDAP, 0o600):
+            changed.add('openvpn')
+    else:
+        for path in (OPENVPN_CONF, OPENVPN_LDAP):
+            if remove(path):
+                changed.add('openvpn')
+
     return ctx, changed

debian-package/src/usr/lib/vpn-router/vpnrouter_auth.py

@@ -0,0 +1,207 @@
+"""The tokens of the router's web service, for the web service and vpn-router-token.
+
+Tokens are HS256 JWTs signed with WEB_KEY, a key the web service makes on its first run on the
+router and that never leaves it. /auth issues an access token naming the user in sub, valid ACCESS_SECONDS, and
+a refresh token valid REFRESH_SECONDS; vpn-router-token issues tokens with the admin claim and
+any lifetime. A token without the admin claim must expire within MAX_LIFETIME seconds.
+
+Every failure of the database raises DatabaseError, the only database exception callers see.
+
+Refresh tokens are random strings kept in PostgreSQL as their SHA-256, in the database of
+web.database, by default the vpn-router database on this host that provision() creates on the
+web service's first run, with the password in DATABASE_SECRET. A refresh token is used once: refreshing replaces it.
+"""
+
+import base64
+import contextlib
+import hashlib
+import hmac
+import json
+import pathlib
+import secrets
+import subprocess
+import time
+
+import psycopg
+
+WEB_KEY = pathlib.Path('/etc/vpn-router/web.key')
+DATABASE_SECRET = pathlib.Path('/etc/vpn-router/database.secret')
+
+#: The role and the database provision() creates when web.database is not set.
+DATABASE = 'vpn-router'
+
+#: The user operations of vpnrouter_users after which the user's refresh tokens are revoked.
+REVOKING = ('passwd', 'disable', 'unlock', 'remove')
+
+#: Seconds an access token of /auth and a refresh token are valid.
+ACCESS_SECONDS = 300
+REFRESH_SECONDS = 8 * 3600
+
+#: The longest a token without the admin claim may be valid, from now, in seconds.
+MAX_LIFETIME = 600
+
+#: Seconds of clock difference accepted between a token and this router.
+LEEWAY = 60
+
+SCHEMA = 'CREATE TABLE IF NOT EXISTS refresh_tokens (hash text PRIMARY KEY, name text NOT NULL, expires timestamptz NOT NULL)'
+
+
+class DatabaseError(Exception):
+    """The database of the refresh tokens failed."""
+
+
+def key():
+    """The signing key, or '' while the web service has not made it."""
+    try:
+        return WEB_KEY.read_text(encoding='utf-8').strip()
+    except OSError:
+        return ''
+
+
+def ensure_key():
+    """Make the signing key when it is missing."""
+    if not key():
+        WEB_KEY.parent.mkdir(parents=True, exist_ok=True)
+        WEB_KEY.touch(mode=0o600)
+        WEB_KEY.chmod(0o600)
+        WEB_KEY.write_text(secrets.token_urlsafe(36) + '\n', encoding='utf-8')
+
+
+def b64encode(data):
+    return base64.urlsafe_b64encode(data).rstrip(b'=').decode()
+
+
+def b64decode(part):
+    return base64.urlsafe_b64decode(part + '=' * (-len(part) % 4))
+
+
+def sign(claims, secret):
+    head = b64encode(json.dumps({'alg': 'HS256', 'typ': 'JWT'}).encode())
+    body = b64encode(json.dumps(claims).encode())
+    signature = hmac.new(secret.encode(), f'{head}.{body}'.encode(), hashlib.sha256).digest()
+    return f'{head}.{body}.{b64encode(signature)}'
+
+
+def verify(token, secret):
+    """The claims of a token signed with HS256 by the secret, or None when it is not acceptable.
+
+    Only HS256 is accepted, so a token cannot choose a weaker algorithm. The token must expire
+    (exp); without the admin claim not later than MAX_LIFETIME seconds from now, so that a stolen
+    token is of little use. LEEWAY seconds of clock difference are allowed.
+    """
+    try:
+        header, payload, signature = token.split('.')
+        if json.loads(b64decode(header)).get('alg') != 'HS256':
+            return None
+        expected = hmac.new(secret.encode(), f'{header}.{payload}'.encode(), hashlib.sha256).digest()
+        if not hmac.compare_digest(expected, b64decode(signature)):
+            return None
+        claims = json.loads(b64decode(payload))
+        now = time.time()
+        expires = claims['exp']
+        if isinstance(expires, bool) or not isinstance(expires, (int, float)) or expires < now - LEEWAY:
+            return None
+        if claims.get('admin') is not True and expires > now + MAX_LIFETIME + LEEWAY:
+            return None
+        not_before = claims.get('nbf')
+        if not_before is not None and (isinstance(not_before, bool) or not isinstance(not_before, (int, float)) or not_before > now + LEEWAY):
+            return None
+    except (ValueError, KeyError, TypeError, AttributeError):
+        return None
+    return claims
+
+
+def admin_token(seconds):
+    """A token with the admin claim, valid for the seconds given."""
+    secret = key()
+    if not secret:
+        raise OSError(f'{WEB_KEY} does not exist; vpn-router-web makes it when it starts')
+    now = int(time.time())
+    return {'token': sign({'admin': True, 'iat': now, 'exp': now + seconds}, secret), 'expires': now + seconds}
+
+
+def connection_string(cfg):
+    """web.database, or the database provision() creates; its password, of URL-safe characters, is written unquoted."""
+    configured = cfg['web']['database'].strip()
+    if configured:
+        return configured
+    password = DATABASE_SECRET.read_text(encoding='utf-8').strip()
+    return f'host=localhost dbname={DATABASE} user={DATABASE} password={password}'
+
+
+@contextlib.contextmanager
+def database(cfg):
+    """A connection to the database of the refresh tokens, with their table, closed at the end; DatabaseError on any failure."""
+    try:
+        with psycopg.connect(connection_string(cfg), autocommit=True) as connection:
+            connection.execute(SCHEMA)
+            yield connection
+    except psycopg.Error as exc:
+        raise DatabaseError(f'the token database failed: {exc}') from exc
+
+
+def digest(token):
+    return hashlib.sha256(token.encode()).hexdigest()
+
+
+def issue(cfg, name):
+    """An access token and a refresh token for the user, as /auth answers them."""
+    secret = key()
+    now = int(time.time())
+    refresh = secrets.token_urlsafe(32)
+    with database(cfg) as connection:
+        connection.execute('DELETE FROM refresh_tokens WHERE expires < now()')
+        connection.execute("INSERT INTO refresh_tokens VALUES (%s, %s, now() + make_interval(secs => %s))",
+                           (digest(refresh), name, REFRESH_SECONDS))
+    return {
+        'access_token': sign({'sub': name, 'iat': now, 'exp': now + ACCESS_SECONDS}, secret),
+        'token_type': 'Bearer',
+        'expires_in': ACCESS_SECONDS,
+        'refresh_token': refresh,
+        'refresh_expires_in': REFRESH_SECONDS,
+    }
+
+
+def take(cfg, token):
+    """Remove an unexpired refresh token and return the user it was issued to, or None."""
+    if not isinstance(token, str) or not token:
+        return None
+    with database(cfg) as connection:
+        row = connection.execute('DELETE FROM refresh_tokens WHERE hash = %s RETURNING name, expires > now()',
+                                 (digest(token),)).fetchone()
+    return row[0] if row and row[1] else None
+
+
+def revoke(cfg, name):
+    """Remove all refresh tokens of the user."""
+    with database(cfg) as connection:
+        connection.execute('DELETE FROM refresh_tokens WHERE name = %s', (name,))
+
+
+def psql(sql, name):
+    """The output of an SQL statement, read from standard input, run by psql as postgres with name as the variable v1."""
+    args = ['runuser', '-u', 'postgres', '--', 'psql', '-X', '-q', '-t', '-A', '-v', 'ON_ERROR_STOP=1', '-v', f'v1={name}']
+    result = subprocess.run(args, input=sql, capture_output=True, text=True)
+    if result.returncode != 0:
+        raise OSError(f'psql failed: {result.stderr.strip()}')
+    return result.stdout.strip()
+
+
+def provision(cfg):
+    """Create the role, its password file and the database when web.database is not set and they are missing.
+
+    The password, of URL-safe characters, reaches psql on standard input, not on its command line.
+    """
+    if cfg['web']['database'].strip():
+        return
+    if not psql("SELECT 1 FROM pg_roles WHERE rolname = :'v1'", DATABASE):
+        DATABASE_SECRET.unlink(missing_ok=True)
+        psql("CREATE ROLE :\"v1\" LOGIN", DATABASE)
+    if not DATABASE_SECRET.is_file():
+        password = secrets.token_urlsafe(32)
+        psql(f"ALTER ROLE :\"v1\" PASSWORD '{password}'", DATABASE)
+        DATABASE_SECRET.touch(mode=0o600)
+        DATABASE_SECRET.chmod(0o600)
+        DATABASE_SECRET.write_text(password + '\n', encoding='utf-8')
+    if not psql("SELECT 1 FROM pg_database WHERE datname = :'v1'", DATABASE):
+        psql("CREATE DATABASE :\"v1\" OWNER :\"v1\"", DATABASE)

debian-package/src/usr/lib/vpn-router/vpnrouter_ldap.py

@@ -0,0 +1,254 @@
+"""The router's user directory in the local slapd.
+
+The package keeps its own database, dc=vpn-router,dc=local in /var/lib/ldap/vpn-router,
+next to the one slapd's installation creates. Root manages it through the local socket
+with SASL EXTERNAL: the database's root DN is root's peer-credential DN, so there is no
+administrator password. Users and groups are changed by the user-manager service account,
+which the ACLs allow to write ou=users and ou=groups and to set, never read, passwords and
+TOTP secrets.
+
+The password policy locks an account for LOCKOUT_SECONDS after LOCKOUT_FAILURES failed binds;
+a successful bind clears the count. slapd's otp overlay is configured on the database, so a
+bind of a user with a TOTP token takes the password followed by a current code.
+cn=totp,ou=policies holds the TOTP parameters the token of every user points to.
+"""
+
+import os
+import pathlib
+import pwd
+import re
+import secrets
+
+import ldap
+import ldap.modlist
+import ldap.sasl
+
+URI = 'ldapi:///'
+BASE = 'dc=vpn-router,dc=local'
+USERS = f'ou=users,{BASE}'
+GROUPS = f'ou=groups,{BASE}'
+SERVICES = f'ou=services,{BASE}'
+POLICIES = f'ou=policies,{BASE}'
+DEFAULT_POLICY = f'cn=default,{POLICIES}'
+TOTP_PARAMS = f'cn=totp,{POLICIES}'
+MANAGER = f'cn=user-manager,{SERVICES}'
+ROOT = 'gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth'
+
+DB_DIRECTORY = pathlib.Path('/var/lib/ldap/vpn-router')
+SECRET_DIR = pathlib.Path('/etc/vpn-router/ldap')
+MANAGER_SECRET = SECRET_DIR / 'user-manager.secret'
+SLAPD_DEFAULTS = pathlib.Path('/etc/default/slapd')
+SLAPD_SERVICES = 'SLAPD_SERVICES="ldapi:///"'
+
+MODULES = ('argon2', 'ppolicy', 'otp')
+PASSWORD_HASH = '{ARGON2}'
+
+#: Failed binds that lock an account, within LOCKOUT_SECONDS, and the seconds it stays locked.
+LOCKOUT_FAILURES = 5
+LOCKOUT_SECONDS = 900
+
+#: The overlays of the database, in the order they are configured.
+OVERLAYS = ('ppolicy', 'otp')
+
+#: The package's own schema, under an OID of the UUID arc 2.25 (ITU-T X.667): attribute types
+#: under .1 and object classes under .2. vpnRouterAccount, auxiliary, adds properties of the
+#: package to a user of either kind: mail, for a Manager-only account, vpnRouterScope,
+#: authorization scopes, of which service marks a service account, and displayName and
+#: description, the labels of a service account.
+SCHEMA_OID = '2.25.98477352595735834532404256531620013001'
+SCHEMA_NAME = 'vpn-router'
+SCHEMA_ATTRIBUTE_TYPES = (
+    f"( {SCHEMA_OID}.1.1 NAME 'vpnRouterScope' DESC 'VPN router: an authorization scope of the account' "
+    "EQUALITY caseIgnoreMatch SUBSTR caseIgnoreSubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{256} )",
+)
+SCHEMA_OBJECT_CLASSES = (
+    f"( {SCHEMA_OID}.2.1 NAME 'vpnRouterAccount' DESC 'VPN router: properties of a user or a Manager-only account' "
+    "SUP top AUXILIARY MAY ( mail $ vpnRouterScope $ displayName $ description ) )",
+)
+
+#: The TOTP parameters: digits, the OID of HMAC-SHA1, seconds per time step, and the time steps accepted before and after now.
+TOTP_DIGITS = 6
+TOTP_ALGORITHM = '1.2.840.113549.2.7'
+TOTP_PERIOD = 30
+TOTP_WINDOW = 1
+
+#: The ACLs of the database, in order.
+ACCESS = (
+    f'to attrs=userPassword by dn.exact="{MANAGER}" =wx by self =wx by anonymous auth by * none',
+    f'to attrs=oathSecret by dn.exact="{MANAGER}" =w by * none',
+    f'to dn.subtree="{USERS}" by dn.exact="{MANAGER}" manage by * read',
+    f'to dn.subtree="{GROUPS}" by dn.exact="{MANAGER}" manage by * read',
+    'to * by * read',
+)
+
+#: The index of an ordered cn=config value: {0}, {1}, ...
+ORDER = re.compile(rb'^\{\d+\}')
+
+
+def unordered(values):
+    """Values of an ordered cn=config attribute without their {n} prefixes."""
+    return [ORDER.sub(b'', value).decode() for value in values]
+
+
+def root():
+    """A connection to slapd as root, through the local socket."""
+    connection = ldap.initialize(URI)
+    connection.sasl_interactive_bind_s('', ldap.sasl.external())
+    return connection
+
+
+def manager():
+    """A connection to slapd as the user-manager service account."""
+    connection = ldap.initialize(URI)
+    connection.simple_bind_s(MANAGER, MANAGER_SECRET.read_text(encoding='utf-8'))
+    return connection
+
+
+def ensure_services():
+    """Make slapd listen on the local socket only. Returns True when the setting changed."""
+    text = SLAPD_DEFAULTS.read_text(encoding='utf-8')
+    lines = text.splitlines(keepends=True)
+    updated = ''.join(SLAPD_SERVICES + '\n' if line.startswith('SLAPD_SERVICES=') else line for line in lines)
+    if not any(line.startswith('SLAPD_SERVICES=') for line in lines):
+        updated = text.rstrip('\n') + '\n' + SLAPD_SERVICES + '\n'
+    if updated == text:
+        return False
+    SLAPD_DEFAULTS.write_text(updated, encoding='utf-8')
+    return True
+
+
+def _ensure_modules(connection):
+    dn, attributes = connection.search_s('cn=config', ldap.SCOPE_ONELEVEL, '(objectClass=olcModuleList)', ['olcModuleLoad'])[0]
+    loaded = unordered(attributes.get('olcModuleLoad', []))
+    missing = [name for name in MODULES if name not in loaded and f'{name}.la' not in loaded]
+    if missing:
+        connection.modify_s(dn, [(ldap.MOD_ADD, 'olcModuleLoad', [name.encode() for name in missing])])
+
+
+def _ensure_schema(connection):
+    """Add the package's schema to cn=config when missing, else set its definitions that differ."""
+    found = connection.search_s('cn=schema,cn=config', ldap.SCOPE_ONELEVEL, f'(cn=*{SCHEMA_NAME})',
+                                ['olcAttributeTypes', 'olcObjectClasses'])
+    attributes = {'olcAttributeTypes': [value.encode() for value in SCHEMA_ATTRIBUTE_TYPES],
+                  'olcObjectClasses': [value.encode() for value in SCHEMA_OBJECT_CLASSES]}
+    if not found:
+        connection.add_s(f'cn={SCHEMA_NAME},cn=schema,cn=config', ldap.modlist.addModlist(
+            dict(attributes, objectClass=[b'olcSchemaConfig'], cn=[SCHEMA_NAME.encode()])))
+        return
+    dn, current = found[0]
+    changes = [(ldap.MOD_REPLACE, name, values) for name, values in attributes.items()
+               if unordered(current.get(name, [])) != [value.decode() for value in values]]
+    if changes:
+        connection.modify_s(dn, changes)
+
+
+def _ensure_password_hash(connection):
+    dn = 'olcDatabase={-1}frontend,cn=config'
+    current = connection.search_s(dn, ldap.SCOPE_BASE, attrlist=['olcPasswordHash'])[0][1].get('olcPasswordHash', [])
+    if current != [PASSWORD_HASH.encode()]:
+        connection.modify_s(dn, [(ldap.MOD_REPLACE, 'olcPasswordHash', [PASSWORD_HASH.encode()])])
+
+
+def _ensure_database(connection):
+    """The DN of the package's database in cn=config, created when missing; its ACLs are kept as ACCESS."""
+    found = connection.search_s('cn=config', ldap.SCOPE_ONELEVEL, f'(olcSuffix={BASE})', ['olcAccess'])
+    if not found:
+        DB_DIRECTORY.mkdir(mode=0o700, parents=True, exist_ok=True)
+        openldap = pwd.getpwnam('openldap')
+        os.chown(DB_DIRECTORY, openldap.pw_uid, openldap.pw_gid)
+        connection.add_s('olcDatabase=mdb,cn=config', ldap.modlist.addModlist({
+            'objectClass': [b'olcDatabaseConfig', b'olcMdbConfig'],
+            'olcDatabase': [b'mdb'],
+            'olcSuffix': [BASE.encode()],
+            'olcDbDirectory': [str(DB_DIRECTORY).encode()],
+            'olcRootDN': [ROOT.encode()],
+            'olcDbIndex': [b'objectClass eq', b'uid eq'],
+            'olcAccess': [f'{{{index}}}{rule}'.encode() for index, rule in enumerate(ACCESS)],
+        }))
+        found = connection.search_s('cn=config', ldap.SCOPE_ONELEVEL, f'(olcSuffix={BASE})', ['olcAccess'])
+    dn, attributes = found[0]
+    if unordered(attributes.get('olcAccess', [])) != list(ACCESS):
+        connection.modify_s(dn, [(ldap.MOD_REPLACE, 'olcAccess', [f'{{{index}}}{rule}'.encode() for index, rule in enumerate(ACCESS)])])
+    return dn
+
+
+def _ensure_overlays(connection, database):
+    """Add the overlays of OVERLAYS the database lacks; the otp overlay has no settings of its own."""
+    found = connection.search_s(database, ldap.SCOPE_ONELEVEL, '(objectClass=olcOverlayConfig)', ['olcOverlay'])
+    present = {ORDER.sub(b'', attributes['olcOverlay'][0]).decode() for _, attributes in found}
+    for name in OVERLAYS:
+        if name in present:
+            continue
+        record = {'objectClass': [b'olcOverlayConfig'], 'olcOverlay': [name.encode()]}
+        if name == 'ppolicy':
+            record = {'objectClass': [b'olcOverlayConfig', b'olcPPolicyConfig'], 'olcOverlay': [b'ppolicy'],
+                      'olcPPolicyDefault': [DEFAULT_POLICY.encode()]}
+        connection.add_s(f'olcOverlay={name},{database}', ldap.modlist.addModlist(record))
+
+
+def _ensure_entry(connection, dn, attributes):
+    """Add the entry when it is missing, else set the given attributes that differ."""
+    try:
+        current = connection.search_s(dn, ldap.SCOPE_BASE, attrlist=list(attributes))[0][1]
+    except ldap.NO_SUCH_OBJECT:
+        connection.add_s(dn, ldap.modlist.addModlist(attributes))
+        return
+    changes = [(ldap.MOD_REPLACE, name, values) for name, values in attributes.items()
+               if name != 'objectClass' and sorted(current.get(name, [])) != sorted(values)]
+    if changes:
+        connection.modify_s(dn, changes)
+
+
+def _ensure_manager(connection):
+    SECRET_DIR.mkdir(mode=0o700, parents=True, exist_ok=True)
+    os.chmod(SECRET_DIR, 0o700)
+    created = False
+    try:
+        connection.search_s(MANAGER, ldap.SCOPE_BASE, attrlist=['1.1'])
+    except ldap.NO_SUCH_OBJECT:
+        connection.add_s(MANAGER, ldap.modlist.addModlist({
+            'objectClass': [b'organizationalRole', b'simpleSecurityObject'],
+            'cn': [b'user-manager'],
+            'userPassword': [b'{CRYPT}!'],
+        }))
+        created = True
+    if created or not MANAGER_SECRET.is_file():
+        password = secrets.token_urlsafe(32)
+        connection.passwd_s(MANAGER, None, password)
+        MANAGER_SECRET.touch(mode=0o600)
+        MANAGER_SECRET.chmod(0o600)
+        MANAGER_SECRET.write_text(password, encoding='utf-8')
+
+
+def provision():
+    """Create or bring up to date the modules, the package's schema, database, ACLs, password policy, entries and service account."""
+    connection = root()
+    try:
+        _ensure_modules(connection)
+        _ensure_schema(connection)
+        _ensure_password_hash(connection)
+        database = _ensure_database(connection)
+        _ensure_overlays(connection, database)
+        _ensure_entry(connection, BASE, {'objectClass': [b'dcObject', b'organization'], 'dc': [b'vpn-router'], 'o': [b'vpn-router']})
+        for unit in (USERS, GROUPS, SERVICES, POLICIES):
+            _ensure_entry(connection, unit, {'objectClass': [b'organizationalUnit'], 'ou': [unit.split(',')[0][3:].encode()]})
+        _ensure_entry(connection, DEFAULT_POLICY, {
+            'objectClass': [b'organizationalRole', b'pwdPolicy'],
+            'cn': [b'default'],
+            'pwdAttribute': [b'userPassword'],
+            'pwdLockout': [b'TRUE'],
+            'pwdMaxFailure': [str(LOCKOUT_FAILURES).encode()],
+            'pwdLockoutDuration': [str(LOCKOUT_SECONDS).encode()],
+            'pwdFailureCountInterval': [str(LOCKOUT_SECONDS).encode()],
+        })
+        _ensure_entry(connection, TOTP_PARAMS, {
+            'objectClass': [b'organizationalRole', b'oathTOTPParams'],
+            'cn': [b'totp'],
+            'oathOTPLength': [str(TOTP_DIGITS).encode()],
+            'oathHMACAlgorithm': [TOTP_ALGORITHM.encode()],
+            'oathTOTPTimeStepPeriod': [str(TOTP_PERIOD).encode()],
+            'oathTOTPTimeStepWindow': [str(TOTP_WINDOW).encode()],
+        })
+        _ensure_manager(connection)
+    finally:
+        connection.unbind_s()

debian-package/src/usr/lib/vpn-router/vpnrouter_metrics.py

@@ -0,0 +1,207 @@
+"""The metrics of the router in PostgreSQL, for the metrics service and the web service.
+
+The metrics service writes one detail row per minute to the four detail tables, of TABLES:
+metrics_system (CPU, memory, disk and load averages), metrics_network (bytes per second
+received and sent per interface), metrics_vpn (clients per kind) and metrics_site_to_site (per
+site-to-site connection: up, 1 or 0, bytes and packets per second received and sent, and the
+seconds its IKE SA has been up). Each detail table has four
+summary tables, of LEVELS: <table>_hourly from the details, <table>_daily from the hourly,
+<table>_monthly from the daily and <table>_yearly from the monthly rows. A summary row has the
+start of its period, in UTC, and the average and the maximum of each value over the period: of
+the details for the hourly rows, else the average of the averages and the maximum of the maxima
+below. A period is summarized once it is complete; summarize() catches up every complete period
+after the last summarized one, so a stopped service loses no summary of what was written.
+
+collect_garbage() deletes the rows older than the retention of their level, metrics.keep_*;
+the service runs it on start and once a day. history() answers the views of the Metrics tab.
+
+The tables are in the database of the refresh tokens (vpnrouter_auth.connection_string) and are
+created when missing. Every failure of the database raises DatabaseError.
+"""
+
+import contextlib
+
+import psycopg
+
+import vpnrouter_auth
+
+#: The detail tables: their key columns besides time, and their value columns.
+TABLES = {
+    'metrics_system': ((), ('cpu', 'memory_used', 'memory_total', 'disk_free', 'disk_total', 'load1', 'load5', 'load15')),
+    'metrics_network': (('interface',), ('rx', 'tx')),
+    'metrics_vpn': (('kind',), ('clients',)),
+    'metrics_site_to_site': (('connection',), ('up', 'rx', 'tx', 'packets_rx', 'packets_tx', 'uptime')),
+}
+
+#: The summary levels, from the finest: the suffix of the tables, the period of date_trunc and the retention setting.
+LEVELS = (
+    ('hourly', 'hour', 'keep_hourly'),
+    ('daily', 'day', 'keep_daily'),
+    ('monthly', 'month', 'keep_monthly'),
+    ('yearly', 'year', 'keep_yearly'),
+)
+
+#: The kinds of VPN clients of metrics_vpn.
+VPN_KINDS = ('p2s', 'openvpn', 'wireguard')
+
+#: The time slots of the site-to-site availability of each view of history(): the summary level read (None: the
+#: details), the slot as PostgreSQL starts it from time, and its length.
+UP_SLOTS = {
+    'hour': (None, "date_bin('5 minutes', time, timestamptz '2000-01-01 00:00:00+00')", '5 minutes'),
+    'day': (None, "date_trunc('hour', time, 'UTC')", '1 hour'),
+    'month': ('hourly', "date_trunc('day', time, 'UTC')", '1 day'),
+    'year': ('monthly', "date_trunc('month', time, 'UTC')", '1 month'),
+}
+
+#: The views of history(): the summary level read (None: the details), the bucket and the span.
+RANGES = {
+    'hour': (None, '1 minute', '1 hour'),
+    'day': (None, '10 minutes', '1 day'),
+    'month': ('hourly', '6 hours', '1 month'),
+    'year': ('monthly', '1 month', '1 year'),
+}
+
+
+class DatabaseError(Exception):
+    """The metrics database failed."""
+
+
+def schema():
+    """The statements that create the detail and the summary tables."""
+    statements = []
+    for table, (keys, values) in TABLES.items():
+        key_columns = ''.join(f', {key} text NOT NULL' for key in keys)
+        primary = ', '.join(('time', *keys))
+        detail = ''.join(f', {value} double precision' for value in values)
+        statements.append(f'CREATE TABLE IF NOT EXISTS {table} (time timestamptz NOT NULL{key_columns}{detail}, '
+                          f'PRIMARY KEY ({primary}))')
+        summary = ''.join(f', {value}_avg double precision, {value}_max double precision' for value in values)
+        for suffix, _, _ in LEVELS:
+            statements.append(f'CREATE TABLE IF NOT EXISTS {table}_{suffix} (time timestamptz NOT NULL{key_columns}{summary}, '
+                              f'PRIMARY KEY ({primary}))')
+    return statements
+
+
+@contextlib.contextmanager
+def database(cfg):
+    """A connection to the metrics database, with its tables, closed at the end; DatabaseError on any failure."""
+    try:
+        with psycopg.connect(vpnrouter_auth.connection_string(cfg), autocommit=True) as connection:
+            for statement in schema():
+                connection.execute(statement)
+            yield connection
+    except (psycopg.Error, OSError) as exc:
+        raise DatabaseError(f'the metrics database failed: {exc}') from exc
+
+
+def write(connection, system, network, vpn, site_to_site):
+    """Write a sample as the detail rows of the current minute.
+
+    system maps the values of metrics_system to numbers; network maps an interface to its rx and
+    tx; vpn maps a kind of VPN_KINDS to its clients; site_to_site maps a connection to the values
+    of metrics_site_to_site. A value that is None is written as NULL.
+    """
+    with connection.transaction():
+        minute = connection.execute("SELECT date_trunc('minute', now(), 'UTC')").fetchone()[0]
+        names = TABLES['metrics_system'][1]
+        connection.execute(f"INSERT INTO metrics_system (time, {', '.join(names)}) "
+                           f"VALUES (%s{', %s' * len(names)}) ON CONFLICT DO NOTHING",
+                           (minute, *(system.get(name) for name in names)))
+        for interface, (rx, tx) in network.items():
+            connection.execute('INSERT INTO metrics_network (time, interface, rx, tx) VALUES (%s, %s, %s, %s) '
+                               'ON CONFLICT DO NOTHING', (minute, interface, rx, tx))
+        for kind, clients in vpn.items():
+            connection.execute('INSERT INTO metrics_vpn (time, kind, clients) VALUES (%s, %s, %s) ON CONFLICT DO NOTHING',
+                               (minute, kind, clients))
+        names = TABLES['metrics_site_to_site'][1]
+        for name, values in site_to_site.items():
+            connection.execute(f"INSERT INTO metrics_site_to_site (time, connection, {', '.join(names)}) "
+                               f"VALUES (%s, %s{', %s' * len(names)}) ON CONFLICT DO NOTHING",
+                               (minute, name, *(values.get(value) for value in names)))
+
+
+def summarize(connection, level):
+    """Summarize the complete periods of a level of LEVELS that are not summarized yet, for every detail table."""
+    index = [suffix for suffix, _, _ in LEVELS].index(level)
+    _, period, _ = LEVELS[index]
+    for table, (keys, values) in TABLES.items():
+        target = f'{table}_{level}'
+        source = table if index == 0 else f'{table}_{LEVELS[index - 1][0]}'
+        if index == 0:
+            aggregates = ', '.join(f'avg({value}), max({value})' for value in values)
+        else:
+            aggregates = ', '.join(f'avg({value}_avg), max({value}_max)' for value in values)
+        columns = ', '.join(('time', *keys, *(f'{value}_{kind}' for value in values for kind in ('avg', 'max'))))
+        groups = ''.join(f', {key}' for key in keys)
+        connection.execute(
+            f"INSERT INTO {target} ({columns}) "
+            f"SELECT date_trunc('{period}', time, 'UTC') AS period{groups}, {aggregates} FROM {source} "
+            f"WHERE time < date_trunc('{period}', now(), 'UTC') "
+            f"AND time >= coalesce((SELECT max(time) FROM {target}) + interval '1 {period}', '-infinity') "
+            f"GROUP BY period{groups} ON CONFLICT DO NOTHING")
+
+
+def collect_garbage(connection, cfg):
+    """Delete the rows older than the retention of their level: metrics.keep_details, keep_hourly and so on."""
+    metrics = cfg['metrics']
+    for table in TABLES:
+        connection.execute(f'DELETE FROM {table} WHERE time < now() - %s::interval', (metrics['keep_details'],))
+        for suffix, _, keep in LEVELS:
+            connection.execute(f'DELETE FROM {table}_{suffix} WHERE time < now() - %s::interval', (metrics[keep],))
+
+
+def history(connection, view):
+    """The points of a view of RANGES, oldest first, each with the start of its bucket in Unix seconds.
+
+    A point has the average and the maximum of each value of metrics_system, network, which maps
+    each interface to its rx and tx, clients, which maps each kind of VPN client to its clients,
+    and site_to_site, which maps each site-to-site connection to the values of
+    metrics_site_to_site; a value without data is None.
+
+    site_to_site_up is the availability of each site-to-site connection in the time slots of UP_SLOTS, oldest first:
+    for each slot its start in Unix seconds and, per connection, the percentage of its samples in which the connection
+    was up, 0 to 100; the summaries' averages of up give it for the month and the year.
+    """
+    level, bucket, span = RANGES[view]
+    up_level, up_start, up_slot = UP_SLOTS[view]
+
+    def rows(table):
+        keys, values = TABLES[table]
+        source = table if level is None else f'{table}_{level}'
+        if level is None:
+            aggregates = ', '.join(f'avg({value}), max({value})' for value in values)
+        else:
+            aggregates = ', '.join(f'avg({value}_avg), max({value}_max)' for value in values)
+        start = (f"date_bin('{bucket}', time, timestamptz '2000-01-01 00:00:00+00')" if view != 'year'
+                 else "date_trunc('month', time, 'UTC')")
+        groups = ''.join(f', {key}' for key in keys)
+        return connection.execute(
+            f"SELECT extract(epoch FROM {start})::bigint AS bucket{groups}, {aggregates} FROM {source} "
+            f"WHERE time >= now() - interval '{span}' GROUP BY bucket{groups} ORDER BY bucket").fetchall()
+
+    def pairs(row, values):
+        return {value: {'avg': row[2 * index], 'max': row[2 * index + 1]} for index, value in enumerate(values)}
+
+    points = {}
+
+    def point(start):
+        return points.setdefault(start, {'time': start, 'network': {}, 'clients': {}, 'site_to_site': {}})
+
+    for row in rows('metrics_system'):
+        point(row[0]).update(pairs(row[1:], TABLES['metrics_system'][1]))
+    for row in rows('metrics_network'):
+        point(row[0])['network'][row[1]] = pairs(row[2:], TABLES['metrics_network'][1])
+    for row in rows('metrics_vpn'):
+        point(row[0])['clients'][row[1]] = pairs(row[2:], TABLES['metrics_vpn'][1])['clients']
+    for row in rows('metrics_site_to_site'):
+        point(row[0])['site_to_site'][row[1]] = pairs(row[2:], TABLES['metrics_site_to_site'][1])
+    empty = {value: {'avg': None, 'max': None} for value in TABLES['metrics_system'][1]}
+
+    slots = {}
+    for start, name, percent in connection.execute(
+            f"SELECT extract(epoch FROM {up_start})::bigint AS slot, connection, avg({'up' if up_level is None else 'up_avg'}) * 100 "
+            f"FROM {'metrics_site_to_site' if up_level is None else f'metrics_site_to_site_{up_level}'} "
+            f"WHERE time >= now() - interval '{span}' GROUP BY slot, connection ORDER BY slot").fetchall():
+        slots.setdefault(start, {'time': start, 'connections': {}})['connections'][name] = percent
+    return {'range': view, 'bucket': bucket, 'points': [{**empty, **points[start]} for start in sorted(points)],
+            'site_to_site_up': {'slot': up_slot, 'points': [slots[start] for start in sorted(slots)]}}

debian-package/src/usr/lib/vpn-router/vpnrouter_net.py

@@ -0,0 +1,176 @@
+"""Interfaces and addresses of the router, read from the system.
+
+Only the Python standard library is used, so the web service and its bootstrap server can use
+this module too.
+"""
+
+import pathlib
+import subprocess
+
+#: The values of web.listen and of the Router Manager's listen: the interfaces a service listens on.
+LISTEN_CHOICES = ('internal', 'external', 'both')
+
+#: The interfaces a service listens on when it is not set.
+DEFAULT_LISTEN = 'both'
+
+#: The address every service also listens on, where the HTTPS proxy calls it.
+LOOPBACK = '127.0.0.1'
+
+
+class ConfigError(Exception):
+    """A setting is present but malformed."""
+
+
+def ip_output(*args):
+    """The output of an ip command, or '' when it fails."""
+    try:
+        result = subprocess.run(['ip', *args], capture_output=True, text=True)
+    except OSError:
+        return ''
+    return result.stdout if result.returncode == 0 else ''
+
+
+def detect_interfaces():
+    """Work out the external and internal interface names.
+
+    Only reached in auto mode, where the operator has accepted that a wrong
+    guess is possible. The external interface is whichever one carries the
+    default route; the internal one is the only other addressed interface.
+    """
+    tokens = ip_output('route', 'get', '1.1.1.1').split()
+    external = ''
+    for index, token in enumerate(tokens):
+        if token == 'dev' and index + 1 < len(tokens):
+            external = tokens[index + 1]
+            break
+    if not external:
+        raise ConfigError('auto mode: cannot determine the external interface')
+
+    candidates = [name for name in _addressed_interfaces() if name != external]
+    if len(candidates) != 1:
+        raise ConfigError(
+            f'auto mode: expected one internal interface, found {len(candidates)}; '
+            'use mode = interfaces and name them')
+    return external, candidates[0]
+
+
+def _is_virtual(name):
+    """True for an interface the kernel creates without a device behind it.
+
+    Loopback, bridges, veth pairs and tunnels live under /sys/devices/virtual,
+    which is where the Docker bridge is; a NIC of the virtual machine does not.
+    """
+    link = pathlib.Path('/sys/class/net') / name
+    return '/devices/virtual/' in str(link.resolve())
+
+
+def _addressed_interfaces():
+    """Names of the NICs holding at least one IPv4 address; virtual interfaces are left out."""
+    names = []
+    for line in ip_output('-o', '-4', 'addr', 'show').splitlines():
+        fields = line.split()
+        if len(fields) >= 4 and fields[1] not in names and not _is_virtual(fields[1]):
+            names.append(fields[1])
+    return names
+
+
+def interface_addresses(name, setting):
+    """IPv4 addresses configured on an interface.
+
+    The interface name is configuration; its addresses are read from the
+    system. Looking up the addresses of a named interface is deterministic -
+    unlike working out which interface to use, which is a guess the package
+    deliberately does not make.
+    """
+    if not name:
+        return []
+    if not (pathlib.Path('/sys/class/net') / name).exists():
+        raise ConfigError(f'{setting}: interface {name!r} does not exist')
+
+    result = subprocess.run(
+        ['ip', '-o', '-4', 'addr', 'show', 'dev', name],
+        capture_output=True, text=True,
+    )
+    if result.returncode != 0:
+        raise ConfigError(f'{setting}: cannot read addresses of {name!r}')
+
+    addresses = []
+    for line in result.stdout.splitlines():
+        fields = line.split()
+        if len(fields) >= 4:
+            addresses.append(fields[3].split('/')[0])
+    return addresses
+
+
+def internal_address(mode, internal, int_addr):
+    """The router's address on the internal network, as general.mode directs, or '' when it is not known yet.
+
+    manual takes local.int_addr; interfaces reads the address of interfaces.internal; auto also
+    works out which interface that is.
+    """
+    if mode == 'manual':
+        return int_addr
+    try:
+        if mode == 'auto':
+            internal = detect_interfaces()[1]
+        addresses = interface_addresses(internal, 'interfaces.internal')
+    except ConfigError:
+        return ''
+    return addresses[0] if addresses else ''
+
+
+def external_address(mode, external):
+    """The router's address on the external interface, as general.mode directs, or '' when it is not known yet.
+
+    auto works out which interface that is; the other modes read the address of interfaces.external.
+    """
+    try:
+        if mode == 'auto':
+            external = detect_interfaces()[0]
+        addresses = interface_addresses(external, 'interfaces.external')
+    except ConfigError:
+        return ''
+    return addresses[0] if addresses else ''
+
+
+def _section(configuration, name):
+    """A section of a configuration, or {} when it is missing or not an object."""
+    value = configuration.get(name)
+    return value if isinstance(value, dict) else {}
+
+
+def listen_addresses(configuration, choice):
+    """The addresses a service listens on for choice of LISTEN_CHOICES, from vpn-router.json read as configuration.
+
+    The internal address comes first; an address not known yet is left out. A choice that is not
+    one of LISTEN_CHOICES is DEFAULT_LISTEN. LOOPBACK is not among them; web_listen adds it.
+    """
+    if choice not in LISTEN_CHOICES:
+        choice = DEFAULT_LISTEN
+    general, interfaces, local = (_section(configuration, name) for name in ('general', 'interfaces', 'local'))
+    mode = str(general.get('mode') or 'manual')
+    addresses = []
+    if choice in ('internal', 'both'):
+        addresses.append(internal_address(mode, str(interfaces.get('internal') or ''), str(local.get('int_addr') or '')))
+    if choice in ('external', 'both'):
+        addresses.append(external_address(mode, str(interfaces.get('external') or '')))
+    return [address for address in addresses if address]
+
+
+def web_listen(configuration, default_port):
+    """Where the web service listens, from vpn-router.json read as configuration, as ([(address, port), ...], '').
+
+    The addresses are those of the interfaces web.listen names (listen_addresses), then
+    LOOPBACK, the port web.port, else default_port. While web.enabled is false or no address of
+    an interface is known, the answer is ([], the reason).
+    """
+    web = _section(configuration, 'web')
+    if web.get('enabled', True) is False:
+        return [], 'the web service is disabled in the configuration'
+    port = web.get('port', default_port)
+    if isinstance(port, bool) or not isinstance(port, int) or not 1 <= port <= 65535:
+        port = default_port
+    addresses = listen_addresses(configuration, web.get('listen', DEFAULT_LISTEN))
+    if not addresses:
+        return [], 'the addresses to listen on are not known yet'
+    return [(address, port) for address in [*addresses, LOOPBACK]], ''

debian-package/src/usr/lib/vpn-router/vpnrouter_serve.py

@@ -0,0 +1,46 @@
+"""The listening loop of the web service and of its bootstrap server.
+
+Only the Python standard library is used, so the bootstrap server, which must run whatever else
+is broken, can use it.
+"""
+
+import threading
+import time
+from http.server import ThreadingHTTPServer
+
+
+def serve(wanted, handler, log, interval):
+    """Work out where to listen every interval seconds with wanted, and run an HTTP server of handler at each place.
+
+    wanted answers ([(address, port), ...], '') or ([], the reason not to listen). A server whose
+    place is no longer wanted is stopped, and a new one listens at each new place; a place that
+    cannot be bound is tried again the next round. Every new message is logged with log. Does
+    not return.
+    """
+    servers, reported = {}, None
+    while True:
+        targets, reason = wanted()
+        for target in [target for target in servers if target not in targets]:
+            server, thread = servers.pop(target)
+            server.shutdown()
+            server.server_close()
+            thread.join()
+        messages = [reason] if reason else []
+        for target in targets:
+            if target in servers:
+                continue
+            try:
+                server = ThreadingHTTPServer(target, handler)
+            except OSError as exc:
+                messages.append(f'cannot listen on {target[0]}:{target[1]}: {exc}')
+                continue
+            thread = threading.Thread(target=server.serve_forever, name=f'server {target[0]}', daemon=True)
+            thread.start()
+            servers[target] = (server, thread)
+        if servers:
+            messages.append('serving on ' + ', '.join(f'http://{address}:{port}' for address, port in servers))
+        message = '; '.join(messages)
+        if message and message != reported:
+            log(message)
+            reported = message
+        time.sleep(interval)

debian-package/src/usr/lib/vpn-router/vpnrouter_status.py

@@ -0,0 +1,145 @@
+"""What the router's VPN systems report, for the web service and the metrics service.
+
+Each reader asks the running system once and answers None when it cannot be asked: swanctl for
+the IKE SAs and the state and traffic of the site-to-site connections, wg for the WireGuard
+peers and the OpenVPN server's status file for its clients.
+"""
+
+import subprocess
+import time
+
+COMMAND_TIMEOUT = 5
+
+OPENVPN_STATUS = '/run/openvpn-server/status-vpn-router.log'
+
+#: Seconds since its last handshake within which a WireGuard peer counts as connected.
+WIREGUARD_CONNECTED_SECONDS = 180
+
+
+def run(*args):
+    """The finished process of a command, or None when it cannot be run."""
+    try:
+        return subprocess.run(args, capture_output=True, text=True, timeout=COMMAND_TIMEOUT)
+    except (OSError, subprocess.TimeoutExpired):
+        return None
+
+
+def swanctl(command):
+    """The entries of a swanctl list command, or None when swanctl cannot be asked.
+
+    The --pretty output has one item per line: "name {" opens a section, "key = [" a list
+    whose items follow one per line, "}" and "]" close them, and "key = value" is a pair
+    whose value may contain spaces. Every "... event {" section at the top holds one
+    connection or SA; each becomes an entry with its name under "name". The closing
+    "... reply {" section is left out. Values stay strings, as swanctl prints them.
+    Output with more closing than opening lines gives None.
+    """
+    result = run('swanctl', command, '--pretty')
+    if result is None or result.returncode != 0:
+        return None
+    events = []
+    stack = []
+    for line in result.stdout.splitlines():
+        line = line.strip()
+        if not line:
+            continue
+        if line in ('}', ']'):
+            if not stack:
+                return None
+            stack.pop()
+        elif not stack:
+            section = {}
+            if line.endswith(' event {'):
+                events.append(section)
+            stack.append(section)
+        elif isinstance(stack[-1], list):
+            stack[-1].append(line)
+        elif line.endswith(' {'):
+            stack[-1][line[:-2]] = section = {}
+            stack.append(section)
+        elif line.endswith(' = ['):
+            stack[-1][line[:-4]] = items = []
+            stack.append(items)
+        else:
+            key, _, value = line.partition(' = ')
+            stack[-1][key] = value
+    return [dict(body, name=name) for event in events for name, body in event.items()]
+
+
+def established_each(connections):
+    """The number of established IKE SAs of each of the connections by name, None for each when swanctl cannot be asked."""
+    sas = swanctl('--list-sas')
+    return {name: None if sas is None else sum(1 for sa in sas if sa['name'] == name and sa.get('state') == 'ESTABLISHED')
+            for name in connections}
+
+
+def _count(text):
+    """A counter swanctl prints as a string, as an integer; 0 when it is not a whole number."""
+    return int(text) if isinstance(text, str) and text.isdigit() else 0
+
+
+def site_to_site_tunnels(connections):
+    """The state of each of the connections by name, or None when swanctl cannot be asked.
+
+    Each has up, 1 while it has an established IKE SA and 0 otherwise; uptime, the seconds since
+    its oldest established IKE SA was established, None while it is down; and bytes_in,
+    bytes_out, packets_in and packets_out, the sums over the child SAs of its established IKE
+    SAs. The sums start again from zero when a child SA is rekeyed.
+    """
+    sas = swanctl('--list-sas')
+    if sas is None:
+        return None
+    tunnels = {}
+    for name in connections:
+        up = [sa for sa in sas if sa['name'] == name and sa.get('state') == 'ESTABLISHED']
+        children = [child for sa in up for child in (sa.get('child-sas') or {}).values() if isinstance(child, dict)]
+        tunnels[name] = {
+            'up': 1 if up else 0,
+            'uptime': max((_count(sa.get('established')) for sa in up), default=None),
+            **{key.replace('-', '_'): sum(_count(child.get(key)) for child in children)
+               for key in ('bytes-in', 'bytes-out', 'packets-in', 'packets-out')},
+        }
+    return tunnels
+
+
+def established(connection):
+    """The number of established IKE SAs of the connection, or None when swanctl cannot be asked."""
+    return established_each([connection])[connection]
+
+
+def wireguard_peers():
+    """One entry per WireGuard peer, with its handshake age and traffic, or None when wg0 is down."""
+    result = run('wg', 'show', 'wg0', 'dump')
+    if result is None or result.returncode != 0:
+        return None
+    now = int(time.time())
+    peers = []
+    for line in result.stdout.splitlines()[1:]:
+        fields = line.split('\t')
+        if len(fields) < 7:
+            continue
+        handshake = int(fields[4])
+        peers.append({
+            'handshake_seconds_ago': now - handshake if handshake else None,
+            'received_bytes': int(fields[5]),
+            'sent_bytes': int(fields[6]),
+        })
+    return peers
+
+
+def wireguard_connected():
+    """The number of WireGuard peers with a handshake in the last WIREGUARD_CONNECTED_SECONDS, or None when wg0 is down."""
+    peers = wireguard_peers()
+    if peers is None:
+        return None
+    return sum(1 for peer in peers
+               if peer['handshake_seconds_ago'] is not None and peer['handshake_seconds_ago'] < WIREGUARD_CONNECTED_SECONDS)
+
+
+def openvpn_clients():
+    """The number of connected OpenVPN clients, from the server's status file, or None when it cannot be read."""
+    try:
+        with open(OPENVPN_STATUS, encoding='utf-8') as handle:
+            return sum(1 for line in handle if line.startswith('CLIENT_LIST,'))
+    except OSError:
+        return None

debian-package/src/usr/lib/vpn-router/vpnrouter_users.py

@@ -0,0 +1,647 @@
+"""The users and groups of the router's directory, for vpn-router-user, vpn-router-group and the web service.
+
+A user is an entry in ou=users of the router's directory, of one of two kinds, fixed when it is
+added. A VPN user is an inetOrgPerson with a first name, a last name and an e-mail address, whose
+password OpenVPN checks, and a client certificate of the router CA with the user name as its
+common name, which OpenVPN and IKEv2 accept. A Manager-only account is an account,
+simpleSecurityObject and vpnRouterAccount (the package's schema) with a name, a password and
+optionally an e-mail address, no names, no certificate and so no VPN access; it is added when
+neither the first nor the last name is given, and can sign in to the web service. A service
+account is a Manager-only account with the scope SERVICE_SCOPE in vpnRouterScope, which another
+system signs in with to use a router service, and which cannot sign in to the web service; it has
+a display name and a description (labels), in displayName and description of vpnRouterAccount. The
+internal service accounts are those the router creates and manages for its own services, labels
+included (internal_accounts); vpn-router-setup keeps the metrics service's.
+Users exist whether or not a P2S system is enabled.
+
+A group is a groupOfNames in ou=groups with a name, which follows the rules of user names, an
+optional description and at least one member; a group cannot be empty, so the last member
+cannot leave it and a user who is the last member of a group cannot be removed. Groups only
+record membership; nothing on the router reads them.
+
+A user may have a TOTP token: the user entry itself is an oathTOTPUser and oathTOTPToken whose
+oathTOTPToken points to itself, with the secret in oathSecret and the parameters of
+vpnrouter_ldap.TOTP_PARAMS. Enrolment stores nothing until a code of the new secret is
+verified: otp_new makes a secret and its otpauth URI, and otp_enrol stores the secret given a
+code of it. slapd's otp overlay then takes the password followed by a code at every bind.
+
+A user is disabled when pwdAccountLockedTime holds LOCKED, and locked when the password
+policy has set it after failed binds, for vpnrouter_ldap.LOCKOUT_SECONDS; unlock ends that.
+login() checks a user's password and code with a bind. Members of ADMIN_GROUP manage the
+router through the web service. The module knows only the directory; the callers revoke a
+user's refresh tokens after the operations of vpnrouter_auth.REVOKING.
+
+Every operation runs in a session(), which holds the lock that serializes changes and the
+connection of the user manager account. A request that cannot be done raises Failure, whose
+status says why: 2 the request is invalid, 3 the user does not exist, 4 the user exists, 5 a
+login failed, 1 anything else. Answers are dicts with the keys of the JSON answers.
+"""
+
+import base64
+import calendar
+import contextlib
+import fcntl
+import pathlib
+import re
+import subprocess
+import sys
+import time
+
+import ldap
+import ldap.filter
+import ldap.modlist
+import pyotp
+
+import vpnrouter
+import vpnrouter_ldap
+
+#: A user name is one lowercase DNS label, as it is also the common name of the certificate.
+NAME = re.compile(r'^[a-z][a-z0-9-]{0,30}[a-z0-9]$|^[a-z]$')
+PASSWORD_MIN = 8
+PASSWORD_MAX = 128
+#: The longest first or last name, in characters.
+PERSON_NAME_MAX = 64
+#: An e-mail address: one @, no spaces, and a dot in the domain.
+EMAIL = re.compile(r'^[^@\s]+@[^@\s]+\.[^@\s]+$')
+EMAIL_MAX = 254
+#: The keys of the person attributes and their LDAP attributes.
+PERSON = (('first_name', 'givenName'), ('last_name', 'sn'), ('email', 'mail'))
+#: The keys of the labels of a service account and their LDAP attributes.
+LABELS = (('display_name', 'displayName'), ('description', 'description'))
+#: The attributes read for a user.
+ATTRIBUTES = ['uid', 'objectClass', 'pwdAccountLockedTime', 'givenName', 'sn', 'mail', 'oathTOTPToken', 'vpnRouterScope',
+              'displayName', 'description']
+#: The vpnRouterScope of a service account.
+SERVICE_SCOPE = 'service'
+#: The object classes of a VPN user and of a Manager-only account.
+VPN_USER_CLASSES = [b'inetOrgPerson']
+MANAGER_ONLY_CLASSES = [b'account', b'simpleSecurityObject', b'vpnRouterAccount']
+#: The readable attributes of a TOTP token; oathSecret is never readable.
+TOKEN_ATTRIBUTES = ['oathTOTPToken', 'oathTOTPParams', 'oathSecretTime', 'oathTOTPLastTimeStep', 'oathTOTPTimeStepDrift',
+                    'oathLastLogin', 'oathLastFailure', 'oathFailureCount']
+#: A TOTP secret as otp_new makes it: 32 base32 characters, 160 bits.
+TOTP_SECRET = re.compile(r'^[A-Z2-7]{32}$')
+#: The attributes read for a group.
+GROUP_ATTRIBUTES = ['cn', 'description', 'member']
+#: The longest group description, in characters.
+DESCRIPTION_MAX = 256
+#: The value of pwdAccountLockedTime that disables an account until it is enabled.
+LOCKED = b'000001010000Z'
+#: The group whose members manage the router through the web service.
+ADMIN_GROUP = 'admins'
+#: The digits of a TOTP code given to login().
+CODE = re.compile(r'^\d{6}$')
+LOCK_FILE = pathlib.Path('/run/vpn-router/users.lock')
+
+
+class Failure(Exception):
+    """A request that cannot be done; status is 2 invalid, 3 no such user, 4 the user exists, 5 failed login, 1 other."""
+
+    def __init__(self, message, status=1):
+        super().__init__(message)
+        self.status = status
+
+
+@contextlib.contextmanager
+def session():
+    """The connection of the user manager account, used under the lock that serializes changes."""
+    LOCK_FILE.parent.mkdir(parents=True, exist_ok=True)
+    with open(LOCK_FILE, 'w') as lock:
+        fcntl.flock(lock, fcntl.LOCK_EX)
+        connection = vpnrouter_ldap.manager()
+        try:
+            yield connection
+        finally:
+            connection.unbind_s()
+
+
+def check_name(name):
+    if not isinstance(name, str) or not NAME.match(name):
+        raise Failure('a user name is 1 to 32 lowercase letters, digits and hyphens, starting with a letter', 2)
+
+
+def check_password(value):
+    if not isinstance(value, str) or '\n' in value or '\r' in value:
+        raise Failure('the password must be a string of one line', 2)
+    if not PASSWORD_MIN <= len(value) <= PASSWORD_MAX:
+        raise Failure(f'the password must have {PASSWORD_MIN} to {PASSWORD_MAX} characters', 2)
+
+
+def person(values, required):
+    """The person attributes of values ({first_name, last_name, email}), checked, as {LDAP attribute: value}.
+
+    A key whose value is None is not given; with required, all three must be given.
+    """
+    found = {}
+    for key, attribute in PERSON:
+        value = values.get(key)
+        if value is None:
+            if required:
+                raise Failure(f'{key} is required', 2)
+            continue
+        if not isinstance(value, str):
+            raise Failure(f'{key} must be a string', 2)
+        value = value.strip()
+        if key == 'email':
+            if len(value) > EMAIL_MAX or not EMAIL.match(value):
+                raise Failure(f'{value!r} is not an e-mail address', 2)
+        elif not value or len(value) > PERSON_NAME_MAX or any(not character.isprintable() for character in value):
+            raise Failure(f'the {key.replace("_", " ")} must have 1 to {PERSON_NAME_MAX} printable characters', 2)
+        found[attribute] = value
+    return found
+
+
+def labels(values):
+    """The labels of a service account in values ({display_name, description}), checked, as {LDAP attribute: value}.
+
+    A key whose value is None is not given; '' means none. A display name has at most PERSON_NAME_MAX printable
+    characters and a description at most DESCRIPTION_MAX.
+    """
+    found = {}
+    name = values.get('display_name')
+    if name is not None:
+        if not isinstance(name, str):
+            raise Failure('display_name must be a string', 2)
+        name = name.strip()
+        if len(name) > PERSON_NAME_MAX or any(not character.isprintable() for character in name):
+            raise Failure(f'the display name must have at most {PERSON_NAME_MAX} printable characters', 2)
+        found['displayName'] = name
+    if values.get('description') is not None:
+        found['description'] = check_description(values['description'])
+    return found
+
+
+def dn_of(name):
+    return f'uid={name},{vpnrouter_ldap.USERS}'
+
+
+def certificate(name):
+    return vpnrouter.USER_CERT_DIR / f'{name}_cert.pem', vpnrouter.USER_CERT_DIR / f'{name}_key.pem'
+
+
+def entry(connection, name):
+    """The attributes of the user's entry; Failure with status 3 when there is none."""
+    check_name(name)
+    try:
+        return connection.search_s(dn_of(name), ldap.SCOPE_BASE, attrlist=ATTRIBUTES)[0][1]
+    except ldap.NO_SUCH_OBJECT:
+        raise Failure(f'user {name} does not exist', 3)
+
+
+def simple_ca(*args):
+    result = subprocess.run([sys.executable, str(vpnrouter.SIMPLE_CA), *args], capture_output=True, text=True)
+    if result.returncode != 0:
+        raise Failure(f'simple-ca {args[0]} failed: {result.stderr.strip() or result.stdout.strip()}')
+
+
+def publish_crl():
+    """Give strongSwan the CRL after a revocation; OpenVPN reads it at every connection."""
+    if vpnrouter.install_file(vpnrouter.PKI_DIR / 'crl.pem', vpnrouter.CRL_DEST, 0o644):
+        subprocess.run(['swanctl', '--load-creds'], capture_output=True, text=True)
+
+
+def locked_since(value):
+    """The time a pwdAccountLockedTime value names, in seconds since the epoch, or 0 when it names none."""
+    try:
+        return calendar.timegm(time.strptime(value[:14].decode(), '%Y%m%d%H%M%S'))
+    except ValueError:
+        return 0
+
+
+def manager_only(attributes):
+    """Whether the entry is a Manager-only account."""
+    return b'account' in (value.lower() for value in attributes.get('objectClass', []))
+
+
+def service(attributes):
+    """Whether the entry is a service account."""
+    return SERVICE_SCOPE.encode() in (value.lower() for value in attributes.get('vpnRouterScope', []))
+
+
+def internal_accounts():
+    """The internal service accounts, which the router creates and manages for its own services, by name, each with
+    the labels the router gives it ({display_name, description}): the one metrics.service_account names while metrics
+    are enabled."""
+    try:
+        metrics = vpnrouter.load_config()['metrics']
+    except SystemExit:
+        return {}
+    if not metrics['enabled']:
+        return {}
+    return {metrics['service_account']: {
+        'display_name': 'Metrics',
+        'description': 'The account an OpenTelemetry Collector signs in with to read /metrics of the metrics service.',
+    }}
+
+
+def summary(attributes):
+    """A user as the answers show it; a person attribute or a label the entry does not have is None.
+
+    internal is whether it is an internal service account (internal_accounts).
+    """
+    name = attributes['uid'][0].decode()
+    lock = attributes.get('pwdAccountLockedTime', [None])[0]
+    user = {
+        'name': name,
+        'manager_only': manager_only(attributes),
+        'service': service(attributes),
+        'internal': service(attributes) and name in internal_accounts(),
+        'enabled': lock != LOCKED,
+        'locked': lock not in (None, LOCKED) and locked_since(lock) > time.time() - vpnrouter_ldap.LOCKOUT_SECONDS,
+        'certificate': certificate(name)[0].is_file(),
+        'otp': 'oathTOTPToken' in attributes,
+    }
+    for key, attribute in PERSON + LABELS:
+        values = attributes.get(attribute)
+        user[key] = values[0].decode() if values else None
+    return user
+
+
+def common_name(attributes, name):
+    """The cn of an entry: the first and the last name, or the user name without them."""
+    full = ' '.join(value for value in (attributes.get('givenName'), attributes.get('sn')) if value)
+    return full or name
+
+
+def list_users(connection):
+    found = connection.search_s(vpnrouter_ldap.USERS, ldap.SCOPE_ONELEVEL, '(|(objectClass=inetOrgPerson)(objectClass=account))',
+                                ATTRIBUTES)
+    return {'users': sorted((summary(attributes) for _, attributes in found), key=lambda user: user['name'])}
+
+
+def add(connection, name, password, values, service=False):
+    """Add a user: a VPN user with its certificate when values gives all three person attributes, a Manager-only account
+    with at most the e-mail address when it gives neither name, and, with service, a service account, with at most the
+    e-mail address and its labels (labels)."""
+    check_name(name)
+    details = person(values, required=False)
+    vpn_user = 'givenName' in details or 'sn' in details
+    if service and vpn_user:
+        raise Failure('a service account has no first or last name', 2)
+    if service:
+        details.update({attribute: value for attribute, value in labels(values).items() if value})
+    if vpn_user and len(details) != len(PERSON):
+        raise Failure('give first_name, last_name and email for a VPN user, or at most email for a Manager-only account', 2)
+    check_password(password)
+    if vpn_user and not (vpnrouter.PKI_DIR / 'ca_key.pem').is_file():
+        raise Failure('the router CA does not exist; vpn-router-setup creates it once a platform is chosen')
+    if vpn_user:
+        record = {'objectClass': VPN_USER_CLASSES, 'uid': [name.encode()], 'cn': [common_name(details, name).encode()]}
+    else:
+        # simpleSecurityObject requires a userPassword; passwd_s sets the real one at once.
+        record = {'objectClass': MANAGER_ONLY_CLASSES, 'uid': [name.encode()], 'userPassword': [b'{CRYPT}!']}
+        if service:
+            record['vpnRouterScope'] = [SERVICE_SCOPE.encode()]
+    record.update({attribute: [value.encode()] for attribute, value in details.items()})
+    try:
+        connection.add_s(dn_of(name), ldap.modlist.addModlist(record))
+    except ldap.ALREADY_EXISTS:
+        raise Failure(f'user {name} exists', 4)
+    connection.passwd_s(dn_of(name), None, password)
+    if vpn_user:
+        vpnrouter.USER_CERT_DIR.mkdir(mode=0o700, parents=True, exist_ok=True)
+        for path in certificate(name):
+            path.unlink(missing_ok=True)
+        simple_ca('make-cert', '--ca-dir', str(vpnrouter.PKI_DIR), '--cert-dir', str(vpnrouter.USER_CERT_DIR), name)
+    return summary(entry(connection, name))
+
+
+def update(connection, name, values):
+    """Change the person attributes values gives; the cn follows the first and the last name.
+
+    A Manager-only account has only the e-mail address, which '' removes; a service account also its labels
+    (update_service).
+    """
+    current = entry(connection, name)
+    if service(current):
+        return update_service(connection, name, values)
+    if manager_only(current):
+        if values.get('first_name') is not None or values.get('last_name') is not None:
+            raise Failure(f'user {name} is a Manager-only account, which has no first or last name', 2)
+        email = values.get('email')
+        if email is None:
+            raise Failure('give email', 2)
+        if isinstance(email, str) and email.strip() == '':
+            if 'mail' in current:
+                connection.modify_s(dn_of(name), [(ldap.MOD_DELETE, 'mail', None)])
+        else:
+            connection.modify_s(dn_of(name), [(ldap.MOD_REPLACE, 'mail', [person({'email': email}, required=False)['mail'].encode()])])
+        return summary(entry(connection, name))
+    details = person(values, required=False)
+    if not details:
+        raise Failure('give at least one of first_name, last_name and email', 2)
+    names = {attribute: current[attribute][0].decode() for attribute in ('givenName', 'sn') if current.get(attribute)}
+    names.update({attribute: details[attribute] for attribute in ('givenName', 'sn') if attribute in details})
+    changes = [(ldap.MOD_REPLACE, attribute, [value.encode()]) for attribute, value in details.items()]
+    changes.append((ldap.MOD_REPLACE, 'cn', [common_name(names, name).encode()]))
+    connection.modify_s(dn_of(name), changes)
+    return summary(entry(connection, name))
+
+
+def update_service(connection, name, values):
+    """Change any of the e-mail address and the labels (labels) of a service account that values gives; '' removes one."""
+    if not service(entry(connection, name)):
+        raise Failure(f'{name} is not a service account', 3)
+    changes = [(ldap.MOD_REPLACE, attribute, [value.encode()] if value else []) for attribute, value in labels(values).items()]
+    email = values.get('email')
+    if email is not None:
+        address = '' if isinstance(email, str) and email.strip() == '' else person({'email': email}, required=False)['mail']
+        changes.append((ldap.MOD_REPLACE, 'mail', [address.encode()] if address else []))
+    if not changes:
+        raise Failure('give email, display_name or description', 2)
+    connection.modify_s(dn_of(name), changes)
+    return summary(entry(connection, name))
+
+
+def passwd(connection, name, password):
+    check_password(password)
+    entry(connection, name)
+    connection.passwd_s(dn_of(name), None, password)
+    return summary(entry(connection, name))
+
+
+def disable(connection, name):
+    if entry(connection, name).get('pwdAccountLockedTime') != [LOCKED]:
+        connection.modify_s(dn_of(name), [(ldap.MOD_REPLACE, 'pwdAccountLockedTime', [LOCKED])])
+    return summary(entry(connection, name))
+
+
+def enable(connection, name):
+    if entry(connection, name).get('pwdAccountLockedTime') == [LOCKED]:
+        connection.modify_s(dn_of(name), [(ldap.MOD_DELETE, 'pwdAccountLockedTime', None)])
+    return summary(entry(connection, name))
+
+
+def unlock(connection, name):
+    """End a lockout after failed binds and clear the failure count; a disabled user stays disabled."""
+    entry(connection, name)
+    current = connection.search_s(dn_of(name), ldap.SCOPE_BASE, attrlist=['pwdAccountLockedTime', 'pwdFailureTime'])[0][1]
+    changes = [(ldap.MOD_DELETE, 'pwdFailureTime', None)] if 'pwdFailureTime' in current else []
+    if 'pwdAccountLockedTime' in current and current['pwdAccountLockedTime'] != [LOCKED]:
+        changes.append((ldap.MOD_DELETE, 'pwdAccountLockedTime', None))
+    if changes:
+        connection.modify_s(dn_of(name), changes)
+    return summary(entry(connection, name))
+
+
+def login(name, password, code):
+    """Check the user's password, and the TOTP code of an enrolled user, with a bind as the user.
+
+    A wrong name, password or code, a locked or a disabled user give the same Failure, status 5.
+    """
+    refused = Failure('the name, the password or the code is wrong', 5)
+    if not isinstance(name, str) or not NAME.match(name) or not isinstance(password, str) or not password:
+        raise refused
+    if code is not None and code != '' and (not isinstance(code, str) or not CODE.match(code)):
+        raise refused
+    connection = ldap.initialize(vpnrouter_ldap.URI)
+    try:
+        connection.simple_bind_s(dn_of(name), password + (code or ''))
+    except ldap.INVALID_CREDENTIALS:
+        raise refused
+    finally:
+        connection.unbind_s()
+
+
+def is_admin(connection, name):
+    """Whether the user exists and is a member of ADMIN_GROUP."""
+    try:
+        entry(connection, name)
+    except Failure:
+        return False
+    return ADMIN_GROUP in groups_of(connection, name)
+
+
+def totp(secret):
+    return pyotp.TOTP(secret, digits=vpnrouter_ldap.TOTP_DIGITS, interval=vpnrouter_ldap.TOTP_PERIOD)
+
+
+def otp_new(connection, name):
+    """A new TOTP secret for the user and its otpauth URI, issued by the router's wan.local_fqdn; nothing is stored."""
+    entry(connection, name)
+    secret = pyotp.random_base32()
+    issuer = vpnrouter.load_config()['wan']['local_fqdn'].strip()
+    return {'name': name, 'secret': secret, 'uri': totp(secret).provisioning_uri(name=name, issuer_name=issuer or None)}
+
+
+def otp_enrol(connection, name, secret, code):
+    """Store the secret of otp_new as the user's TOTP token, replacing one the user has, when code is a code of it."""
+    current = entry(connection, name)
+    if not isinstance(secret, str) or not TOTP_SECRET.match(secret):
+        raise Failure('the secret must be 32 base32 characters, as a new secret has them', 2)
+    if not isinstance(code, str) or not re.fullmatch(rf'\d{{{vpnrouter_ldap.TOTP_DIGITS}}}', code):
+        raise Failure(f'the code must be {vpnrouter_ldap.TOTP_DIGITS} digits', 2)
+    if not totp(secret).verify(code, valid_window=vpnrouter_ldap.TOTP_WINDOW):
+        raise Failure('the code does not match the secret; check the clock of the authenticator', 2)
+    changes = [] if 'oathTOTPToken' in current else [(ldap.MOD_ADD, 'objectClass', [b'oathTOTPUser', b'oathTOTPToken'])]
+    changes += [
+        (ldap.MOD_REPLACE, 'oathTOTPToken', [dn_of(name).encode()]),
+        (ldap.MOD_REPLACE, 'oathTOTPParams', [vpnrouter_ldap.TOTP_PARAMS.encode()]),
+        (ldap.MOD_REPLACE, 'oathSecret', [base64.b32decode(secret)]),
+        (ldap.MOD_REPLACE, 'oathSecretTime', [time.strftime('%Y%m%d%H%M%SZ', time.gmtime()).encode()]),
+    ]
+    connection.modify_s(dn_of(name), changes)
+    return summary(entry(connection, name))
+
+
+def otp_remove(connection, name):
+    """Remove the user's TOTP token, when the user has one."""
+    if 'oathTOTPToken' in entry(connection, name):
+        token = connection.search_s(dn_of(name), ldap.SCOPE_BASE, attrlist=TOKEN_ATTRIBUTES)[0][1]
+        changes = [(ldap.MOD_DELETE, 'objectClass', [b'oathTOTPUser', b'oathTOTPToken']),
+                   (ldap.MOD_DELETE, 'oathSecret', None)]
+        changes += [(ldap.MOD_DELETE, attribute, None) for attribute in token]
+        connection.modify_s(dn_of(name), changes)
+    return summary(entry(connection, name))
+
+
+def remove(connection, name):
+    """Remove the user from its groups and the directory, and revoke the certificate.
+
+    A user who is the last member of a group is not removed.
+    """
+    entry(connection, name)
+    groups = groups_of(connection, name)
+    alone = sorted(group for group, members in groups.items() if members == [name])
+    if alone:
+        raise Failure(f'user {name} is the last member of {", ".join(alone)}; remove the group first', 2)
+    for group in groups:
+        connection.modify_s(group_dn(group), [(ldap.MOD_DELETE, 'member', [dn_of(name).encode()])])
+    cert, key = certificate(name)
+    if cert.is_file():
+        simple_ca('revoke-cert', '--ca-dir', str(vpnrouter.PKI_DIR), str(cert))
+        simple_ca('make-crl', '--ca-dir', str(vpnrouter.PKI_DIR), '--days', str(vpnrouter.CRL_DAYS))
+        publish_crl()
+    cert.unlink(missing_ok=True)
+    key.unlink(missing_ok=True)
+    connection.delete_s(dn_of(name))
+    return {'name': name, 'removed': True}
+
+
+def profile(connection, name):
+    """The OpenVPN client profile of the user, with the CA, the certificate and the key inline."""
+    entry(connection, name)
+    cfg = vpnrouter.load_config()
+    if not cfg['openvpn']['enabled']:
+        raise Failure('OpenVPN is not enabled', 2)
+    cert, key = certificate(name)
+    if not cert.is_file() or not key.is_file():
+        raise Failure(f'user {name} has no certificate')
+    fqdn = cfg['wan']['local_fqdn'].strip()
+    text = '\n'.join([
+        'client',
+        'dev tun',
+        f'proto {cfg["openvpn"]["protocol"]}',
+        f'remote {fqdn} {cfg["openvpn"]["port"]}',
+        'nobind',
+        'persist-key',
+        'persist-tun',
+        'remote-cert-tls server',
+        f'verify-x509-name {fqdn} name',
+        'auth-user-pass',
+        'auth-nocache',
+        'verb 3',
+        '<ca>', extract_pem(vpnrouter.CA_CERT_DEST.read_text(encoding='utf-8')), '</ca>',
+        '<cert>', extract_pem(cert.read_text(encoding='utf-8')), '</cert>',
+        '<key>', key.read_text(encoding='utf-8').strip(), '</key>',
+        '',
+    ])
+    return {'name': name, 'profile': text}
+
+
+def group_dn(name):
+    return f'cn={name},{vpnrouter_ldap.GROUPS}'
+
+
+def check_group_name(name):
+    if not isinstance(name, str) or not NAME.match(name):
+        raise Failure('a group name is 1 to 32 lowercase letters, digits and hyphens, starting with a letter', 2)
+
+
+def check_description(value):
+    """The description, stripped; '' means none."""
+    if not isinstance(value, str):
+        raise Failure('the description must be a string', 2)
+    value = value.strip()
+    if len(value) > DESCRIPTION_MAX or any(not character.isprintable() for character in value):
+        raise Failure(f'the description must have at most {DESCRIPTION_MAX} printable characters', 2)
+    return value
+
+
+def member_names(attributes):
+    """The user names of a group's member DNs, sorted."""
+    names = []
+    for value in attributes.get('member', []):
+        dn = value.decode()
+        if dn.startswith('uid=') and dn.endswith(f',{vpnrouter_ldap.USERS}'):
+            names.append(dn[4:-len(f',{vpnrouter_ldap.USERS}')])
+    return sorted(names)
+
+
+def group_summary(attributes):
+    """A group as the answers show it."""
+    values = attributes.get('description')
+    return {
+        'name': attributes['cn'][0].decode(),
+        'description': values[0].decode() if values else '',
+        'members': member_names(attributes),
+    }
+
+
+def group_entry(connection, name):
+    """The attributes of the group's entry; Failure with status 3 when there is none."""
+    check_group_name(name)
+    try:
+        return connection.search_s(group_dn(name), ldap.SCOPE_BASE, attrlist=GROUP_ATTRIBUTES)[0][1]
+    except ldap.NO_SUCH_OBJECT:
+        raise Failure(f'group {name} does not exist', 3)
+
+
+def groups_of(connection, name):
+    """The groups the user is a member of, as {group name: its member names}."""
+    found = connection.search_s(vpnrouter_ldap.GROUPS, ldap.SCOPE_ONELEVEL,
+                                f'(&(objectClass=groupOfNames)(member={ldap.filter.escape_filter_chars(dn_of(name))}))',
+                                GROUP_ATTRIBUTES)
+    return {attributes['cn'][0].decode(): member_names(attributes) for _, attributes in found}
+
+
+def list_groups(connection):
+    found = connection.search_s(vpnrouter_ldap.GROUPS, ldap.SCOPE_ONELEVEL, '(objectClass=groupOfNames)', GROUP_ATTRIBUTES)
+    return {'groups': sorted((group_summary(attributes) for _, attributes in found), key=lambda group: group['name'])}
+
+
+def group_add(connection, name, description, members):
+    """Add a group with a description ('' for none) and at least one member, each an existing user."""
+    check_group_name(name)
+    description = check_description(description if description is not None else '')
+    if not isinstance(members, list) or not members:
+        raise Failure('a group needs at least one member', 2)
+    for member in members:
+        entry(connection, member)
+    record = {'objectClass': [b'groupOfNames'], 'cn': [name.encode()],
+              'member': [dn_of(member).encode() for member in sorted(set(members))]}
+    if description:
+        record['description'] = [description.encode()]
+    try:
+        connection.add_s(group_dn(name), ldap.modlist.addModlist(record))
+    except ldap.ALREADY_EXISTS:
+        raise Failure(f'group {name} exists', 4)
+    return group_summary(group_entry(connection, name))
+
+
+def group_update(connection, name, new_name=None, description=None):
+    """Rename the group or change its description ('' removes it); at least one must be given."""
+    if new_name is None and description is None:
+        raise Failure('give a new name or a description', 2)
+    group_entry(connection, name)
+    if description is not None:
+        description = check_description(description)
+        change = (ldap.MOD_REPLACE, 'description', [description.encode()]) if description else \
+            (ldap.MOD_REPLACE, 'description', [])
+        connection.modify_s(group_dn(name), [change])
+    if new_name is not None and new_name != name:
+        check_group_name(new_name)
+        try:
+            connection.rename_s(group_dn(name), f'cn={new_name}', delold=1)
+        except ldap.ALREADY_EXISTS:
+            raise Failure(f'group {new_name} exists', 4)
+        name = new_name
+    return group_summary(group_entry(connection, name))
+
+
+def group_remove(connection, name):
+    group_entry(connection, name)
+    connection.delete_s(group_dn(name))
+    return {'name': name, 'removed': True}
+
+
+def group_member_add(connection, name, user):
+    attributes = group_entry(connection, name)
+    entry(connection, user)
+    if user not in member_names(attributes):
+        connection.modify_s(group_dn(name), [(ldap.MOD_ADD, 'member', [dn_of(user).encode()])])
+    return group_summary(group_entry(connection, name))
+
+
+def group_member_remove(connection, name, user):
+    """Take the user out of the group; the last member cannot leave."""
+    check_name(user)
+    members = member_names(group_entry(connection, name))
+    if user not in members:
+        raise Failure(f'user {user} is not a member of group {name}', 3)
+    if members == [user]:
+        raise Failure(f'user {user} is the last member of group {name}; remove the group instead', 2)
+    connection.modify_s(group_dn(name), [(ldap.MOD_DELETE, 'member', [dn_of(user).encode()])])
+    return group_summary(group_entry(connection, name))
+
+
+def extract_pem(text):
+    """The PEM blocks of the certificates in a file that also holds their text form."""
+    begin, end = '-----BEGIN CERTIFICATE-----', '-----END CERTIFICATE-----'
+    blocks = []
+    start = text.find(begin)
+    while start != -1:
+        stop = text.index(end, start) + len(end)
+        blocks.append(text[start:stop])
+        start = text.find(begin, stop)
+    return '\n'.join(blocks)

debian-package/src/usr/lib/vpn-router/wait-for-libpq

@@ -0,0 +1,12 @@
+#!/bin/sh
+# Wait until libpq is in the dynamic linker cache, where psycopg looks it up.
+# ExecStartPre= of vpn-router-web.service; TimeoutStartSec= of the unit bounds the wait.
+
+reported=
+until ldconfig -p | grep -q '^[[:space:]]*libpq\.so\.5 '; do
+    if [ -z "$reported" ]; then
+        echo 'waiting for libpq.so.5 in the linker cache'
+        reported=yes
+    fi
+    sleep 2
+done

debian-package/src/usr/sbin/vpn-config-edit

@@ -0,0 +1,132 @@
+#!/usr/bin/python3
+"""Show and change the router's configuration, /etc/vpn-router/vpn-router.json.
+
+show prints the file as indented JSON; the file is not changed. set takes a setting as
+section.option, such as web.port, or as site_to_site.NAME.option for a site-to-site connection,
+which it creates with the defaults (vpnrouter.CONNECTION) when it does not exist, and a value,
+which it converts to the type the package's schema (vpnrouter.SCHEMA) gives the setting: a whole
+number, true or false, a list of strings separated by commas (an empty value is an empty list),
+or a string. remove takes site_to_site.NAME and removes that connection. The configuration with
+the change is checked as vpn-router-setup checks it, and only then written with
+vpnrouter.save_config, holding vpnrouter.config_lock; vpn-router-setup applies it at its next
+run, which set and remove remind of. A failure is a message on standard error; the exit status
+is 2 for an invalid request and 1 for any other failure. -h or --help shows the options.
+
+  vpn-config-edit show
+  vpn-config-edit set SECTION.OPTION VALUE
+  vpn-config-edit set site_to_site.NAME.OPTION VALUE
+  vpn-config-edit remove site_to_site.NAME
+"""
+
+import argparse
+import copy
+import json
+import sys
+
+sys.path.insert(0, '/usr/lib/vpn-router')
+
+import vpnrouter
+
+
+class Failure(Exception):
+    """A request that cannot be done; status is 2 invalid, 1 other."""
+
+    def __init__(self, message, status=1):
+        super().__init__(message)
+        self.status = status
+
+
+class Parser(argparse.ArgumentParser):
+    """An argument parser whose errors are Failures with status 2."""
+
+    def error(self, message):
+        raise Failure(f'{message}; see {self.prog} --help', 2)
+
+
+def read():
+    """The settings in the configuration file, without the defaults; {} when the file does not exist."""
+    try:
+        return vpnrouter.read_settings()
+    except vpnrouter.ConfigError as exc:
+        raise Failure(str(exc))
+
+
+def converted(key, text):
+    """The path of the setting key, section.option or site_to_site.NAME.option, and text as the type the schema gives it."""
+    path = key.split('.')
+    if path[0] == 'site_to_site' and len(path) == 3 and path[2] in vpnrouter.CONNECTION:
+        kind = type(vpnrouter.CONNECTION[path[2]])
+    elif path[0] != 'site_to_site' and len(path) == 2 and path[0] in vpnrouter.SCHEMA and path[1] in vpnrouter.SCHEMA[path[0]]:
+        kind = type(vpnrouter.SCHEMA[path[0]][path[1]])
+    else:
+        raise Failure(f'{key}: unknown setting; settings are section.option, such as web.port, '
+                      'or site_to_site.NAME.option', 2)
+    if kind is bool:
+        if text not in ('true', 'false'):
+            raise Failure(f'{key} takes true or false', 2)
+        return path, text == 'true'
+    if kind is int:
+        try:
+            return path, int(text)
+        except ValueError:
+            raise Failure(f'{key} takes a whole number', 2)
+    if kind is list:
+        return path, [item.strip() for item in text.split(',') if item.strip()]
+    return path, text
+
+
+def main(argv):
+    try:
+        parser = Parser(prog='vpn-config-edit', description='Show and change the router configuration.')
+        commands = parser.add_subparsers(dest='command', required=True, parser_class=Parser)
+        commands.add_parser('show', help='print the configuration as indented JSON')
+        setter = commands.add_parser('set', help='set a setting and write the configuration')
+        setter.add_argument('key', metavar='SECTION.OPTION',
+                            help='the setting, such as web.port or site_to_site.NAME.option')
+        setter.add_argument('value', metavar='VALUE', help='its value; a list separated by commas')
+        remover = commands.add_parser('remove', help='remove a site-to-site connection and write the configuration')
+        remover.add_argument('key', metavar='site_to_site.NAME', help='the connection')
+        options = parser.parse_args(argv)
+
+        if options.command == 'show':
+            print(json.dumps(read(), indent=2))
+            return 0
+
+        with vpnrouter.config_lock():
+            data = read()
+            if options.command == 'set':
+                path, value = converted(options.key, options.value)
+                if path[0] == 'site_to_site':
+                    connections = data.setdefault('site_to_site', {})
+                    connection = connections.setdefault(path[1], copy.deepcopy(vpnrouter.CONNECTION))
+                    if not isinstance(connection, dict):
+                        raise Failure(f'site_to_site.{path[1]} is not an object')
+                    connection[path[2]] = value
+                else:
+                    data.setdefault(path[0], {})[path[1]] = value
+            else:
+                section, _, name = options.key.partition('.')
+                if section != 'site_to_site' or not name or name not in data.get('site_to_site', {}):
+                    raise Failure(f'{options.key}: no such site-to-site connection', 2)
+                del data['site_to_site'][name]
+            try:
+                vpnrouter.validate(vpnrouter.with_defaults(data))
+            except vpnrouter.ConfigError as exc:
+                raise Failure(str(exc), 2)
+            vpnrouter.save_config(data)
+    except Failure as failure:
+        print(f'vpn-config-edit: {failure}', file=sys.stderr)
+        return failure.status
+    except OSError as exc:
+        print(f'vpn-config-edit: {exc}', file=sys.stderr)
+        return 1
+    if options.command == 'set':
+        print(f'{options.key} set to {json.dumps(value)} in {vpnrouter.CONFIG_FILE}.')
+    else:
+        print(f'{options.key} removed from {vpnrouter.CONFIG_FILE}.')
+    print('Apply it with: systemctl restart vpn-router-setup')
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))

debian-package/src/usr/sbin/vpn-router-group

@@ -0,0 +1,120 @@
+#!/usr/bin/python3
+"""Manage the groups of the router's directory from the command line, with vpnrouter_users.
+
+A group has a name, an optional description and at least one member, each a user of the
+directory. The answer is text for people on standard output, and a failure is a message on
+standard error. With --json the answer is JSON on standard output, and a failure is
+{"error": ...} there too. The exit status says what failed: 2 the request is invalid, 3 the
+group or the user does not exist, 4 the group exists, 1 anything else. -h or --help shows the
+commands and their options.
+
+  vpn-router-group [--json] list
+  vpn-router-group [--json] add <name> --member U [--member U ...] [--description D]
+  vpn-router-group [--json] update <name> [--name N] [--description D]
+  vpn-router-group [--json] remove <name>
+  vpn-router-group [--json] member-add <name> <user>
+  vpn-router-group [--json] member-remove <name> <user>
+"""
+
+import argparse
+import json
+import sys
+
+sys.path.insert(0, '/usr/lib/vpn-router')
+
+import ldap
+
+import vpnrouter_users
+from vpnrouter_users import Failure
+
+#: The help line of each command.
+HELP = {
+    'list': 'list the groups with their members',
+    'add': 'add a group with at least one member',
+    'update': 'rename the group or change its description',
+    'remove': 'remove the group; its members are kept',
+    'member-add': 'add a user to the group',
+    'member-remove': 'take a user out of the group; the last member cannot leave',
+}
+
+
+class Parser(argparse.ArgumentParser):
+    """An argument parser whose errors are Failures with status 2, so that --json answers them as JSON."""
+
+    def error(self, message):
+        raise Failure(f'{message}; see {self.prog} --help', 2)
+
+
+def parse(argv):
+    parser = Parser(prog='vpn-router-group', description='Manage the groups of the router.')
+    parser.add_argument('--json', action='store_true', help='answer in JSON')
+    commands = parser.add_subparsers(dest='command', required=True, metavar='command', parser_class=Parser)
+    for command, text in HELP.items():
+        sub = commands.add_parser(command, help=text, description=text[0].upper() + text[1:] + '.')
+        if command != 'list':
+            sub.add_argument('name', help='the group name')
+        if command == 'add':
+            sub.add_argument('--member', dest='members', action='append', required=True, metavar='USER',
+                             help='a member; give it once for each member')
+            sub.add_argument('--description', metavar='TEXT', help='the description')
+        if command == 'update':
+            sub.add_argument('--name', dest='new_name', metavar='NAME', help='the new group name')
+            sub.add_argument('--description', metavar='TEXT', help="the description; '' removes it")
+        if command in ('member-add', 'member-remove'):
+            sub.add_argument('user', help='the user name')
+    return parser.parse_args(argv)
+
+
+def run(connection, options):
+    """The answer of the command the options name."""
+    command = options.command
+    if command == 'list':
+        return vpnrouter_users.list_groups(connection)
+    if command == 'add':
+        return vpnrouter_users.group_add(connection, options.name, options.description, options.members)
+    if command == 'update':
+        return vpnrouter_users.group_update(connection, options.name, options.new_name, options.description)
+    if command == 'remove':
+        return vpnrouter_users.group_remove(connection, options.name)
+    if command == 'member-add':
+        return vpnrouter_users.group_member_add(connection, options.name, options.user)
+    return vpnrouter_users.group_member_remove(connection, options.name, options.user)
+
+
+def text_of(answer):
+    """The answer as text for people."""
+    if answer.get('removed'):
+        return f'group {answer["name"]} removed'
+    groups = answer['groups'] if 'groups' in answer else [answer]
+    if not groups:
+        return 'no groups'
+    head = ('GROUP', 'MEMBERS', 'DESCRIPTION')
+    rows = [(group['name'], ', '.join(group['members']), group['description'] or '-') for group in groups]
+    widths = [max(len(row[column]) for row in (head, *rows)) for column in range(len(head))]
+    return '\n'.join('  '.join(cell.ljust(width) for cell, width in zip(row, widths)).rstrip() for row in (head, *rows))
+
+
+def main(argv):
+    # A request that cannot be parsed is answered in JSON when --json is among the arguments.
+    as_json = '--json' in argv
+    try:
+        options = parse(argv)
+        as_json = options.json
+        with vpnrouter_users.session() as connection:
+            answer = run(connection, options)
+    except Failure as failure:
+        message, status = str(failure), failure.status
+    except (ldap.LDAPError, OSError) as exc:
+        message, status = f'the user directory failed: {exc}', 1
+    else:
+        print(json.dumps(answer) if as_json else text_of(answer))
+        return 0
+    if as_json:
+        print(json.dumps({'error': message}))
+    else:
+        print(f'vpn-router-group: {message}', file=sys.stderr)
+    return status
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))

debian-package/src/usr/sbin/vpn-router-token

@@ -0,0 +1,79 @@
+#!/usr/bin/python3
+"""Make a token with the admin claim for the router's web service, with vpnrouter_auth.
+
+The token is signed with the router's key and is valid for the lifetime given, without a
+maximum. It may call every protected path, and it is the only token that may change the
+admins group. The answer is the token on standard output, and a failure is a message on
+standard error. With --json the answer is {"token", "expires"} on standard output, and a
+failure is {"error": ...} there too. The exit status is 2 for an invalid request and 1 for
+any other failure. -h or --help shows the options.
+
+  vpn-router-token [--json] (--minutes N | --hours N | --days N)
+"""
+
+import argparse
+import json
+import sys
+
+sys.path.insert(0, '/usr/lib/vpn-router')
+
+import vpnrouter_auth
+
+#: The seconds of each unit of the lifetime.
+UNITS = {'minutes': 60, 'hours': 3600, 'days': 86400}
+
+
+class Failure(Exception):
+    """A request that cannot be done; status is 2 invalid, 1 other."""
+
+    def __init__(self, message, status=1):
+        super().__init__(message)
+        self.status = status
+
+
+class Parser(argparse.ArgumentParser):
+    """An argument parser whose errors are Failures with status 2, so that --json answers them as JSON."""
+
+    def error(self, message):
+        raise Failure(f'{message}; see {self.prog} --help', 2)
+
+
+def positive(value):
+    try:
+        number = int(value)
+    except ValueError:
+        number = 0
+    if number < 1:
+        raise argparse.ArgumentTypeError(f'{value!r} is not a positive whole number')
+    return number
+
+
+def main(argv):
+    # A request that cannot be parsed is answered in JSON when --json is among the arguments.
+    as_json = '--json' in argv
+    try:
+        parser = Parser(prog='vpn-router-token', description='Make a token with the admin claim for the web service.')
+        parser.add_argument('--json', action='store_true', help='answer in JSON')
+        lifetime = parser.add_mutually_exclusive_group(required=True)
+        for unit in UNITS:
+            lifetime.add_argument(f'--{unit}', type=positive, metavar='N', help=f'valid for N {unit}')
+        options = parser.parse_args(argv)
+        as_json = options.json
+        seconds = next(getattr(options, unit) * factor for unit, factor in UNITS.items() if getattr(options, unit))
+        answer = vpnrouter_auth.admin_token(seconds)
+    except Failure as failure:
+        message, status = str(failure), failure.status
+    except OSError as exc:
+        message, status = str(exc), 1
+    else:
+        print(json.dumps(answer) if as_json else answer['token'])
+        return 0
+    if as_json:
+        print(json.dumps({'error': message}))
+    else:
+        print(f'vpn-router-token: {message}', file=sys.stderr)
+    return status
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))

debian-package/src/usr/sbin/vpn-router-user

@@ -0,0 +1,165 @@
+#!/usr/bin/python3
+"""Manage the users of the router's directory from the command line, with vpnrouter_users.
+
+Passwords and TOTP secrets are read from the first line of standard input, never from the
+command line. The
+answer is text for people on standard output, and a failure is a message on standard error.
+With --json the answer is JSON on standard output, and a failure is {"error": ...} there too.
+The exit status says what failed: 2 the request is invalid, 3 the user does not exist, 4 the
+user exists, 1 anything else. -h or --help shows the commands and their options.
+
+  vpn-router-user [--json] list
+  vpn-router-user [--json] add <name> --first-name F --last-name L --email E   (password on stdin)
+  vpn-router-user [--json] add <name> [--email E]   (a Manager-only account; password on stdin)
+  vpn-router-user [--json] add <name> --service [--display-name N] [--description D] [--email E]
+                                                  (a service account; password on stdin)
+  vpn-router-user [--json] update <name> [--first-name F] [--last-name L] [--email E]
+  vpn-router-user [--json] update <name> [--display-name N] [--description D] [--email E]   (a service account)
+  vpn-router-user [--json] passwd <name>       (password on stdin)
+  vpn-router-user [--json] disable <name>
+  vpn-router-user [--json] enable <name>
+  vpn-router-user [--json] unlock <name>
+  vpn-router-user [--json] remove <name>
+  vpn-router-user [--json] profile <name>
+  vpn-router-user [--json] otp-new <name>
+  vpn-router-user [--json] otp-enrol <name> <code>   (secret on stdin)
+  vpn-router-user [--json] otp-remove <name>
+"""
+
+import argparse
+import json
+import sys
+
+sys.path.insert(0, '/usr/lib/vpn-router')
+
+import ldap
+
+import vpnrouter
+import vpnrouter_auth
+import vpnrouter_users
+from vpnrouter_users import Failure
+
+#: The help line of each command.
+HELP = {
+    'list': 'list the users',
+    'add': 'add a VPN user with the three person options and issue the certificate, or without the names a '
+           'Manager-only account, or with --service a service account, each with an optional e-mail address; the '
+           'password is read from standard input',
+    'update': "change the first name, the last name or the e-mail address; a Manager-only account has only the "
+              "e-mail address, which '' removes",
+    'passwd': 'set the password, read from standard input',
+    'disable': 'lock the user out of OpenVPN',
+    'enable': 'let the user in again',
+    'unlock': 'end a lockout after failed logins',
+    'remove': 'revoke the certificate and remove the user',
+    'profile': 'print the OpenVPN client profile',
+    'otp-new': 'make a TOTP secret and its otpauth URI; nothing is stored',
+    'otp-enrol': 'store the TOTP secret read from standard input, when the code is a code of it',
+    'otp-remove': 'remove the TOTP token',
+}
+
+#: The help of the person options.
+PERSON_HELP = {'first_name': 'the first name', 'last_name': 'the last name', 'email': 'the e-mail address',
+               'display_name': "a service account's display name; '' removes it",
+               'description': "a service account's description; '' removes it"}
+
+
+class Parser(argparse.ArgumentParser):
+    """An argument parser whose errors are Failures with status 2, so that --json answers them as JSON."""
+
+    def error(self, message):
+        raise Failure(f'{message}; see {self.prog} --help', 2)
+
+
+def parse(argv):
+    parser = Parser(prog='vpn-router-user', description='Manage the users of the router.')
+    parser.add_argument('--json', action='store_true', help='answer in JSON')
+    commands = parser.add_subparsers(dest='command', required=True, metavar='command', parser_class=Parser)
+    for command, text in HELP.items():
+        sub = commands.add_parser(command, help=text, description=text[0].upper() + text[1:] + '.')
+        if command != 'list':
+            sub.add_argument('name', help='the user name')
+        if command in ('add', 'update'):
+            for key, _ in vpnrouter_users.PERSON + vpnrouter_users.LABELS:
+                sub.add_argument(f'--{key.replace("_", "-")}', dest=key, metavar='TEXT', help=PERSON_HELP[key])
+        if command == 'add':
+            sub.add_argument('--service', action='store_true', help='add a service account, which cannot sign in to the web service')
+        if command == 'otp-enrol':
+            sub.add_argument('code', help='a code of the secret from the authenticator')
+    return parser.parse_args(argv)
+
+
+def password():
+    """The password or the TOTP secret from the first line of standard input."""
+    return sys.stdin.readline().rstrip('\n')
+
+
+def run(connection, options, secret):
+    """The answer of the command the options name; secret is the password of add and passwd, the TOTP secret of otp-enrol."""
+    command = options.command
+    if command == 'list':
+        return vpnrouter_users.list_users(connection)
+    values = {key: getattr(options, key, None) for key, _ in vpnrouter_users.PERSON + vpnrouter_users.LABELS}
+    if command == 'add':
+        return vpnrouter_users.add(connection, options.name, secret, values, service=options.service)
+    if command == 'update':
+        return vpnrouter_users.update(connection, options.name, values)
+    if command == 'passwd':
+        return vpnrouter_users.passwd(connection, options.name, secret)
+    if command == 'otp-enrol':
+        return vpnrouter_users.otp_enrol(connection, options.name, secret, options.code)
+    return getattr(vpnrouter_users, command.replace('-', '_'))(connection, options.name)
+
+
+def text_of(answer):
+    """The answer as text for people."""
+    if 'profile' in answer:
+        return answer['profile'].rstrip('\n')
+    if 'secret' in answer:
+        return f'secret  {answer["secret"]}\nuri     {answer["uri"]}'
+    if answer.get('removed'):
+        return f'user {answer["name"]} removed'
+    users = answer['users'] if 'users' in answer else [answer]
+    if not users:
+        return 'no users'
+    head = ('USER', 'KIND', 'STATE', 'CERTIFICATE', 'OTP', 'FIRST NAME', 'LAST NAME', 'E-MAIL')
+    rows = [(user['name'], 'internal' if user['internal'] else 'service' if user['service'] else 'manager' if user['manager_only'] else 'vpn',
+             'disabled' if not user['enabled'] else 'locked' if user['locked'] else 'enabled',
+             'issued' if user['certificate'] else 'missing',
+             'enrolled' if user['otp'] else '-', user['first_name'] or '-', user['last_name'] or '-', user['email'] or '-')
+            for user in users]
+    widths = [max(len(row[column]) for row in (head, *rows)) for column in range(len(head))]
+    return '\n'.join('  '.join(cell.ljust(width) for cell, width in zip(row, widths)).rstrip() for row in (head, *rows))
+
+
+def main(argv):
+    # A request that cannot be parsed is answered in JSON when --json is among the arguments.
+    as_json = '--json' in argv
+    try:
+        options = parse(argv)
+        as_json = options.json
+        if options.command != 'list':
+            vpnrouter_users.check_name(options.name)
+        secret = password() if options.command in ('add', 'passwd', 'otp-enrol') else None
+        with vpnrouter_users.session() as connection:
+            answer = run(connection, options, secret)
+        if options.command in vpnrouter_auth.REVOKING:
+            vpnrouter_auth.revoke(vpnrouter.load_config(), options.name)
+    except Failure as failure:
+        message, status = str(failure), failure.status
+    except (ldap.LDAPError, OSError) as exc:
+        message, status = f'the user directory failed: {exc}', 1
+    except vpnrouter_auth.DatabaseError as exc:
+        message, status = str(exc), 1
+    else:
+        print(json.dumps(answer) if as_json else text_of(answer))
+        return 0
+    if as_json:
+        print(json.dumps({'error': message}))
+    else:
+        print(f'vpn-router-user: {message}', file=sys.stderr)
+    return status
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))

debian-package/src/usr/share/doc/vpn-router/vpn-router.json.example

@@ -8,20 +8,23 @@
     "internal": "eth1"
   },
   "wan": {
-    "local_fqdn": "router.example.com",
+    "local_fqdn": "router.example.com"
-    "local_id_mode": "fqdn"
   },
   "local": {
     "cidrs": ["10.0.0.0/24"],
     "int_addr": "10.1.1.4",
     "int_gateway_ip": "10.1.1.1"
   },
-  "remote": {
+  "site_to_site": {
-    "addrs": ["peer.example.net"],
+    "remote-site": {
-    "id": "peer.example.net",
+      "addrs": ["peer.example.net"],
-    "cidrs": ["192.168.0.0/24"],
+      "id": "peer.example.net",
-    "psk": "change-me",
+      "cidrs": ["192.168.0.0/24"],
-    "psk_file": ""
+      "local_cidrs": ["10.0.0.0/24"],
+      "local_id_mode": "fqdn",
+      "psk": "change-me",
+      "psk_file": ""
+    }
   },
   "p2s": {
     "enabled": false,
@@ -33,8 +36,43 @@
     "address": "192.168.200.1/24",
     "listen_port": 51820
   },
+  "openvpn": {
+    "enabled": false,
+    "address_pool": "172.16.1.0/24",
+    "port": 1194,
+    "protocol": "udp"
+  },
   "docker": {
     "mode": "auto",
     "exposed_ports": ["443"]
+  },
+  "web": {
+    "enabled": true,
+    "listen": "both",
+    "port": 7081,
+    "database": "",
+    "trusted_proxies": [],
+    "status_content": "basic"
+  },
+  "https": {
+    "enabled": true,
+    "email": "",
+    "web_port": 7444,
+    "manager_port": 7443
+  },
+  "metrics": {
+    "enabled": true,
+    "listen": "internal",
+    "port": 9464,
+    "https_port": 9465,
+    "service_account": "metrics",
+    "otlp_endpoint": "",
+    "otlp_username": "",
+    "otlp_password": "",
+    "keep_details": "24 hours",
+    "keep_hourly": "1 month",
+    "keep_daily": "3 months",
+    "keep_monthly": "1 year",
+    "keep_yearly": "10 years"
   }
 }

debian-package/src/usr/share/vpn-router/templates/Caddyfile.j2

@@ -0,0 +1,22 @@
+# Written by vpn-router-setup from /etc/vpn-router/vpn-router.json; edits are overwritten.
+{
+	auto_https disable_redirects
+{%- if https_email %}
+	email {{ https_email }}
+{%- endif %}
+}
+{% if web_enabled %}
+{{ local_fqdn }}:{{ https_web_port }} {
+	reverse_proxy {{ loopback }}:{{ web_port }}
+}
+{% endif %}
+{%- if manager_port %}
+{{ local_fqdn }}:{{ https_manager_port }} {
+	reverse_proxy {{ loopback }}:{{ manager_port }}
+}
+{% endif %}
+{%- if metrics_enabled %}
+{{ local_fqdn }}:{{ metrics_https_port }} {
+	reverse_proxy {{ loopback }}:{{ metrics_port }}
+}
+{% endif %}

debian-package/src/usr/share/vpn-router/templates/openvpn-ldap.conf.j2

@@ -0,0 +1,12 @@
+<LDAP>
+	URL		{{ ldap_uri }}
+	Timeout		5
+	TLSEnable	no
+	FollowReferrals	no
+</LDAP>
+
+<Authorization>
+	BaseDN		"{{ ldap_users }}"
+	SearchFilter	"(&(objectClass=inetOrgPerson)(uid=%u))"
+	RequireGroup	false
+</Authorization>

debian-package/src/usr/share/vpn-router/templates/openvpn-server.conf.j2

@@ -0,0 +1,30 @@
+port {{ openvpn_port }}
+proto {{ openvpn_protocol }}
+dev {{ openvpn_device }}
+dev-type tun
+topology subnet
+server {{ openvpn_network }} {{ openvpn_netmask }}
+
+ca {{ ca_cert }}
+cert {{ server_cert }}
+key {{ server_key }}
+crl-verify {{ crl }}
+dh none
+remote-cert-tls client
+
+plugin /usr/lib/openvpn/openvpn-auth-ldap.so {{ openvpn_ldap_conf }}
+script-security 2
+auth-user-pass-verify /usr/lib/vpn-router/openvpn-same-user via-env
+
+{% for network, netmask in openvpn_routes -%}
+push "route {{ network }} {{ netmask }}"
+{% endfor -%}
+{% for address in p2s_dns_list -%}
+push "dhcp-option DNS {{ address }}"
+{% endfor -%}
+
+keepalive 10 120
+{% if openvpn_protocol == 'udp' -%}
+explicit-exit-notify 1
+{% endif -%}
+verb 3

debian-package/src/usr/share/vpn-router/templates/remote-site.conf.j2

@@ -1,46 +0,0 @@
-connections {
-  remote-site {
-    version = 2
-    proposals = aes256-sha256-modp2048,aes256-sha1-modp1024
-    reauth_time = 28800
-    unique = replace
-
-    local_addrs  = {{ local_addrs }}
-    remote_addrs = {{ remote_addrs }}
-
-    local {
-      id = {{ local_id }}
-      auth = psk
-    }
-
-    remote {
-      id = {{ remote_id }}
-      auth = psk
-    }
-
-    children {
-      site2site {
-        mode = tunnel
-        local_ts = {{ local_ts }}
-        remote_ts = {{ remote_cidrs }}
-        esp_proposals = aes256-sha256,aes256-sha1
-        life_time = 3600
-        dpd_action = restart
-        start_action = start
-        close_action = none
-      }
-    }
-
-    rekey_time = 10800
-    dpd_delay = 10
-    dpd_timeout = 120
-  }
-}
-
-secrets {
-  ike-psk {
-    id-1 = {{ local_id }}
-    id-2 = {{ remote_id }}
-    secret = "{{ psk }}"
-  }
-}

debian-package/src/usr/share/vpn-router/templates/site-to-site.conf.j2

@@ -0,0 +1,50 @@
+connections {
+{%- for connection in connections if connection.ready %}
+  {{ connection.name }} {
+    version = 2
+    proposals = aes256-sha256-modp2048,aes256-sha1-modp1024
+    reauth_time = 28800
+    unique = replace
+
+    local_addrs  = {{ local_addrs }}
+    remote_addrs = {{ connection.remote_addrs }}
+
+    local {
+      id = {{ connection.local_id }}
+      auth = psk
+    }
+
+    remote {
+      id = {{ connection.remote_id }}
+      auth = psk
+    }
+
+    children {
+      {{ connection.name }} {
+        mode = tunnel
+        local_ts = {{ connection.local_ts }}
+        remote_ts = {{ connection.remote_cidrs }}
+        esp_proposals = aes256-sha256,aes256-sha1
+        life_time = 3600
+        dpd_action = restart
+        start_action = start
+        close_action = none
+      }
+    }
+
+    rekey_time = 10800
+    dpd_delay = 10
+    dpd_timeout = 120
+  }
+{%- endfor %}
+}
+
+secrets {
+{%- for connection in connections if connection.ready %}
+  ike-psk-{{ connection.name }} {
+    id-1 = {{ connection.local_id }}
+    id-2 = {{ connection.remote_id }}
+    secret = "{{ connection.psk }}"
+  }
+{%- endfor %}
+}

examples/azure/router.tf

@@ -15,13 +15,17 @@ locals {
     # Contents of /etc/vpn-router/vpn-router.json.
     config = {
       general = { platform = "azure", mode = "auto" }
-      wan     = { local_fqdn = var.local_fqdn, local_id_mode = var.local_id_mode }
+      wan     = { local_fqdn = var.local_fqdn }
-      local   = { cidrs = [for c in split(",", var.local_cidrs) : trimspace(c) if trimspace(c) != ""] }
+      local   = { cidrs = local.local_cidr_list }
-      remote = {
+      site_to_site = {
-        addrs = [for a in split(",", var.remote_addrs) : trimspace(a) if trimspace(a) != ""]
+        "remote-site" = {
-        id    = var.remote_id
+          addrs         = [for a in split(",", var.remote_addrs) : trimspace(a) if trimspace(a) != ""]
-        cidrs = local.remote_cidr_list
+          id            = var.remote_id
-        psk   = var.psk
+          cidrs         = local.remote_cidr_list
+          local_cidrs   = local.local_cidr_list
+          local_id_mode = var.local_id_mode
+          psk           = var.psk
+        }
       }
       p2s       = { enabled = var.p2s_enabled, address_pool = var.p2s_address_pool, ca_name = "VPN Router CA" }
       wireguard = { enabled = var.wireguard_enabled, address = var.wireguard_address, listen_port = var.wireguard_listen_port }

examples/azure/routes.tf

@@ -1,5 +1,6 @@
 locals {
   remote_cidr_list = [for c in split(",", var.remote_cidrs) : trimspace(c) if trimspace(c) != ""]
+  local_cidr_list  = [for c in split(",", var.local_cidrs) : trimspace(c) if trimspace(c) != ""]
 }
 
 resource "azurerm_route_table" "router" {

examples/gcp/router.tf

@@ -15,13 +15,17 @@ locals {
     # Contents of /etc/vpn-router/vpn-router.json.
     config = {
       general = { platform = "gcp", mode = "auto" }
-      wan     = { local_fqdn = var.local_fqdn, local_id_mode = var.local_id_mode }
+      wan     = { local_fqdn = var.local_fqdn }
-      local   = { cidrs = [for c in split(",", var.local_cidrs) : trimspace(c) if trimspace(c) != ""] }
+      local   = { cidrs = local.local_cidr_list }
-      remote = {
+      site_to_site = {
-        addrs = [for a in split(",", var.remote_addrs) : trimspace(a) if trimspace(a) != ""]
+        "remote-site" = {
-        id    = var.remote_id
+          addrs         = [for a in split(",", var.remote_addrs) : trimspace(a) if trimspace(a) != ""]
-        cidrs = local.remote_cidr_list
+          id            = var.remote_id
-        psk   = var.psk
+          cidrs         = local.remote_cidr_list
+          local_cidrs   = local.local_cidr_list
+          local_id_mode = var.local_id_mode
+          psk           = var.psk
+        }
       }
       p2s       = { enabled = var.p2s_enabled, address_pool = var.p2s_address_pool, ca_name = "VPN Router CA" }
       wireguard = { enabled = var.wireguard_enabled, address = var.wireguard_address, listen_port = var.wireguard_listen_port }

examples/gcp/routes.tf

@@ -1,5 +1,6 @@
 locals {
   remote_cidr_list = [for c in split(",", var.remote_cidrs) : trimspace(c) if trimspace(c) != ""]
+  local_cidr_list  = [for c in split(",", var.local_cidrs) : trimspace(c) if trimspace(c) != ""]
 }
 
 resource "google_compute_route" "to_remote" {

examples/install.sh

@@ -47,19 +47,22 @@ cat > /etc/vpn-router/vpn-router.json <<EOF
     "internal": "${INTERNAL_INTERFACE}"
   },
   "wan": {
-    "local_fqdn": "${LOCAL_FQDN}",
+    "local_fqdn": "${LOCAL_FQDN}"
-    "local_id_mode": "${LOCAL_ID_MODE}"
   },
   "local": {
     "cidrs": ${LOCAL_CIDRS},
     "int_addr": "${INT_ADDR}",
     "int_gateway_ip": "${INT_GATEWAY_IP}"
   },
-  "remote": {
+  "site_to_site": {
-    "addrs": ${REMOTE_ADDRS},
+    "remote-site": {
-    "id": "${REMOTE_ID}",
+      "addrs": ${REMOTE_ADDRS},
-    "cidrs": ${REMOTE_CIDRS},
+      "id": "${REMOTE_ID}",
-    "psk_file": "/etc/vpn-router/psk"
+      "cidrs": ${REMOTE_CIDRS},
+      "local_cidrs": ${LOCAL_CIDRS},
+      "local_id_mode": "${LOCAL_ID_MODE}",
+      "psk_file": "/etc/vpn-router/psk"
+    }
   },
   "p2s": {
     "enabled": ${P2S_ENABLED},

release.py

@@ -0,0 +1,88 @@
+#!/usr/bin/env python3
+"""Commit everything, tag the version of the newest changelog entry and push main with the tags.
+
+The tag is the version of the first entry of debian-package/debian/changelog, without the
+Debian revision 1: vpn-router (0.5.1-1) is tagged 0.5.1 and (0.5.1-2) 0.5.1-2. The tag
+starts the release workflow.
+"""
+
+import argparse
+import pathlib
+import shlex
+import subprocess
+import sys
+
+ROOT = pathlib.Path(__file__).resolve().parent
+CHANGELOG = ROOT / 'debian-package' / 'debian' / 'changelog'
+
+
+#: Print the git commands instead of running them; set by -n/--dry-run.
+DRY_RUN = False
+
+
+def git(*args):
+    """Run git in the repository, or print the command on a dry run; stop with git's exit status when it fails."""
+    command = ['git', *args]
+    if DRY_RUN:
+        print(shlex.join(command))
+        return
+    result = subprocess.run(command, cwd=ROOT)
+    if result.returncode != 0:
+        sys.exit(result.returncode)
+
+
+def summary():
+    """The summary line of the first changelog entry, or None when it has none.
+
+    The summary is the entry's first text line: indented by two spaces, not a "*" item,
+    with a blank line between it and the header and a blank line after it.
+    """
+    lines = CHANGELOG.read_text(encoding='utf-8').splitlines()
+    if len(lines) < 4 or lines[1].strip() or lines[3].strip():
+        return None
+    line = lines[2]
+    text = line.strip()
+    if not line.startswith('  ') or line.startswith('   ') or not text or text.startswith('*') or line.startswith(' -- '):
+        return None
+    return text
+
+
+def version():
+    """The tag of the first changelog entry: its version, without the Debian revision when that is 1."""
+    first = CHANGELOG.read_text(encoding='utf-8').splitlines()[0]
+    start, end = first.find('('), first.find(')')
+    if start == -1 or end < start:
+        sys.exit(f'{CHANGELOG}: the first line has no version: {first}')
+    full = first[start + 1:end]
+    upstream, _, revision = full.rpartition('-')
+    return upstream if upstream and revision == '1' else full
+
+
+def main():
+    global DRY_RUN
+    parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    parser.add_argument('-m', '--message', help='commit message; default: the summary line of the newest changelog entry, a line without "*" between blank lines right after its header')
+    parser.add_argument('-n', '--dry-run', action='store_true', help='print the git commands instead of running them')
+    parser.add_argument('--no-push', action='store_true', help='commit and tag, but do not push')
+    args = parser.parse_args()
+    DRY_RUN = args.dry_run
+
+    message = args.message or summary()
+    if not message:
+        sys.exit(f'no commit message: pass -m, or start the newest entry of {CHANGELOG} with a summary line '
+                 'without "*", indented by two spaces, with a blank line above and below it')
+    tag = version()
+    if subprocess.run(['git', 'rev-parse', '-q', '--verify', f'refs/tags/{tag}'], cwd=ROOT,
+                      capture_output=True).returncode == 0:
+        sys.exit(f'tag {tag} exists; add a new changelog entry first')
+
+    git('add', '-A')
+    git('commit', '-m', message)
+    git('tag', tag)
+    if not args.no_push:
+        git('push', 'origin', 'main', '--tags')
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main())