feature: /config shows the router's PostgreSQL clusters and their settings, and /postgresql their databases with their sizes.

ADR.md

@@ -27,67 +27,79 @@ port is reachable from the external interface only when it is in `exposed_ports`
 in `DOCKER-USER` are not touched, because the package owns only its own chain. Restarting Docker
 restores the rules. The rules cover IPv4 only.
 
-## ADR-2: A status web service of the standard library, bound to the internal address
+## ADR-2: A small status web service, on the interfaces web.listen chooses
 
-Status: Accepted; amended by ADR-7
+Status: Accepted
 
 Context: The router has no way to see its state without a shell. A check from the protected
-network, by a person or by a monitor, needs a page or an endpoint. The router is an
+network, by a person or by a monitor, needs a page or an endpoint, and so does the platform that
-internet-facing host, so what the service shows and where it listens matter.
+deploys the router, which reaches it on its public address. The router is an internet-facing
+host, so what the service shows and where it listens matter.
 
-Decision: The `vpn-router-web` unit runs one Python script that uses only the standard library, in
+Decision: The `vpn-router-web` unit runs one Python script on the standard library's
-the way of a small test server. It binds to the address of the internal interface and to
+`http.server`, with the package's own modules and the Python packages of the Ubuntu archive that
-`web.port`, on by default at 8080. `/` is for people, in plain text for `curl` and `wget` and as
+the package depends on, and no web framework. It listens on `web.port`, 7081 by default, on the
-a Tailwind CSS page for a browser, and `/status` is the same status as JSON. It shows the
+address of the internal interface, of the external interface or of both, as `web.listen` sets
-state of the services, the package version and the state of the site-to-site, road-warrior and
+(`internal`, `external` or `both`, the default), with a server for each address, and is on
-WireGuard features, and nothing from the configuration. `vpn-router-setup` resolves the
+unless `web.enabled` is `false` (ADR-7). `/` is for people, in plain text for `curl` and `wget`
-interface address, writes it with the port and the features to `/run/vpn-router/web.json`,
+and as a Tailwind CSS page for a browser, and `/status` is the same status as JSON. Both show as
-opens the port on the internal interface in UFW and restarts the service, so the service does
+much as `web.status_content` says, since the router is reachable from the internet and the
-not need the package's configuration code, which needs Jinja2.
+operator decides how much it tells strangers: `basic`, the default because it tells the least,
+the overall status and the Client IP, `standard` also the package versions and the number of
+active and other services, and `full` also each service and the state of the site-to-site, road-warrior, OpenVPN and
+WireGuard features. The Client IP is the address the request came from, through the trusted
+proxies, so a person can see the address the router sees. Nothing comes from the configuration.
+`/status` names its content, so clients such as manage-ui and the Router Manager show what it
+has. `vpn-router-setup` opens the
+port on both interfaces in UFW, so the choice of `web.listen` is made by the service alone, and
+writes the features and the facts it resolves to `/run/vpn-router/web.json`.
 
-Consequences: Nothing outside the internal network can reach it, since it listens on the internal
+Consequences: By default the service is reachable from the internet on the external address;
-address only and the firewall opens the port on the internal interface only. It has no
+`web.listen` set to `internal` keeps it to the internal network. It serves plain HTTP itself;
-authentication, which is acceptable because it is read only and shows no configuration. It runs
+HTTPS is Caddy's (ADR-21), so on the plain HTTP port the passwords of `/auth` and the tokens
-as root, because `swanctl` and `wg` need it, with `NoNewPrivileges`, `ProtectSystem=strict`,
+cross the internet unencrypted. A change of `web.listen`
-`ProtectHome` and `PrivateTmp`. The Tailwind style comes from a CDN, so a browser on a network
+needs no run of setup, since the firewall does not depend on it; the package's upgrade runs
-without internet shows the unstyled page, and `curl` is unaffected. A change of the internal
+setup, which writes the rules. `/` and `/status` have no authentication, which is acceptable because they are
-address is picked up at the next run of `vpn-router-setup`.
+read only and show no configuration. It runs as root, because `swanctl` and `wg` need it, with `NoNewPrivileges`,
+`ProtectSystem=strict`, `ProtectHome` and `PrivateTmp`. The Tailwind style comes from a CDN, so a
+browser on a network without internet shows the unstyled page, and `curl` is unaffected.
 
-## ADR-3: /config is authenticated with a short-lived JWT signed with a shared secret
+## ADR-3: The protected paths take JWTs that only the router signs
 
-Status: Accepted; amended by ADR-7
+Status: Accepted
 
-Context: A client such as manage-ui needs the configuration and the resolved state of the
+Context: Clients such as the Router Manager, and people with `curl`, need the configuration,
-router, which `/` and `/status` must not show. The service has only the standard library, listens
+the users and the groups of the router, which `/` and `/status` must not show. Both people and
-on plain HTTP on the internal network, and its configuration file is readable by root only.
+programs sign in with a user of the directory. A secret shared with the platform that deploys
+the router would let whoever holds it, and every copy of the deployment's state, make tokens.
+The service listens on plain HTTP.
 
-Decision: `/config` needs a JWT in an `Authorization: Bearer` header. The service verifies HS256
+Decision: `/config`, `/users` and `/groups` need a JWT in an `Authorization: Bearer` header,
-tokens with `hmac` and `hashlib`, with the secret `web.jwt_secret`, and accepts no other
+HS256, verified with `hmac` and `hashlib`; no other algorithm is accepted. The key is
-algorithm, because verifying RS256 or ES256 needs a library outside the standard library. The
+`/etc/vpn-router/web.key`, random, made by the web service on its first run and readable by root
-token must carry `exp`, not more than ten minutes ahead, so that the client makes a fresh token for
+only; it is never configured or output. The router itself issues
-each use and a leaked one expires. Without a secret the page is not served. The secrets in the
+the tokens: `POST /auth` checks a user's password and TOTP code with a bind (ADR-14) and answers
-configuration shown are masked by name (`psk`, and options with `secret`, `password`, `token` or a
+an access token naming the user in `sub`, valid for 5 minutes, and a refresh token valid for 8
-`_key` ending in the name).
+hours (ADR-15); `vpn-router-token` makes tokens with the `admin` claim (ADR-13). A token without
+`admin` must expire within ten minutes, so a leaked one is of little use. Without the key the
+paths and `/auth` are not served. The configuration shown has no secret options (ADR-8).
 
-Consequences: The client and the router share one secret, and whoever holds it can make tokens,
+Consequences: Nothing outside the router can make a token, so a token is either issued to a
-so it is long and random; it is checked to have at least 32 characters. The secret is in the
+user who signed in or made by root on the router. A client keeps the refresh token to stay
-configuration file and in `/run/vpn-router/web.json`, both readable by root only. The token and
+signed in. A token of `vpn-router-token` can be revoked only by replacing the key, which ends
-the answer cross the network unencrypted, so the page is for a trusted internal network; HTTPS
+every token. Tokens, passwords and answers cross the network unencrypted, so the API is for a
-would need a certificate that the package does not have. The masking is by option name, so a new
+trusted internal network; HTTPS would need a certificate that the package does not have.
-secret option needs a name that matches.
 
 ## ADR-4: Users are kept in a local OpenLDAP directory
 
 Status: Accepted
 
 Context: OpenVPN needs user accounts with passwords, and the users are meant to cover every P2S
-system of the router. The options weighed were an external OIDC provider (Entra ID, Keycloak)
+system of the router, including IKEv2 EAP-TLS and WireGuard, which an OIDC provider cannot
-with openvpn-auth-oauth2, an OIDC server embedded on the router, a file of password hashes, and
+authenticate. Everything the router runs comes from the Ubuntu archive. slapd, its Python module
-OpenLDAP on the router. OIDC for OpenVPN needs a browser on the client and a program from outside
+`python3-ldap` and the OpenVPN plugin `openvpn-auth-ldap` are all in the archive; the plugin
-the Ubuntu archive, and it cannot authenticate IKEv2 EAP-TLS or WireGuard. slapd, its Python
+checks a password by binding as the user, so the hash scheme is slapd's alone. An OIDC provider
-module `python3-ldap` and the OpenVPN plugin `openvpn-auth-ldap` are all in the archive; the
+can read the directory through its LDAP connector.
-plugin checks a password by binding as the user, so the hash scheme is slapd's alone. A directory
-can also be read later by an OIDC provider through its LDAP connector.
 
 Decision: The users are entries in slapd. The package keeps its own database,
 `dc=vpn-router,dc=local` in `/var/lib/ldap/vpn-router` (the path the slapd AppArmor profile
@@ -101,19 +113,18 @@ slapd listens on the local socket only. Provisioning checks every step before ch
 runs at every start of `vpn-router-setup`.
 
 Consequences: The package depends on `slapd`, `python3-ldap`, `openvpn` and
-`openvpn-auth-ldap`, and `vpn-router-setup` and `vpn-router-user` import `python3-ldap`, the first
+`openvpn-auth-ldap`, and `vpn-router-setup` and `vpn-router-user` import `python3-ldap`. The
-library outside the standard library besides `jinja2`. The service account's password is a
+service account's password is a root-only file. The directory is user data: it is kept on purge,
-root-only file. The directory is user data: it is kept when the features are switched off and
+and it is backed up with `slapcat`, not by copying files.
-on purge, and it is backed up with `slapcat`, not by copying files.
 
 ## ADR-5: OpenVPN clients need a certificate and the password of the user it names
 
 Status: Accepted
 
-Context: The router CA already issues client certificates for IKEv2. OpenVPN can check a
+Context: The router CA issues client certificates for IKEv2. OpenVPN can check a certificate, a
-certificate, a password, or both. With both, a certificate alone and a password alone are not
+password, or both. With both, a certificate alone and a password alone are not enough, but
-enough, but `openvpn-auth-ldap` checks only the password of the user name given at login, which
+`openvpn-auth-ldap` checks only the password of the user name given at login, which need not be
-need not be the user of the certificate.
+the user of the certificate.
 
 Decision: OpenVPN checks the certificate against the CA and the CRL, the password with
 `openvpn-auth-ldap`, and runs `openvpn-same-user` as `auth-user-pass-verify` with `via-env`,
@@ -130,47 +141,439 @@ checks certificates only; removing a user revokes the certificate for both. A us
 certificate expires after the CA's default of a year and is renewed by removing and adding the
 user again.
 
-## ADR-6: The web service manages users by running vpn-router-user
+## ADR-6: The web service and vpn-router-user share one module for users
 
 Status: Accepted
 
-Context: The web service has only the standard library (ADR-2), and managing users needs
+Context: Users are managed from the command line, by an operator at a terminal, and through
-`python3-ldap` and `simple-ca`. Its unit is sandboxed with `ProtectSystem=strict`.
+`/users` of the web service, by the Router Manager. Both must apply the same rules to names,
+passwords and person attributes, and both change the directory and the CA. `simple-ca` keeps
+the CA's state in one file, so two changes must not run at the same time. The web unit is
+sandboxed with `ProtectSystem=strict`.
 
-Decision: `/users` runs `vpn-router-user` and returns its JSON answer, mapping its exit status
+Decision: `vpnrouter_users.py` holds the user operations, the checks and a session that takes a
-to the HTTP status. Passwords go to the program on standard input, so they never appear in a
+lock and the connection of the user manager account. The web service imports it and maps a
-command line. The unit allows writing the paths the program needs: `/run/vpn-router` for its
+`Failure` to the HTTP status; `vpn-router-user` imports it and adds the argument parsing and
-lock, `/etc/vpn-router/pki` for the certificates and the CA's state, and `/etc/swanctl/x509crl`
+the output, text for people and JSON with `--json`. The web unit allows writing the paths the
-for the published CRL. A lock serializes the program's runs, because `simple-ca` keeps its state
+operations need: `/run/vpn-router` for the lock, its `RuntimeDirectory`, which systemd creates
-in one file.
+before the service starts, also before setup has run, and keeps when it stops, as setup's state
+files are there too; `/etc/vpn-router/pki` for the certificates and the CA's state, and
+`/etc/swanctl/x509crl` for the published CRL.
 
-Consequences: The web service still imports nothing outside the standard library, and the same
+Consequences: The rules exist once. A request starts no process, and a password never leaves
-program serves the command line and the API. Each request starts a process, which is fine for
+the process that received it. The web service imports `python3-ldap` and the package's
-user administration. The API is protected by the token of `/config`, so whoever has the secret
+configuration module. The API is protected by the tokens of ADR-3, so only members of `admins`
-can manage users.
+and the holders of an admin token manage users.
 
 ## ADR-7: The web service starts on its own and reads its inputs at each request
 
 Status: Accepted
 
-Context: The web service took the address to listen on, the port, the token secret and the
+Context: The status page is the tool to see a problem of `vpn-router-setup`, so it must run
-features from `/run/vpn-router/web.json`, written by `vpn-router-setup`, and ended with a failure
+whatever state setup is in, also before setup has run and while it fails. The addresses,
-while the file or the internal address was missing. It was ordered after `vpn-router-setup`, so a
+`web.listen`, `web.port` and `web.enabled` can change at any time.
-problem in the setup kept the status page, the tool to see that problem, from running.
 
 Decision: The service depends on nothing to start and does not end on its own. A discovery
-thread works out, every 30 seconds, the internal address from `vpn-router.json` as
+thread works out, every 30 seconds, the addresses of the interfaces `web.listen` names from
-`general.mode` directs, with the discovery code in `vpnrouter_net.py`, which uses only the
+`vpn-router.json` as `general.mode` directs, with the discovery code in `vpnrouter_net.py`,
-standard library and which `vpnrouter.py` uses too, and `web.port` and `web.enabled` from the
+which `vpnrouter.py` uses too, and `web.port` and `web.enabled` from the same file. It runs an
-same file. It runs the HTTP server in a thread of its own at that address and port, stops it and
+HTTP server in a thread of its own at each address and the port, stops one and starts a new one
-starts a new one when they change, and runs none while the service is disabled, the address is
+when they change, and runs none while the service is disabled or no address is known; an address
-not known or it cannot be bound. Each request reads the token secret from `vpn-router.json` and
+that cannot be bound is tried again the next round. Each request
-the features and resolved facts from `web.json`, which setup still writes, because they need the
+reads the signing key and `vpn-router.json`, and the features and resolved facts from
-configuration code; while that file is missing the router shows no features. The unit is not
+`web.json`, which setup writes when it applies the configuration, so the service shows what setup
+applied; while that file is missing the router shows no features. The unit is not
 ordered after `vpn-router-setup`, and setup does not restart it.
 
-Consequences: The status page answers as soon as the internal address exists, whatever state
+Consequences: The status page answers as soon as an address exists, whatever state setup is in,
-setup is in, and follows a changed address, port or `web.enabled` within 30 seconds without a
+and follows a changed address, `web.listen`, port or `web.enabled` within 30 seconds without a
-restart. `web.json` no longer holds the address, the port or the secret, so the secret is only in
+restart. A new signing key applies to the next request. The interfaces are read every 30
-`vpn-router.json`. A change of the secret applies to the next request. The interfaces are read
+seconds, which costs a few `ip` commands.
-every 30 seconds, which costs a few `ip` commands.
+
+## ADR-8: /config leaves secret options out
+
+Status: Accepted
+
+Context: A secret has no use outside the router, so a client never needs it, and showing even
+that a secret is set tells a reader more than it needs.
+
+Decision: The configuration that `vpn-router-setup` writes for the web service has no secret
+options at all. They are recognised by name: `psk`, `database`, whose connection string may hold
+a password, and options with `secret`, `password`, `token` or a `_key` ending in the name.
+
+Consequences: No secret reaches the web service's state file or any answer of the API. A client
+cannot tell whether a secret is set. A new secret option needs a name that matches.
+
+## ADR-9: The Router Manager is a separate package with a small Python server
+
+Status: Accepted
+
+Context: The router needs a web application to manage it, which may run on the router or on
+another Linux system. The router's web service issues tokens through `/auth` (ADR-3) and sends
+no CORS headers, so a browser cannot call it directly, and tokens kept in the page could be read
+by its scripts.
+
+Decision: `vpn-router-manager` is a Debian package of its own, built in the repository
+`linux-cloud-router-manager`. A React and MUI application in the style of manage-ui is built
+with Vite when the package is built and shipped as static files. A Python server, using the
+Ubuntu packages `python3-requests` and `python3-psycopg`, serves them, signs users in through the
+router's `/auth`, keeps their tokens in a server-side session in PostgreSQL behind an
+`HttpOnly` cookie, and passes every other `/api/<path>` call to the router's `/<path>` with the
+session's access token, refreshing it as needed; it holds no logic of the router's own and no
+key of it. It reads its settings from `/etc/vpn-router-manager/manager.json` at each request; on
+the router the settings not given come from `vpn-router.json`, and it listens on the internal
+address, the external address or both, as its own `listen` sets, like the status web service,
+with the same discovery thread. `vpn-router-setup` opens the manager's port on the internal and
+the external interface while the manager is installed, so the firewall stays in one place.
+
+Consequences: The router needs no Node.js; the package needs only `python3` and packages of the
+Ubuntu archive. Tokens never reach the browser, and sessions survive a restart of the manager.
+The router decides what a user may do; the manager only signs in. The application checks the
+shape of every answer of the router. The router's firewall depends on whether the manager is
+installed, and a changed manager port needs `vpn-router-setup` to run again.
+
+## ADR-10: Users and their certificates do not depend on a P2S system being enabled
+
+Status: Accepted
+
+Context: Users are prepared before a P2S system is switched on, and the Router Manager manages
+them on any configured router, with or without one.
+
+Decision: Once a platform is chosen, `vpn-router-setup` always provisions the user directory in
+slapd and the CA with its CRL; the server certificate follows once `wan.local_fqdn` is set.
+`/users` and `vpn-router-user` rely only on the directory and the CA. Only the OpenVPN client
+profile needs OpenVPN to be enabled.
+
+Consequences: slapd always holds the directory and the CA always exists on a configured router,
+even when no P2S system uses them. An operator who wants a CA of their own must place it before
+choosing the platform.
+
+## ADR-11: Groups are groupOfNames entries that never become empty
+
+Status: Accepted
+
+Context: Users are to be organised in groups. Groups record membership only; nothing on the
+router reads them. LDAP clients that read groups, such as `openvpn-auth-ldap` and the LDAP
+connectors of OIDC providers, expect `groupOfNames` with member DNs, and OpenLDAP's core schema
+requires a `groupOfNames` to have at least one `member`.
+
+Decision: A group is `cn=<name>,ou=groups,dc=vpn-router,dc=local`, a `groupOfNames` with the
+DNs of its members and an optional `description`. A group is created with at least one member,
+and taking out its last member is refused; such a group is removed instead. Removing a user
+takes it out of its groups and is refused while the user is the last member of one. The group
+operations are in `vpnrouter_users.py` with the user operations, so they share the session and
+its lock; `vpn-router-group` is a tool of its own beside `vpn-router-user`, and the web service
+serves them under `/groups`. Group names follow the rules of user names.
+
+Consequences: A group always satisfies the schema without a placeholder member, and an empty
+group cannot exist. Removing a user or a member can need another step first. A later use of
+groups, such as allowing OpenVPN only to some, can read them as they are.
+
+## ADR-12: TOTP secrets are kept with the OATH-LDAP schema of slapd's otp module
+
+Status: Accepted
+
+Context: Users are enrolled for TOTP one-time passwords, and enrolled users must give a code
+whenever they log in, to OpenVPN and to `/auth`. Ubuntu's slapd ships the `otp` module, which
+holds the OATH-LDAP schema and an overlay that checks a code appended to the password at bind
+time; both OpenVPN (`openvpn-auth-ldap`) and `/auth` check passwords with a bind. Ubuntu
+packages `python3-pyotp`. An authenticator app is enrolled from a QR code of an `otpauth://` URI.
+
+Decision: The `otp` module is loaded and its overlay is configured on the package's database,
+after ppolicy, so every bind of an enrolled user takes the password followed by a current code
+and slapd records the time step used. The TOTP token is the user entry itself: enrolment adds `oathTOTPUser` and `oathTOTPToken` to it, with
+`oathTOTPToken` naming the entry, the secret in `oathSecret` and `oathTOTPParams` naming one
+shared parameter entry, `cn=totp,ou=policies`, with the parameters authenticator apps assume
+(6 digits, HMAC-SHA1, 30 seconds). An ACL lets the user manager account set `oathSecret` and
+no one read it. Enrolment is confirmed: `otp_new` makes a secret with `pyotp` and returns it
+with its URI, storing nothing, and `otp_enrol` stores the secret it is given back only when a
+code of it verifies, so a secret is never stored that the user's app does not produce. The
+router answers the URI; the client renders the QR code.
+
+Consequences: Removing a user removes its token. One check covers OpenVPN and `/auth`, and
+an enrolled user types the code after the password in the OpenVPN client. The secret passes
+through the client between `otp_new` and `otp_enrol`, so the router keeps no pending state;
+enrolling again replaces the secret only once a code of the new one verifies.
+
+## ADR-13: Members of admins manage the router; only CLI tokens change admins
+
+Status: Accepted
+
+Context: Signing in through `/auth` (ADR-3) is open to every user of the directory, most of
+whom only use OpenVPN. Finer permissions per operation are not needed. Somebody must be able to
+manage the router before any administrator exists and after every administrator lost access,
+and administrators must not be able to grant themselves or others the administrator role over
+the API.
+
+Decision: A token naming a user in `sub` may call the protected paths only while the user is a
+member of the group `admins`, read at each request; other users get 403. `vpn-router-token`, run
+as root on the router, makes tokens with the claim `admin: true` and a lifetime given at
+creation, without a maximum, signed with the router's key; `/auth` never issues the claim. An
+admin token may call everything, and only an admin token may change `admins`: create it, rename a
+group to it, change or remove it, add or take out its members, or remove a user who is a member
+of it. The first member is added with `vpn-router-group` on the router, which acts as root.
+
+Consequences: Removing a user from `admins` takes effect at the next request, without waiting
+for the user's tokens to expire. Membership of `admins` changes only through root on the router.
+An admin token is as powerful as root over the API and is revoked only by replacing the key, so
+it is given a short lifetime unless it is meant for automation.
+
+## ADR-14: Failed logins lock the account in slapd; /auth is rate limited per client
+
+Status: Accepted
+
+Context: `/auth` is open to the internal network and checks passwords, so it invites guessing;
+OpenVPN checks passwords from the internet. Both check with a bind, so slapd sees every attempt.
+ppolicy locks an account after a number of failed binds for a fixed time; it has no growing
+lockout. A Router Manager signs all its users in from one address.
+
+Decision: The default password policy locks an account for 15 minutes after 5 failed binds
+within 15 minutes; a successful bind clears the count; `vpn-router-user unlock` and
+`POST /users/<name>/unlock` end a lockout and clear the count, and leave a disabled user disabled.
+`/auth` answers the same 401 for a wrong name, password or code and for a locked or disabled
+user. `/auth` and `/auth/refresh` allow 10 calls a minute per client address, counted in memory,
+and answer 429 with `Retry-After` beyond that. The client address is the peer, or the last
+`X-Forwarded-For` entry for a peer in `web.trusted_proxies`, by default the address the call came
+in on, which is the address a manager on the router calls from.
+
+Consequences: One lockout covers OpenVPN and `/auth`, kept by slapd alone. Someone who knows a
+user name can lock the user out for 15 minutes at a time. The rate limit restarts with the
+service. A manager elsewhere than on the router has its address added to `web.trusted_proxies`,
+or its users share one limit.
+
+## ADR-15: Refresh tokens are kept in PostgreSQL
+
+Status: Accepted
+
+Context: An access token lives 5 minutes, and a client stays signed in with a refresh token for
+8 hours; refresh tokens must be revocable when a user signs out, is disabled, changes the
+password or is removed, and must survive a restart of the service. The Router Manager keeps its
+sessions in PostgreSQL too.
+
+Decision: Refresh tokens are random strings, stored as their SHA-256 with the user name and the
+expiry in the table `refresh_tokens`, through psycopg 3 (`python3-psycopg`). The database is
+`web.database`, or by default the database and role `vpn-router` on the router, which the web
+service creates on its first run (ADR-16) with `runuser -u postgres -- psql` and a random password in
+`/etc/vpn-router/database.secret`, reached over TCP on `localhost`, as Ubuntu's `pg_hba.conf`
+takes peer authentication on the local socket. A refresh token is used once: refreshing deletes
+it and issues a new one, after checking that the user is neither disabled nor locked.
+
+Consequences: The package depends on `postgresql` and `python3-psycopg`. A stolen database holds
+no usable token. Changing a user's password, disabling, unlocking or removing the user, and
+signing out, end the user's sessions at once; an access token already issued lives out its
+5 minutes.
+
+## ADR-16: Each service creates what it needs on its first run
+
+Status: Accepted
+
+Context: The package manager configures packages in an order of its own; on a new VM the Router
+Manager was configured before the PostgreSQL server, so a database created at installation was
+missing. PostgreSQL may also start after the services at boot.
+
+Decision: The web service makes its signing key and creates the refresh tokens' database itself,
+and the Router Manager its sessions database; neither `vpn-router-setup` nor a maintainer script
+creates them. Each service does it before it listens, retrying every 15 seconds and logging each
+failure, then listens. `vpnrouter_auth` turns every failure of the database into
+`DatabaseError`, the only database exception the web service and `vpn-router-user` handle.
+
+psycopg (Ubuntu ships only its pure-Python implementation) finds libpq through the dynamic linker
+cache, which dpkg refreshes with its deferred `ldconfig` only at the end of an installation, so a
+service started during the installation cannot import it yet; a failed import cannot be repeated
+in the same process either, as psycopg caches the lookup. The services therefore import psycopg
+plainly, and their units wait instead: `ExecStartPre=wait-for-libpq`, a shell script of each
+package, waits until `ldconfig -p` lists `libpq.so.5`, within `TimeoutStartSec=5min`. The
+maintainer scripts start the units with `systemctl --no-block` (debhelper's start is turned off),
+because a start that waited would hold dpkg before its `ldconfig`.
+
+Consequences: The order of installation and of the start of PostgreSQL and of the linker cache
+does not matter; a database that cannot be created is retried until the cause is gone, and the
+reason is in the service's log. Outside an installation the wait ends at once. The web unit may
+write `/etc/vpn-router` for the key and the database password.
+
+## ADR-17: Bootstrap diagnostics run only until the first setup succeeds
+
+Status: Accepted
+
+Context: A new VM is reachable only once its setup has worked, and the reason of a failure must
+be readable from the internal network before an SSH session is possible. A service that reports
+the setup cannot be one of the services being set up: those import packages that may not load yet
+(psycopg could not find libpq before `ldconfig` ran), and they would fail before reporting
+anything. Once the router has been set up, SSH is the way to diagnose it. The more a diagnostic
+page does, the more ways it has to fail itself.
+
+Decision: Each package ships a bootstrap service of the Python standard library alone, a script
+that answers every request with the plain-text output of a fixed list of commands: the setup
+state, the services, the failed units, the addresses and the end of the logs. The router's
+listens on 17081 and the Router Manager's on 17080, beside the ports of the services, 7081 and
+7080, on every address. The units are never enabled and not started at installation;
+`vpn-router-setup` starts them while `/var/lib/vpn-router/setup.done` does not exist, and opens
+their ports on the internal interface. The first successful run writes that file, closes the ports
+and stops them; a failed run leaves them running, so the reason stays readable. Setup records its
+state, step and failure in `/run/vpn-router/setup.json` at every step; `vpnrouter.fail` ends with
+a `SystemExit` carrying its message, so setup records it.
+
+Consequences: The diagnostics need nothing the setup provides and cannot collide with the real
+services. After the first success they never run again, also not after an upgrade or a reboot.
+The page shows log lines to whoever reaches the internal network during the first setup.
+
+## ADR-18: Manager-only accounts are a second kind of user; the package can create the first admin
+
+Status: Accepted
+
+Context: The Router Manager needs a first administrator before anyone has added users, as other
+packages create a default user from their debconf answers. Such an account, and others that only
+manage the router, have no person behind them and need no VPN access. A VPN user is an
+`inetOrgPerson`, which requires a last name and a common name, and its certificate needs the
+names. The Manager is configured before the router's directory exists during an installation.
+
+Decision: A user is one of two kinds in `ou=users`, fixed when it is added: a VPN user, an
+`inetOrgPerson` with the three person attributes and a certificate, or a Manager-only account,
+an `account`, `simpleSecurityObject` and `vpnRouterAccount` (ADR-19) with the name, the password,
+an optional e-mail address for notifications, and no certificate. `add` with all three person
+attributes adds a VPN user and without the first and the last name a Manager-only account; one
+name without the rest is refused. The CLI and the API add both kinds; the Manager's dialog
+requires the names unless Manager access only is chosen, which offers the e-mail address only.
+Answers carry `manager_only`, and `update` changes only the e-mail address of a Manager-only
+account. The Manager's package asks `admin_username` and `admin_password`; with both
+it writes them to a root-only file, clears the answers, and `vpn-router-setup` creates the account
+after it provisions the directory, adds it to `admins` (creating the group when missing) and
+removes the file. Without a name nothing is created. `vpnrouter_users` knows only the directory,
+so setup can use it before PostgreSQL's driver can be loaded; `web` and `vpn-router-user` revoke
+refresh tokens after the operations of `vpnrouter_auth.REVOKING`.
+
+Consequences: A Manager-only account cannot become a VPN user, as LDAP cannot change an entry's
+structural class and its password hash cannot be copied; it is removed and added again instead.
+It has no certificate, so OpenVPN and road-warrior access refuse it. The administrator's password
+is on disk only until setup has run.
+
+## ADR-19: The package has an LDAP schema of its own under a UUID OID
+
+Status: Accepted
+
+Context: A Manager-only account (ADR-18) is an `account`, which allows no `mail`, yet notifications
+need an address for it; no standard auxiliary class allows `mail` except `extensibleObject`, which
+allows every attribute. Authorization scopes for users of both kinds may follow. An attribute or
+class of its own needs an OID, and the package has no registered arc.
+
+Decision: The package defines its schema, `cn=vpn-router,cn=schema,cn=config`, which
+`vpnrouter_ldap.provision` adds when missing and keeps as `SCHEMA_ATTRIBUTE_TYPES` and
+`SCHEMA_OBJECT_CLASSES`. Its OID is `2.25.98477352595735834532404256531620013001`, made from a
+random UUID under the arc 2.25 (ITU-T X.667), which needs no registration; attribute types are
+under `.1` and object classes under `.2`. The first version has the attribute `vpnRouterScope`
+(multi-valued, case-insensitive strings) and the auxiliary class `vpnRouterAccount`,
+`MAY ( mail $ vpnRouterScope )`. Manager-only accounts have the class from their creation, so an
+e-mail address can be added later without changing their classes; VPN users get it once scopes are
+used.
+
+Consequences: The schema check stays strict. New attributes extend `vpnRouterAccount`, and their
+OIDs continue the arcs. The scope `service` marks a service account (ADR-23). The OID must never
+be reused for anything else.
+
+## ADR-20: Users manage their own account through /account, proving who they are again
+
+Status: Accepted
+
+Context: Users who are not members of `admins` can sign in but get 403 on every path, so they
+cannot change their password or enrol an authenticator without an administrator. A session
+token alone is a weak proof for such changes: whoever holds an open session could lock the user
+out. A wrong current password must not look like an ended session to the Manager, which signs a
+user out on 401.
+
+Decision: `/account` serves the user named in the token's `sub`, admin or not: `GET` the user
+and whether they are a member of `admins`, `PUT /account/password`, and `POST` and
+`PUT /account/otp` to enrol or replace the TOTP enrolment. The routes do not take a user name.
+Each change sends the current password and, for an enrolled user, a current code, checked with a
+bind as the user like `/auth`, so it counts toward the lockout (ADR-14); a wrong one is 403. A
+new password is checked before the bind, so an invalid one does not use up a code. A user cannot
+remove their own enrolment; `/users` and `vpn-router-user` do. Tokens of `vpn-router-token` name
+no user and get 403 there.
+
+Consequences: slapd's `otp` overlay accepts each code once, so a change right after signing in
+needs the next code. Changing the password revokes the user's refresh tokens, so the user's
+Manager session ends with its access token. An enrolled user who has lost the authenticator
+needs an administrator.
+
+## ADR-21: Caddy from the Ubuntu archive serves HTTPS with a certificate from Let's Encrypt
+
+Status: Accepted
+
+Context: The status web service and the Router Manager are reachable from the internet (ADR-2),
+and `/auth` and the Manager's sign-in carry passwords. Both are small servers on the standard
+library's `http.server`, which would need certificates, their renewal and a TLS setup of their
+own. Browsers trust a public CA, not the router's.
+
+Decision: Caddy, the `caddy` package of the Ubuntu archive, is the HTTPS proxy. While
+`https.enabled` is `true` (the default) and `wan.local_fqdn` is set, `vpn-router-setup` writes
+`/etc/caddy/vpn-router.Caddyfile`: a site for `wan.local_fqdn` on `https.web_port` (7444) for the
+web service and, while the Router Manager is installed, one on `https.manager_port` (7443) for
+the manager, each passing calls to the service on `127.0.0.1`. Caddy obtains and renews the
+certificate from Let's Encrypt with the HTTP challenge on port 80, which the firewall opens on
+the external interface; `auto_https disable_redirects` keeps port 80 to the challenge. The
+package's drop-in for `caddy.service` points it at that file and keeps it from starting before
+the file exists, so `/etc/caddy/Caddyfile`, a configuration file of the `caddy` package, is left
+unchanged. Both services always listen on `127.0.0.1` as well as on the interfaces their
+`listen` settings name, and trust `X-Forwarded-For` from it, which Caddy sets from the peer and
+never takes from the client.
+
+Consequences: HTTPS needs `wan.local_fqdn` in public DNS pointing at the router and port 80
+reachable from the internet; until then Caddy retries and the HTTPS ports do not answer. The
+certificate and the ACME account are in `/var/lib/caddy`. Plain HTTP stays as the `listen`
+settings set it, so `internal` keeps passwords off the internet. Caddy's admin endpoint listens
+on `localhost:2019`, which `caddy reload` uses. Port 80 and the HTTPS ports are not available
+to Docker containers while HTTPS is on. The Manager's HTTPS port is a setting of the router, not
+of the manager.
+
+## ADR-22: Metrics are kept in PostgreSQL and served in the Prometheus format, pushed with OTLP
+
+Status: Accepted
+
+Context: The router should keep its CPU, memory, disk, network, load and VPN client metrics for
+hour, day, month and year views, as Proxmox VE does, and expose them to OpenTelemetry. Ubuntu
+noble has no OpenTelemetry Python SDK and no OpenTelemetry Collector package. OTLP itself is
+push-only; a collector reads a scraped endpoint in the Prometheus format with its `prometheus`
+receiver. The router already runs PostgreSQL for the refresh tokens (ADR-15).
+
+Decision: A service of its own, `vpn-router-metrics`, takes a sample every minute with `psutil`
+(`python3-psutil`) and the VPN readers it shares with the web service (`vpnrouter_status`). It
+keeps the samples in the router's database, in three detail tables (`metrics_system`,
+`metrics_network`, `metrics_vpn`). Each detail table has hourly, daily, monthly and yearly summary
+tables, with the average and the maximum of each value per period, written when the clock passes a
+period from the level below with `INSERT ... ON CONFLICT DO NOTHING`, so a stopped service catches
+up. Garbage collection runs on start and once a day, not with every write, and deletes the rows
+older than the retention of their level, `metrics.keep_*`, PostgreSQL intervals. The service serves
+the last sample as `/metrics` in the Prometheus text format with `prometheus_client`
+(`python3-prometheus-client`) and pushes it as OTLP/HTTP JSON, built with the standard library, to
+`metrics.otlp_endpoint`. The names follow the OpenTelemetry semantic conventions. The web service
+answers the views from the tables at `/metrics/history`, protected like `/config`, for the Router
+Manager.
+
+Consequences: The metrics live in the database of the refresh tokens and are dropped with it on
+purge. A view shows only what its level keeps, so a shorter retention empties the older part of a
+view. A month view needs the hourly rows and a year view the monthly rows. `/metrics` has only the
+last sample; history is not scraped.
+
+## ADR-23: /metrics takes HTTP Basic authentication of a service account that setup owns
+
+Status: Accepted
+
+Context: An OpenTelemetry Collector's `prometheus` receiver authenticates with HTTP Basic, not with
+the router's tokens, which last minutes (ADR-3). The scraper should not be able to sign in to the
+API or the Manager, and its password should be checked and locked out like any other. The account
+belongs to the package's metrics, not to the people using the router.
+
+Decision: A service account is a Manager-only account (ADR-18) with `vpnRouterScope` `service`.
+`vpn-router-setup` owns it: while metrics are enabled it adds the account `metrics.service_account`
+names, `metrics` by default, with a random password it writes to `/etc/vpn-router/metrics.secret`
+(root only), and sets a new random password, ending a lockout, when the file is missing or no
+longer holds the password. `vpn-router-user` and `/users` cannot add one. `/auth` refuses it
+with the message of a wrong password. `/metrics` accepts HTTP Basic only for that account, and
+only when it is a service account; the password is checked with a bind as the account, so the
+password policy's lockout applies (ADR-14). TLS is Caddy's, on `metrics.https_port` (ADR-21);
+plain HTTP is served on the interfaces `metrics.listen` names, `internal` by default.
+
+Consequences: Every scrape binds to slapd once. The collector's operator copies the password from
+the router; deleting the file and running setup changes it. A user who already has the name is not
+taken over; setup fails until `metrics.service_account` names another. A disabled account stays
+disabled. A scraper on the external interface over plain HTTP sends the password in the clear, so
+`metrics.listen` stays `internal` unless HTTPS is used.

README.md

@@ -9,6 +9,8 @@ A Debian package that configures a Linux host or virtual machine as a VPN router
 - optional WireGuard endpoint
 - routing and NAT for a protected subnet, TCP MSS clamping, and UFW firewall rules
 - firewall rules for the ports Docker publishes, when Docker is installed
+- metrics of the router kept in PostgreSQL, served for an OpenTelemetry Collector and pushed
+  over OTLP
 
 Nothing in the package is specific to a cloud provider. It is built on the external and
 internal NIC names, and a `mode` setting says how much of the rest you are supplying and
@@ -85,8 +87,10 @@ that deploy a router on Azure or GCP.
 ## Configuration
 
 `/etc/vpn-router/vpn-router.json` is JSON, mode 0600, and is created on first install. It
-is managed with `ucf`, so it is yours to edit: a later `dpkg-reconfigure` applies what
+is managed with `ucf`, so it is yours to edit: a later `dpkg-reconfigure` applies the answers
-changed and asks before touching anything you altered by hand. A complete example is
+given and asks before touching anything you altered by hand, and an upgrade asks no questions
+and leaves the file as it is. Settings the file leaves out take their defaults when it is read, so a
+file written by cloud-init or by hand needs only the settings that differ. A complete example is
 installed at `/usr/share/doc/vpn-router/vpn-router.json.example`.
 
 The file is an object of sections, each an object of settings. A section or setting that
@@ -121,9 +125,29 @@ must have the type shown below; an unknown section or setting is an error.
 | `openvpn` | `protocol` | string | `udp` | `udp` or `tcp` |
 | `docker` | `mode` | string | `auto` | `auto` secures Docker when its service is installed, `on` requires it, `off` removes the rules |
 | `docker` | `exposed_ports` | list | | TCP ports and port ranges (`8000:8100`) reachable from the external interface; at most 15, a range counting as two |
-| `web` | `enabled` | boolean | `true` | The status web service on the internal address |
+| `web` | `enabled` | boolean | `true` | The status web service |
-| `web` | `port` | integer | `8080` | TCP port of the status web service |
+| `web` | `listen` | string | `both` | The interfaces the status web service listens on: `internal`, `external` or `both` |
-| `web` | `jwt_secret` | string | | Secret, at least 32 characters, that signs the tokens for `/config`; without it `/config` is not served |
+| `web` | `port` | integer | `7081` | TCP port of the status web service |
+| `web` | `database` | string | | A PostgreSQL connection string for the refresh tokens; without it, the `vpn-router` database on this host that the web service creates |
+| `web` | `trusted_proxies` | list | the address the call came in on | Addresses whose `X-Forwarded-For` header names the client for the `/auth` rate limit and the Client IP of the status |
+| `web` | `status_content` | string | `basic` | What `/` and `/status` show: `basic`, `standard` or `full` (see Status web service) |
+| `https` | `enabled` | boolean | `true` | Caddy serves the status web service and the Router Manager over HTTPS for `wan.local_fqdn` |
+| `https` | `email` | string | | The e-mail address of the Let's Encrypt account |
+| `https` | `web_port` | integer | `7444` | HTTPS port of the status web service |
+| `https` | `manager_port` | integer | `7443` | HTTPS port of the Router Manager |
+| `metrics` | `enabled` | boolean | `true` | The metrics service: collection, history and `/metrics` |
+| `metrics` | `listen` | string | `internal` | The interfaces `/metrics` is served on: `internal`, `external` or `both` |
+| `metrics` | `port` | integer | `9464` | TCP port of `/metrics` |
+| `metrics` | `https_port` | integer | `9465` | HTTPS port of `/metrics`, served by Caddy while `https.enabled` is true |
+| `metrics` | `service_account` | string | `metrics` | The service account `/metrics` takes, which `vpn-router-setup` adds and keeps |
+| `metrics` | `otlp_endpoint` | string | | An OTLP/HTTP URL the metrics are pushed to every minute, for example `https://collector.example.com:4318/v1/metrics` |
+| `metrics` | `otlp_username` | string | | The user name of HTTP Basic authentication at the OTLP endpoint |
+| `metrics` | `otlp_password` | string | | Its password |
+| `metrics` | `keep_details` | string | `24 hours` | How long the per-minute metrics are kept |
+| `metrics` | `keep_hourly` | string | `1 month` | How long the hourly summaries are kept |
+| `metrics` | `keep_daily` | string | `3 months` | How long the daily summaries are kept |
+| `metrics` | `keep_monthly` | string | `1 year` | How long the monthly summaries are kept |
+| `metrics` | `keep_yearly` | string | `10 years` | How long the yearly summaries are kept |
 
 A minimal site-to-site configuration in `auto` mode:
 
@@ -231,6 +255,24 @@ vpn-router-setup` applies a changed configuration while Docker runs. With `docke
 `on`, the service fails when Docker is not installed or its daemon is not running; with
 `off` it removes the rules.
 
+## Editing the configuration
+
+`vpn-config-edit`, run as root, shows and changes `/etc/vpn-router/vpn-router.json`:
+
+```sh
+vpn-config-edit show
+vpn-config-edit set web.status_content standard
+vpn-config-edit set web.port 7082
+vpn-config-edit set web.trusted_proxies 127.0.0.1,10.0.0.4
+```
+
+`show` prints the file as indented JSON and does not change it. `set` takes a setting as
+`section.option` and converts the value to the setting's type: a whole number, `true` or
+`false`, a list separated by commas (an empty value is an empty list), or a string; an unknown
+setting is refused. It checks the whole configuration as `vpn-router-setup` does, and only then
+writes the file, indented, with mode 0600, and reminds that `vpn-router-setup` applies it. A
+refused request exits with status 2, any other failure with 1.
+
 ## Applying changes
 
 ```sh
@@ -272,12 +314,12 @@ ca_cert.pem  ca_key.pem  ca_bundle.pem  <label>_cert.pem  <label>_key.pem  crl.p
 `<label>` is the first component of the router FQDN, so `router.example.com` gives
 `router_cert.pem`.
 
-If road-warrior or OpenVPN access is enabled and this directory is empty, a certificate
+Once a platform is chosen and while this directory is empty, a certificate authority and a CRL
-authority, a server certificate and a CRL valid for ten years are created there
+valid for ten years are created there automatically, whether or not road-warrior or OpenVPN
-automatically; a CA without a CRL gets one. User certificates are issued by
+access is enabled. A CA of the router's own (with `ca_key.pem`) without a CRL gets one, and
-`vpn-router-user` into `users/`. To use your own instead, place the
+without a server certificate gets one once `wan.local_fqdn` is set. User certificates are issued
-files in that layout before enabling the feature; nothing is generated or overwritten when
+by `vpn-router-user` into `users/`. To use your own CA instead, place the files in that layout
-material is already present.
+before choosing the platform; nothing present is overwritten.
 
 Either way, the package copies what each consumer needs into place: the CA into
 `/etc/swanctl/x509ca/`, the server certificate and key into `/etc/swanctl/x509/` and
@@ -308,60 +350,148 @@ certificate names:
 - the certificate is checked against the CA and the CRL (`crl-verify`, read at every
   connection);
 - the password is checked by `openvpn-auth-ldap`, which binds to the user directory as the
-  user;
+  user; a user enrolled for TOTP types the password followed by the current 6-digit code;
 - `/usr/lib/vpn-router/openvpn-same-user` refuses a login whose user name is not the common
   name of the certificate.
 
 ### The user directory
 
-While OpenVPN or road-warrior access is configured, `vpn-router-setup` keeps a directory of
+Once a platform is chosen, `vpn-router-setup` keeps a directory of users in the local slapd,
-users in the local slapd:
+whether or not OpenVPN or road-warrior access is enabled, so users can be managed before either
+is switched on:
 
 - its own database, `dc=vpn-router,dc=local` in `/var/lib/ldap/vpn-router`, next to the one the
   slapd package creates; root manages it through the local socket, with no administrator
   password;
-- `ou=users` holds one `inetOrgPerson` per user, named by `uid`;
+- `ou=users` holds one entry per user, named by `uid`: an `inetOrgPerson` for a VPN user, an
-- `cn=user-manager,ou=services` is the account that changes users; it may write `ou=users` and
+  `account` for a Manager-only account (see Managing users);
-  set passwords, never read them, and its password is in
+- the package's own schema, `cn=vpn-router,cn=schema,cn=config`, under the OID
+  `2.25.98477352595735834532404256531620013001` of the UUID arc: the auxiliary class
+  `vpnRouterAccount` with `MAY ( mail $ vpnRouterScope )`, which Manager-only and service accounts
+  have, and the attribute `vpnRouterScope`, authorization scopes, of which `service` marks a
+  service account;
+- `ou=groups` holds one `groupOfNames` per group, named by `cn`, with its members' DNs in
+  `member` and an optional `description`;
+- `cn=user-manager,ou=services` is the account that changes users and groups; it may write
+  `ou=users` and `ou=groups` and set passwords, never read them, and its password is in
   `/etc/vpn-router/ldap/user-manager.secret` (root only);
 - passwords are stored as ARGON2 hashes, and a disabled user is locked with the password policy
   (`pwdAccountLockedTime`);
+- the password policy locks an account for 15 minutes after 5 failed binds within 15 minutes,
+  whether they come from OpenVPN or from `/auth`; a successful bind clears the count, and
+  `vpn-router-user unlock` ends a lockout;
+- slapd's `otp` overlay is configured on the database; a user enrolled for TOTP is also an
+  `oathTOTPUser` and `oathTOTPToken` whose `oathTOTPToken` names the entry itself, with the
+  secret in `oathSecret`, which the user manager account may set and no one can read, and
+  `oathTOTPParams` naming `cn=totp,ou=policies` (6 digits, HMAC-SHA1, 30 second steps, one step
+  of tolerance). A bind of an enrolled user takes the password followed by a current code;
 - slapd listens on its local socket only (`SLAPD_SERVICES` in `/etc/default/slapd`).
 
-The directory is kept when OpenVPN and road-warrior access are switched off, and on purge.
+The directory is kept on purge.
 
 ### Managing users
 
-`vpn-router-user` manages the users. A user is the directory entry and a client certificate
+`vpn-router-user` manages the users. A user is a directory entry of one of three kinds, fixed when
-with the user name as its common name, in `/etc/vpn-router/pki/users`. Passwords are read from
+it is added:
-the first line of standard input. The answer is JSON.
+
+- a VPN user, an `inetOrgPerson` with a first name, a last name and an e-mail address
+  (`givenName`, `sn` and `mail`; `cn` is the first and the last name), and a client certificate
+  with the user name as its common name, in `/etc/vpn-router/pki/users`;
+- a Manager-only account, an `account`, `simpleSecurityObject` and `vpnRouterAccount` with the
+  name, the password and an optional e-mail address, for notifications: no names and no
+  certificate, so no VPN access. It can sign in to `/auth` and, as a member of `admins`, manage
+  the router. `add` without the first and the last name adds one; its names cannot be set later;
+- a service account, a Manager-only account whose `vpnRouterScope` is `service`, which
+  `vpn-router-setup` adds and keeps for the metrics service (see Metrics); it cannot sign in to
+  `/auth`, and `vpn-router-user` and `/users` cannot add one.
+
+Passwords and TOTP secrets are read from the first line of standard input. The answer is text: a
+table of the users, the client profile, a TOTP secret or a short message, and a failure is a
+message on standard error.
+`--json`, before the command, gives the answer as JSON on standard output, and a failure as
+`{"error": ...}` there too; the web service uses it. `-h` or `--help` shows the commands, and
+after a command its options.
 
 ```sh
-echo 's3cr3t-pass' | vpn-router-user add alice
+echo 's3cr3t-pass' | vpn-router-user add alice --first-name Alice --last-name Smith --email alice@example.com
+echo 'm4nager-pass' | vpn-router-user add ops
+vpn-router-user update alice --email a.smith@example.com
 vpn-router-user list
-vpn-router-user profile alice
+vpn-router-user profile alice > alice.ovpn
+vpn-router-user --json list
 echo 'n3w-pass' | vpn-router-user passwd alice
 vpn-router-user disable alice
 vpn-router-user enable alice
+vpn-router-user unlock alice
 vpn-router-user remove alice
+vpn-router-user otp-new alice
+echo 'JBSWY3DPEHPK3PXPJBSWY3DPEHPK3PXP' | vpn-router-user otp-enrol alice 123456
+vpn-router-user otp-remove alice
 ```
 
 | Command | Effect |
 | --- | --- |
-| `list` | The users, whether each is enabled and has a certificate |
+| `list` | The users with their kind, names and e-mail addresses, whether each is enabled or locked, has a certificate and is enrolled for TOTP |
-| `add <name>` | Adds the user and issues the certificate |
+| `add <name> --first-name F --last-name L --email E` | Adds a VPN user and issues the certificate |
+| `add <name> [--email E]` | Adds a Manager-only account, with the e-mail address when given; one name without the other two options is refused |
+| `update <name> [--first-name F] [--last-name L] [--email E]` | Changes the names or the e-mail address |
 | `passwd <name>` | Sets the password |
-| `disable <name>`, `enable <name>` | Locks the user out of OpenVPN, and lets the user in again |
+| `disable <name>`, `enable <name>` | Locks the user out of OpenVPN and `/auth`, and lets the user in again |
-| `remove <name>` | Revokes the certificate, renews the CRL and removes the user |
+| `unlock <name>` | Ends a lockout after failed logins and clears the failure count; a disabled user stays disabled |
+| `remove <name>` | Takes the user out of its groups, revokes the certificate, renews the CRL and removes the user; refused while the user is the last member of a group |
 | `profile <name>` | The OpenVPN client profile, with the CA, the certificate and the key inline |
+| `otp-new <name>` | A new TOTP secret and its `otpauth://` URI, issued by `wan.local_fqdn`, for an authenticator app; nothing is stored |
+| `otp-enrol <name> <code>` | Stores the secret read from standard input when the code is a current code of it, replacing a secret the user has |
+| `otp-remove <name>` | Removes the user's TOTP secret |
 
 A user name is 1 to 32 lowercase letters, digits and hyphens, starting with a letter; a
-password has 8 to 128 characters. The exit status is 0 on success, 2 for an invalid request,
+password has 8 to 128 characters; a first or last name has 1 to 64 printable characters; an
-3 when the user does not exist, 4 when the user exists and 1 for any other failure.
+e-mail address has one `@`, no spaces and a dot in its domain; a TOTP secret is 32 base32
+characters, as `otp-new` makes it, and a code 6 digits. Once enrolled, a user logs in to OpenVPN
+and `/auth` with the password followed by a current code. In the answers `manager_only` tells the
+kind, and a Manager-only account has its names as `null`; `update` changes only its e-mail
+address, and `--email ''` removes it. Changing a user's password, disabling, unlocking or removing the user also revokes its
+refresh tokens of `/auth`. The exit status is 0 on success, 2 for an invalid request, 3 when the
+user does not exist, 4 when the user exists and 1 for any other failure.
+
+When the Router Manager is installed with an administrator (its debconf questions
+`vpn-router-manager/admin_username` and `admin_password`), its package leaves the answers in
+`/etc/vpn-router-manager/admin.json` (root only). `vpn-router-setup` then creates that user as a
+Manager-only account, adds it to the group `admins`, which it creates when missing, and removes
+the file.
 
 The certificate is also accepted by road-warrior (IKEv2 EAP-TLS) access, which checks
 certificates only, so `disable` does not affect it; `remove` revokes the certificate for both.
 
+### Managing groups
+
+`vpn-router-group` manages the groups. A group has a name, with the rules of a user name, an
+optional description of at most 256 printable characters, and at least one member, each a user
+of the directory. Groups record membership only; nothing on the router reads them. The output,
+`--json`, `-h` and `--help` work as for `vpn-router-user`.
+
+```sh
+vpn-router-group add ops --member alice --member bob --description 'Operations team'
+vpn-router-group list
+vpn-router-group update ops --name netops
+vpn-router-group update netops --description ''
+vpn-router-group member-add netops carol
+vpn-router-group member-remove netops bob
+vpn-router-group remove netops
+```
+
+| Command | Effect |
+| --- | --- |
+| `list` | The groups with their members and descriptions |
+| `add <name> --member U [--member U ...] [--description D]` | Adds the group with its members |
+| `update <name> [--name N] [--description D]` | Renames the group or sets its description; `''` removes it |
+| `remove <name>` | Removes the group; its members are kept |
+| `member-add <name> <user>` | Adds the user to the group |
+| `member-remove <name> <user>` | Takes the user out of the group; refused for the last member |
+
+The exit status is 0 on success, 2 for an invalid request, 3 when the group or the user does
+not exist, 4 when the group exists and 1 for any other failure.
+
 ## Platform modules
 
 Platform-specific values do not belong in the core, so they are Python modules in
@@ -405,21 +535,61 @@ modules shipped with the package; the configuration file accepts any module inst
 
 ## Status web service
 
-The `vpn-router-web` service shows the state of the router on its internal address and
+The `vpn-router-web` service shows the state of the router on the addresses of the interfaces
-`web.port` (8080 by default). It uses only the Python standard library, serves HTTP without
+`web.listen` names, the internal, the external or both (the default), and on `127.0.0.1`, where
-authentication, and shows no configuration: no key, identity, address or subnet.
+Caddy calls it (see HTTPS), with `web.port` (7081 by default). Its pages `/` and `/status` need no authentication and show no
+configuration: no key, identity, subnet, or address but the client's. They show as much as
+`web.status_content` says, read at each request:
+
+| `status_content` | `/` and `/status` show |
+|---|---|
+| `basic` (default) | The overall status (`ok` or `degraded`), the time and the Client IP |
+| `standard` | Also the `vpn-router` version, the `vpn-router-manager` version or that it is not installed, and the number of active services and of services in any other state, without their names |
+| `full` | Also each service and its state, Docker, and the state of site-to-site, road-warrior, OpenVPN and WireGuard |
+
+The Client IP is the address the request came from: the last entry of `X-Forwarded-For` for a
+request from one of `web.trusted_proxies`, by default the address the request came in on, so
+behind Caddy or a Router Manager on the router it is the browser's. `/status` names the content in
+`content` and the Client IP in `client_ip`; `/` keeps the time and the link to `/status`.
+
+The service first makes its signing key and the refresh tokens' database, retrying every 15
+seconds, for example while PostgreSQL is not running yet, and logs each failure; then it listens.
+
+### Bootstrap diagnostics
+
+Until a run of `vpn-router-setup` has succeeded, every run starts `vpn-router-bootstrap`, which
+serves diagnostics as plain text on port 17081 of every address, and the firewall opens 17081 and
+17080 (the Router Manager's bootstrap) on the internal interface. The first successful run stops
+the bootstrap services and closes their ports, and writes `/var/lib/vpn-router/setup.done`, after
+which they never run again; a failed run leaves them running, so the reason stays readable until
+a later run succeeds. The unit is never enabled.
+
+The page, read again every 10 seconds, shows the state, the step and the failure of the last
+setup run (`/run/vpn-router/setup.json`, which setup writes at every step), the state of the
+router's services, the failed units, the addresses and routes, `cloud-init status`, and the end of
+the logs of `vpn-router-setup` and `vpn-router-web`:
+
+```bash
+curl http://10.0.0.4:17081/
+```
 
 | Page | Content |
 | --- | --- |
 | `/` | The status for people: plain text for `curl` and `wget`, a page styled with Tailwind CSS for a browser, in the browser's light or dark scheme, that reloads itself every minute (the style sheet is loaded from a CDN, so the browser needs internet access; the text is readable without it) |
 | `/status` | The same status as JSON |
-| `/config` | The configuration with its secrets masked, and what the router resolved and runs, as JSON; needs a token |
+| `/auth` | Signing in with a user's password and TOTP code, refreshing and signing out |
+| `/config` | The configuration without its secret options, and what the router resolved and runs, as JSON; needs a token |
 | `/users` | The users of OpenVPN and road-warrior access; needs a token |
+| `/groups` | The groups of users; needs a token |
+| `/account` | The account of the signed-in user: its details, its password and its TOTP enrolment; needs a user's token |
 
-The status holds the state of `vpn-router-setup`, of `strongswan` when site-to-site or road-warrior
+The status holds the state of `vpn-router-setup`, `slapd` and each PostgreSQL cluster unit
-access is configured, of `wg-quick@wg0` when WireGuard is, of `openvpn-server@vpn-router` when
+(`postgresql@<version>-<cluster>`, or `postgresql` as `no cluster` without one), of
-OpenVPN is, and of `slapd` when either OpenVPN or road-warrior access is; the number of connected
+`vpn-router-manager` when that package is installed, of `strongswan` when site-to-site or
-OpenVPN clients, from the server's status file, which OpenVPN rewrites every minute; the package version; whether
+road-warrior access is configured, of `wg-quick@wg0` when WireGuard is, and of
+`openvpn-server@vpn-router` when OpenVPN is; the number of connected
+OpenVPN clients, from the server's status file, which OpenVPN rewrites every minute; the package version and,
+as `manager_version`, the installed version of `vpn-router-manager` (`null`, shown as not installed, without it); whether
 Docker is installed, and when it is, the state of `docker.service` and of `vpn-router-docker`
 (the Docker rules), which do not change `status`; whether the
 site-to-site tunnel is established; the number of connected road-warrior (IPsec P2S) clients; and
@@ -427,8 +597,62 @@ the WireGuard peers with the age of their last handshake and their traffic. `sta
 all these services are active, and `degraded` otherwise.
 
 ```bash
-curl http://10.0.0.4:8080/
+curl http://10.0.0.4:7081/
-curl http://10.0.0.4:8080/status
+curl http://10.0.0.4:7081/status
+```
+
+### Tokens and signing in
+
+`/config`, `/users`, `/groups` and `/account` need a JWT in an `Authorization: Bearer` header, signed with
+HS256 by the key the web service makes on its first run, `/etc/vpn-router/web.key` (root only).
+The key never leaves the router; while it does not exist these paths and `/auth` are not served
+(404). A missing, expired or wrongly signed token, or one of another algorithm, gets 401 with
+`WWW-Authenticate: Bearer`. There are two kinds of tokens:
+
+- an access token of `/auth` names a user in `sub` and is valid for 5 minutes; every user may
+  call `/account`, and for the other paths the user must be a member of the group `admins`,
+  otherwise the request gets 403;
+- a token of `vpn-router-token`, run as root on the router, has the claim `admin` and is valid
+  for the lifetime given, without a maximum. It names no user, so `/account` refuses it with 403;
+  it may call everything else, and only such a token may
+  change the group `admins`: create it, rename a group to it, change or remove it, add or take
+  out its members, or remove a user who is a member of it.
+
+The first member of `admins` is added on the router:
+
+```sh
+vpn-router-group add admins --member alice --description 'Router administrators'
+vpn-router-token --hours 1
+vpn-router-token --json --days 30
+```
+
+`POST /auth` signs a user in. Request and answer bodies are JSON:
+
+| Request | Body | Answer |
+| --- | --- | --- |
+| `POST /auth` | `{"name", "password", "code"}`; `code` only for a user enrolled for TOTP | the tokens |
+| `POST /auth/refresh` | `{"refresh_token"}` | new tokens; the refresh token given is used up |
+| `POST /auth/logout` | `{"refresh_token"}` | `{"revoked": true}`, or `false` for a token not known |
+
+The tokens are `{"access_token", "token_type": "Bearer", "expires_in": 300, "refresh_token",
+"refresh_expires_in": 28800}`. A wrong name, password or code, and a locked or disabled user,
+get 401 with the same message; a refresh token is refused once its user is disabled, locked or
+removed. Changing a user's password, disabling, unlocking or removing the user revokes the
+user's refresh tokens. Five failed logins lock the account for 15 minutes (see the user
+directory). `/auth` and `/auth/refresh` take at most 10 calls a minute from one client address,
+and answer 429 with `Retry-After` beyond that; for a call from one of `web.trusted_proxies`, by
+default the address the call came in on, the client address is the last entry of
+`X-Forwarded-For`.
+
+Refresh tokens are kept, as their SHA-256, in PostgreSQL: by default in the database and role
+`vpn-router` on the router, which the web service creates on its first run with a password in
+`/etc/vpn-router/database.secret` (root only), reached over TCP on `localhost`; `web.database`
+names another database. Tokens and passwords travel over plain HTTP on the internal network.
+
+```bash
+curl -H 'Content-Type: application/json' -d '{"name": "alice", "password": "s3cr3t-pass", "code": "123456"}' \
+     http://10.0.0.4:7081/auth
+curl -H "Authorization: Bearer $TOKEN" http://10.0.0.4:7081/config
 ```
 
 ### The /config page
@@ -437,82 +661,238 @@ curl http://10.0.0.4:8080/status
 
 | Key | Content |
 | --- | --- |
-| `configuration` | `vpn-router.json` with the defaults applied and the secrets masked (`psk`, `jwt_secret`, and any option named like a password, token or key) |
+| `configuration` | `vpn-router.json` with the defaults applied, without its secret options (`psk`, `database`, and any option named like a secret, password, token or key) |
 | `network` | The external and internal interface, their addresses, the internal gateway, the local and remote IKE identities and subnets, and the IPv4 routes as `ip -j -4 route` prints them |
 | `ipsec` | `connections` and `sas`: one object per connection and per IKE SA, read from `swanctl --list-conns --pretty` and `swanctl --list-sas --pretty`, with the connection's name under `name` and swanctl's keys and string values |
 | `wireguard` | The public key and port of `wg0`, and each peer's public key, endpoint, allowed IPs, handshake age and traffic; no private or preshared key |
 | `docker` | `docker.exposed_ports` and the rules of the `VPN-ROUTER-DOCKER` chain |
+| `postgresql` | `clusters`: each PostgreSQL cluster of `pg_lsclusters` with its version, name, port, status and data directory, and for one that is online its `settings`: `server_version`, `listen_addresses`, `port`, `max_connections`, `shared_buffers`, `effective_cache_size`, `work_mem`, `password_encryption`, `ssl` and `timezone`; `null` when `pg_lsclusters` fails |
 
-The page needs a JWT in an `Authorization: Bearer` header, signed with HS256 by `web.jwt_secret`.
+### The /postgresql page
-The token must have an `exp` claim, not more than ten minutes from now (a minute of clock
-difference is allowed); `nbf` is checked when present. Other algorithms are refused, with 401 and
-`WWW-Authenticate: Bearer`. While `web.jwt_secret` is not set the page is not served (404). The
-token and the page travel over plain HTTP on the internal network, and the token can be used
-until it expires, so keep the lifetime short. A client makes a token from the secret with only
-HMAC-SHA256; with Python:
 
-```python
+`/postgresql`, with the same token as `/config`, shows the databases of the PostgreSQL clusters on the
-import base64, hashlib, hmac, json, time
+router:
 
-def b64(data):
+- **Content:** `clusters` lists each cluster with its version, name, port and status, its `databases` (name, owner, encoding and size in bytes, `pg_database_size`) and their total `size`.
-    return base64.urlsafe_b64encode(data).rstrip(b'=').decode()
+- **A cluster that is not online:** `databases` and `size` are `null`.
-
+- **Collection:** the web service reads it with `psql` as the `postgres` user at each request.
-secret = 'the web.jwt_secret value'
-head = b64(json.dumps({'alg': 'HS256', 'typ': 'JWT'}).encode())
-body = b64(json.dumps({'exp': int(time.time()) + 300}).encode())
-sign = b64(hmac.new(secret.encode(), f'{head}.{body}'.encode(), hashlib.sha256).digest())
-print(f'{head}.{body}.{sign}')
-```
-
-```bash
-curl -H "Authorization: Bearer $TOKEN" http://10.0.0.4:8080/config
-```
 
 ### The /users API
 
-`/users` manages users with the same token as `/config`, by running `vpn-router-user`. Request
+`/users` manages users with the same token as `/config`, with the module `vpn-router-user` uses
+too, `/usr/lib/vpn-router/vpnrouter_users.py`, so both apply the same rules. Request
 and answer bodies are JSON; a request body is at most 4096 bytes.
 
 | Request | Body | Answer |
 | --- | --- | --- |
-| `GET /users` | | `{"users": [{"name", "enabled", "certificate"}, ...]}` |
+| `GET /users` | | `{"users": [{"name", "manager_only", "enabled", "locked", "certificate", "otp", "first_name", "last_name", "email"}, ...]}` |
-| `POST /users` | `{"name": ..., "password": ...}` | 201 and the user |
+| `POST /users` | `{"name", "password", "first_name", "last_name", "email"}` for a VPN user; `{"name", "password", "email"?}` for a Manager-only account | 201 and the user |
+| `PUT /users/<name>` | any of `{"first_name", "last_name", "email"}`; for a Manager-only account only `{"email"}`, `""` removing it | the user |
 | `PUT /users/<name>/password` | `{"password": ...}` | the user |
 | `POST /users/<name>/disable` | | the user |
 | `POST /users/<name>/enable` | | the user |
+| `POST /users/<name>/unlock` | | the user |
 | `DELETE /users/<name>` | | `{"name", "removed": true}` |
 | `GET /users/<name>/profile` | | `{"name", "profile"}`, the OpenVPN client profile |
+| `POST /users/<name>/otp` | | `{"name", "secret", "uri"}`, a new TOTP secret; nothing is stored |
+| `PUT /users/<name>/otp` | `{"secret", "code"}` | the user, once the code matches the secret |
+| `DELETE /users/<name>/otp` | | the user |
 
 A failure is `{"error": ...}` with 400 for an invalid request, 404 for a user that does not
-exist, 409 for a user that exists and 500 otherwise. While neither OpenVPN nor road-warrior
+exist, 409 for a user that exists and 500 otherwise. `/users` works whether or not OpenVPN or
-access is configured, `/users` answers 404. Passwords and private keys cross the internal
+road-warrior access is enabled; the profile needs OpenVPN. Passwords, private keys and TOTP
-network in plain HTTP, like the token.
+secrets cross the internal network in plain HTTP, like the token.
 
 ```bash
 curl -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
-     -d '{"name": "alice", "password": "s3cr3t-pass"}' http://10.0.0.4:8080/users
+     -d '{"name": "alice", "password": "s3cr3t-pass", "first_name": "Alice", "last_name": "Smith", "email": "alice@example.com"}' \
+     http://10.0.0.4:7081/users
 ```
 
+### The /account API
+
+`/account` serves the user the access token names, whether or not the user is a member of
+`admins`, with the same module as `/users`:
+
+| Request | Body | Answer |
+| --- | --- | --- |
+| `GET /account` | | the user as `/users` shows it, and `"admin"`, whether the user is a member of `admins` |
+| `PUT /account/password` | `{"current_password", "current_code", "password"}` | the user |
+| `POST /account/otp` | | `{"name", "secret", "uri"}`, a new TOTP secret; nothing is stored |
+| `PUT /account/otp` | `{"current_password", "current_code", "secret", "code"}` | the user, once the code matches the secret |
+
+`current_code` is needed only for a user enrolled for TOTP. A wrong current password or code gets
+403 and counts as a failed login; slapd accepts each code once, so a code used to sign in is not
+accepted again, and a change right after signing in waits for the next code. A user enrols or
+replaces the enrolment; removing it is left to `/users` and `vpn-router-user`. Changing the
+password revokes the user's refresh tokens. Other failures are those of `/users`.
+
+### The /groups API
+
+`/groups` manages groups with the same token and the same module as `/users`.
+
+| Request | Body | Answer |
+| --- | --- | --- |
+| `GET /groups` | | `{"groups": [{"name", "description", "members": [...]}, ...]}` |
+| `POST /groups` | `{"name", "members": [...]}`, and `"description"` | 201 and the group |
+| `PUT /groups/<name>` | any of `{"name", "description"}`; `""` removes the description | the group |
+| `DELETE /groups/<name>` | | `{"name", "removed": true}` |
+| `POST /groups/<name>/members` | `{"user": ...}` | the group |
+| `DELETE /groups/<name>/members/<user>` | | the group |
+
+A failure is `{"error": ...}` with 400 for an invalid request, among them taking out the last
+member, 404 for a group or user that does not exist, 409 for a group that exists and 500
+otherwise. `DELETE /users/<name>` is refused with 400 while the user is the last member of a
+group.
+
 The service starts on its own, without waiting for `vpn-router-setup`, and keeps running. Every
-30 seconds it works out the internal address from `vpn-router.json` as `general.mode` directs,
+30 seconds it reads `web.listen`, `web.port` and `web.enabled` from `vpn-router.json` and works
-and reads `web.port` and `web.enabled`; it listens there, moves when the address or the port
+out the addresses of the interfaces `web.listen` names as `general.mode` directs; it listens on
-changes, and stops listening while `web.enabled` is `false` or the address is not known. Each
+each, moves when an address or the port changes, and stops listening while `web.enabled` is
-request reads the token secret from `vpn-router.json`, and the features and the facts for
+`false` or no address is known. A change of `web.listen` takes effect within 30 seconds, or when
-`/config` from `/run/vpn-router/web.json` (readable by root only), which `vpn-router-setup` writes;
+the service restarts. Each request reads the signing key and `vpn-router.json`, and the features
-before that file exists the page shows no features. `vpn-router-setup` opens `web.port` on the
+and the facts for `/config` from `/run/vpn-router/web.json` (readable by root only), which
-internal interface in the firewall.
+`vpn-router-setup` writes; before that file exists the page shows no features.
+`vpn-router-setup` opens `web.port` on the internal and the external interface in the firewall.
+
+## HTTPS
+
+Caddy, from Ubuntu's `caddy` package, serves the status web service on `https.web_port` (7444 by
+default), while the Router Manager is installed the manager on `https.manager_port` (7443), and
+while metrics are enabled the metrics service on `metrics.https_port` (9465), all for `wan.local_fqdn`, while `https.enabled` is `true` and `wan.local_fqdn` is set. It obtains
+and renews the certificate from Let's Encrypt with the HTTP challenge, so `wan.local_fqdn` must
+resolve to the router's public address and port 80 must be reachable from the internet; the
+certificate and the ACME account are kept in `/var/lib/caddy`. Port 80 answers the challenge only;
+it does not redirect to HTTPS.
+
+`vpn-router-setup` writes Caddy's configuration to `/etc/caddy/vpn-router.Caddyfile` (root, group
+`caddy`, the group Caddy runs as, mode 0640), which the
+package's drop-in `/usr/lib/systemd/system/caddy.service.d/vpn-router.conf` makes `caddy.service`
+read; `caddy.service` does not start before the file exists, and `/etc/caddy/Caddyfile` is not
+used. Setup starts Caddy, reloads it when the file changes, and stops it and removes the file when
+HTTPS is off. In the firewall it opens port 80 on the external interface and the HTTPS ports on the
+internal and the external interface. Caddy calls both services on `127.0.0.1`, where they listen
+whatever `web.listen` and the manager's `listen` say, and passes the client's address in
+`X-Forwarded-For`, which the manager trusts from `127.0.0.1` and the web service by default, as
+the address the call came in on; a `web.trusted_proxies` that is set must list `127.0.0.1`. The plain HTTP ports stay as
+`web.listen` and the manager's `listen` set them. `docker.exposed_ports` may not take port 80 or
+the HTTPS ports while `https.enabled` is `true`.
+
+```bash
+curl https://router.example.com:7444/status
+```
+
+## Metrics
+
+While `metrics.enabled` is `true`, `vpn-router-setup` runs `vpn-router-metrics`, which every minute
+takes a sample of the router with `psutil`:
+
+- the CPU utilization;
+- the memory used and in total;
+- the free and the total space of `/`;
+- the bytes received and sent per second on the external and the internal interface;
+- the 1, 5 and 15 minute load averages;
+- the VPN clients of each enabled VPN system: road-warrior (IPsec P2S) and OpenVPN clients,
+  WireGuard peers with a handshake in the last 3 minutes, and the site-to-site tunnel, 1 while it is
+  up.
+
+### History
+
+The samples are kept in the router's PostgreSQL database, the one of the refresh tokens
+(`web.database`), in three detail tables, one row per minute, and each is summarized hourly, daily,
+monthly and yearly. A summary row has the start of its period, in UTC, and the average and the
+maximum of each value:
+
+| Level | Tables | Kept |
+|---|---|---|
+| Details | `metrics_system`, `metrics_network`, `metrics_vpn` | `metrics.keep_details` |
+| Hourly | `metrics_system_hourly`, `metrics_network_hourly`, `metrics_vpn_hourly` | `metrics.keep_hourly` |
+| Daily | `metrics_system_daily`, `metrics_network_daily`, `metrics_vpn_daily` | `metrics.keep_daily` |
+| Monthly | `metrics_system_monthly`, `metrics_network_monthly`, `metrics_vpn_monthly` | `metrics.keep_monthly` |
+| Yearly | `metrics_system_yearly`, `metrics_network_yearly`, `metrics_vpn_yearly` | `metrics.keep_yearly` |
+
+How it works:
+
+- **Summaries:** a period is summarized once it is complete, from the level below. A stopped service catches up on the periods it missed, as long as their rows are still kept.
+- **Garbage collection:** runs on start and once a day. It deletes the rows older than their retention, a count and a unit such as `24 hours`, `3 months` or `10 years`.
+- **History API:** `GET /metrics/history?range=hour|day|month|year` of the status web service, protected like `/config`, answers the views of the Router Manager's Metrics tab:
+
+  | Range | Bucket | Source |
+  |---|---|---|
+  | `hour` | 1 minute | details |
+  | `day` | 10 minutes | details |
+  | `month` | 6 hours | hourly |
+  | `year` | 1 month | monthly |
+
+### Scraping
+
+`GET /metrics` of the metrics service serves the last sample in the Prometheus text format, written by `prometheus_client`.
+
+- **Metric names:** the OpenTelemetry semantic convention names in their Prometheus form:
+  - `system_cpu_utilization`;
+  - `system_memory_usage_bytes` and `system_memory_limit_bytes`;
+  - `system_filesystem_usage_bytes` and `system_filesystem_utilization`;
+  - `system_network_io_bytes_total`;
+  - `system_cpu_load_average_1m`, `_5m` and `_15m`;
+  - `vpn_router_clients`.
+- **Listening:** on the interfaces `metrics.listen` names, `internal` by default, at `metrics.port`, and on `127.0.0.1`. While HTTPS is on, Caddy serves it at `metrics.https_port` for `wan.local_fqdn`. `vpn-router-setup` opens both ports on the internal and the external interface.
+- **Authentication:** HTTP Basic as the service account `metrics.service_account` names, `metrics` by default. The password is checked with a bind to the user directory, so 5 failed attempts lock the account as for any user.
+- **The account:** `vpn-router-setup` owns it while metrics are enabled.
+  - It adds the account with a random password, which it writes to `/etc/vpn-router/metrics.secret` (root only). Give that password to the collector.
+  - When the file is missing or no longer holds the account's password, setup sets a new random password and ends any lockout. Delete the file and restart `vpn-router-setup` to change the password.
+  - A disabled account is left as it is.
+  - A user of that name who is not a service account is a setup failure.
+
+An OpenTelemetry Collector scrapes it with its `prometheus` receiver:
+
+```yaml
+receivers:
+  prometheus:
+    config:
+      scrape_configs:
+        - job_name: vpn-router
+          scrape_interval: 60s
+          scheme: https
+          basic_auth:
+            username: metrics
+            password_file: /etc/otelcol/vpn-router-metrics.secret   # a copy of /etc/vpn-router/metrics.secret
+          static_configs:
+            - targets: ['router.example.com:9465']
+```
+
+### Pushing
+
+When `metrics.otlp_endpoint` is set, every sample is also pushed there as an OTLP/HTTP JSON request.
+
+- **Content:** the semantic convention names (`system.cpu.utilization`, `system.network.io` and so on), and the resource attributes `service.name` `vpn-router` and `host.name`.
+- **Authentication:** HTTP Basic with `metrics.otlp_username` and `metrics.otlp_password` when they are set.
+- **Failures:** a failed push is logged and not repeated; the next sample is pushed a minute later.
+
+## Router Manager
+
+The `vpn-router-manager` package, built in the repository `linux-cloud-router-manager`, is a web
+application that shows the state of the router and manages its users through the status web
+service. It can be installed on the router or on another Linux system. While it is installed on
+the router, `vpn-router-setup` opens its port (`port` in `/etc/vpn-router-manager/manager.json`,
+7080 by default, HTTP; HTTPS is Caddy's, on `https.manager_port`) on the internal and the external
+interface; restart `vpn-router-setup` after changing it. The manager's `listen` chooses the
+interfaces it listens on.
 
 ## Removal
 
 `apt-get remove` stops `vpn-router-docker`, which removes the Docker rules. It terminates the
-tunnels and takes `wg-quick@wg0` and `openvpn-server@vpn-router` down, but leaves the
+tunnels and takes `wg-quick@wg0`, `openvpn-server@vpn-router`, `caddy` and `vpn-router-metrics`
+down, but leaves the
 configuration in place. `strongswan` is left enabled, since it is a shared service that
 may be in use by something else.
 
 `apt-get purge` additionally removes the generated files and the firewall rules, restoring
 `/etc/ufw/before.rules` to its original content, and hands `vpn-router.json` back to `ucf`
-before deleting it so a later reinstall starts clean. Key material and users are never
+before deleting it so a later reinstall starts clean. It also removes the web service's
-deleted: `/etc/vpn-router/pki`, `/etc/vpn-router/ldap`, the user directory in slapd, the
+signing key, the password file of the metrics service account, and the database of the refresh
+tokens and the metrics, its role and their password
+file when the package created them, and Caddy's configuration `/etc/caddy/vpn-router.Caddyfile`; the certificate in
+`/var/lib/caddy` belongs to the `caddy` package. Key material and users are never deleted: `/etc/vpn-router/pki`, `/etc/vpn-router/ldap`, the user directory in slapd, the
 WireGuard key pair and everything under `/etc/swanctl` are left alone.
 
 ## Building

debian-package/debian/changelog

@@ -1,3 +1,389 @@
+vpn-router (0.17.0-1) unstable; urgency=medium
+
+  feature: /config shows the router's PostgreSQL clusters and their settings, and /postgresql their databases with their sizes.
+
+  * /config has a postgresql section: each cluster of pg_lsclusters with
+    its version, name, port, status and data directory, and the main
+    settings of each online cluster.
+  * /postgresql, protected like /config, lists the databases of each
+    online cluster with owner, encoding and size, and the total size.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 21:00:00 +0200
+
+vpn-router (0.16.0-1) unstable; urgency=medium
+
+  feature: the router keeps its metrics in PostgreSQL, serves them for an OpenTelemetry Collector and pushes them over OTLP.
+
+  * vpn-router-metrics takes a sample every minute: CPU, memory, free
+    space of /, bytes per second on the external and the internal
+    interface, the load averages, and the VPN clients of each enabled
+    system.
+  * The samples are kept in metrics_system, metrics_network and
+    metrics_vpn, and summarized hourly, daily, monthly and yearly in
+    their _hourly, _daily, _monthly and _yearly tables; once a day the
+    rows older than metrics.keep_details, keep_hourly, keep_daily,
+    keep_monthly and keep_yearly are deleted.
+  * /metrics serves the last sample in the Prometheus format on the
+    interfaces metrics.listen names (internal by default) at
+    metrics.port, and over HTTPS by Caddy at metrics.https_port, with
+    HTTP Basic authentication of the service account
+    metrics.service_account names (metrics by default).
+  * metrics.otlp_endpoint pushes every sample as OTLP/HTTP JSON, with
+    metrics.otlp_username and metrics.otlp_password.
+  * /metrics/history of the web service answers the hour, day, month
+    and year views for the Router Manager.
+  * vpn-router-setup adds and keeps that service account, which cannot
+    sign in to /auth, with a random password in
+    /etc/vpn-router/metrics.secret, set anew when the file is missing.
+  * Depends on python3-psutil and python3-prometheus-client.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 19:00:00 +0200
+
+vpn-router (0.15.0-1) unstable; urgency=medium
+
+  feature: vpn-config-edit shows and changes the router's configuration.
+
+  * vpn-config-edit show prints /etc/vpn-router/vpn-router.json as
+    indented JSON.
+  * vpn-config-edit set SECTION.OPTION VALUE converts the value to the
+    setting's type, checks the whole configuration as vpn-router-setup
+    does, writes the file indented with mode 0600, and reminds that
+    vpn-router-setup applies it.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 18:00:00 +0200
+
+vpn-router (0.14.1-1) unstable; urgency=medium
+
+  fix: web.status_content is basic by default.
+
+  * Without web.status_content, or with a value that is not allowed, /
+    and /status show only the overall status, the time and the Client
+    IP; set it to standard or full to show more.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 17:00:00 +0200
+
+vpn-router (0.14.0-1) unstable; urgency=medium
+
+  feature: the status page shows the Client IP, and web.status_content sets how much it shows.
+
+  * / and /status show the Client IP, the address the request came from:
+    the last X-Forwarded-For entry for a request from a trusted proxy, so
+    behind Caddy or the Router Manager the browser's; /status has it as
+    client_ip.
+  * web.status_content sets what / and /status show: basic the overall
+    status, the time and the Client IP; standard also the package
+    versions and the number of active and failed services; full, the
+    default, everything as before. /status names it in content.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 16:00:00 +0200
+
+vpn-router (0.13.0-1) unstable; urgency=medium
+
+  feature: Caddy serves the status page and the Router Manager over HTTPS with a certificate from Let's Encrypt.
+
+  * The package depends on caddy. While https.enabled is true, the
+    default, and wan.local_fqdn is set, vpn-router-setup writes
+    /etc/caddy/vpn-router.Caddyfile and runs Caddy, which serves the
+    status page on https.web_port (7444) and the Router Manager, when it
+    is installed, on https.manager_port (7443) for wan.local_fqdn, and
+    obtains the certificate from Let's Encrypt with the HTTP challenge.
+  * The package's drop-in for caddy.service makes Caddy read that file
+    and keeps it from starting before the file exists. The file belongs
+    to root and the group caddy, which Caddy runs as, with mode 0640.
+  * The firewall opens port 80 on the external interface and the HTTPS
+    ports on both interfaces while HTTPS is on.
+  * The web service also listens on 127.0.0.1, where Caddy calls it.
+  * docker.exposed_ports may not take port 80 or the HTTPS ports while
+    HTTPS is on.
+  * Removing the package stops and disables caddy; purging it removes
+    /etc/caddy/vpn-router.Caddyfile.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 15:00:00 +0200
+
+vpn-router (0.12.3-1) unstable; urgency=medium
+
+  fix: the web service listens on the internal address, the external address or both.
+
+  * web.listen in vpn-router.json sets the interfaces the web service
+    listens on: internal, external or both, the default. It is read
+    every 30 seconds, so a change takes effect without running setup,
+    at the latest when vpn-router-web restarts.
+  * The firewall opens the ports of the web service and of the Router
+    Manager on the internal and the external interface.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 13:00:00 +0200
+
+vpn-router (0.12.2-1) unstable; urgency=medium
+
+  fix: the web service manages users also when it started before setup.
+
+  * vpn-router-web creates /run/vpn-router when it starts, so the user
+    operations take their lock there; started before vpn-router-setup had
+    made the directory, they failed with "Read-only file system".
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 12:00:00 +0200
+
+vpn-router (0.12.1-1) unstable; urgency=medium
+
+  fix: the status page shows whether the Router Manager runs.
+
+  * When vpn-router-manager is installed, the services show the state of
+    vpn-router-manager, and status is degraded while it is not active.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 11:00:00 +0200
+
+vpn-router (0.12.0-1) unstable; urgency=medium
+
+  feature: every user manages their own account through /account.
+
+  * /account serves the signed-in user, admin or not: GET shows the user
+    and whether they are a member of admins; PUT /account/password sets
+    the password and POST and PUT /account/otp enrol or replace the TOTP
+    enrolment, given the current password and, when enrolled, a current
+    code. A wrong one is 403 and counts as a failed login. Tokens of
+    vpn-router-token get 403 there.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 10:00:00 +0200
+
+vpn-router (0.11.0-1) unstable; urgency=medium
+
+  feature: the status page shows the Router Manager's version.
+
+  * / and /status show the installed version of vpn-router-manager, as
+    manager_version in JSON, or that it is not installed.
+  * The services show the state of each PostgreSQL cluster, as
+    postgresql@<version>-<cluster>, or postgresql: no cluster without one.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 09:00:00 +0200
+
+vpn-router (0.10.0-1) unstable; urgency=medium
+
+  feature: Manager-only accounts, and the Router Manager's first administrator.
+
+  * vpn-router-user add without the person options, and POST /users without
+    them, add a Manager-only account: a name and a password, no names, no
+    e-mail address and no certificate, so no VPN access. Giving only some of
+    the three is refused. The answers show the kind as manager_only, list
+    shows a KIND column, and update refuses a Manager-only account.
+  * vpn-router-setup creates the administrator the Router Manager's package
+    leaves in /etc/vpn-router-manager/admin.json as a Manager-only account in
+    the group admins, which it creates when missing, and removes the file.
+  * The directory has the package's own schema, with the auxiliary class
+    vpnRouterAccount and the attribute vpnRouterScope. A Manager-only account
+    can have an e-mail address: add --email, update --email ('' removes it),
+    and email in POST and PUT /users.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 08:00:00 +0200
+
+vpn-router (0.9.2-1) unstable; urgency=medium
+
+  fix: the web service starts once libpq can be found.
+
+  * The web service unit waits, with wait-for-libpq, until libpq is in the
+    linker cache before it starts the service, for at most 5 minutes; the
+    service imports psycopg at start again.
+  * The package starts the web service without waiting for it.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 06:00:00 +0200
+
+vpn-router (0.9.1-1) unstable; urgency=medium
+
+  fix: the web service waits for the PostgreSQL driver instead of failing at start.
+
+  * The web service imports psycopg as the first step of its
+    initialization, retrying every 15 seconds and logging the failure while
+    libpq cannot be found yet during an installation.
+  * vpn-router-user and vpn-router-token load without psycopg; a command
+    that needs the token database then reports that the driver cannot be
+    loaded, as it reports every failure of that database.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 05:00:00 +0200
+
+vpn-router (0.9.0-1) unstable; urgency=medium
+
+  feature: bootstrap diagnostics on port 17081 during the first setup.
+
+  * vpn-router-bootstrap serves, on port 17081, the setup state, the
+    services, the failed units, the addresses, cloud-init status and the end
+    of the logs of setup and the web service, as plain text.
+  * vpn-router-setup starts it, and the Router Manager's bootstrap on port
+    17080, until a run has succeeded, and opens both ports on the internal
+    interface; the first successful run writes
+    /var/lib/vpn-router/setup.done, closes the ports and stops them. A
+    failed run leaves them running.
+  * The web service no longer has a starting phase: it creates its key and
+    database, then listens.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 04:00:00 +0200
+
+vpn-router (0.8.0-1) unstable; urgency=medium
+
+  feature: the web service creates its key and database itself and shows its starting phase.
+
+  * vpn-router-web makes its signing key and creates the refresh tokens'
+    database on its first run, retrying every 15 seconds until PostgreSQL
+    answers; vpn-router-setup no longer creates them.
+  * Until then / and /status show the web service's phase and its failure,
+    with the state, the step and the failure of the last run of
+    vpn-router-setup, and every page answers 503.
+  * vpn-router-setup records its state, current step and failure in
+    /run/vpn-router/setup.json.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 03:00:00 +0200
+
+vpn-router (0.7.1-1) unstable; urgency=medium
+
+  fix: a failing user directory or token database no longer leaves the tunnels down.
+
+  * vpn-router-setup brings up strongSwan and WireGuard before it provisions
+    the user directory and the token database. A failure of either no
+    longer stops the other steps; setup reports it and fails at the end.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 02:00:00 +0200
+
+vpn-router (0.7.0-1) unstable; urgency=medium
+
+  feature: users sign in to the web service, and only members of admins manage the router.
+
+  * POST /auth signs a user in with the password, and the TOTP code for an
+    enrolled user, and answers a 5 minute access token and an 8 hour refresh
+    token; /auth/refresh renews them and /auth/logout revokes the refresh
+    token.
+  * /config, /users and /groups take tokens of users in the group admins,
+    and tokens of the new vpn-router-token, which has an admin claim and a
+    chosen lifetime. Only these may change the group admins.
+  * The tokens are signed with /etc/vpn-router/web.key, which
+    vpn-router-setup makes on the router; web.jwt_secret is ignored.
+  * Five failed logins lock an account for 15 minutes, for OpenVPN and
+    /auth. vpn-router-user unlock and POST /users/<name>/unlock end the
+    lockout, and the users show locked.
+  * /auth takes at most 10 calls a minute per client address; for calls from
+    web.trusted_proxies the address is read from X-Forwarded-For.
+  * Users enrolled for TOTP log in to OpenVPN with the password followed by
+    the code.
+  * Refresh tokens are kept in PostgreSQL, in the database web.database
+    names or in the vpn-router database vpn-router-setup creates. The
+    package depends on postgresql and python3-psycopg.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 01:00:00 +0200
+
+vpn-router (0.6.0-1) unstable; urgency=medium
+
+  feature: users can be enrolled for TOTP one-time passwords.
+
+  * vpn-router-user otp-new makes a TOTP secret and its otpauth URI,
+    otp-enrol stores the secret, read from standard input, given a code of
+    it, and otp-remove removes it. list and every answer show whether a user
+    is enrolled, as otp.
+  * The web service enrols with POST, PUT and DELETE /users/<name>/otp.
+  * The secret is kept in the user's entry with the OATH-LDAP schema of
+    slapd's otp module; only the user manager account may set it, and no
+    one can read it. Logins do not ask for the codes.
+  * The package depends on python3-pyotp.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 24 Oct 2026 23:30:00 +0200
+
+vpn-router (0.5.9-1) unstable; urgency=medium
+
+  fix: package upgrades no longer ask the configuration questions.
+
+  * The configuration questions are asked only on a fresh installation and
+    by dpkg-reconfigure.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 24 Oct 2026 21:00:00 +0200
+
+vpn-router (0.5.8-1) unstable; urgency=medium
+
+  fix: an upgrade keeps the configuration and asks nothing.
+
+  * postinst passes on only the debconf answers that were given. The
+    defaults of questions never asked, such as platform none, no longer
+    replace settings of vpn-router.json.
+  * Without an answer that changes it, the configuration ucf is given is
+    vpn-router.json byte for byte, so an upgrade has no conflict to ask
+    about; settings the file leaves out keep their defaults when it is read.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 24 Oct 2026 20:00:00 +0200
+
+vpn-router (0.5.7-1) unstable; urgency=medium
+
+  feature: users have a first name, a last name and an e-mail address.
+
+  * vpn-router-user add needs --first-name, --last-name and --email, kept as
+    givenName, sn and mail, with cn set to the first and the last name.
+    vpn-router-user update changes them, and list and every answer show
+    them as first_name, last_name and email.
+  * POST /users needs first_name, last_name and email, and PUT
+    /users/<name> changes them.
+  * vpn-router-user answers in text: a table of the users, the client
+    profile or a short message, with failures on standard error. --json
+    gives the JSON answers. -h and --help show the commands and their
+    options.
+  * The web service manages users with vpnrouter_users, the module
+    vpn-router-user uses, instead of running vpn-router-user for each
+    request.
+  * Groups of users, kept as groupOfNames entries in ou=groups with a
+    description and at least one member. vpn-router-group lists, adds,
+    renames, describes and removes them and adds and takes out members, and
+    the web service serves the same under /groups. Removing a user takes it
+    out of its groups and is refused while it is the last member of one.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 24 Oct 2026 19:00:00 +0200
+
+vpn-router (0.5.6-1) unstable; urgency=medium
+
+  update: the status web service listens on port 7081 by default.
+
+  * web.port defaults to 7081.
+  * vpn-router-setup opens the Router Manager's port, 7080 by default, on
+    the internal interface.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 24 Oct 2026 18:00:00 +0200
+
+vpn-router (0.5.5-1) unstable; urgency=medium
+
+  feature: users and their certificates no longer need a P2S system enabled.
+
+  * vpn-router-setup provisions the user directory, the CA and its CRL once
+    a platform is chosen, whether or not OpenVPN or road-warrior access is
+    enabled. The server certificate is issued once wan.local_fqdn is set,
+    also for an existing CA that has none.
+  * /users and vpn-router-user manage users and issue and revoke their
+    certificates without a P2S system; only the OpenVPN client profile
+    needs OpenVPN.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 24 Oct 2026 16:00:00 +0200
+
+vpn-router (0.5.4-1) unstable; urgency=medium
+
+  feature: the firewall lets the Router Manager in on the internal interface.
+
+  * While vpn-router-manager is installed, vpn-router-setup opens its port,
+    port in /etc/vpn-router-manager/manager.json or 8090, on the internal
+    interface. Purging the package removes the rule.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 24 Oct 2026 13:00:00 +0200
+
+vpn-router (0.5.3-1) unstable; urgency=medium
+
+  fix: the status always shows the state of slapd.
+
+  * /status and the status page show the state of slapd whether or not
+    OpenVPN or road-warrior access is configured, since the package always
+    runs it.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 24 Oct 2026 12:00:00 +0200
+
+vpn-router (0.5.2-1) unstable; urgency=medium
+
+  fix: /config no longer shows secret options.
+
+  * The configuration shown by /config leaves out the secret options (psk,
+    jwt_secret, and options named like a password, token or key) instead of
+    showing them masked.
+
+ -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 24 Oct 2026 11:00:00 +0200
+
 vpn-router (0.5.1-1) unstable; urgency=medium
 
   fix: the web service works in every state of the router.

debian-package/debian/config

@@ -2,6 +2,12 @@
 set -e
 . /usr/share/debconf/confmodule
 
+# An upgrade (configure with the old version) asks nothing; vpn-router.json holds the
+# configuration, and dpkg-reconfigure asks the questions again.
+if [ "$1" = "configure" ] && [ -n "$2" ]; then
+    exit 0
+fi
+
 # Every question has a default, so a non-interactive install with nothing
 # preseeded completes without prompting. The defaults are platform "none" and
 # mode "manual", which together mean: install the files, configure nothing.

debian-package/debian/control

@@ -21,18 +21,25 @@ Depends: ${misc:Depends},
          python3,
          python3-jinja2,
          python3-ldap,
+         python3-pyotp,
+         python3-psycopg,
+         python3-psutil,
+         python3-prometheus-client,
+         postgresql,
          slapd,
          openvpn,
          openvpn-auth-ldap,
          iproute2,
-         systemd-resolved
+         systemd-resolved,
+         caddy
 Description: Linux VPN router with IPSec, road-warrior, OpenVPN and WireGuard support
  Configures a Linux host or virtual machine as a VPN router: site-to-site
  IKEv2 IPSec (strongSwan swanctl), optional road-warrior access using EAP-TLS,
  optional OpenVPN with a certificate and a password per user, kept in a local
  LDAP directory, optional WireGuard, and the routing, NAT, MSS clamping and
- firewall rules a router needs. Nothing in the package is specific to a cloud
+ firewall rules a router needs. It keeps metrics of the router in PostgreSQL
- provider.
+ and serves them for an OpenTelemetry Collector. Nothing in the package is
+ specific to a cloud provider.
  .
  Settings live in /etc/vpn-router/vpn-router.json, which is created on first
  install and owned by the administrator afterwards. Editing it and restarting

debian-package/debian/install

@@ -2,14 +2,26 @@ src/etc/sysctl.d/99-vpn-router.conf                   etc/sysctl.d/
 src/usr/lib/vpn-router/vpnrouter.py                   usr/lib/vpn-router/
 src/usr/lib/vpn-router/vpnrouter_ldap.py              usr/lib/vpn-router/
 src/usr/lib/vpn-router/vpnrouter_net.py               usr/lib/vpn-router/
+src/usr/lib/vpn-router/vpnrouter_serve.py             usr/lib/vpn-router/
+src/usr/lib/vpn-router/vpnrouter_users.py             usr/lib/vpn-router/
+src/usr/lib/vpn-router/vpnrouter_auth.py              usr/lib/vpn-router/
+src/usr/lib/vpn-router/vpnrouter_metrics.py           usr/lib/vpn-router/
+src/usr/lib/vpn-router/vpnrouter_status.py            usr/lib/vpn-router/
 src/usr/lib/vpn-router/openvpn-same-user              usr/lib/vpn-router/
 src/usr/sbin/vpn-router-user                          usr/sbin/
+src/usr/sbin/vpn-router-group                         usr/sbin/
+src/usr/sbin/vpn-router-token                         usr/sbin/
+src/usr/sbin/vpn-config-edit                          usr/sbin/
 src/usr/lib/vpn-router/simple-ca                      usr/lib/vpn-router/
 src/usr/lib/vpn-router/configure                      usr/lib/vpn-router/
 src/usr/lib/vpn-router/docker-rules                   usr/lib/vpn-router/
 src/usr/lib/vpn-router/setup                          usr/lib/vpn-router/
 src/usr/lib/vpn-router/web                            usr/lib/vpn-router/
+src/usr/lib/vpn-router/metrics                        usr/lib/vpn-router/
+src/usr/lib/vpn-router/bootstrap                      usr/lib/vpn-router/
+src/usr/lib/vpn-router/wait-for-libpq                 usr/lib/vpn-router/
 src/usr/lib/vpn-router/generate-config                usr/lib/vpn-router/
 src/usr/lib/vpn-router/vpnrouter_platforms/*.py       usr/lib/vpn-router/vpnrouter_platforms/
 src/usr/share/vpn-router/templates/*                  usr/share/vpn-router/templates/
+src/usr/lib/systemd/system/caddy.service.d/vpn-router.conf usr/lib/systemd/system/caddy.service.d/
 src/usr/share/doc/vpn-router/vpn-router.json.example  usr/share/doc/vpn-router/

debian-package/debian/postinst

@@ -2,28 +2,39 @@
 set -e
 . /usr/share/debconf/confmodule
 
+# The answer to a question in ANSWER, or "" when the question has not been seen; the
+# default debconf holds for an unseen question is not an answer.
+answer() {
+    ANSWER=""
+    db_fget "$1" seen
+    if [ "$RET" = "true" ]; then
+        db_get "$1"
+        ANSWER="$RET"
+    fi
+}
+
 case "$1" in
     configure)
         # --- Read the installer's answers ---
-        db_get vpn-router/platform;         VPN_ROUTER_PLATFORM="$RET"
+        answer vpn-router/platform;         VPN_ROUTER_PLATFORM="$ANSWER"
-        db_get vpn-router/mode;             VPN_ROUTER_MODE="$RET"
+        answer vpn-router/mode;             VPN_ROUTER_MODE="$ANSWER"
-        db_get vpn-router/int_addr;         VPN_ROUTER_INT_ADDR="$RET"
+        answer vpn-router/int_addr;         VPN_ROUTER_INT_ADDR="$ANSWER"
-        db_get vpn-router/external_interface; VPN_ROUTER_EXTERNAL_INTERFACE="$RET"
+        answer vpn-router/external_interface; VPN_ROUTER_EXTERNAL_INTERFACE="$ANSWER"
-        db_get vpn-router/internal_interface; VPN_ROUTER_INTERNAL_INTERFACE="$RET"
+        answer vpn-router/internal_interface; VPN_ROUTER_INTERNAL_INTERFACE="$ANSWER"
-        db_get vpn-router/local_fqdn;       VPN_ROUTER_LOCAL_FQDN="$RET"
+        answer vpn-router/local_fqdn;       VPN_ROUTER_LOCAL_FQDN="$ANSWER"
-        db_get vpn-router/local_id_mode;    VPN_ROUTER_LOCAL_ID_MODE="$RET"
+        answer vpn-router/local_id_mode;    VPN_ROUTER_LOCAL_ID_MODE="$ANSWER"
-        db_get vpn-router/local_cidrs;      VPN_ROUTER_LOCAL_CIDRS="$RET"
+        answer vpn-router/local_cidrs;      VPN_ROUTER_LOCAL_CIDRS="$ANSWER"
-        db_get vpn-router/int_gateway_ip;   VPN_ROUTER_INT_GATEWAY_IP="$RET"
+        answer vpn-router/int_gateway_ip;   VPN_ROUTER_INT_GATEWAY_IP="$ANSWER"
-        db_get vpn-router/remote_addrs;     VPN_ROUTER_REMOTE_ADDRS="$RET"
+        answer vpn-router/remote_addrs;     VPN_ROUTER_REMOTE_ADDRS="$ANSWER"
-        db_get vpn-router/remote_id;        VPN_ROUTER_REMOTE_ID="$RET"
+        answer vpn-router/remote_id;        VPN_ROUTER_REMOTE_ID="$ANSWER"
-        db_get vpn-router/remote_cidrs;     VPN_ROUTER_REMOTE_CIDRS="$RET"
+        answer vpn-router/remote_cidrs;     VPN_ROUTER_REMOTE_CIDRS="$ANSWER"
-        db_get vpn-router/psk;              VPN_ROUTER_PSK="$RET"
+        answer vpn-router/psk;              VPN_ROUTER_PSK="$ANSWER"
-        db_get vpn-router/p2s_enabled;      VPN_ROUTER_P2S_ENABLED="$RET"
+        answer vpn-router/p2s_enabled;      VPN_ROUTER_P2S_ENABLED="$ANSWER"
-        db_get vpn-router/p2s_address_pool; VPN_ROUTER_P2S_ADDRESS_POOL="$RET"
+        answer vpn-router/p2s_address_pool; VPN_ROUTER_P2S_ADDRESS_POOL="$ANSWER"
-        db_get vpn-router/p2s_ca_name;      VPN_ROUTER_P2S_CA_NAME="$RET"
+        answer vpn-router/p2s_ca_name;      VPN_ROUTER_P2S_CA_NAME="$ANSWER"
-        db_get vpn-router/wg_enabled;       VPN_ROUTER_WG_ENABLED="$RET"
+        answer vpn-router/wg_enabled;       VPN_ROUTER_WG_ENABLED="$ANSWER"
-        db_get vpn-router/wg_address;       VPN_ROUTER_WG_ADDRESS="$RET"
+        answer vpn-router/wg_address;       VPN_ROUTER_WG_ADDRESS="$ANSWER"
-        db_get vpn-router/wg_listen_port;   VPN_ROUTER_WG_LISTEN_PORT="$RET"
+        answer vpn-router/wg_listen_port;   VPN_ROUTER_WG_LISTEN_PORT="$ANSWER"
 
         export VPN_ROUTER_PLATFORM VPN_ROUTER_MODE VPN_ROUTER_INT_ADDR \
                VPN_ROUTER_EXTERNAL_INTERFACE \
@@ -59,6 +70,12 @@ case "$1" in
         # The firewall is not touched here. Enabling it, allowing SSH and
         # setting the forward policy all belong to vpn-router-setup, which does
         # them only once the configuration says this host is a router.
+
+        # The web service waits for libpq in the linker cache, which dpkg
+        # refreshes after this script, so it is started without waiting for it.
+        if [ -d /run/systemd/system ]; then
+            systemctl --no-block restart vpn-router-web.service >/dev/null 2>&1 || true
+        fi
         ;;
 esac
 

debian-package/debian/postrm

@@ -11,10 +11,10 @@ strip_ufw_blocks() {
 
     tmp="$(mktemp)"
     awk '
-        /^# (IPSEC|WIREGUARD|OPENVPN|P2S DNS|WEB|ROUTER FORWARD|ROUTER NAT|ROUTER MSS) RULES START$/ {
+        /^# (IPSEC|WIREGUARD|OPENVPN|P2S DNS|WEB|BOOTSTRAP|MANAGER|HTTPS|METRICS|ROUTER FORWARD|ROUTER NAT|ROUTER MSS) RULES START$/ {
             skip = 1; pending = 0; next
         }
-        /^# (IPSEC|WIREGUARD|OPENVPN|P2S DNS|WEB|ROUTER FORWARD|ROUTER NAT|ROUTER MSS) RULES END$/ {
+        /^# (IPSEC|WIREGUARD|OPENVPN|P2S DNS|WEB|BOOTSTRAP|MANAGER|HTTPS|METRICS|ROUTER FORWARD|ROUTER NAT|ROUTER MSS) RULES END$/ {
             skip = 0; next
         }
         skip { next }
@@ -49,7 +49,19 @@ case "$1" in
               /etc/wireguard/wg0.conf \
               /etc/openvpn/server/vpn-router.conf \
               /etc/openvpn/server/vpn-router-ldap.conf \
-              /etc/vpn-router/vpn-router.json
+              /etc/caddy/vpn-router.Caddyfile \
+              /etc/vpn-router/vpn-router.json \
+              /etc/vpn-router/web.key \
+              /etc/vpn-router/metrics.secret \
+              /var/lib/vpn-router/setup.done
+        rmdir --ignore-fail-on-non-empty /var/lib/vpn-router 2>/dev/null || true
+
+        # The refresh tokens' database and role, when the package created them.
+        if [ -f /etc/vpn-router/database.secret ] && command -v psql >/dev/null 2>&1; then
+            runuser -u postgres -- psql -X -q -c 'DROP DATABASE IF EXISTS "vpn-router"' || true
+            runuser -u postgres -- psql -X -q -c 'DROP ROLE IF EXISTS "vpn-router"' || true
+        fi
+        rm -f /etc/vpn-router/database.secret
 
         # Key material and users are not package state: /etc/vpn-router/pki,
         # /etc/vpn-router/ldap, the user directory in slapd, the WireGuard key

debian-package/debian/prerm

@@ -11,10 +11,11 @@ case "$1" in
             swanctl --terminate --ike road-warrior >/dev/null 2>&1 || true
         fi
 
-        # This package enabled wg-quick@wg0 and openvpn-server@vpn-router, so
+        # vpn-router-setup enabled wg-quick@wg0, openvpn-server@vpn-router,
-        # it takes them down again. Use the debhelper wrappers rather than
+        # caddy and vpn-router-metrics, so the package takes them down again.
-        # systemctl, so the script behaves on a machine without systemd.
+        # Use the debhelper wrappers rather than systemctl, so the script
-        for unit in wg-quick@wg0.service openvpn-server@vpn-router.service; do
+        # behaves on a machine without systemd.
+        for unit in wg-quick@wg0.service openvpn-server@vpn-router.service caddy.service vpn-router-metrics.service; do
             if command -v deb-systemd-invoke >/dev/null 2>&1; then
                 deb-systemd-invoke stop "$unit" >/dev/null 2>&1 || true
             fi

debian-package/debian/rules

@@ -5,4 +5,6 @@
 override_dh_installsystemd:
 	dh_installsystemd --name=vpn-router-setup
 	dh_installsystemd --name=vpn-router-docker
-	dh_installsystemd --name=vpn-router-web
+	dh_installsystemd --name=vpn-router-web --no-start
+	dh_installsystemd --name=vpn-router-metrics --no-enable --no-start
+	dh_installsystemd --name=vpn-router-bootstrap --no-enable --no-start

debian-package/debian/vpn-router.vpn-router-bootstrap.service

@@ -0,0 +1,14 @@
+[Unit]
+Description=VPN Router bootstrap diagnostics page
+Documentation=file:/usr/share/doc/vpn-router/README.md
+
+# Never enabled: vpn-router-setup starts it until a setup has succeeded, and stops it after that.
+[Service]
+Type=simple
+ExecStart=/usr/bin/python3 /usr/lib/vpn-router/bootstrap
+Restart=on-failure
+RestartSec=5
+NoNewPrivileges=yes
+ProtectSystem=strict
+ProtectHome=yes
+PrivateTmp=yes

debian-package/debian/vpn-router.vpn-router-metrics.service

@@ -0,0 +1,22 @@
+[Unit]
+Description=VPN Router metrics service
+Documentation=file:/usr/share/doc/vpn-router/README.md
+After=network-online.target postgresql.service vpn-router-web.service
+Wants=network-online.target
+
+[Service]
+Type=simple
+ExecStartPre=/usr/lib/vpn-router/wait-for-libpq
+ExecStart=/usr/bin/python3 /usr/lib/vpn-router/metrics
+TimeoutStartSec=5min
+Restart=on-failure
+RestartSec=15
+NoNewPrivileges=yes
+ProtectSystem=strict
+RuntimeDirectory=vpn-router
+RuntimeDirectoryPreserve=yes
+ProtectHome=yes
+PrivateTmp=yes
+
+[Install]
+WantedBy=multi-user.target

debian-package/debian/vpn-router.vpn-router-setup.service

@@ -1,8 +1,8 @@
 [Unit]
 Description=VPN Router Setup
 Documentation=file:/usr/share/doc/vpn-router/README.md
-Wants=network-online.target
+Wants=network-online.target postgresql.service
-After=network-online.target docker.service
+After=network-online.target docker.service postgresql.service
 
 [Service]
 Type=oneshot

debian-package/debian/vpn-router.vpn-router-web.service

@@ -1,17 +1,21 @@
 [Unit]
 Description=VPN Router status web service
 Documentation=file:/usr/share/doc/vpn-router/README.md
-After=network-online.target
+After=network-online.target postgresql.service
 Wants=network-online.target
 
 [Service]
 Type=simple
+ExecStartPre=/usr/lib/vpn-router/wait-for-libpq
 ExecStart=/usr/bin/python3 /usr/lib/vpn-router/web
+TimeoutStartSec=5min
 Restart=on-failure
 RestartSec=15
 NoNewPrivileges=yes
 ProtectSystem=strict
-ReadWritePaths=-/run/vpn-router -/etc/vpn-router/pki -/etc/swanctl/x509crl
+RuntimeDirectory=vpn-router
+RuntimeDirectoryPreserve=yes
+ReadWritePaths=-/etc/vpn-router -/etc/swanctl/x509crl
 ProtectHome=yes
 PrivateTmp=yes
 

debian-package/src/usr/lib/systemd/system/caddy.service.d/vpn-router.conf

@@ -0,0 +1,10 @@
+# Caddy is the HTTPS proxy of vpn-router: it reads the configuration vpn-router-setup writes, and
+# does not start until that exists.
+[Unit]
+ConditionPathExists=/etc/caddy/vpn-router.Caddyfile
+
+[Service]
+ExecStart=
+ExecStart=/usr/bin/caddy run --environ --adapter caddyfile --config /etc/caddy/vpn-router.Caddyfile
+ExecReload=
+ExecReload=/usr/bin/caddy reload --adapter caddyfile --config /etc/caddy/vpn-router.Caddyfile --force

debian-package/src/usr/lib/vpn-router/bootstrap

@@ -0,0 +1,55 @@
+#!/usr/bin/python3
+"""Serve diagnostics of the router's first setup on PORT, with the Python standard library alone.
+
+Run by vpn-router-bootstrap.service, which vpn-router-setup starts until a setup has succeeded and
+then stops. Every request on any address answers, as plain text, the output of COMMANDS run at that
+moment: the state of the last setup run, the state of the services, the addresses, and the end of
+the logs.
+"""
+
+import subprocess
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+
+PORT = 17081
+
+#: The title of each section and its command.
+COMMANDS = (
+    ('Setup state', ['cat', '/run/vpn-router/setup.json']),
+    ('Services', ['systemctl', 'is-active', 'vpn-router-setup', 'vpn-router-web', 'slapd', 'postgresql',
+                  'strongswan', 'openvpn-server@vpn-router', 'wg-quick@wg0']),
+    ('Failed units', ['systemctl', '--failed', '--no-legend', '--plain']),
+    ('Addresses', ['ip', '-brief', 'address']),
+    ('Routes', ['ip', 'route']),
+    ('cloud-init', ['cloud-init', 'status', '--long']),
+    ('vpn-router-setup log', ['journalctl', '--unit', 'vpn-router-setup', '--lines', '60', '--no-pager', '--output', 'short-iso']),
+    ('vpn-router-web log', ['journalctl', '--unit', 'vpn-router-web', '--lines', '30', '--no-pager', '--output', 'short-iso']),
+)
+
+
+def report():
+    """The output of every command of COMMANDS under its title."""
+    sections = []
+    for title, command in COMMANDS:
+        try:
+            result = subprocess.run(command, capture_output=True, text=True, timeout=10)
+            text = (result.stdout + result.stderr).rstrip()
+        except (OSError, subprocess.TimeoutExpired) as exc:
+            text = f'cannot run {command[0]}: {exc}'
+        sections.append(f'== {title} ==\n{text}\n')
+    return '\n'.join(sections)
+
+
+class Handler(BaseHTTPRequestHandler):
+    def do_GET(self):
+        body = report().encode()
+        self.send_response(200)
+        self.send_header('Content-Type', 'text/plain; charset=utf-8')
+        self.send_header('Content-Length', str(len(body)))
+        self.send_header('Cache-Control', 'no-store')
+        self.send_header('Refresh', '10')
+        self.end_headers()
+        self.wfile.write(body)
+
+
+if __name__ == '__main__':
+    ThreadingHTTPServer(('', PORT), Handler).serve_forever()

debian-package/src/usr/lib/vpn-router/generate-config

@@ -6,18 +6,15 @@ Called from postinst, which hands the result to ucf(1). ucf compares it with
 this script never writes to /etc and never has to worry about destroying an
 administrator's edits.
 
-The candidate is built in three layers, each overriding the one before:
+The candidate is the current /etc/vpn-router/vpn-router.json as it is, or the
-
+shipped defaults when there is none, with every debconf answer that was given
-  1. the shipped defaults
+written over it. Without an answer that changes it, the candidate is the current
-  2. the current contents of /etc/vpn-router/vpn-router.json, if it exists
+file byte for byte, so
-  3. any debconf answer that was actually given
+an upgrade or a reconfiguration that changes nothing gives ucf nothing to ask;
-
+defaults the file leaves out are applied by vpnrouter.load_config() when the
-Layer 2 is what makes reconfiguration safe: a setting the administrator edited
+file is read. An empty answer means "leave alone", which is why clearing the
-by hand, and that debconf has nothing to say about, is carried through
+pre-shared key from the debconf database after installation does not blank it
-unchanged, so ucf only sees a diff where something really changed. An empty
+on the next dpkg-reconfigure.
-answer means "leave alone", which is why clearing the pre-shared key from the
-debconf database after installation does not blank it on the next
-dpkg-reconfigure.
 
 Answers arrive as VPN_ROUTER_* environment variables and are converted to the
 type of the setting they fill: comma-separated lists become arrays, "true" and
@@ -62,8 +59,21 @@ def main():
         return 2
     output = sys.argv[1]
 
-    cfg = vpnrouter.load_config()
+    try:
+        current = vpnrouter.CONFIG_FILE.read_text(encoding='utf-8')
+    except FileNotFoundError:
+        current = None
+    if current is None:
+        cfg = vpnrouter.load_config()
+    else:
+        try:
+            cfg = json.loads(current)
+        except ValueError as exc:
+            vpnrouter.fail(f'cannot parse {vpnrouter.CONFIG_FILE}: {exc}')
+        if not isinstance(cfg, dict):
+            vpnrouter.fail(f'cannot parse {vpnrouter.CONFIG_FILE}: the top level is not an object')
 
+    answered = False
     for variable, (section, option) in ANSWERS.items():
         value = os.environ.get(variable)
         if not value:
@@ -76,11 +86,19 @@ def main():
             value = int(value) if value.isdigit() else value
         elif isinstance(default, list):
             value = [item.strip() for item in value.split(',') if item.strip()]
-        cfg[section][option] = value
+        if not isinstance(cfg.get(section, {}), dict):
+            vpnrouter.fail(f'cannot parse {vpnrouter.CONFIG_FILE}: {section} is not an object')
+        if cfg.setdefault(section, {}).get(option) != value:
+            cfg[section][option] = value
+            answered = True
 
+    # Without a change the candidate is the current file byte for byte, as ucf compares checksums.
     with open(output, 'w', encoding='utf-8') as handle:
-        json.dump(cfg, handle, indent=2)
+        if current is not None and not answered:
-        handle.write('\n')
+            handle.write(current)
+        else:
+            json.dump(cfg, handle, indent=2)
+            handle.write('\n')
     os.chmod(output, 0o600)
     return 0
 

debian-package/src/usr/lib/vpn-router/metrics

@@ -0,0 +1,414 @@
+#!/usr/bin/python3
+"""Collect the metrics of the router, keep them in PostgreSQL, serve them for scraping and push them with OTLP.
+
+Run by vpn-router-metrics.service while metrics.enabled is true; vpn-router-setup enables and
+disables the unit.
+
+Every SAMPLE_SECONDS, on the minute, a sample is taken with psutil: the CPU utilization since the
+previous sample, the memory used and in total, the free and the total space of /, the bytes
+received and sent per second on the external and the internal interface, and the load averages;
+and the VPN clients of each enabled VPN system (vpnrouter_status): road-warrior (p2s) and
+OpenVPN clients, WireGuard peers with a recent handshake, and the site-to-site tunnel, 1 while it
+is up. The interfaces and the enabled systems are those vpn-router-setup resolved, read from
+/run/vpn-router/web.json.
+
+The sample is written to the detail tables of vpnrouter_metrics. When the clock has passed into
+a new hour, day, month or year, the complete periods are summarized, and once a day, after the
+daily summary, the rows older than their retention are deleted; both also run on start. A
+failure of the database is logged and tried again with the next sample.
+
+GET /metrics answers the last sample in the Prometheus text format, written by
+prometheus_client, with OpenTelemetry semantic convention names in their Prometheus form, for an
+OpenTelemetry Collector's prometheus receiver. It needs HTTP Basic authentication as the service
+account metrics.service_account names, which vpn-router-setup keeps with the password in
+/etc/vpn-router/metrics.secret, checked with a bind to the router's directory, so failed attempts
+lock the account as for any user. The service listens on the interfaces metrics.listen
+names and on the loopback address, where Caddy serves it over HTTPS at metrics.https_port; the
+address of a call through Caddy is the last entry of its X-Forwarded-For header.
+
+When metrics.otlp_endpoint is set, every sample is also pushed there as an OTLP/HTTP JSON
+ExportMetricsServiceRequest, with HTTP Basic authentication when metrics.otlp_username is set.
+"""
+
+import base64
+import datetime
+import json
+import os
+import signal
+import socket
+import sys
+import threading
+import time
+import urllib.error
+import urllib.request
+from http.server import BaseHTTPRequestHandler
+
+sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
+
+import ldap
+import psutil
+from prometheus_client import CONTENT_TYPE_LATEST, generate_latest
+from prometheus_client.core import CollectorRegistry, CounterMetricFamily, GaugeMetricFamily
+
+import vpnrouter
+import vpnrouter_metrics
+import vpnrouter_net
+import vpnrouter_serve
+import vpnrouter_status
+import vpnrouter_users
+
+STATE_FILE = '/run/vpn-router/web.json'
+CONFIG_FILE = '/etc/vpn-router/vpn-router.json'
+
+SAMPLE_SECONDS = 60
+
+#: Seconds between two discoveries of where the service should listen.
+DISCOVERY_SECONDS = 30
+
+#: Seconds an OTLP push may take.
+OTLP_TIMEOUT = 10
+
+#: The mount point whose space is measured.
+MOUNTPOINT = '/'
+
+#: The last sample, as sample() makes it, or None before the first.
+_latest = {'sample': None}
+_latest_lock = threading.Lock()
+
+
+def log(message):
+    print(f'vpn-router-metrics: {message}', file=sys.stderr, flush=True)
+
+
+def read_json(path):
+    """The object in a JSON file, or {} when the file is missing or holds no object."""
+    try:
+        with open(path, encoding='utf-8') as handle:
+            data = json.load(handle)
+    except (OSError, ValueError):
+        return {}
+    return data if isinstance(data, dict) else {}
+
+
+def section(configuration, name):
+    """A section of the configuration, or {} when it is missing or not an object."""
+    value = configuration.get(name)
+    return value if isinstance(value, dict) else {}
+
+
+def clients(state):
+    """The VPN clients of each enabled VPN system, None where it cannot be asked."""
+    counts = {}
+    if state.get('p2s'):
+        counts['p2s'] = vpnrouter_status.established('road-warrior')
+    if state.get('openvpn'):
+        counts['openvpn'] = vpnrouter_status.openvpn_clients()
+    if state.get('wireguard'):
+        counts['wireguard'] = vpnrouter_status.wireguard_connected()
+    if state.get('site_to_site'):
+        count = vpnrouter_status.established('remote-site')
+        counts['site_to_site'] = None if count is None else int(count > 0)
+    return counts
+
+
+def sample(previous):
+    """A sample of the router, with the rates since the previous sample, None for the first."""
+    state = read_json(STATE_FILE)
+    network = section(state, 'network')
+    names = [name for name in dict.fromkeys((network.get('external_interface'), network.get('internal_interface')))
+             if isinstance(name, str) and name]
+    counters = psutil.net_io_counters(pernic=True)
+    memory = psutil.virtual_memory()
+    disk = psutil.disk_usage(MOUNTPOINT)
+    load1, load5, load15 = os.getloadavg()
+    now = time.time()
+    current = {
+        'time': now,
+        'monotonic': time.monotonic(),
+        'system': {
+            'cpu': psutil.cpu_percent(interval=None),
+            'memory_used': memory.total - memory.available,
+            'memory_total': memory.total,
+            'disk_free': disk.free,
+            'disk_total': disk.total,
+            'load1': load1,
+            'load5': load5,
+            'load15': load15,
+        },
+        'counters': {name: (counters[name].bytes_recv, counters[name].bytes_sent) for name in names if name in counters},
+        'clients': clients(state),
+        'rates': {},
+    }
+    if previous is not None:
+        elapsed = current['monotonic'] - previous['monotonic']
+        for name, (received, sent) in current['counters'].items():
+            if name in previous['counters'] and elapsed > 0:
+                before_received, before_sent = previous['counters'][name]
+                current['rates'][name] = (max(received - before_received, 0) / elapsed, max(sent - before_sent, 0) / elapsed)
+    return current
+
+
+def periods(moment):
+    """The hour, the day, the month and the year of a UTC moment, keyed by the levels of vpnrouter_metrics.LEVELS."""
+    return {'hourly': moment.strftime('%Y-%m-%dT%H'), 'daily': moment.strftime('%Y-%m-%d'),
+            'monthly': moment.strftime('%Y-%m'), 'yearly': moment.strftime('%Y')}
+
+
+def store(cfg, current, done):
+    """Write the sample, summarize the periods the clock has passed and collect the garbage once a day.
+
+    done holds the periods summarized so far, and garbage the day of the last garbage collection;
+    it is updated only for what succeeded.
+    """
+    now = periods(datetime.datetime.now(datetime.timezone.utc))
+    with vpnrouter_metrics.database(cfg) as connection:
+        vpnrouter_metrics.write(connection, current['system'], current['rates'], current['clients'])
+        for level, _, _ in vpnrouter_metrics.LEVELS:
+            if done.get(level) != now[level]:
+                vpnrouter_metrics.summarize(connection, level)
+                done[level] = now[level]
+        if done.get('garbage') != now['daily']:
+            vpnrouter_metrics.collect_garbage(connection, cfg)
+            done['garbage'] = now['daily']
+
+
+def attributes(**values):
+    """OTLP attributes of string values."""
+    return [{'key': key, 'value': {'stringValue': value}} for key, value in values.items()]
+
+
+def otlp_request(current, started):
+    """The OTLP/HTTP JSON ExportMetricsServiceRequest of a sample; started is when the counters were first read, in Unix seconds."""
+    stamp = str(int(current['time'] * 1e9))
+    start = str(int(started * 1e9))
+    system = current['system']
+
+    def gauge(name, unit, points):
+        return {'name': name, 'unit': unit, 'gauge': {'dataPoints': [
+            {'timeUnixNano': stamp, 'asDouble': float(value), 'attributes': attributes(**labels)}
+            for value, labels in points if value is not None]}}
+
+    metrics = [
+        gauge('system.cpu.utilization', '1', [(system['cpu'] / 100, {})]),
+        gauge('system.memory.usage', 'By', [(system['memory_used'], {'state': 'used'})]),
+        gauge('system.memory.limit', 'By', [(system['memory_total'], {})]),
+        gauge('system.filesystem.usage', 'By', [(system['disk_free'], {'mountpoint': MOUNTPOINT, 'state': 'free'}),
+                                               (system['disk_total'] - system['disk_free'], {'mountpoint': MOUNTPOINT, 'state': 'used'})]),
+        gauge('system.filesystem.utilization', '1',
+              [(1 - system['disk_free'] / system['disk_total'] if system['disk_total'] else None, {'mountpoint': MOUNTPOINT})]),
+        gauge('system.cpu.load_average.1m', '{thread}', [(system['load1'], {})]),
+        gauge('system.cpu.load_average.5m', '{thread}', [(system['load5'], {})]),
+        gauge('system.cpu.load_average.15m', '{thread}', [(system['load15'], {})]),
+        gauge('vpn_router.clients', '{client}', [(count, {'kind': kind}) for kind, count in current['clients'].items()]),
+        {'name': 'system.network.io', 'unit': 'By', 'sum': {
+            'aggregationTemporality': 2, 'isMonotonic': True, 'dataPoints': [
+                {'startTimeUnixNano': start, 'timeUnixNano': stamp, 'asInt': str(value),
+                 'attributes': attributes(device=name, direction=direction)}
+                for name, counters in current['counters'].items()
+                for value, direction in zip(counters, ('receive', 'transmit'))]}},
+    ]
+    return {'resourceMetrics': [{
+        'resource': {'attributes': attributes(**{'service.name': 'vpn-router', 'host.name': socket.gethostname()})},
+        'scopeMetrics': [{'scope': {'name': 'vpn-router-metrics'}, 'metrics': metrics}],
+    }]}
+
+
+def push(metrics, current, started):
+    """Push the sample to metrics.otlp_endpoint; raises OSError when the collector does not take it."""
+    request = urllib.request.Request(metrics['otlp_endpoint'], data=json.dumps(otlp_request(current, started)).encode(),
+                                     headers={'Content-Type': 'application/json'}, method='POST')
+    if metrics['otlp_username']:
+        credentials = f"{metrics['otlp_username']}:{metrics['otlp_password']}".encode()
+        request.add_header('Authorization', 'Basic ' + base64.b64encode(credentials).decode())
+    with urllib.request.urlopen(request, timeout=OTLP_TIMEOUT) as response:
+        response.read()
+
+
+def collector_loop():
+    """Take a sample on every minute, store it and push it; every new failure is logged once."""
+    previous, done, reported = None, {}, {}
+    started = time.time()
+    psutil.cpu_percent(interval=None)
+
+    def report(kind, message):
+        if message != reported.get(kind):
+            if message:
+                log(message)
+            elif reported.get(kind):
+                log(f'{kind} works again')
+            reported[kind] = message
+
+    while True:
+        time.sleep(SAMPLE_SECONDS - time.time() % SAMPLE_SECONDS)
+        try:
+            cfg = vpnrouter.load_config()
+        except SystemExit as exc:
+            report('configuration', str(exc.code))
+            continue
+        report('configuration', '')
+        current = sample(previous)
+        previous = current
+        with _latest_lock:
+            _latest['sample'] = current
+        try:
+            store(cfg, current, done)
+            report('database', '')
+        except vpnrouter_metrics.DatabaseError as exc:
+            report('database', str(exc))
+        metrics = cfg['metrics']
+        if metrics['otlp_endpoint']:
+            try:
+                push(metrics, current, started)
+                report('OTLP push', '')
+            except (OSError, ValueError) as exc:
+                report('OTLP push', f"pushing to {metrics['otlp_endpoint']} failed: {exc}")
+
+
+class Sample:
+    """A prometheus_client collector of a sample."""
+
+    def __init__(self, current):
+        self.current = current
+
+    def collect(self):
+        system = self.current['system']
+        yield GaugeMetricFamily('system_cpu_utilization', 'CPU utilization since the previous sample, 0 to 1',
+                                value=system['cpu'] / 100)
+        memory = GaugeMetricFamily('system_memory_usage_bytes', 'Memory in use', labels=['state'])
+        memory.add_metric(['used'], system['memory_used'])
+        yield memory
+        yield GaugeMetricFamily('system_memory_limit_bytes', 'Total memory', value=system['memory_total'])
+        usage = GaugeMetricFamily('system_filesystem_usage_bytes', 'Space of the file system', labels=['mountpoint', 'state'])
+        usage.add_metric([MOUNTPOINT, 'free'], system['disk_free'])
+        usage.add_metric([MOUNTPOINT, 'used'], system['disk_total'] - system['disk_free'])
+        yield usage
+        if system['disk_total']:
+            utilization = GaugeMetricFamily('system_filesystem_utilization', 'Used part of the file system, 0 to 1',
+                                            labels=['mountpoint'])
+            utilization.add_metric([MOUNTPOINT], 1 - system['disk_free'] / system['disk_total'])
+            yield utilization
+        for period in ('1m', '5m', '15m'):
+            yield GaugeMetricFamily(f'system_cpu_load_average_{period}', f'Load average over {period}',
+                                    value=system[f'load{period[:-1]}'])
+        network = CounterMetricFamily('system_network_io_bytes', 'Bytes received and sent', labels=['device', 'direction'])
+        for name, (received, sent) in self.current['counters'].items():
+            network.add_metric([name, 'receive'], received)
+            network.add_metric([name, 'transmit'], sent)
+        yield network
+        vpn = GaugeMetricFamily('vpn_router_clients', 'Connected VPN clients; site_to_site is 1 while the tunnel is up',
+                                labels=['kind'])
+        for kind, count in self.current['clients'].items():
+            if count is not None:
+                vpn.add_metric([kind], count)
+        yield vpn
+
+
+def authorized(header, account):
+    """Whether an Authorization header holds the password of the service account; LDAP errors propagate."""
+    if not header.startswith('Basic '):
+        return False
+    try:
+        name, _, password = base64.b64decode(header[6:].strip(), validate=True).decode().partition(':')
+    except ValueError:
+        return False
+    if name != account:
+        return False
+    try:
+        with vpnrouter_users.session() as connection:
+            if not vpnrouter_users.service(vpnrouter_users.entry(connection, name)):
+                return False
+        vpnrouter_users.login(name, password, None)
+    except vpnrouter_users.Failure:
+        return False
+    return True
+
+
+class Handler(BaseHTTPRequestHandler):
+    protocol_version = 'HTTP/1.1'
+
+    def client(self):
+        """The client address: the last X-Forwarded-For entry of a call through Caddy on the loopback address, else the peer."""
+        peer = self.client_address[0]
+        forwarded = self.headers.get('X-Forwarded-For', '')
+        if peer == vpnrouter_net.LOOPBACK and forwarded.strip():
+            return forwarded.split(',')[-1].strip()
+        return peer
+
+    def answer(self, send_body):
+        self.principal = None
+        extra = []
+        path = self.path.split('?', 1)[0]
+        account = str(section(read_json(CONFIG_FILE), 'metrics').get('service_account',
+                                                                       vpnrouter.SCHEMA['metrics']['service_account'])).strip()
+        with _latest_lock:
+            current = _latest['sample']
+        if path != '/metrics':
+            body, kind, code = b'Not found\n', 'text/plain; charset=utf-8', 404
+        elif not account:
+            body, kind, code = b'metrics.service_account is not set\n', 'text/plain; charset=utf-8', 503
+        else:
+            try:
+                allowed = authorized(self.headers.get('Authorization', ''), account)
+            except (ldap.LDAPError, OSError) as exc:
+                allowed = None
+                body, kind, code = f'the user directory failed: {exc}\n'.encode(), 'text/plain; charset=utf-8', 500
+            if allowed is False:
+                body, kind, code = b'Unauthorized\n', 'text/plain; charset=utf-8', 401
+                extra.append(('WWW-Authenticate', 'Basic realm="vpn-router metrics", charset="UTF-8"'))
+            elif allowed:
+                self.principal = account
+                if current is None:
+                    body, kind, code = b'no sample yet\n', 'text/plain; charset=utf-8', 503
+                else:
+                    registry = CollectorRegistry()
+                    registry.register(Sample(current))
+                    body, kind, code = generate_latest(registry), CONTENT_TYPE_LATEST, 200
+        self.send_response(code)
+        self.send_header('Content-Type', kind)
+        self.send_header('Content-Length', str(len(body)))
+        self.send_header('Cache-Control', 'no-store')
+        for name, value in extra:
+            self.send_header(name, value)
+        self.end_headers()
+        if send_body:
+            self.wfile.write(body)
+
+    def do_GET(self):
+        self.answer(True)
+
+    def do_HEAD(self):
+        self.answer(False)
+
+    def log_message(self, format, *args):
+        principal = getattr(self, 'principal', None) or '-'
+        sys.stderr.write('%s - %s [%s] %s\n' % (self.client(), principal, self.log_date_time_string(), format % args))
+
+
+def wanted():
+    """Where the service listens: the addresses of the interfaces metrics.listen names and LOOPBACK, at metrics.port."""
+    configuration = read_json(CONFIG_FILE)
+    metrics = section(configuration, 'metrics')
+    defaults = vpnrouter.SCHEMA['metrics']
+    if metrics.get('enabled', defaults['enabled']) is False:
+        return [], 'metrics are disabled in the configuration'
+    port = metrics.get('port', defaults['port'])
+    if isinstance(port, bool) or not isinstance(port, int) or not 1 <= port <= 65535:
+        port = defaults['port']
+    addresses = vpnrouter_net.listen_addresses(configuration, metrics.get('listen', defaults['listen']))
+    if not addresses:
+        return [], 'the addresses to listen on are not known yet'
+    return [(address, port) for address in [*addresses, vpnrouter_net.LOOPBACK]], ''
+
+
+def main():
+    signal.signal(signal.SIGTERM, lambda *_: sys.exit(0))
+    threading.Thread(target=collector_loop, name='collector', daemon=True).start()
+    try:
+        vpnrouter_serve.serve(wanted, Handler, log, DISCOVERY_SECONDS)
+    except KeyboardInterrupt:
+        pass
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main())

debian-package/src/usr/lib/vpn-router/setup

@@ -5,18 +5,27 @@ Regenerates the configuration files first, then does everything that needs
 the live system: routes, DNS-derived identity, WireGuard keys, firewall rules
 and service reloads. Idempotent - a second run changes nothing.
 
+Until a run has succeeded (SETUP_DONE), every run starts the bootstrap
+services, BOOTSTRAP_UNITS, which show the progress and the failure recorded in
+SETUP_STATE on BOOTSTRAP_PORTS, and the firewall opens those ports on the
+internal interface. The first successful run writes SETUP_DONE, closes the
+ports and stops the bootstrap services; a failed run leaves them running.
+
 Interface names come from the configuration and are never guessed. The
 firewall and NAT rules are written against them, so choosing the wrong one
 would be a silent and security-relevant mistake.
 """
 
+import grp
 import json
 import os
 import pathlib
 import re
+import secrets
 import socket
 import subprocess
 import sys
+import time
 
 sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
 
@@ -24,6 +33,7 @@ import ldap
 
 import vpnrouter
 import vpnrouter_ldap
+import vpnrouter_users
 from vpnrouter import fail
 
 BEFORE_RULES = pathlib.Path('/etc/ufw/before.rules')
@@ -31,10 +41,38 @@ UFW_DEFAULTS = pathlib.Path('/etc/default/ufw')
 WG_DIR       = pathlib.Path('/etc/wireguard')
 WEB_STATE    = pathlib.Path('/run/vpn-router/web.json')
 
+#: The state, the current step and the failure of the last run, which the bootstrap services show.
+SETUP_STATE  = pathlib.Path('/run/vpn-router/setup.json')
+
+#: Written once a run has succeeded; until then setup runs the bootstrap services and opens their ports.
+SETUP_DONE   = pathlib.Path('/var/lib/vpn-router/setup.done')
+
+#: The units of the bootstrap services, and their ports: the Router Manager's and the router's.
+BOOTSTRAP_UNITS = ('vpn-router-bootstrap.service', 'vpn-router-manager-bootstrap.service')
+BOOTSTRAP_PORTS = (17080, 17081)
+
+#: The Router Manager, vpn-router-manager, when it is installed, and its settings.
+MANAGER        = pathlib.Path('/usr/lib/vpn-router-manager/manager')
+MANAGER_CONFIG = pathlib.Path('/etc/vpn-router-manager/manager.json')
+
+#: The default admin of the Router Manager, {"username", "password"}, which its package writes from
+#: its debconf answers; setup creates the account and removes the file.
+MANAGER_ADMIN  = pathlib.Path('/etc/vpn-router-manager/admin.json')
+MANAGER_PORT   = 7080
+
+#: The password of the metrics service account, which setup makes and keeps (root only).
+METRICS_SECRET = pathlib.Path('/etc/vpn-router/metrics.secret')
+
+#: The configuration of Caddy, the HTTPS proxy, which the package's drop-in for caddy.service names;
+#: Caddy runs as the user and group CADDY_GROUP, which the caddy package creates.
+CADDYFILE   = pathlib.Path('/etc/caddy/vpn-router.Caddyfile')
+CADDY_UNIT  = 'caddy.service'
+CADDY_GROUP = 'caddy'
+
 FILTER_ANCHOR = '# End required lines'
 
-#: Configuration options whose values are masked wherever the configuration is shown.
+#: Configuration options left out wherever the configuration is shown.
-SECRET_OPTION = re.compile(r'(^psk$)|secret|password|token|_key$')
+SECRET_OPTION = re.compile(r'(^psk$)|secret|password|token|_key$|^database$')
 
 BLOCKS = (
     ('# IPSEC RULES START',         '# IPSEC RULES END'),
@@ -42,6 +80,10 @@ BLOCKS = (
     ('# OPENVPN RULES START',       '# OPENVPN RULES END'),
     ('# P2S DNS RULES START',       '# P2S DNS RULES END'),
     ('# WEB RULES START',           '# WEB RULES END'),
+    ('# BOOTSTRAP RULES START',     '# BOOTSTRAP RULES END'),
+    ('# MANAGER RULES START',       '# MANAGER RULES END'),
+    ('# HTTPS RULES START',         '# HTTPS RULES END'),
+    ('# METRICS RULES START',       '# METRICS RULES END'),
     ('# ROUTER FORWARD RULES START', '# ROUTER FORWARD RULES END'),
     ('# ROUTER NAT RULES START',    '# ROUTER NAT RULES END'),
     ('# ROUTER MSS RULES START',    '# ROUTER MSS RULES END'),
@@ -129,6 +171,15 @@ def is_configured(ctx):
                 vpnrouter.openvpn_ready(ctx)))
 
 
+def manager_port():
+    """The port of the Router Manager: port in its settings, or MANAGER_PORT when it is not set or not a port."""
+    try:
+        port = json.loads(MANAGER_CONFIG.read_text(encoding='utf-8')).get('port')
+    except (OSError, ValueError, AttributeError):
+        return MANAGER_PORT
+    return port if isinstance(port, int) and not isinstance(port, bool) and 1 <= port <= 65535 else MANAGER_PORT
+
+
 def filter_block(ctx):
     lines = [
         '',
@@ -177,14 +228,55 @@ def filter_block(ctx):
                 ]
         lines += ['# P2S DNS RULES END']
 
-    if ctx['web_enabled'] and ctx['int_iface']:
+    # The web service, the Router Manager and the metrics service choose their interfaces themselves, so their ports are open on both.
+    interfaces = [name for name in (ctx['int_iface'], ctx['wan_iface']) if name]
+    if ctx['web_enabled'] and interfaces:
         lines += [
             '',
             '# WEB RULES START',
-            f'-A ufw-before-input -i {ctx["int_iface"]} -p tcp --dport {ctx["web_port"]} -j ACCEPT',
+            *(f'-A ufw-before-input -i {name} -p tcp --dport {ctx["web_port"]} -j ACCEPT' for name in interfaces),
             '# WEB RULES END',
         ]
 
+    if not SETUP_DONE.exists() and ctx['int_iface']:
+        lines += [
+            '',
+            '# BOOTSTRAP RULES START',
+            *(f'-A ufw-before-input -i {ctx["int_iface"]} -p tcp --dport {port} -j ACCEPT' for port in BOOTSTRAP_PORTS),
+            '# BOOTSTRAP RULES END',
+        ]
+
+    if MANAGER.exists() and interfaces:
+        port = manager_port()
+        lines += [
+            '',
+            '# MANAGER RULES START',
+            *(f'-A ufw-before-input -i {name} -p tcp --dport {port} -j ACCEPT' for name in interfaces),
+            '# MANAGER RULES END',
+        ]
+
+    if vpnrouter.metrics_ready(ctx) and interfaces:
+        lines += [
+            '',
+            '# METRICS RULES START',
+            *(f'-A ufw-before-input -i {name} -p tcp --dport {ctx["metrics_port"]} -j ACCEPT' for name in interfaces),
+            '# METRICS RULES END',
+        ]
+
+    # Let's Encrypt reaches the HTTP challenge on the external interface; the HTTPS ports are open on both.
+    if vpnrouter.https_ready(ctx) and interfaces:
+        ports = [port for wanted, port in ((ctx['web_enabled'], ctx['https_web_port']),
+                                           (MANAGER.exists(), ctx['https_manager_port']),
+                                           (vpnrouter.metrics_ready(ctx), ctx['metrics_https_port'])) if wanted]
+        lines += [
+            '',
+            '# HTTPS RULES START',
+            *([f'-A ufw-before-input -i {ctx["wan_iface"]} -p tcp --dport {vpnrouter.ACME_HTTP_PORT} -j ACCEPT']
+              if ctx['wan_iface'] else []),
+            *(f'-A ufw-before-input -i {name} -p tcp --dport {port} -j ACCEPT' for name in interfaces for port in ports),
+            '# HTTPS RULES END',
+        ]
+
     if vpnrouter.routing_ready(ctx):
         lines += ['', '# ROUTER FORWARD RULES START']
         for subnet in ctx['local_cidr_list']:
@@ -323,19 +415,93 @@ def manage_wireguard(ctx, changed):
         run(['systemctl', 'restart', unit], check=False)
 
 
-def manage_directory(ctx):
+def manage_directory():
-    """Provision the user directory in slapd while a P2S system that has users is configured.
+    """Provision the user directory in slapd, whether or not a P2S system is configured; the problem, or ''."""
-
-    The directory is left in place when they are switched off, so the users are kept.
-    """
-    if not (vpnrouter.p2s_ready(ctx) or vpnrouter.openvpn_ready(ctx)):
-        return
     if vpnrouter_ldap.ensure_services() or not unit_is_active('slapd'):
-        run(['systemctl', 'restart', 'slapd'])
+        result = run(['systemctl', 'restart', 'slapd'], check=False)
+        if result.returncode != 0:
+            return f'slapd cannot be restarted: {result.stderr.strip()}'
     try:
         vpnrouter_ldap.provision()
-    except ldap.LDAPError as exc:
+    except Exception as exc:
-        fail(f'the user directory cannot be provisioned: {exc}')
+        return f'the user directory cannot be provisioned: {exc!r}'
+    return ''
+
+
+def manage_admin():
+    """Create the Router Manager's default admin from MANAGER_ADMIN, then remove the file; the problem, or ''.
+
+    The admin is a Manager-only account and a member of the admins group, which is created with it
+    when it does not exist.
+    """
+    if not MANAGER_ADMIN.exists():
+        return ''
+    try:
+        answers = json.loads(MANAGER_ADMIN.read_text(encoding='utf-8'))
+        name, password = answers['username'], answers['password']
+        with vpnrouter_users.session() as connection:
+            vpnrouter_users.add(connection, name, password, {})
+            try:
+                vpnrouter_users.group_member_add(connection, vpnrouter_users.ADMIN_GROUP, name)
+            except vpnrouter_users.Failure as failure:
+                if failure.status != 3:
+                    raise
+                vpnrouter_users.group_add(connection, vpnrouter_users.ADMIN_GROUP, 'Router administrators', [name])
+        MANAGER_ADMIN.unlink()
+    except (OSError, ValueError, KeyError, TypeError, ldap.LDAPError, vpnrouter_users.Failure) as exc:
+        return f'the Router Manager admin cannot be created from {MANAGER_ADMIN}: {exc}'
+    return ''
+
+
+def manage_metrics_account(ctx):
+    """Keep the service account metrics.service_account names, whose password is in METRICS_SECRET; the problem, or ''.
+
+    While metrics are enabled, setup owns the account: it adds it as a service account with a random password when
+    it is missing, and gives it a new random password, ending any lockout, when METRICS_SECRET is missing or does not
+    hold its password. A disabled account is left as it is. An account of that name that is not a service account is
+    left alone and is a problem.
+    """
+    if not vpnrouter.metrics_ready(ctx):
+        return ''
+    name = ctx['metrics_service_account']
+    try:
+        vpnrouter_users.check_name(name)
+        try:
+            password = METRICS_SECRET.read_text(encoding='utf-8').strip()
+        except OSError:
+            password = ''
+        with vpnrouter_users.session() as connection:
+            try:
+                current = vpnrouter_users.entry(connection, name)
+            except vpnrouter_users.Failure as failure:
+                if failure.status != 3:
+                    raise
+                current = None
+            if current is not None and not vpnrouter_users.service(current):
+                return f'metrics.service_account: {name} is a user, not a service account'
+            # An administrator who disabled the account meant it; its password is left as it is.
+            if current is not None and not vpnrouter_users.summary(current)['enabled']:
+                return ''
+            valid = False
+            if current is not None and password:
+                try:
+                    vpnrouter_users.login(name, password, None)
+                    valid = True
+                except vpnrouter_users.Failure:
+                    pass
+            if not valid:
+                password = secrets.token_urlsafe(32)
+                METRICS_SECRET.touch(mode=0o600)
+                METRICS_SECRET.chmod(0o600)
+                METRICS_SECRET.write_text(password + '\n', encoding='utf-8')
+                if current is None:
+                    vpnrouter_users.add(connection, name, password, {}, service=True)
+                else:
+                    vpnrouter_users.passwd(connection, name, password)
+                    vpnrouter_users.unlock(connection, name)
+    except (OSError, ldap.LDAPError, vpnrouter_users.Failure) as exc:
+        return f'the metrics service account cannot be kept: {exc}'
+    return ''
 
 
 def manage_openvpn(ctx, changed):
@@ -353,6 +519,47 @@ def manage_openvpn(ctx, changed):
         run(['systemctl', 'restart', unit], check=False)
 
 
+def manage_caddy(ctx):
+    """Run Caddy, the HTTPS proxy of the web service, the Router Manager and the metrics service, while HTTPS is ready, and stop it when not.
+
+    CADDYFILE serves the web service on https.web_port, while the manager is installed the
+    manager on https.manager_port, and while metrics are enabled the metrics service on
+    metrics.https_port, all for wan.local_fqdn with a certificate from Let's Encrypt, and all
+    calling the service on its loopback address.
+    """
+    if not vpnrouter.https_ready(ctx):
+        vpnrouter.remove(CADDYFILE)
+        if unit_is_active(CADDY_UNIT):
+            run(['systemctl', 'disable', '--now', CADDY_UNIT], check=False)
+        return
+
+    try:
+        group = grp.getgrnam(CADDY_GROUP).gr_gid
+    except KeyError:
+        fail(f'the group {CADDY_GROUP} does not exist; is the caddy package installed?')
+    caddy_ctx = dict(ctx, manager_port=manager_port() if MANAGER.exists() else 0)
+    changed = vpnrouter.render(vpnrouter._jinja_env(), caddy_ctx, 'Caddyfile.j2', CADDYFILE, 0o640)
+    os.chown(CADDYFILE, 0, group)
+    if not unit_is_active(CADDY_UNIT):
+        run(['systemctl', 'enable', '--now', CADDY_UNIT], check=False)
+    elif changed:
+        run(['systemctl', 'reload', CADDY_UNIT], check=False)
+
+
+def manage_metrics(ctx):
+    """Run the metrics service while metrics are enabled, and stop it when not.
+
+    The service reads its settings itself, so it is restarted only to run the code of a new package version, which
+    every run of setup after an upgrade does.
+    """
+    unit = 'vpn-router-metrics.service'
+    if not vpnrouter.metrics_ready(ctx):
+        run(['systemctl', 'disable', '--now', unit], check=False)
+        return
+    run(['systemctl', 'enable', unit], check=False)
+    run(['systemctl', 'restart', unit], check=False)
+
+
 def run_platform_setup(cfg, ctx):
     """Let the platform module apply its own state, if it defines any."""
     module = vpnrouter.platform_module(cfg)
@@ -381,14 +588,13 @@ def refresh_web(cfg, ctx):
     """Hand the web service the features and facts it shows.
 
     The service reads the file at each request: the features of the router,
-    the configuration with its secrets masked and the facts resolved from the
+    the configuration without its secret options and the facts resolved from
-    system, which it cannot work out with the standard library alone. The file
+    the system, which it cannot work out with the standard library alone. The
-    is readable by root only. The service follows a changed address or port
+    file is readable by root only. The service follows a changed address or
-    itself.
+    port itself.
     """
-    masked = {
+    public = {
-        section: {option: '********' if SECRET_OPTION.search(option) and value else value
+        section: {option: value for option, value in options.items() if not SECRET_OPTION.search(option)}
-                  for option, value in options.items()}
         for section, options in cfg.items()
     }
     WEB_STATE.parent.mkdir(parents=True, exist_ok=True)
@@ -399,8 +605,7 @@ def refresh_web(cfg, ctx):
         'p2s': bool(vpnrouter.p2s_ready(ctx)),
         'wireguard': bool(vpnrouter.wg_ready(ctx)),
         'openvpn': bool(vpnrouter.openvpn_ready(ctx)),
-        'users': bool(vpnrouter.p2s_ready(ctx) or vpnrouter.openvpn_ready(ctx)),
+        'configuration': public,
-        'configuration': masked,
         'network': {
             'external_interface': ctx['wan_iface'],
             'internal_interface': ctx['int_iface'],
@@ -415,25 +620,62 @@ def refresh_web(cfg, ctx):
     }) + '\n', encoding='utf-8')
 
 
+def progress(state, step, error=None):
+    """Record where setup is in SETUP_STATE: running, deferred, done or failed, the step, and the error of a failure."""
+    SETUP_STATE.parent.mkdir(parents=True, exist_ok=True)
+    SETUP_STATE.write_text(json.dumps({
+        'state': state,
+        'step': step,
+        'error': error,
+        'time': time.strftime('%Y-%m-%dT%H:%M:%S%z'),
+    }) + '\n', encoding='utf-8')
+
+
+def start_bootstrap():
+    """Start the bootstrap services while no run has succeeded; a unit that is not installed is skipped."""
+    if not SETUP_DONE.exists():
+        for unit in BOOTSTRAP_UNITS:
+            run(['systemctl', 'start', '--no-block', unit], check=False)
+
+
+def end_bootstrap(ctx):
+    """After the first successful run: write SETUP_DONE, close the bootstrap ports and stop the bootstrap services."""
+    if SETUP_DONE.exists():
+        return
+    SETUP_DONE.parent.mkdir(parents=True, exist_ok=True)
+    SETUP_DONE.touch()
+    if setup_ufw(ctx):
+        run(['ufw', 'reload'], check=False)
+    for unit in BOOTSTRAP_UNITS:
+        run(['systemctl', 'stop', unit], check=False)
+
+
 def main():
+    start_bootstrap()
+    progress('running', 'configuration')
     cfg = vpnrouter.load_config()
     ctx, changed = vpnrouter.configure(cfg)
 
     if ctx is None:
         print('vpn-router: platform is none, configuration deferred; '
               f'set general.platform in {vpnrouter.CONFIG_FILE}')
+        progress('deferred', 'configuration', f'general.platform is none; set it in {vpnrouter.CONFIG_FILE}')
         return 0
 
+    progress('running', 'routes')
     if vpnrouter.routing_ready(ctx):
         apply_routes(ctx)
 
     if ctx['local_id_mode'] == 'public_ip' and ctx['local_fqdn']:
+        progress('running', 'public address')
         address = resolve_public_ip(ctx['local_fqdn'])
         ctx, more = vpnrouter.configure(cfg, local_id_override=address)
         changed |= more
 
+    progress('running', 'WireGuard keys')
     setup_wireguard(ctx)
 
+    progress('running', 'firewall')
     rules_changed = setup_ufw(ctx)
     if is_configured(ctx):
         status = run(['ufw', 'status'], check=False)
@@ -446,19 +688,66 @@ def main():
     elif rules_changed:
         run(['ufw', 'reload'], check=False)
 
-    manage_directory(ctx)
+    progress('running', 'strongSwan')
     manage_strongswan(ctx, changed)
+    progress('running', 'WireGuard')
     manage_wireguard(ctx, changed)
+
+    # The tunnels are up before the directory and the admin, whose problem ends setup only after every other step.
+    failed_step = 'user directory'
+    progress('running', failed_step)
+    problem = manage_directory()
+    if not problem:
+        failed_step = 'Router Manager admin'
+        progress('running', failed_step)
+        problem = manage_admin()
+    if not problem:
+        failed_step = 'metrics service account'
+        progress('running', failed_step)
+        problem = manage_metrics_account(ctx)
+    progress('running', 'OpenVPN')
     manage_openvpn(ctx, changed)
+    progress('running', 'HTTPS proxy')
+    manage_caddy(ctx)
 
     if 'resolved' in changed:
+        progress('running', 'resolver')
         run(['systemctl', 'restart', 'systemd-resolved'], check=False)
 
+    progress('running', 'platform module')
     run_platform_setup(cfg, ctx)
+    progress('running', 'Docker rules')
     refresh_docker_rules()
+    progress('running', 'web service state')
     refresh_web(cfg, ctx)
+    progress('running', 'metrics service')
+    manage_metrics(ctx)
+    if problem:
+        progress('failed', failed_step, problem)
+        print(f'vpn-router: error: {problem}', file=sys.stderr)
+        return 1
+    progress('done', 'done')
+    end_bootstrap(ctx)
     return 0
 
 
+def record_failure(error):
+    """Mark the run failed in the step SETUP_STATE holds."""
+    try:
+        step = json.loads(SETUP_STATE.read_text(encoding='utf-8')).get('step')
+    except (OSError, ValueError):
+        step = None
+    progress('failed', step, error)
+
+
 if __name__ == '__main__':
-    sys.exit(main())
+    # A failing step ends with SystemExit carrying its message (vpnrouter.fail).
+    try:
+        code = main()
+    except SystemExit as exc:
+        record_failure(str(exc.code))
+        raise
+    except Exception as exc:
+        record_failure(repr(exc))
+        raise
+    sys.exit(code)

debian-package/src/usr/lib/vpn-router/vpnrouter.py

@@ -12,6 +12,7 @@ import re
 import shutil
 import subprocess
 import sys
+import urllib.parse
 
 import jinja2
 
@@ -21,7 +22,7 @@ if str(LIB_DIR) not in sys.path:
 
 import vpnrouter_ldap
 import vpnrouter_platforms
-from vpnrouter_net import ConfigError, detect_interfaces, interface_addresses, ip_output
+from vpnrouter_net import DEFAULT_LISTEN, LISTEN_CHOICES, LOOPBACK, ConfigError, detect_interfaces, interface_addresses, ip_output
 
 CONFIG_FILE  = pathlib.Path('/etc/vpn-router/vpn-router.json')
 TEMPLATE_DIR = pathlib.Path('/usr/share/vpn-router/templates')
@@ -54,11 +55,26 @@ SCHEMA = {
     'wireguard':  {'enabled': False, 'address': '', 'listen_port': 51820},
     'openvpn':    {'enabled': False, 'address_pool': '', 'port': 1194, 'protocol': 'udp'},
     'docker':     {'mode': 'auto', 'exposed_ports': []},
-    'web':        {'enabled': True, 'port': 8080, 'jwt_secret': ''},
+    'web':        {'enabled': True, 'listen': DEFAULT_LISTEN, 'port': 7081, 'database': '', 'trusted_proxies': [],
+                   'status_content': 'basic'},
+    'https':      {'enabled': True, 'email': '', 'web_port': 7444, 'manager_port': 7443},
+    'metrics':    {'enabled': True, 'listen': 'internal', 'port': 9464, 'https_port': 9465, 'service_account': 'metrics',
+                   'otlp_endpoint': '', 'otlp_username': '', 'otlp_password': '',
+                   'keep_details': '24 hours', 'keep_hourly': '1 month', 'keep_daily': '3 months',
+                   'keep_monthly': '1 year', 'keep_yearly': '10 years'},
 }
 
-#: The shortest secret that signs the tokens of the status web service.
+#: The retention settings of the metrics, from the details to the yearly summaries.
-WEB_SECRET_MIN = 32
+METRICS_KEEP = ('keep_details', 'keep_hourly', 'keep_daily', 'keep_monthly', 'keep_yearly')
+
+#: A retention period as PostgreSQL reads an interval: a count and a unit.
+METRICS_INTERVAL = re.compile(r'^[1-9]\d{0,4} (hour|day|month|year)s?$')
+
+#: The port of Let's Encrypt's HTTP challenge, which Caddy answers while HTTPS is on.
+ACME_HTTP_PORT = 80
+
+#: What / and /status of the web service show, web.status_content, from the least to the most.
+STATUS_CONTENTS = ('basic', 'standard', 'full')
 
 DOCKER_MODES = ('auto', 'on', 'off')
 
@@ -98,8 +114,8 @@ def warn(message):
 
 
 def fail(message):
-    print(f'vpn-router: error: {message}', file=sys.stderr)
+    """End with status 1; SystemExit carries the message, which Python prints to standard error."""
-    sys.exit(1)
+    sys.exit(f'vpn-router: error: {message}')
 
 
 def load_config(path=None):
@@ -262,9 +278,56 @@ def validate(cfg):
     if isinstance(web_port, bool) or not isinstance(web_port, int) or not 1 <= web_port <= 65535:
         raise ConfigError(f'web.port: {web_port!r} is not a valid port')
 
-    web_secret = _get(cfg, 'web', 'jwt_secret')
+    web_listen = _get(cfg, 'web', 'listen')
-    if web_secret and len(web_secret) < WEB_SECRET_MIN:
+    if web_listen not in LISTEN_CHOICES:
-        raise ConfigError(f'web.jwt_secret: use at least {WEB_SECRET_MIN} characters')
+        raise ConfigError(
+            f'web.listen: {web_listen!r} is not one of {", ".join(LISTEN_CHOICES)}')
+
+    status_content = _get(cfg, 'web', 'status_content')
+    if status_content not in STATUS_CONTENTS:
+        raise ConfigError(
+            f'web.status_content: {status_content!r} is not one of {", ".join(STATUS_CONTENTS)}')
+
+    https_ports = []
+    for option in ('web_port', 'manager_port'):
+        port = _get(cfg, 'https', option)
+        if isinstance(port, bool) or not isinstance(port, int) or not 1 <= port <= 65535:
+            raise ConfigError(f'https.{option}: {port!r} is not a valid port')
+        if port == ACME_HTTP_PORT:
+            raise ConfigError(f'https.{option}: {port} is the port of the HTTP challenge')
+        https_ports.append(port)
+    if https_ports[0] == https_ports[1]:
+        raise ConfigError('https.web_port and https.manager_port must differ')
+
+    for item in _get_list(cfg, 'web', 'trusted_proxies'):
+        _check_address(item, 'web.trusted_proxies')
+
+    taken = {web_port: 'web.port', ACME_HTTP_PORT: 'the HTTP challenge',
+             https_ports[0]: 'https.web_port', https_ports[1]: 'https.manager_port'}
+    for option in ('port', 'https_port'):
+        port = _get(cfg, 'metrics', option)
+        if isinstance(port, bool) or not isinstance(port, int) or not 1 <= port <= 65535:
+            raise ConfigError(f'metrics.{option}: {port!r} is not a valid port')
+        if port in taken:
+            raise ConfigError(f'metrics.{option}: {port} is the port of {taken[port]}')
+        taken[port] = f'metrics.{option}'
+
+    metrics_listen = _get(cfg, 'metrics', 'listen')
+    if metrics_listen not in LISTEN_CHOICES:
+        raise ConfigError(
+            f'metrics.listen: {metrics_listen!r} is not one of {", ".join(LISTEN_CHOICES)}')
+
+    endpoint = _get(cfg, 'metrics', 'otlp_endpoint')
+    if endpoint and urllib.parse.urlsplit(endpoint).scheme not in ('http', 'https'):
+        raise ConfigError(f'metrics.otlp_endpoint: {endpoint!r} is not an http or https URL')
+    if bool(_get(cfg, 'metrics', 'otlp_username')) != bool(_get(cfg, 'metrics', 'otlp_password')):
+        raise ConfigError('metrics.otlp_username and metrics.otlp_password are given together')
+
+    for option in METRICS_KEEP:
+        keep = _get(cfg, 'metrics', option)
+        if not METRICS_INTERVAL.match(keep):
+            raise ConfigError(
+                f'metrics.{option}: {keep!r} is not a count and a unit, such as "24 hours" or "3 months"')
 
     docker_mode = _get(cfg, 'docker', 'mode')
     if docker_mode not in DOCKER_MODES:
@@ -279,6 +342,13 @@ def validate(cfg):
             raise ConfigError(
                 f'docker.exposed_ports: {item!r} is not a port or a port range')
         weight += 2 if ':' in item else 1
+        first, _, last = item.partition(':')
+        if _get(cfg, 'https', 'enabled'):
+            for port in (ACME_HTTP_PORT, *https_ports):
+                if int(first) <= port <= int(last or first):
+                    raise ConfigError(
+                        f'docker.exposed_ports: {item!r} takes port {port}, which Caddy serves while '
+                        'https.enabled is true')
     if weight > DOCKER_PORT_LIMIT:
         raise ConfigError(
             f'docker.exposed_ports: at most {DOCKER_PORT_LIMIT} ports are allowed, '
@@ -447,6 +517,15 @@ def build_context(cfg, local_id_override=None):
         'ldap_users':      vpnrouter_ldap.USERS,
         'web_enabled':     _get(cfg, 'web', 'enabled'),
         'web_port':        _get(cfg, 'web', 'port'),
+        'https_enabled':   _get(cfg, 'https', 'enabled'),
+        'https_email':     _get(cfg, 'https', 'email'),
+        'https_web_port':  _get(cfg, 'https', 'web_port'),
+        'https_manager_port': _get(cfg, 'https', 'manager_port'),
+        'metrics_enabled': _get(cfg, 'metrics', 'enabled'),
+        'metrics_port':    _get(cfg, 'metrics', 'port'),
+        'metrics_https_port': _get(cfg, 'metrics', 'https_port'),
+        'metrics_service_account': _get(cfg, 'metrics', 'service_account'),
+        'loopback':        LOOPBACK,
     }
 
 
@@ -476,6 +555,16 @@ def openvpn_ready(ctx):
     return bool(ctx['openvpn_enabled'] and ctx['openvpn_pool'] and ctx['local_fqdn'])
 
 
+def https_ready(ctx):
+    """Caddy serves HTTPS once it is enabled and wan.local_fqdn names the certificate."""
+    return bool(ctx['https_enabled'] and ctx['local_fqdn'])
+
+
+def metrics_ready(ctx):
+    """The metrics service collects, keeps and serves the metrics while metrics.enabled is true."""
+    return bool(ctx['metrics_enabled'])
+
+
 def _jinja_env():
     return jinja2.Environment(
         loader=jinja2.FileSystemLoader(str(TEMPLATE_DIR)),
@@ -565,24 +654,28 @@ def _simple_ca(*args):
 
 
 def provision_pki(ctx):
-    """Create the CA, server certificate and CRL when the PKI directory is empty; a CA without a CRL gets one."""
+    """Create the CA and CRL when the PKI directory is empty, and the server certificate once wan.local_fqdn is set.
+
+    A CA of the router's own (ca_key.pem present) without a CRL gets one, and without a
+    certificate for wan.local_fqdn gets one too; nothing present is overwritten.
+    """
     PKI_DIR.mkdir(parents=True, exist_ok=True)
     os.chmod(PKI_DIR, 0o700)
-
-    if any(PKI_DIR.iterdir()):
-        if (PKI_DIR / 'ca_key.pem').is_file() and not (PKI_DIR / 'crl.pem').is_file():
-            _simple_ca('make-crl', '--ca-dir', str(PKI_DIR), '--days', str(CRL_DAYS))
-        return
-    if not ctx['p2s_ca_name']:
-        warn('p2s.ca_name is empty, not creating a local CA')
-        return
     if not SIMPLE_CA.is_file():
         fail(f'{SIMPLE_CA} is missing')
 
-    _simple_ca('make-ca', '--ca-dir', str(PKI_DIR), ctx['p2s_ca_name'])
+    if not any(PKI_DIR.iterdir()):
-    _simple_ca('make-cert', '--ca-dir', str(PKI_DIR),
+        if not ctx['p2s_ca_name']:
-               ctx['local_fqdn'], ctx['local_fqdn'])
+            warn('p2s.ca_name is empty, not creating a local CA')
-    _simple_ca('make-crl', '--ca-dir', str(PKI_DIR), '--days', str(CRL_DAYS))
+            return
+        _simple_ca('make-ca', '--ca-dir', str(PKI_DIR), ctx['p2s_ca_name'])
+    if not (PKI_DIR / 'ca_key.pem').is_file():
+        return
+    if not (PKI_DIR / 'crl.pem').is_file():
+        _simple_ca('make-crl', '--ca-dir', str(PKI_DIR), '--days', str(CRL_DAYS))
+    label = ctx['local_fqdn'].split('.', 1)[0] if ctx['local_fqdn'] else ''
+    if label and not (PKI_DIR / f'{label}_cert.pem').is_file():
+        _simple_ca('make-cert', '--ca-dir', str(PKI_DIR), ctx['local_fqdn'], ctx['local_fqdn'])
 
 
 def distribute_pki(ctx):
@@ -628,8 +721,7 @@ def configure(cfg=None, local_id_override=None):
 
     changed = set()
 
-    if p2s_ready(ctx) or openvpn_ready(ctx):
+    provision_pki(ctx)
-        provision_pki(ctx)
     if distribute_pki(ctx):
         changed.add('swanctl')
 

debian-package/src/usr/lib/vpn-router/vpnrouter_auth.py

@@ -0,0 +1,207 @@
+"""The tokens of the router's web service, for the web service and vpn-router-token.
+
+Tokens are HS256 JWTs signed with WEB_KEY, a key the web service makes on its first run on the
+router and that never leaves it. /auth issues an access token naming the user in sub, valid ACCESS_SECONDS, and
+a refresh token valid REFRESH_SECONDS; vpn-router-token issues tokens with the admin claim and
+any lifetime. A token without the admin claim must expire within MAX_LIFETIME seconds.
+
+Every failure of the database raises DatabaseError, the only database exception callers see.
+
+Refresh tokens are random strings kept in PostgreSQL as their SHA-256, in the database of
+web.database, by default the vpn-router database on this host that provision() creates on the
+web service's first run, with the password in DATABASE_SECRET. A refresh token is used once: refreshing replaces it.
+"""
+
+import base64
+import contextlib
+import hashlib
+import hmac
+import json
+import pathlib
+import secrets
+import subprocess
+import time
+
+import psycopg
+
+WEB_KEY = pathlib.Path('/etc/vpn-router/web.key')
+DATABASE_SECRET = pathlib.Path('/etc/vpn-router/database.secret')
+
+#: The role and the database provision() creates when web.database is not set.
+DATABASE = 'vpn-router'
+
+#: The user operations of vpnrouter_users after which the user's refresh tokens are revoked.
+REVOKING = ('passwd', 'disable', 'unlock', 'remove')
+
+#: Seconds an access token of /auth and a refresh token are valid.
+ACCESS_SECONDS = 300
+REFRESH_SECONDS = 8 * 3600
+
+#: The longest a token without the admin claim may be valid, from now, in seconds.
+MAX_LIFETIME = 600
+
+#: Seconds of clock difference accepted between a token and this router.
+LEEWAY = 60
+
+SCHEMA = 'CREATE TABLE IF NOT EXISTS refresh_tokens (hash text PRIMARY KEY, name text NOT NULL, expires timestamptz NOT NULL)'
+
+
+class DatabaseError(Exception):
+    """The database of the refresh tokens failed."""
+
+
+def key():
+    """The signing key, or '' while the web service has not made it."""
+    try:
+        return WEB_KEY.read_text(encoding='utf-8').strip()
+    except OSError:
+        return ''
+
+
+def ensure_key():
+    """Make the signing key when it is missing."""
+    if not key():
+        WEB_KEY.parent.mkdir(parents=True, exist_ok=True)
+        WEB_KEY.touch(mode=0o600)
+        WEB_KEY.chmod(0o600)
+        WEB_KEY.write_text(secrets.token_urlsafe(36) + '\n', encoding='utf-8')
+
+
+def b64encode(data):
+    return base64.urlsafe_b64encode(data).rstrip(b'=').decode()
+
+
+def b64decode(part):
+    return base64.urlsafe_b64decode(part + '=' * (-len(part) % 4))
+
+
+def sign(claims, secret):
+    head = b64encode(json.dumps({'alg': 'HS256', 'typ': 'JWT'}).encode())
+    body = b64encode(json.dumps(claims).encode())
+    signature = hmac.new(secret.encode(), f'{head}.{body}'.encode(), hashlib.sha256).digest()
+    return f'{head}.{body}.{b64encode(signature)}'
+
+
+def verify(token, secret):
+    """The claims of a token signed with HS256 by the secret, or None when it is not acceptable.
+
+    Only HS256 is accepted, so a token cannot choose a weaker algorithm. The token must expire
+    (exp); without the admin claim not later than MAX_LIFETIME seconds from now, so that a stolen
+    token is of little use. LEEWAY seconds of clock difference are allowed.
+    """
+    try:
+        header, payload, signature = token.split('.')
+        if json.loads(b64decode(header)).get('alg') != 'HS256':
+            return None
+        expected = hmac.new(secret.encode(), f'{header}.{payload}'.encode(), hashlib.sha256).digest()
+        if not hmac.compare_digest(expected, b64decode(signature)):
+            return None
+        claims = json.loads(b64decode(payload))
+        now = time.time()
+        expires = claims['exp']
+        if isinstance(expires, bool) or not isinstance(expires, (int, float)) or expires < now - LEEWAY:
+            return None
+        if claims.get('admin') is not True and expires > now + MAX_LIFETIME + LEEWAY:
+            return None
+        not_before = claims.get('nbf')
+        if not_before is not None and (isinstance(not_before, bool) or not isinstance(not_before, (int, float)) or not_before > now + LEEWAY):
+            return None
+    except (ValueError, KeyError, TypeError, AttributeError):
+        return None
+    return claims
+
+
+def admin_token(seconds):
+    """A token with the admin claim, valid for the seconds given."""
+    secret = key()
+    if not secret:
+        raise OSError(f'{WEB_KEY} does not exist; vpn-router-web makes it when it starts')
+    now = int(time.time())
+    return {'token': sign({'admin': True, 'iat': now, 'exp': now + seconds}, secret), 'expires': now + seconds}
+
+
+def connection_string(cfg):
+    """web.database, or the database provision() creates; its password, of URL-safe characters, is written unquoted."""
+    configured = cfg['web']['database'].strip()
+    if configured:
+        return configured
+    password = DATABASE_SECRET.read_text(encoding='utf-8').strip()
+    return f'host=localhost dbname={DATABASE} user={DATABASE} password={password}'
+
+
+@contextlib.contextmanager
+def database(cfg):
+    """A connection to the database of the refresh tokens, with their table, closed at the end; DatabaseError on any failure."""
+    try:
+        with psycopg.connect(connection_string(cfg), autocommit=True) as connection:
+            connection.execute(SCHEMA)
+            yield connection
+    except psycopg.Error as exc:
+        raise DatabaseError(f'the token database failed: {exc}') from exc
+
+
+def digest(token):
+    return hashlib.sha256(token.encode()).hexdigest()
+
+
+def issue(cfg, name):
+    """An access token and a refresh token for the user, as /auth answers them."""
+    secret = key()
+    now = int(time.time())
+    refresh = secrets.token_urlsafe(32)
+    with database(cfg) as connection:
+        connection.execute('DELETE FROM refresh_tokens WHERE expires < now()')
+        connection.execute("INSERT INTO refresh_tokens VALUES (%s, %s, now() + make_interval(secs => %s))",
+                           (digest(refresh), name, REFRESH_SECONDS))
+    return {
+        'access_token': sign({'sub': name, 'iat': now, 'exp': now + ACCESS_SECONDS}, secret),
+        'token_type': 'Bearer',
+        'expires_in': ACCESS_SECONDS,
+        'refresh_token': refresh,
+        'refresh_expires_in': REFRESH_SECONDS,
+    }
+
+
+def take(cfg, token):
+    """Remove an unexpired refresh token and return the user it was issued to, or None."""
+    if not isinstance(token, str) or not token:
+        return None
+    with database(cfg) as connection:
+        row = connection.execute('DELETE FROM refresh_tokens WHERE hash = %s RETURNING name, expires > now()',
+                                 (digest(token),)).fetchone()
+    return row[0] if row and row[1] else None
+
+
+def revoke(cfg, name):
+    """Remove all refresh tokens of the user."""
+    with database(cfg) as connection:
+        connection.execute('DELETE FROM refresh_tokens WHERE name = %s', (name,))
+
+
+def psql(sql, name):
+    """The output of an SQL statement, read from standard input, run by psql as postgres with name as the variable v1."""
+    args = ['runuser', '-u', 'postgres', '--', 'psql', '-X', '-q', '-t', '-A', '-v', 'ON_ERROR_STOP=1', '-v', f'v1={name}']
+    result = subprocess.run(args, input=sql, capture_output=True, text=True)
+    if result.returncode != 0:
+        raise OSError(f'psql failed: {result.stderr.strip()}')
+    return result.stdout.strip()
+
+
+def provision(cfg):
+    """Create the role, its password file and the database when web.database is not set and they are missing.
+
+    The password, of URL-safe characters, reaches psql on standard input, not on its command line.
+    """
+    if cfg['web']['database'].strip():
+        return
+    if not psql("SELECT 1 FROM pg_roles WHERE rolname = :'v1'", DATABASE):
+        DATABASE_SECRET.unlink(missing_ok=True)
+        psql("CREATE ROLE :\"v1\" LOGIN", DATABASE)
+    if not DATABASE_SECRET.is_file():
+        password = secrets.token_urlsafe(32)
+        psql(f"ALTER ROLE :\"v1\" PASSWORD '{password}'", DATABASE)
+        DATABASE_SECRET.touch(mode=0o600)
+        DATABASE_SECRET.chmod(0o600)
+        DATABASE_SECRET.write_text(password + '\n', encoding='utf-8')
+    if not psql("SELECT 1 FROM pg_database WHERE datname = :'v1'", DATABASE):
+        psql("CREATE DATABASE :\"v1\" OWNER :\"v1\"", DATABASE)

debian-package/src/usr/lib/vpn-router/vpnrouter_ldap.py

@@ -3,8 +3,14 @@
 The package keeps its own database, dc=vpn-router,dc=local in /var/lib/ldap/vpn-router,
 next to the one slapd's installation creates. Root manages it through the local socket
 with SASL EXTERNAL: the database's root DN is root's peer-credential DN, so there is no
-administrator password. Users are changed by the user-manager service account, which the
+administrator password. Users and groups are changed by the user-manager service account,
-ACLs allow to write ou=users and to set, never read, passwords.
+which the ACLs allow to write ou=users and ou=groups and to set, never read, passwords and
+TOTP secrets.
+
+The password policy locks an account for LOCKOUT_SECONDS after LOCKOUT_FAILURES failed binds;
+a successful bind clears the count. slapd's otp overlay is configured on the database, so a
+bind of a user with a TOTP token takes the password followed by a current code.
+cn=totp,ou=policies holds the TOTP parameters the token of every user points to.
 """
 
 import os
@@ -20,9 +26,11 @@ import ldap.sasl
 URI = 'ldapi:///'
 BASE = 'dc=vpn-router,dc=local'
 USERS = f'ou=users,{BASE}'
+GROUPS = f'ou=groups,{BASE}'
 SERVICES = f'ou=services,{BASE}'
 POLICIES = f'ou=policies,{BASE}'
 DEFAULT_POLICY = f'cn=default,{POLICIES}'
+TOTP_PARAMS = f'cn=totp,{POLICIES}'
 MANAGER = f'cn=user-manager,{SERVICES}'
 ROOT = 'gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth'
 
@@ -32,13 +40,43 @@ MANAGER_SECRET = SECRET_DIR / 'user-manager.secret'
 SLAPD_DEFAULTS = pathlib.Path('/etc/default/slapd')
 SLAPD_SERVICES = 'SLAPD_SERVICES="ldapi:///"'
 
-MODULES = ('argon2', 'ppolicy')
+MODULES = ('argon2', 'ppolicy', 'otp')
 PASSWORD_HASH = '{ARGON2}'
 
+#: Failed binds that lock an account, within LOCKOUT_SECONDS, and the seconds it stays locked.
+LOCKOUT_FAILURES = 5
+LOCKOUT_SECONDS = 900
+
+#: The overlays of the database, in the order they are configured.
+OVERLAYS = ('ppolicy', 'otp')
+
+#: The package's own schema, under an OID of the UUID arc 2.25 (ITU-T X.667): attribute types
+#: under .1 and object classes under .2. vpnRouterAccount, auxiliary, adds properties of the
+#: package to a user of either kind: mail, for a Manager-only account, and vpnRouterScope,
+#: authorization scopes, of which service marks a service account.
+SCHEMA_OID = '2.25.98477352595735834532404256531620013001'
+SCHEMA_NAME = 'vpn-router'
+SCHEMA_ATTRIBUTE_TYPES = (
+    f"( {SCHEMA_OID}.1.1 NAME 'vpnRouterScope' DESC 'VPN router: an authorization scope of the account' "
+    "EQUALITY caseIgnoreMatch SUBSTR caseIgnoreSubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{256} )",
+)
+SCHEMA_OBJECT_CLASSES = (
+    f"( {SCHEMA_OID}.2.1 NAME 'vpnRouterAccount' DESC 'VPN router: properties of a user or a Manager-only account' "
+    "SUP top AUXILIARY MAY ( mail $ vpnRouterScope ) )",
+)
+
+#: The TOTP parameters: digits, the OID of HMAC-SHA1, seconds per time step, and the time steps accepted before and after now.
+TOTP_DIGITS = 6
+TOTP_ALGORITHM = '1.2.840.113549.2.7'
+TOTP_PERIOD = 30
+TOTP_WINDOW = 1
+
 #: The ACLs of the database, in order.
 ACCESS = (
     f'to attrs=userPassword by dn.exact="{MANAGER}" =wx by self =wx by anonymous auth by * none',
+    f'to attrs=oathSecret by dn.exact="{MANAGER}" =w by * none',
     f'to dn.subtree="{USERS}" by dn.exact="{MANAGER}" manage by * read',
+    f'to dn.subtree="{GROUPS}" by dn.exact="{MANAGER}" manage by * read',
     'to * by * read',
 )
 
@@ -86,6 +124,23 @@ def _ensure_modules(connection):
         connection.modify_s(dn, [(ldap.MOD_ADD, 'olcModuleLoad', [name.encode() for name in missing])])
 
 
+def _ensure_schema(connection):
+    """Add the package's schema to cn=config when missing, else set its definitions that differ."""
+    found = connection.search_s('cn=schema,cn=config', ldap.SCOPE_ONELEVEL, f'(cn=*{SCHEMA_NAME})',
+                                ['olcAttributeTypes', 'olcObjectClasses'])
+    attributes = {'olcAttributeTypes': [value.encode() for value in SCHEMA_ATTRIBUTE_TYPES],
+                  'olcObjectClasses': [value.encode() for value in SCHEMA_OBJECT_CLASSES]}
+    if not found:
+        connection.add_s(f'cn={SCHEMA_NAME},cn=schema,cn=config', ldap.modlist.addModlist(
+            dict(attributes, objectClass=[b'olcSchemaConfig'], cn=[SCHEMA_NAME.encode()])))
+        return
+    dn, current = found[0]
+    changes = [(ldap.MOD_REPLACE, name, values) for name, values in attributes.items()
+               if unordered(current.get(name, [])) != [value.decode() for value in values]]
+    if changes:
+        connection.modify_s(dn, changes)
+
+
 def _ensure_password_hash(connection):
     dn = 'olcDatabase={-1}frontend,cn=config'
     current = connection.search_s(dn, ldap.SCOPE_BASE, attrlist=['olcPasswordHash'])[0][1].get('olcPasswordHash', [])
@@ -116,13 +171,18 @@ def _ensure_database(connection):
     return dn
 
 
-def _ensure_overlay(connection, database):
+def _ensure_overlays(connection, database):
-    if not connection.search_s(database, ldap.SCOPE_ONELEVEL, '(objectClass=olcPPolicyConfig)', ['1.1']):
+    """Add the overlays of OVERLAYS the database lacks; the otp overlay has no settings of its own."""
-        connection.add_s(f'olcOverlay=ppolicy,{database}', ldap.modlist.addModlist({
+    found = connection.search_s(database, ldap.SCOPE_ONELEVEL, '(objectClass=olcOverlayConfig)', ['olcOverlay'])
-            'objectClass': [b'olcOverlayConfig', b'olcPPolicyConfig'],
+    present = {ORDER.sub(b'', attributes['olcOverlay'][0]).decode() for _, attributes in found}
-            'olcOverlay': [b'ppolicy'],
+    for name in OVERLAYS:
-            'olcPPolicyDefault': [DEFAULT_POLICY.encode()],
+        if name in present:
-        }))
+            continue
+        record = {'objectClass': [b'olcOverlayConfig'], 'olcOverlay': [name.encode()]}
+        if name == 'ppolicy':
+            record = {'objectClass': [b'olcOverlayConfig', b'olcPPolicyConfig'], 'olcOverlay': [b'ppolicy'],
+                      'olcPPolicyDefault': [DEFAULT_POLICY.encode()]}
+        connection.add_s(f'olcOverlay={name},{database}', ldap.modlist.addModlist(record))
 
 
 def _ensure_entry(connection, dn, attributes):
@@ -160,22 +220,33 @@ def _ensure_manager(connection):
 
 
 def provision():
-    """Create or bring up to date the modules, database, ACLs, password policy, entries and service account."""
+    """Create or bring up to date the modules, the package's schema, database, ACLs, password policy, entries and service account."""
     connection = root()
     try:
         _ensure_modules(connection)
+        _ensure_schema(connection)
         _ensure_password_hash(connection)
         database = _ensure_database(connection)
-        _ensure_overlay(connection, database)
+        _ensure_overlays(connection, database)
         _ensure_entry(connection, BASE, {'objectClass': [b'dcObject', b'organization'], 'dc': [b'vpn-router'], 'o': [b'vpn-router']})
-        for unit in (USERS, SERVICES, POLICIES):
+        for unit in (USERS, GROUPS, SERVICES, POLICIES):
             _ensure_entry(connection, unit, {'objectClass': [b'organizationalUnit'], 'ou': [unit.split(',')[0][3:].encode()]})
         _ensure_entry(connection, DEFAULT_POLICY, {
             'objectClass': [b'organizationalRole', b'pwdPolicy'],
             'cn': [b'default'],
             'pwdAttribute': [b'userPassword'],
             'pwdLockout': [b'TRUE'],
-            'pwdMaxFailure': [b'0'],
+            'pwdMaxFailure': [str(LOCKOUT_FAILURES).encode()],
+            'pwdLockoutDuration': [str(LOCKOUT_SECONDS).encode()],
+            'pwdFailureCountInterval': [str(LOCKOUT_SECONDS).encode()],
+        })
+        _ensure_entry(connection, TOTP_PARAMS, {
+            'objectClass': [b'organizationalRole', b'oathTOTPParams'],
+            'cn': [b'totp'],
+            'oathOTPLength': [str(TOTP_DIGITS).encode()],
+            'oathHMACAlgorithm': [TOTP_ALGORITHM.encode()],
+            'oathTOTPTimeStepPeriod': [str(TOTP_PERIOD).encode()],
+            'oathTOTPTimeStepWindow': [str(TOTP_WINDOW).encode()],
         })
         _ensure_manager(connection)
     finally:

debian-package/src/usr/lib/vpn-router/vpnrouter_metrics.py

@@ -0,0 +1,173 @@
+"""The metrics of the router in PostgreSQL, for the metrics service and the web service.
+
+The metrics service writes one detail row per minute to the three detail tables, of TABLES:
+metrics_system (CPU, memory, disk and load averages), metrics_network (bytes per second
+received and sent per interface) and metrics_vpn (clients per kind). Each detail table has four
+summary tables, of LEVELS: <table>_hourly from the details, <table>_daily from the hourly,
+<table>_monthly from the daily and <table>_yearly from the monthly rows. A summary row has the
+start of its period, in UTC, and the average and the maximum of each value over the period: of
+the details for the hourly rows, else the average of the averages and the maximum of the maxima
+below. A period is summarized once it is complete; summarize() catches up every complete period
+after the last summarized one, so a stopped service loses no summary of what was written.
+
+collect_garbage() deletes the rows older than the retention of their level, metrics.keep_*;
+the service runs it on start and once a day. history() answers the views of the Metrics tab.
+
+The tables are in the database of the refresh tokens (vpnrouter_auth.connection_string) and are
+created when missing. Every failure of the database raises DatabaseError.
+"""
+
+import contextlib
+
+import psycopg
+
+import vpnrouter_auth
+
+#: The detail tables: their key columns besides time, and their value columns.
+TABLES = {
+    'metrics_system': ((), ('cpu', 'memory_used', 'memory_total', 'disk_free', 'disk_total', 'load1', 'load5', 'load15')),
+    'metrics_network': (('interface',), ('rx', 'tx')),
+    'metrics_vpn': (('kind',), ('clients',)),
+}
+
+#: The summary levels, from the finest: the suffix of the tables, the period of date_trunc and the retention setting.
+LEVELS = (
+    ('hourly', 'hour', 'keep_hourly'),
+    ('daily', 'day', 'keep_daily'),
+    ('monthly', 'month', 'keep_monthly'),
+    ('yearly', 'year', 'keep_yearly'),
+)
+
+#: The kinds of VPN clients of metrics_vpn; site_to_site is 1 while the tunnel is up.
+VPN_KINDS = ('p2s', 'openvpn', 'wireguard', 'site_to_site')
+
+#: The views of history(): the summary level read (None: the details), the bucket and the span.
+RANGES = {
+    'hour': (None, '1 minute', '1 hour'),
+    'day': (None, '10 minutes', '1 day'),
+    'month': ('hourly', '6 hours', '1 month'),
+    'year': ('monthly', '1 month', '1 year'),
+}
+
+
+class DatabaseError(Exception):
+    """The metrics database failed."""
+
+
+def schema():
+    """The statements that create the detail and the summary tables."""
+    statements = []
+    for table, (keys, values) in TABLES.items():
+        key_columns = ''.join(f', {key} text NOT NULL' for key in keys)
+        primary = ', '.join(('time', *keys))
+        detail = ''.join(f', {value} double precision' for value in values)
+        statements.append(f'CREATE TABLE IF NOT EXISTS {table} (time timestamptz NOT NULL{key_columns}{detail}, '
+                          f'PRIMARY KEY ({primary}))')
+        summary = ''.join(f', {value}_avg double precision, {value}_max double precision' for value in values)
+        for suffix, _, _ in LEVELS:
+            statements.append(f'CREATE TABLE IF NOT EXISTS {table}_{suffix} (time timestamptz NOT NULL{key_columns}{summary}, '
+                              f'PRIMARY KEY ({primary}))')
+    return statements
+
+
+@contextlib.contextmanager
+def database(cfg):
+    """A connection to the metrics database, with its tables, closed at the end; DatabaseError on any failure."""
+    try:
+        with psycopg.connect(vpnrouter_auth.connection_string(cfg), autocommit=True) as connection:
+            for statement in schema():
+                connection.execute(statement)
+            yield connection
+    except (psycopg.Error, OSError) as exc:
+        raise DatabaseError(f'the metrics database failed: {exc}') from exc
+
+
+def write(connection, system, network, vpn):
+    """Write a sample as the detail rows of the current minute.
+
+    system maps the values of metrics_system to numbers; network maps an interface to its rx and
+    tx; vpn maps a kind of VPN_KINDS to its clients. A value that is None is written as NULL.
+    """
+    with connection.transaction():
+        minute = connection.execute("SELECT date_trunc('minute', now(), 'UTC')").fetchone()[0]
+        names = TABLES['metrics_system'][1]
+        connection.execute(f"INSERT INTO metrics_system (time, {', '.join(names)}) "
+                           f"VALUES (%s{', %s' * len(names)}) ON CONFLICT DO NOTHING",
+                           (minute, *(system.get(name) for name in names)))
+        for interface, (rx, tx) in network.items():
+            connection.execute('INSERT INTO metrics_network (time, interface, rx, tx) VALUES (%s, %s, %s, %s) '
+                               'ON CONFLICT DO NOTHING', (minute, interface, rx, tx))
+        for kind, clients in vpn.items():
+            connection.execute('INSERT INTO metrics_vpn (time, kind, clients) VALUES (%s, %s, %s) ON CONFLICT DO NOTHING',
+                               (minute, kind, clients))
+
+
+def summarize(connection, level):
+    """Summarize the complete periods of a level of LEVELS that are not summarized yet, for every detail table."""
+    index = [suffix for suffix, _, _ in LEVELS].index(level)
+    _, period, _ = LEVELS[index]
+    for table, (keys, values) in TABLES.items():
+        target = f'{table}_{level}'
+        source = table if index == 0 else f'{table}_{LEVELS[index - 1][0]}'
+        if index == 0:
+            aggregates = ', '.join(f'avg({value}), max({value})' for value in values)
+        else:
+            aggregates = ', '.join(f'avg({value}_avg), max({value}_max)' for value in values)
+        columns = ', '.join(('time', *keys, *(f'{value}_{kind}' for value in values for kind in ('avg', 'max'))))
+        groups = ''.join(f', {key}' for key in keys)
+        connection.execute(
+            f"INSERT INTO {target} ({columns}) "
+            f"SELECT date_trunc('{period}', time, 'UTC') AS period{groups}, {aggregates} FROM {source} "
+            f"WHERE time < date_trunc('{period}', now(), 'UTC') "
+            f"AND time >= coalesce((SELECT max(time) FROM {target}) + interval '1 {period}', '-infinity') "
+            f"GROUP BY period{groups} ON CONFLICT DO NOTHING")
+
+
+def collect_garbage(connection, cfg):
+    """Delete the rows older than the retention of their level: metrics.keep_details, keep_hourly and so on."""
+    metrics = cfg['metrics']
+    for table in TABLES:
+        connection.execute(f'DELETE FROM {table} WHERE time < now() - %s::interval', (metrics['keep_details'],))
+        for suffix, _, keep in LEVELS:
+            connection.execute(f'DELETE FROM {table}_{suffix} WHERE time < now() - %s::interval', (metrics[keep],))
+
+
+def history(connection, view):
+    """The points of a view of RANGES, oldest first, each with the start of its bucket in Unix seconds.
+
+    A point has the average and the maximum of each value of metrics_system, network, which maps
+    each interface to its rx and tx, and clients, which maps each kind of VPN client to its
+    clients; a value without data is None.
+    """
+    level, bucket, span = RANGES[view]
+
+    def rows(table):
+        keys, values = TABLES[table]
+        source = table if level is None else f'{table}_{level}'
+        if level is None:
+            aggregates = ', '.join(f'avg({value}), max({value})' for value in values)
+        else:
+            aggregates = ', '.join(f'avg({value}_avg), max({value}_max)' for value in values)
+        start = (f"date_bin('{bucket}', time, timestamptz '2000-01-01 00:00:00+00')" if view != 'year'
+                 else "date_trunc('month', time, 'UTC')")
+        groups = ''.join(f', {key}' for key in keys)
+        return connection.execute(
+            f"SELECT extract(epoch FROM {start})::bigint AS bucket{groups}, {aggregates} FROM {source} "
+            f"WHERE time >= now() - interval '{span}' GROUP BY bucket{groups} ORDER BY bucket").fetchall()
+
+    def pairs(row, values):
+        return {value: {'avg': row[2 * index], 'max': row[2 * index + 1]} for index, value in enumerate(values)}
+
+    points = {}
+
+    def point(start):
+        return points.setdefault(start, {'time': start, 'network': {}, 'clients': {}})
+
+    for row in rows('metrics_system'):
+        point(row[0]).update(pairs(row[1:], TABLES['metrics_system'][1]))
+    for row in rows('metrics_network'):
+        point(row[0])['network'][row[1]] = pairs(row[2:], TABLES['metrics_network'][1])
+    for row in rows('metrics_vpn'):
+        point(row[0])['clients'][row[1]] = pairs(row[2:], TABLES['metrics_vpn'][1])['clients']
+    empty = {value: {'avg': None, 'max': None} for value in TABLES['metrics_system'][1]}
+    return {'range': view, 'bucket': bucket, 'points': [{**empty, **points[start]} for start in sorted(points)]}

debian-package/src/usr/lib/vpn-router/vpnrouter_net.py

@@ -1,11 +1,21 @@
 """Interfaces and addresses of the router, read from the system.
 
-Only the Python standard library is used, so the web service can use this module too.
+Only the Python standard library is used, so the web service and its bootstrap server can use
+this module too.
 """
 
 import pathlib
 import subprocess
 
+#: The values of web.listen and of the Router Manager's listen: the interfaces a service listens on.
+LISTEN_CHOICES = ('internal', 'external', 'both')
+
+#: The interfaces a service listens on when it is not set.
+DEFAULT_LISTEN = 'both'
+
+#: The address every service also listens on, where the HTTPS proxy calls it.
+LOOPBACK = '127.0.0.1'
+
 
 class ConfigError(Exception):
     """A setting is present but malformed."""
@@ -107,3 +117,60 @@ def internal_address(mode, internal, int_addr):
     except ConfigError:
         return ''
     return addresses[0] if addresses else ''
+
+
+def external_address(mode, external):
+    """The router's address on the external interface, as general.mode directs, or '' when it is not known yet.
+
+    auto works out which interface that is; the other modes read the address of interfaces.external.
+    """
+    try:
+        if mode == 'auto':
+            external = detect_interfaces()[0]
+        addresses = interface_addresses(external, 'interfaces.external')
+    except ConfigError:
+        return ''
+    return addresses[0] if addresses else ''
+
+
+def _section(configuration, name):
+    """A section of a configuration, or {} when it is missing or not an object."""
+    value = configuration.get(name)
+    return value if isinstance(value, dict) else {}
+
+
+def listen_addresses(configuration, choice):
+    """The addresses a service listens on for choice of LISTEN_CHOICES, from vpn-router.json read as configuration.
+
+    The internal address comes first; an address not known yet is left out. A choice that is not
+    one of LISTEN_CHOICES is DEFAULT_LISTEN. LOOPBACK is not among them; web_listen adds it.
+    """
+    if choice not in LISTEN_CHOICES:
+        choice = DEFAULT_LISTEN
+    general, interfaces, local = (_section(configuration, name) for name in ('general', 'interfaces', 'local'))
+    mode = str(general.get('mode') or 'manual')
+    addresses = []
+    if choice in ('internal', 'both'):
+        addresses.append(internal_address(mode, str(interfaces.get('internal') or ''), str(local.get('int_addr') or '')))
+    if choice in ('external', 'both'):
+        addresses.append(external_address(mode, str(interfaces.get('external') or '')))
+    return [address for address in addresses if address]
+
+
+def web_listen(configuration, default_port):
+    """Where the web service listens, from vpn-router.json read as configuration, as ([(address, port), ...], '').
+
+    The addresses are those of the interfaces web.listen names (listen_addresses), then
+    LOOPBACK, the port web.port, else default_port. While web.enabled is false or no address of
+    an interface is known, the answer is ([], the reason).
+    """
+    web = _section(configuration, 'web')
+    if web.get('enabled', True) is False:
+        return [], 'the web service is disabled in the configuration'
+    port = web.get('port', default_port)
+    if isinstance(port, bool) or not isinstance(port, int) or not 1 <= port <= 65535:
+        port = default_port
+    addresses = listen_addresses(configuration, web.get('listen', DEFAULT_LISTEN))
+    if not addresses:
+        return [], 'the addresses to listen on are not known yet'
+    return [(address, port) for address in [*addresses, LOOPBACK]], ''

debian-package/src/usr/lib/vpn-router/vpnrouter_serve.py

@@ -0,0 +1,46 @@
+"""The listening loop of the web service and of its bootstrap server.
+
+Only the Python standard library is used, so the bootstrap server, which must run whatever else
+is broken, can use it.
+"""
+
+import threading
+import time
+from http.server import ThreadingHTTPServer
+
+
+def serve(wanted, handler, log, interval):
+    """Work out where to listen every interval seconds with wanted, and run an HTTP server of handler at each place.
+
+    wanted answers ([(address, port), ...], '') or ([], the reason not to listen). A server whose
+    place is no longer wanted is stopped, and a new one listens at each new place; a place that
+    cannot be bound is tried again the next round. Every new message is logged with log. Does
+    not return.
+    """
+    servers, reported = {}, None
+    while True:
+        targets, reason = wanted()
+        for target in [target for target in servers if target not in targets]:
+            server, thread = servers.pop(target)
+            server.shutdown()
+            server.server_close()
+            thread.join()
+        messages = [reason] if reason else []
+        for target in targets:
+            if target in servers:
+                continue
+            try:
+                server = ThreadingHTTPServer(target, handler)
+            except OSError as exc:
+                messages.append(f'cannot listen on {target[0]}:{target[1]}: {exc}')
+                continue
+            thread = threading.Thread(target=server.serve_forever, name=f'server {target[0]}', daemon=True)
+            thread.start()
+            servers[target] = (server, thread)
+        if servers:
+            messages.append('serving on ' + ', '.join(f'http://{address}:{port}' for address, port in servers))
+        message = '; '.join(messages)
+        if message and message != reported:
+            log(message)
+            reported = message
+        time.sleep(interval)

debian-package/src/usr/lib/vpn-router/vpnrouter_status.py

@@ -0,0 +1,111 @@
+"""What the router's VPN systems report, for the web service and the metrics service.
+
+Each reader asks the running system once and answers None when it cannot be asked: swanctl for
+the IKE SAs, wg for the WireGuard peers and the OpenVPN server's status file for its clients.
+"""
+
+import subprocess
+import time
+
+COMMAND_TIMEOUT = 5
+
+OPENVPN_STATUS = '/run/openvpn-server/status-vpn-router.log'
+
+#: Seconds since its last handshake within which a WireGuard peer counts as connected.
+WIREGUARD_CONNECTED_SECONDS = 180
+
+
+def run(*args):
+    """The finished process of a command, or None when it cannot be run."""
+    try:
+        return subprocess.run(args, capture_output=True, text=True, timeout=COMMAND_TIMEOUT)
+    except (OSError, subprocess.TimeoutExpired):
+        return None
+
+
+def swanctl(command):
+    """The entries of a swanctl list command, or None when swanctl cannot be asked.
+
+    The --pretty output has one item per line: "name {" opens a section, "key = [" a list
+    whose items follow one per line, "}" and "]" close them, and "key = value" is a pair
+    whose value may contain spaces. Every "... event {" section at the top holds one
+    connection or SA; each becomes an entry with its name under "name". The closing
+    "... reply {" section is left out. Values stay strings, as swanctl prints them.
+    Output with more closing than opening lines gives None.
+    """
+    result = run('swanctl', command, '--pretty')
+    if result is None or result.returncode != 0:
+        return None
+    events = []
+    stack = []
+    for line in result.stdout.splitlines():
+        line = line.strip()
+        if not line:
+            continue
+        if line in ('}', ']'):
+            if not stack:
+                return None
+            stack.pop()
+        elif not stack:
+            section = {}
+            if line.endswith(' event {'):
+                events.append(section)
+            stack.append(section)
+        elif isinstance(stack[-1], list):
+            stack[-1].append(line)
+        elif line.endswith(' {'):
+            stack[-1][line[:-2]] = section = {}
+            stack.append(section)
+        elif line.endswith(' = ['):
+            stack[-1][line[:-4]] = items = []
+            stack.append(items)
+        else:
+            key, _, value = line.partition(' = ')
+            stack[-1][key] = value
+    return [dict(body, name=name) for event in events for name, body in event.items()]
+
+
+def established(connection):
+    """The number of established IKE SAs of the connection, or None when swanctl cannot be asked."""
+    sas = swanctl('--list-sas')
+    if sas is None:
+        return None
+    return sum(1 for sa in sas if sa['name'] == connection and sa.get('state') == 'ESTABLISHED')
+
+
+def wireguard_peers():
+    """One entry per WireGuard peer, with its handshake age and traffic, or None when wg0 is down."""
+    result = run('wg', 'show', 'wg0', 'dump')
+    if result is None or result.returncode != 0:
+        return None
+    now = int(time.time())
+    peers = []
+    for line in result.stdout.splitlines()[1:]:
+        fields = line.split('\t')
+        if len(fields) < 7:
+            continue
+        handshake = int(fields[4])
+        peers.append({
+            'handshake_seconds_ago': now - handshake if handshake else None,
+            'received_bytes': int(fields[5]),
+            'sent_bytes': int(fields[6]),
+        })
+    return peers
+
+
+def wireguard_connected():
+    """The number of WireGuard peers with a handshake in the last WIREGUARD_CONNECTED_SECONDS, or None when wg0 is down."""
+    peers = wireguard_peers()
+    if peers is None:
+        return None
+    return sum(1 for peer in peers
+               if peer['handshake_seconds_ago'] is not None and peer['handshake_seconds_ago'] < WIREGUARD_CONNECTED_SECONDS)
+
+
+def openvpn_clients():
+    """The number of connected OpenVPN clients, from the server's status file, or None when it cannot be read."""
+    try:
+        with open(OPENVPN_STATUS, encoding='utf-8') as handle:
+            return sum(1 for line in handle if line.startswith('CLIENT_LIST,'))
+    except OSError:
+        return None

debian-package/src/usr/lib/vpn-router/vpnrouter_users.py

@@ -0,0 +1,581 @@
+"""The users and groups of the router's directory, for vpn-router-user, vpn-router-group and the web service.
+
+A user is an entry in ou=users of the router's directory, of one of two kinds, fixed when it is
+added. A VPN user is an inetOrgPerson with a first name, a last name and an e-mail address, whose
+password OpenVPN checks, and a client certificate of the router CA with the user name as its
+common name, which OpenVPN and IKEv2 accept. A Manager-only account is an account,
+simpleSecurityObject and vpnRouterAccount (the package's schema) with a name, a password and
+optionally an e-mail address, no names, no certificate and so no VPN access; it is added when
+neither the first nor the last name is given, and can sign in to the web service. A service
+account is a Manager-only account with the scope SERVICE_SCOPE in vpnRouterScope, which
+vpn-router-setup adds and keeps for the metrics service, as metrics.service_account names it; it
+cannot sign in to the web service. Users exist whether or not a P2S system is enabled.
+
+A group is a groupOfNames in ou=groups with a name, which follows the rules of user names, an
+optional description and at least one member; a group cannot be empty, so the last member
+cannot leave it and a user who is the last member of a group cannot be removed. Groups only
+record membership; nothing on the router reads them.
+
+A user may have a TOTP token: the user entry itself is an oathTOTPUser and oathTOTPToken whose
+oathTOTPToken points to itself, with the secret in oathSecret and the parameters of
+vpnrouter_ldap.TOTP_PARAMS. Enrolment stores nothing until a code of the new secret is
+verified: otp_new makes a secret and its otpauth URI, and otp_enrol stores the secret given a
+code of it. slapd's otp overlay then takes the password followed by a code at every bind.
+
+A user is disabled when pwdAccountLockedTime holds LOCKED, and locked when the password
+policy has set it after failed binds, for vpnrouter_ldap.LOCKOUT_SECONDS; unlock ends that.
+login() checks a user's password and code with a bind. Members of ADMIN_GROUP manage the
+router through the web service. The module knows only the directory; the callers revoke a
+user's refresh tokens after the operations of vpnrouter_auth.REVOKING.
+
+Every operation runs in a session(), which holds the lock that serializes changes and the
+connection of the user manager account. A request that cannot be done raises Failure, whose
+status says why: 2 the request is invalid, 3 the user does not exist, 4 the user exists, 5 a
+login failed, 1 anything else. Answers are dicts with the keys of the JSON answers.
+"""
+
+import base64
+import calendar
+import contextlib
+import fcntl
+import pathlib
+import re
+import subprocess
+import sys
+import time
+
+import ldap
+import ldap.filter
+import ldap.modlist
+import pyotp
+
+import vpnrouter
+import vpnrouter_ldap
+
+#: A user name is one lowercase DNS label, as it is also the common name of the certificate.
+NAME = re.compile(r'^[a-z][a-z0-9-]{0,30}[a-z0-9]$|^[a-z]$')
+PASSWORD_MIN = 8
+PASSWORD_MAX = 128
+#: The longest first or last name, in characters.
+PERSON_NAME_MAX = 64
+#: An e-mail address: one @, no spaces, and a dot in the domain.
+EMAIL = re.compile(r'^[^@\s]+@[^@\s]+\.[^@\s]+$')
+EMAIL_MAX = 254
+#: The keys of the person attributes and their LDAP attributes.
+PERSON = (('first_name', 'givenName'), ('last_name', 'sn'), ('email', 'mail'))
+#: The attributes read for a user.
+ATTRIBUTES = ['uid', 'objectClass', 'pwdAccountLockedTime', 'givenName', 'sn', 'mail', 'oathTOTPToken', 'vpnRouterScope']
+#: The vpnRouterScope of a service account.
+SERVICE_SCOPE = 'service'
+#: The object classes of a VPN user and of a Manager-only account.
+VPN_USER_CLASSES = [b'inetOrgPerson']
+MANAGER_ONLY_CLASSES = [b'account', b'simpleSecurityObject', b'vpnRouterAccount']
+#: The readable attributes of a TOTP token; oathSecret is never readable.
+TOKEN_ATTRIBUTES = ['oathTOTPToken', 'oathTOTPParams', 'oathSecretTime', 'oathTOTPLastTimeStep', 'oathTOTPTimeStepDrift',
+                    'oathLastLogin', 'oathLastFailure', 'oathFailureCount']
+#: A TOTP secret as otp_new makes it: 32 base32 characters, 160 bits.
+TOTP_SECRET = re.compile(r'^[A-Z2-7]{32}$')
+#: The attributes read for a group.
+GROUP_ATTRIBUTES = ['cn', 'description', 'member']
+#: The longest group description, in characters.
+DESCRIPTION_MAX = 256
+#: The value of pwdAccountLockedTime that disables an account until it is enabled.
+LOCKED = b'000001010000Z'
+#: The group whose members manage the router through the web service.
+ADMIN_GROUP = 'admins'
+#: The digits of a TOTP code given to login().
+CODE = re.compile(r'^\d{6}$')
+LOCK_FILE = pathlib.Path('/run/vpn-router/users.lock')
+
+
+class Failure(Exception):
+    """A request that cannot be done; status is 2 invalid, 3 no such user, 4 the user exists, 5 failed login, 1 other."""
+
+    def __init__(self, message, status=1):
+        super().__init__(message)
+        self.status = status
+
+
+@contextlib.contextmanager
+def session():
+    """The connection of the user manager account, used under the lock that serializes changes."""
+    LOCK_FILE.parent.mkdir(parents=True, exist_ok=True)
+    with open(LOCK_FILE, 'w') as lock:
+        fcntl.flock(lock, fcntl.LOCK_EX)
+        connection = vpnrouter_ldap.manager()
+        try:
+            yield connection
+        finally:
+            connection.unbind_s()
+
+
+def check_name(name):
+    if not isinstance(name, str) or not NAME.match(name):
+        raise Failure('a user name is 1 to 32 lowercase letters, digits and hyphens, starting with a letter', 2)
+
+
+def check_password(value):
+    if not isinstance(value, str) or '\n' in value or '\r' in value:
+        raise Failure('the password must be a string of one line', 2)
+    if not PASSWORD_MIN <= len(value) <= PASSWORD_MAX:
+        raise Failure(f'the password must have {PASSWORD_MIN} to {PASSWORD_MAX} characters', 2)
+
+
+def person(values, required):
+    """The person attributes of values ({first_name, last_name, email}), checked, as {LDAP attribute: value}.
+
+    A key whose value is None is not given; with required, all three must be given.
+    """
+    found = {}
+    for key, attribute in PERSON:
+        value = values.get(key)
+        if value is None:
+            if required:
+                raise Failure(f'{key} is required', 2)
+            continue
+        if not isinstance(value, str):
+            raise Failure(f'{key} must be a string', 2)
+        value = value.strip()
+        if key == 'email':
+            if len(value) > EMAIL_MAX or not EMAIL.match(value):
+                raise Failure(f'{value!r} is not an e-mail address', 2)
+        elif not value or len(value) > PERSON_NAME_MAX or any(not character.isprintable() for character in value):
+            raise Failure(f'the {key.replace("_", " ")} must have 1 to {PERSON_NAME_MAX} printable characters', 2)
+        found[attribute] = value
+    return found
+
+
+def dn_of(name):
+    return f'uid={name},{vpnrouter_ldap.USERS}'
+
+
+def certificate(name):
+    return vpnrouter.USER_CERT_DIR / f'{name}_cert.pem', vpnrouter.USER_CERT_DIR / f'{name}_key.pem'
+
+
+def entry(connection, name):
+    """The attributes of the user's entry; Failure with status 3 when there is none."""
+    check_name(name)
+    try:
+        return connection.search_s(dn_of(name), ldap.SCOPE_BASE, attrlist=ATTRIBUTES)[0][1]
+    except ldap.NO_SUCH_OBJECT:
+        raise Failure(f'user {name} does not exist', 3)
+
+
+def simple_ca(*args):
+    result = subprocess.run([sys.executable, str(vpnrouter.SIMPLE_CA), *args], capture_output=True, text=True)
+    if result.returncode != 0:
+        raise Failure(f'simple-ca {args[0]} failed: {result.stderr.strip() or result.stdout.strip()}')
+
+
+def publish_crl():
+    """Give strongSwan the CRL after a revocation; OpenVPN reads it at every connection."""
+    if vpnrouter.install_file(vpnrouter.PKI_DIR / 'crl.pem', vpnrouter.CRL_DEST, 0o644):
+        subprocess.run(['swanctl', '--load-creds'], capture_output=True, text=True)
+
+
+def locked_since(value):
+    """The time a pwdAccountLockedTime value names, in seconds since the epoch, or 0 when it names none."""
+    try:
+        return calendar.timegm(time.strptime(value[:14].decode(), '%Y%m%d%H%M%S'))
+    except ValueError:
+        return 0
+
+
+def manager_only(attributes):
+    """Whether the entry is a Manager-only account."""
+    return b'account' in (value.lower() for value in attributes.get('objectClass', []))
+
+
+def service(attributes):
+    """Whether the entry is a service account."""
+    return SERVICE_SCOPE.encode() in (value.lower() for value in attributes.get('vpnRouterScope', []))
+
+
+def summary(attributes):
+    """A user as the answers show it; a person attribute the entry does not have is None."""
+    name = attributes['uid'][0].decode()
+    lock = attributes.get('pwdAccountLockedTime', [None])[0]
+    user = {
+        'name': name,
+        'manager_only': manager_only(attributes),
+        'service': service(attributes),
+        'enabled': lock != LOCKED,
+        'locked': lock not in (None, LOCKED) and locked_since(lock) > time.time() - vpnrouter_ldap.LOCKOUT_SECONDS,
+        'certificate': certificate(name)[0].is_file(),
+        'otp': 'oathTOTPToken' in attributes,
+    }
+    for key, attribute in PERSON:
+        values = attributes.get(attribute)
+        user[key] = values[0].decode() if values else None
+    return user
+
+
+def common_name(attributes, name):
+    """The cn of an entry: the first and the last name, or the user name without them."""
+    full = ' '.join(value for value in (attributes.get('givenName'), attributes.get('sn')) if value)
+    return full or name
+
+
+def list_users(connection):
+    found = connection.search_s(vpnrouter_ldap.USERS, ldap.SCOPE_ONELEVEL, '(|(objectClass=inetOrgPerson)(objectClass=account))',
+                                ATTRIBUTES)
+    return {'users': sorted((summary(attributes) for _, attributes in found), key=lambda user: user['name'])}
+
+
+def add(connection, name, password, values, service=False):
+    """Add a user: a VPN user with its certificate when values gives all three person attributes, a Manager-only account
+    with at most the e-mail address when it gives neither name, and, with service, which only vpn-router-setup uses, a
+    service account, with at most the e-mail address too."""
+    check_name(name)
+    details = person(values, required=False)
+    vpn_user = 'givenName' in details or 'sn' in details
+    if service and vpn_user:
+        raise Failure('a service account has no first or last name', 2)
+    if vpn_user and len(details) != len(PERSON):
+        raise Failure('give first_name, last_name and email for a VPN user, or at most email for a Manager-only account', 2)
+    check_password(password)
+    if vpn_user and not (vpnrouter.PKI_DIR / 'ca_key.pem').is_file():
+        raise Failure('the router CA does not exist; vpn-router-setup creates it once a platform is chosen')
+    if vpn_user:
+        record = {'objectClass': VPN_USER_CLASSES, 'uid': [name.encode()], 'cn': [common_name(details, name).encode()]}
+    else:
+        # simpleSecurityObject requires a userPassword; passwd_s sets the real one at once.
+        record = {'objectClass': MANAGER_ONLY_CLASSES, 'uid': [name.encode()], 'userPassword': [b'{CRYPT}!']}
+        if service:
+            record['vpnRouterScope'] = [SERVICE_SCOPE.encode()]
+    record.update({attribute: [value.encode()] for attribute, value in details.items()})
+    try:
+        connection.add_s(dn_of(name), ldap.modlist.addModlist(record))
+    except ldap.ALREADY_EXISTS:
+        raise Failure(f'user {name} exists', 4)
+    connection.passwd_s(dn_of(name), None, password)
+    if vpn_user:
+        vpnrouter.USER_CERT_DIR.mkdir(mode=0o700, parents=True, exist_ok=True)
+        for path in certificate(name):
+            path.unlink(missing_ok=True)
+        simple_ca('make-cert', '--ca-dir', str(vpnrouter.PKI_DIR), '--cert-dir', str(vpnrouter.USER_CERT_DIR), name)
+    return summary(entry(connection, name))
+
+
+def update(connection, name, values):
+    """Change the person attributes values gives; the cn follows the first and the last name.
+
+    A Manager-only account has only the e-mail address, which '' removes.
+    """
+    current = entry(connection, name)
+    if manager_only(current):
+        if values.get('first_name') is not None or values.get('last_name') is not None:
+            raise Failure(f'user {name} is a Manager-only account, which has no first or last name', 2)
+        email = values.get('email')
+        if email is None:
+            raise Failure('give email', 2)
+        if isinstance(email, str) and email.strip() == '':
+            if 'mail' in current:
+                connection.modify_s(dn_of(name), [(ldap.MOD_DELETE, 'mail', None)])
+        else:
+            connection.modify_s(dn_of(name), [(ldap.MOD_REPLACE, 'mail', [person({'email': email}, required=False)['mail'].encode()])])
+        return summary(entry(connection, name))
+    details = person(values, required=False)
+    if not details:
+        raise Failure('give at least one of first_name, last_name and email', 2)
+    names = {attribute: current[attribute][0].decode() for attribute in ('givenName', 'sn') if current.get(attribute)}
+    names.update({attribute: details[attribute] for attribute in ('givenName', 'sn') if attribute in details})
+    changes = [(ldap.MOD_REPLACE, attribute, [value.encode()]) for attribute, value in details.items()]
+    changes.append((ldap.MOD_REPLACE, 'cn', [common_name(names, name).encode()]))
+    connection.modify_s(dn_of(name), changes)
+    return summary(entry(connection, name))
+
+
+def passwd(connection, name, password):
+    check_password(password)
+    entry(connection, name)
+    connection.passwd_s(dn_of(name), None, password)
+    return summary(entry(connection, name))
+
+
+def disable(connection, name):
+    if entry(connection, name).get('pwdAccountLockedTime') != [LOCKED]:
+        connection.modify_s(dn_of(name), [(ldap.MOD_REPLACE, 'pwdAccountLockedTime', [LOCKED])])
+    return summary(entry(connection, name))
+
+
+def enable(connection, name):
+    if entry(connection, name).get('pwdAccountLockedTime') == [LOCKED]:
+        connection.modify_s(dn_of(name), [(ldap.MOD_DELETE, 'pwdAccountLockedTime', None)])
+    return summary(entry(connection, name))
+
+
+def unlock(connection, name):
+    """End a lockout after failed binds and clear the failure count; a disabled user stays disabled."""
+    entry(connection, name)
+    current = connection.search_s(dn_of(name), ldap.SCOPE_BASE, attrlist=['pwdAccountLockedTime', 'pwdFailureTime'])[0][1]
+    changes = [(ldap.MOD_DELETE, 'pwdFailureTime', None)] if 'pwdFailureTime' in current else []
+    if 'pwdAccountLockedTime' in current and current['pwdAccountLockedTime'] != [LOCKED]:
+        changes.append((ldap.MOD_DELETE, 'pwdAccountLockedTime', None))
+    if changes:
+        connection.modify_s(dn_of(name), changes)
+    return summary(entry(connection, name))
+
+
+def login(name, password, code):
+    """Check the user's password, and the TOTP code of an enrolled user, with a bind as the user.
+
+    A wrong name, password or code, a locked or a disabled user give the same Failure, status 5.
+    """
+    refused = Failure('the name, the password or the code is wrong', 5)
+    if not isinstance(name, str) or not NAME.match(name) or not isinstance(password, str) or not password:
+        raise refused
+    if code is not None and code != '' and (not isinstance(code, str) or not CODE.match(code)):
+        raise refused
+    connection = ldap.initialize(vpnrouter_ldap.URI)
+    try:
+        connection.simple_bind_s(dn_of(name), password + (code or ''))
+    except ldap.INVALID_CREDENTIALS:
+        raise refused
+    finally:
+        connection.unbind_s()
+
+
+def is_admin(connection, name):
+    """Whether the user exists and is a member of ADMIN_GROUP."""
+    try:
+        entry(connection, name)
+    except Failure:
+        return False
+    return ADMIN_GROUP in groups_of(connection, name)
+
+
+def totp(secret):
+    return pyotp.TOTP(secret, digits=vpnrouter_ldap.TOTP_DIGITS, interval=vpnrouter_ldap.TOTP_PERIOD)
+
+
+def otp_new(connection, name):
+    """A new TOTP secret for the user and its otpauth URI, issued by the router's wan.local_fqdn; nothing is stored."""
+    entry(connection, name)
+    secret = pyotp.random_base32()
+    issuer = vpnrouter.load_config()['wan']['local_fqdn'].strip()
+    return {'name': name, 'secret': secret, 'uri': totp(secret).provisioning_uri(name=name, issuer_name=issuer or None)}
+
+
+def otp_enrol(connection, name, secret, code):
+    """Store the secret of otp_new as the user's TOTP token, replacing one the user has, when code is a code of it."""
+    current = entry(connection, name)
+    if not isinstance(secret, str) or not TOTP_SECRET.match(secret):
+        raise Failure('the secret must be 32 base32 characters, as a new secret has them', 2)
+    if not isinstance(code, str) or not re.fullmatch(rf'\d{{{vpnrouter_ldap.TOTP_DIGITS}}}', code):
+        raise Failure(f'the code must be {vpnrouter_ldap.TOTP_DIGITS} digits', 2)
+    if not totp(secret).verify(code, valid_window=vpnrouter_ldap.TOTP_WINDOW):
+        raise Failure('the code does not match the secret; check the clock of the authenticator', 2)
+    changes = [] if 'oathTOTPToken' in current else [(ldap.MOD_ADD, 'objectClass', [b'oathTOTPUser', b'oathTOTPToken'])]
+    changes += [
+        (ldap.MOD_REPLACE, 'oathTOTPToken', [dn_of(name).encode()]),
+        (ldap.MOD_REPLACE, 'oathTOTPParams', [vpnrouter_ldap.TOTP_PARAMS.encode()]),
+        (ldap.MOD_REPLACE, 'oathSecret', [base64.b32decode(secret)]),
+        (ldap.MOD_REPLACE, 'oathSecretTime', [time.strftime('%Y%m%d%H%M%SZ', time.gmtime()).encode()]),
+    ]
+    connection.modify_s(dn_of(name), changes)
+    return summary(entry(connection, name))
+
+
+def otp_remove(connection, name):
+    """Remove the user's TOTP token, when the user has one."""
+    if 'oathTOTPToken' in entry(connection, name):
+        token = connection.search_s(dn_of(name), ldap.SCOPE_BASE, attrlist=TOKEN_ATTRIBUTES)[0][1]
+        changes = [(ldap.MOD_DELETE, 'objectClass', [b'oathTOTPUser', b'oathTOTPToken']),
+                   (ldap.MOD_DELETE, 'oathSecret', None)]
+        changes += [(ldap.MOD_DELETE, attribute, None) for attribute in token]
+        connection.modify_s(dn_of(name), changes)
+    return summary(entry(connection, name))
+
+
+def remove(connection, name):
+    """Remove the user from its groups and the directory, and revoke the certificate.
+
+    A user who is the last member of a group is not removed.
+    """
+    entry(connection, name)
+    groups = groups_of(connection, name)
+    alone = sorted(group for group, members in groups.items() if members == [name])
+    if alone:
+        raise Failure(f'user {name} is the last member of {", ".join(alone)}; remove the group first', 2)
+    for group in groups:
+        connection.modify_s(group_dn(group), [(ldap.MOD_DELETE, 'member', [dn_of(name).encode()])])
+    cert, key = certificate(name)
+    if cert.is_file():
+        simple_ca('revoke-cert', '--ca-dir', str(vpnrouter.PKI_DIR), str(cert))
+        simple_ca('make-crl', '--ca-dir', str(vpnrouter.PKI_DIR), '--days', str(vpnrouter.CRL_DAYS))
+        publish_crl()
+    cert.unlink(missing_ok=True)
+    key.unlink(missing_ok=True)
+    connection.delete_s(dn_of(name))
+    return {'name': name, 'removed': True}
+
+
+def profile(connection, name):
+    """The OpenVPN client profile of the user, with the CA, the certificate and the key inline."""
+    entry(connection, name)
+    cfg = vpnrouter.load_config()
+    if not cfg['openvpn']['enabled']:
+        raise Failure('OpenVPN is not enabled', 2)
+    cert, key = certificate(name)
+    if not cert.is_file() or not key.is_file():
+        raise Failure(f'user {name} has no certificate')
+    fqdn = cfg['wan']['local_fqdn'].strip()
+    text = '\n'.join([
+        'client',
+        'dev tun',
+        f'proto {cfg["openvpn"]["protocol"]}',
+        f'remote {fqdn} {cfg["openvpn"]["port"]}',
+        'nobind',
+        'persist-key',
+        'persist-tun',
+        'remote-cert-tls server',
+        f'verify-x509-name {fqdn} name',
+        'auth-user-pass',
+        'auth-nocache',
+        'verb 3',
+        '<ca>', extract_pem(vpnrouter.CA_CERT_DEST.read_text(encoding='utf-8')), '</ca>',
+        '<cert>', extract_pem(cert.read_text(encoding='utf-8')), '</cert>',
+        '<key>', key.read_text(encoding='utf-8').strip(), '</key>',
+        '',
+    ])
+    return {'name': name, 'profile': text}
+
+
+def group_dn(name):
+    return f'cn={name},{vpnrouter_ldap.GROUPS}'
+
+
+def check_group_name(name):
+    if not isinstance(name, str) or not NAME.match(name):
+        raise Failure('a group name is 1 to 32 lowercase letters, digits and hyphens, starting with a letter', 2)
+
+
+def check_description(value):
+    """The description, stripped; '' means none."""
+    if not isinstance(value, str):
+        raise Failure('the description must be a string', 2)
+    value = value.strip()
+    if len(value) > DESCRIPTION_MAX or any(not character.isprintable() for character in value):
+        raise Failure(f'the description must have at most {DESCRIPTION_MAX} printable characters', 2)
+    return value
+
+
+def member_names(attributes):
+    """The user names of a group's member DNs, sorted."""
+    names = []
+    for value in attributes.get('member', []):
+        dn = value.decode()
+        if dn.startswith('uid=') and dn.endswith(f',{vpnrouter_ldap.USERS}'):
+            names.append(dn[4:-len(f',{vpnrouter_ldap.USERS}')])
+    return sorted(names)
+
+
+def group_summary(attributes):
+    """A group as the answers show it."""
+    values = attributes.get('description')
+    return {
+        'name': attributes['cn'][0].decode(),
+        'description': values[0].decode() if values else '',
+        'members': member_names(attributes),
+    }
+
+
+def group_entry(connection, name):
+    """The attributes of the group's entry; Failure with status 3 when there is none."""
+    check_group_name(name)
+    try:
+        return connection.search_s(group_dn(name), ldap.SCOPE_BASE, attrlist=GROUP_ATTRIBUTES)[0][1]
+    except ldap.NO_SUCH_OBJECT:
+        raise Failure(f'group {name} does not exist', 3)
+
+
+def groups_of(connection, name):
+    """The groups the user is a member of, as {group name: its member names}."""
+    found = connection.search_s(vpnrouter_ldap.GROUPS, ldap.SCOPE_ONELEVEL,
+                                f'(&(objectClass=groupOfNames)(member={ldap.filter.escape_filter_chars(dn_of(name))}))',
+                                GROUP_ATTRIBUTES)
+    return {attributes['cn'][0].decode(): member_names(attributes) for _, attributes in found}
+
+
+def list_groups(connection):
+    found = connection.search_s(vpnrouter_ldap.GROUPS, ldap.SCOPE_ONELEVEL, '(objectClass=groupOfNames)', GROUP_ATTRIBUTES)
+    return {'groups': sorted((group_summary(attributes) for _, attributes in found), key=lambda group: group['name'])}
+
+
+def group_add(connection, name, description, members):
+    """Add a group with a description ('' for none) and at least one member, each an existing user."""
+    check_group_name(name)
+    description = check_description(description if description is not None else '')
+    if not isinstance(members, list) or not members:
+        raise Failure('a group needs at least one member', 2)
+    for member in members:
+        entry(connection, member)
+    record = {'objectClass': [b'groupOfNames'], 'cn': [name.encode()],
+              'member': [dn_of(member).encode() for member in sorted(set(members))]}
+    if description:
+        record['description'] = [description.encode()]
+    try:
+        connection.add_s(group_dn(name), ldap.modlist.addModlist(record))
+    except ldap.ALREADY_EXISTS:
+        raise Failure(f'group {name} exists', 4)
+    return group_summary(group_entry(connection, name))
+
+
+def group_update(connection, name, new_name=None, description=None):
+    """Rename the group or change its description ('' removes it); at least one must be given."""
+    if new_name is None and description is None:
+        raise Failure('give a new name or a description', 2)
+    group_entry(connection, name)
+    if description is not None:
+        description = check_description(description)
+        change = (ldap.MOD_REPLACE, 'description', [description.encode()]) if description else \
+            (ldap.MOD_REPLACE, 'description', [])
+        connection.modify_s(group_dn(name), [change])
+    if new_name is not None and new_name != name:
+        check_group_name(new_name)
+        try:
+            connection.rename_s(group_dn(name), f'cn={new_name}', delold=1)
+        except ldap.ALREADY_EXISTS:
+            raise Failure(f'group {new_name} exists', 4)
+        name = new_name
+    return group_summary(group_entry(connection, name))
+
+
+def group_remove(connection, name):
+    group_entry(connection, name)
+    connection.delete_s(group_dn(name))
+    return {'name': name, 'removed': True}
+
+
+def group_member_add(connection, name, user):
+    attributes = group_entry(connection, name)
+    entry(connection, user)
+    if user not in member_names(attributes):
+        connection.modify_s(group_dn(name), [(ldap.MOD_ADD, 'member', [dn_of(user).encode()])])
+    return group_summary(group_entry(connection, name))
+
+
+def group_member_remove(connection, name, user):
+    """Take the user out of the group; the last member cannot leave."""
+    check_name(user)
+    members = member_names(group_entry(connection, name))
+    if user not in members:
+        raise Failure(f'user {user} is not a member of group {name}', 3)
+    if members == [user]:
+        raise Failure(f'user {user} is the last member of group {name}; remove the group instead', 2)
+    connection.modify_s(group_dn(name), [(ldap.MOD_DELETE, 'member', [dn_of(user).encode()])])
+    return group_summary(group_entry(connection, name))
+
+
+def extract_pem(text):
+    """The PEM blocks of the certificates in a file that also holds their text form."""
+    begin, end = '-----BEGIN CERTIFICATE-----', '-----END CERTIFICATE-----'
+    blocks = []
+    start = text.find(begin)
+    while start != -1:
+        stop = text.index(end, start) + len(end)
+        blocks.append(text[start:stop])
+        start = text.find(begin, stop)
+    return '\n'.join(blocks)

debian-package/src/usr/lib/vpn-router/wait-for-libpq

@@ -0,0 +1,12 @@
+#!/bin/sh
+# Wait until libpq is in the dynamic linker cache, where psycopg looks it up.
+# ExecStartPre= of vpn-router-web.service; TimeoutStartSec= of the unit bounds the wait.
+
+reported=
+until ldconfig -p | grep -q '^[[:space:]]*libpq\.so\.5 '; do
+    if [ -z "$reported" ]; then
+        echo 'waiting for libpq.so.5 in the linker cache'
+        reported=yes
+    fi
+    sleep 2
+done

debian-package/src/usr/lib/vpn-router/web

@@ -1,8 +1,8 @@
 #!/usr/bin/python3
-"""Serve the status of the router on its internal address.
+"""Serve the status of the router on its internal address, its external address or both, as web.listen sets.
 
-Run by vpn-router-web.service. It needs nothing but the Python standard library; users are
+Run by vpn-router-web.service. Users are managed with vpnrouter_users, the module that
-managed by running vpn-router-user.
+vpn-router-user uses too.
 
 Pages:
   /        the status for people: plain text for curl and wget, a styled page
@@ -10,81 +10,117 @@ Pages:
   /status  the status as JSON
   /config  the configuration without its secrets, and what the router resolved and
            runs, as JSON
-  /users   the users of the P2S systems: list, add, set the password, disable, enable,
+  /users   the users in the router's directory, whether or not a P2S system is enabled: list,
-           remove, and the OpenVPN client profile (see users())
+           add, change, set the password, disable, enable, remove, the OpenVPN client
+           profile, and TOTP enrolment (see users())
+  /groups  the groups of the router's directory: list, add, rename or describe, remove, and
+           add and take out members (see groups())
+  /account the account of the signed-in user: its details, its password and its TOTP
+           enrolment (see account())
+  /auth    sign-in with a user's password and TOTP code, refresh and sign-out (see auth())
+  /metrics/history
+           the metrics the metrics service keeps, for a view: hour, day, month or year (see
+           metrics_history())
+  /postgresql
+           the databases of each PostgreSQL cluster on the router with their sizes, as JSON (see
+           postgresql_databases()); /config has the clusters' settings
 
-/config and /users need an HS256 JWT in an Authorization: Bearer header that is signed
+/config, /metrics/history, /postgresql, /users, /groups and /account need an HS256 JWT in an Authorization: Bearer header signed
-with web.jwt_secret and expires within ten minutes; they are not served (404) while
+with vpnrouter_auth.WEB_KEY (see vpnrouter_auth.verify). A token of /auth names a user in sub;
-web.jwt_secret is not set.
+/account serves that user, and the other paths need a member of vpnrouter_users.ADMIN_GROUP. A
+token of vpn-router-token has the admin claim and names no user; it may call every path but
+/account, and only such a token may change ADMIN_GROUP: create, rename to it, change, remove it,
+add or take out its members, and remove one of its members. These paths and /auth are not
+served (404) while the key does not exist. /auth is limited to AUTH_LIMIT calls per client
+address in AUTH_WINDOW seconds; the address of a call from one of web.trusted_proxies, by
+default the address the call came in on, is the last entry of its X-Forwarded-For header.
 
-/ and /status show only the status: whether the services run, the package version, whether
+/ and /status show only the status, as much as web.status_content says (see shown): basic, the
-Docker is installed, and the state of site-to-site, road-warrior (IPsec P2S), OpenVPN and
+default, the overall status, the time and the client's address (its address as for the rate limit
-WireGuard. No configuration data, key or address is shown.
+below); standard also the package version, the installed version of vpn-router-manager
+(manager_version, null when it is not installed) and the number of active and other services; full
+whether each service runs, whether Docker is installed, and the state of site-to-site, road-warrior
+(IPsec P2S), OpenVPN and WireGuard. No configuration data or key is shown, and no address but the
+client's.
 
-The service needs nothing else to start and does not end on its own. A discovery thread works
+The service first initializes, without listening: it makes the signing key and the refresh
-out the internal address from /etc/vpn-router/vpn-router.json as general.mode directs, with
+tokens' database (vpnrouter_auth), retrying every INIT_RETRY_SECONDS until both are ready and
-web.port and web.enabled, every DISCOVERY_SECONDS, and runs the server thread there: it starts
+logging each failure. Then it listens. Its unit starts it only once wait-for-libpq has found libpq
-it, moves it when the address or the port changes, and stops it while the service is disabled
+in the linker cache, where psycopg looks it up.
-or the address is not known. Every request reads what it needs when it is made: the token secret
+
-from the configuration, and the features of the router and the facts shown by /config from
+The service needs nothing else to start and does not end on its own. Every DISCOVERY_SECONDS it
+works out from /etc/vpn-router/vpn-router.json the addresses of the interfaces web.listen names,
+internal, external or both, as general.mode directs, with web.port and web.enabled
+(vpnrouter_net.web_listen), and runs a server on each (vpnrouter_serve.serve): it starts them,
+moves them when an address or the port changes, and stops them while the service is disabled or
+no address is known. Every request reads what it needs when it is made: the signing
+key, the configuration, and the features of the router and the facts shown by /config from
 /run/vpn-router/web.json, which vpn-router-setup writes. Before setup has written that file
 the router has no features, and /config shows no configuration or interfaces.
 """
 
-import base64
+import collections
-import hashlib
-import hmac
 import json
 import os
 import signal
-import subprocess
 import sys
 import threading
 import time
+import urllib.parse
 from html import escape
-from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+from http.server import BaseHTTPRequestHandler
 
 sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
 
+import ldap
+
+import vpnrouter
+import vpnrouter_auth
+import vpnrouter_metrics
 import vpnrouter_net
+import vpnrouter_serve
+import vpnrouter_users
+from vpnrouter_status import established, openvpn_clients, run, swanctl, wireguard_peers
 
 STATE_FILE = '/run/vpn-router/web.json'
 CONFIG_FILE = '/etc/vpn-router/vpn-router.json'
 
+#: Seconds between two attempts of the initialization.
+INIT_RETRY_SECONDS = 15
+
 #: The port when web.port is not set.
-DEFAULT_PORT = 8080
+DEFAULT_PORT = 7081
 
 #: Seconds between two discoveries of where the service should listen.
 DISCOVERY_SECONDS = 30
-COMMAND_TIMEOUT = 5
 CACHE_SECONDS = 2
 
-#: Seconds of clock difference accepted between a token and this router.
+#: The calls of /auth allowed per client address in AUTH_WINDOW seconds.
-LEEWAY = 60
+AUTH_LIMIT = 10
-
+AUTH_WINDOW = 60
-#: The longest a token may be valid, from now, in seconds.
-MAX_LIFETIME = 600
 
 TAILWIND = 'https://cdn.jsdelivr.net/npm/@tailwindcss/browser@4'
 
 #: Seconds after which the status page in a browser reloads itself.
 REFRESH_SECONDS = 60
 
-USER_TOOL = '/usr/sbin/vpn-router-user'
+#: The PostgreSQL settings /config shows for each cluster.
-OPENVPN_STATUS = '/run/openvpn-server/status-vpn-router.log'
+POSTGRESQL_SETTINGS = ('server_version', 'listen_addresses', 'port', 'max_connections', 'shared_buffers',
-
+                       'effective_cache_size', 'work_mem', 'password_encryption', 'ssl', 'timezone')
-#: Seconds a user command may take; adding a user creates an RSA key.
-USER_TIMEOUT = 120
 
 #: The largest request body accepted, in bytes.
 BODY_LIMIT = 4096
 
-#: The HTTP status for each exit status of vpn-router-user; any other is 500.
+#: The HTTP status for each status of a vpnrouter_users.Failure; any other is 500.
-USER_STATUS = {0: 200, 2: 400, 3: 404, 4: 409}
+USER_STATUS = {2: 400, 3: 404, 4: 409, 5: 401}
 
 _cache = {'at': 0.0, 'data': None}
 _lock = threading.Lock()
 
+#: The times of the recent /auth calls of each client address.
+_auth_calls = collections.defaultdict(collections.deque)
+_auth_lock = threading.Lock()
+
 
 def read_json(path):
     """The object in a JSON file, or {} when the file is missing or holds no object."""
@@ -103,22 +139,12 @@ def section(configuration, name):
 
 
 def current():
-    """What a request needs, read now: the features and facts setup resolved, and the token secret.
+    """What a request needs, read now: the features and facts setup resolved, and the signing key.
 
-    Before setup has run the router has no features, and without a secret /config and /users are
+    Before setup has run the router has no features, and without a key the protected paths and
-    not served.
+    /auth are not served.
     """
-    state = read_json(STATE_FILE)
+    return dict(read_json(STATE_FILE), key=vpnrouter_auth.key())
-    secret = section(read_json(CONFIG_FILE), 'web').get('jwt_secret')
-    return dict(state, jwt_secret=secret if isinstance(secret, str) else '')
-
-
-def run(*args):
-    """The finished process of a command, or None when it cannot be run."""
-    try:
-        return subprocess.run(args, capture_output=True, text=True, timeout=COMMAND_TIMEOUT)
-    except (OSError, subprocess.TimeoutExpired):
-        return None
 
 
 def unit_state(unit):
@@ -126,9 +152,10 @@ def unit_state(unit):
     return result.stdout.strip() if result and result.stdout.strip() else 'unknown'
 
 
-def package_version():
+def package_version(package='vpn-router'):
-    result = run('dpkg-query', '-W', '-f=${Version}', 'vpn-router')
+    """The installed version of the package, or None when it is not installed."""
-    return result.stdout.strip() if result and result.returncode == 0 else None
+    result = run('dpkg-query', '-W', '-f=${Version}', package)
+    return (result.stdout.strip() or None) if result and result.returncode == 0 else None
 
 
 def docker_status():
@@ -143,100 +170,34 @@ def docker_status():
     }
 
 
-def swanctl(command):
+def postgresql_clusters():
-    """The entries of a swanctl list command, or None when swanctl cannot be asked.
+    """The state of each PostgreSQL cluster unit (postgresql@<version>-<name>), or {'postgresql': 'no cluster'} without one.
 
-    The --pretty output has one item per line: "name {" opens a section, "key = [" a list
+    postgresql.service, which only runs /bin/true, is not reported.
-    whose items follow one per line, "}" and "]" close them, and "key = value" is a pair
-    whose value may contain spaces. Every "... event {" section at the top holds one
-    connection or SA; each becomes an entry with its name under "name". The closing
-    "... reply {" section is left out. Values stay strings, as swanctl prints them.
-    Output with more closing than opening lines gives None.
     """
-    result = run('swanctl', command, '--pretty')
+    lines = lines_of('systemctl', 'list-units', 'postgresql@*.service', '--all', '--plain', '--no-legend') or []
-    if result is None or result.returncode != 0:
+    names = [line.split()[0].removesuffix('.service') for line in lines if line.strip()]
-        return None
+    return {name: unit_state(f'{name}.service') for name in names} or {'postgresql': 'no cluster'}
-    events = []
-    stack = []
-    for line in result.stdout.splitlines():
-        line = line.strip()
-        if not line:
-            continue
-        if line in ('}', ']'):
-            if not stack:
-                return None
-            stack.pop()
-        elif not stack:
-            section = {}
-            if line.endswith(' event {'):
-                events.append(section)
-            stack.append(section)
-        elif isinstance(stack[-1], list):
-            stack[-1].append(line)
-        elif line.endswith(' {'):
-            stack[-1][line[:-2]] = section = {}
-            stack.append(section)
-        elif line.endswith(' = ['):
-            stack[-1][line[:-4]] = items = []
-            stack.append(items)
-        else:
-            key, _, value = line.partition(' = ')
-            stack[-1][key] = value
-    return [dict(body, name=name) for event in events for name, body in event.items()]
-
-
-def established(connection):
-    """The number of established IKE SAs of the connection, or None when swanctl cannot be asked."""
-    sas = swanctl('--list-sas')
-    if sas is None:
-        return None
-    return sum(1 for sa in sas if sa['name'] == connection and sa.get('state') == 'ESTABLISHED')
-
-
-def wireguard_peers():
-    """One entry per WireGuard peer, with its handshake age and traffic, or None when wg0 is down."""
-    result = run('wg', 'show', 'wg0', 'dump')
-    if result is None or result.returncode != 0:
-        return None
-    now = int(time.time())
-    peers = []
-    for line in result.stdout.splitlines()[1:]:
-        fields = line.split('\t')
-        if len(fields) < 7:
-            continue
-        handshake = int(fields[4])
-        peers.append({
-            'handshake_seconds_ago': now - handshake if handshake else None,
-            'received_bytes': int(fields[5]),
-            'sent_bytes': int(fields[6]),
-        })
-    return peers
-
-
-def openvpn_clients():
-    """The number of connected OpenVPN clients, from the server's status file, or None when it cannot be read."""
-    try:
-        with open(OPENVPN_STATUS, encoding='utf-8') as handle:
-            return sum(1 for line in handle if line.startswith('CLIENT_LIST,'))
-    except OSError:
-        return None
 
 
 def collect(state):
     """The status document. state says which features the router has."""
-    units = {'vpn-router-setup': unit_state('vpn-router-setup.service')}
+    units = {'vpn-router-setup': unit_state('vpn-router-setup.service'), 'slapd': unit_state('slapd.service'),
+             **postgresql_clusters()}
+    manager_version = package_version('vpn-router-manager')
+    if manager_version:
+        units['vpn-router-manager'] = unit_state('vpn-router-manager.service')
     if state.get('site_to_site') or state.get('p2s'):
         units['strongswan'] = unit_state('strongswan.service')
     if state.get('wireguard'):
         units['wg-quick@wg0'] = unit_state('wg-quick@wg0.service')
     if state.get('openvpn'):
         units['openvpn-server@vpn-router'] = unit_state('openvpn-server@vpn-router.service')
-    if state.get('users'):
-        units['slapd'] = unit_state('slapd.service')
 
     status = {
         'status': 'ok' if all(value == 'active' for value in units.values()) else 'degraded',
         'version': package_version(),
+        'manager_version': manager_version,
         'time': time.strftime('%Y-%m-%dT%H:%M:%S%z'),
         'units': units,
         'docker': docker_status(),
@@ -253,35 +214,24 @@ def collect(state):
     return status
 
 
-def b64decode(part):
+def shown(status, content, client):
-    return base64.urlsafe_b64decode(part + '=' * (-len(part) % 4))
+    """The status document as web.status_content shows it to the client at its address.
 
-
+    Every content has status, time and client_ip, the client's address, and content, which one it is: basic nothing
-def verify(token, secret):
+    more; standard adds version, manager_version and services, the number of active units and of units in every
-    """The claims of a token signed with HS256 by the secret, or None when it is not acceptable.
+    other state; full adds the whole status document of collect. Content that is not one of
-
+    vpnrouter.STATUS_CONTENTS, or not set, is basic.
-    Only HS256 is accepted, so a token cannot choose a weaker algorithm. The token
-    must expire (exp), and not later than MAX_LIFETIME seconds from now, so that a
-    stolen token is of little use; LEEWAY seconds of clock difference are allowed.
     """
-    try:
+    if content not in vpnrouter.STATUS_CONTENTS:
-        header, payload, signature = token.split('.')
+        content = 'basic'
-        if json.loads(b64decode(header)).get('alg') != 'HS256':
+    if content == 'full':
-            return None
+        return {**status, 'content': content, 'client_ip': client}
-        expected = hmac.new(secret.encode(), f'{header}.{payload}'.encode(), hashlib.sha256).digest()
+    document = {'status': status['status'], 'content': content, 'time': status['time'], 'client_ip': client}
-        if not hmac.compare_digest(expected, b64decode(signature)):
+    if content == 'standard':
-            return None
+        active = sum(1 for value in status['units'].values() if value == 'active')
-        claims = json.loads(b64decode(payload))
+        document.update(version=status['version'], manager_version=status['manager_version'],
-        now = time.time()
+                        services={'active': active, 'failed': len(status['units']) - active})
-        expires = claims['exp']
+    return document
-        if isinstance(expires, bool) or not isinstance(expires, (int, float)) or not now - LEEWAY <= expires <= now + MAX_LIFETIME + LEEWAY:
-            return None
-        not_before = claims.get('nbf')
-        if not_before is not None and (isinstance(not_before, bool) or not isinstance(not_before, (int, float)) or not_before > now + LEEWAY):
-            return None
-    except (ValueError, KeyError, TypeError, AttributeError):
-        return None
-    return claims
 
 
 def lines_of(*args):
@@ -322,6 +272,65 @@ def wireguard_details():
     }
 
 
+def postgresql_query(cluster, sql):
+    """The JSON value an SQL statement answers on a cluster (version/name), run by psql as postgres, or None when it fails."""
+    result = run('runuser', '-u', 'postgres', '--', 'psql', '--cluster', cluster, '-X', '-q', '-t', '-A',
+                 '-v', 'ON_ERROR_STOP=1', '-d', 'postgres', '-c', sql)
+    if result is None or result.returncode != 0:
+        return None
+    try:
+        return json.loads(result.stdout)
+    except ValueError:
+        return None
+
+
+def postgresql_clusters_of():
+    """The PostgreSQL clusters of pg_lsclusters, each with version, name, port, status and data_directory, or None when it fails."""
+    lines = lines_of('pg_lsclusters', '--no-header')
+    if lines is None:
+        return None
+    clusters = []
+    for line in lines:
+        fields = line.split()
+        if len(fields) >= 6:
+            clusters.append({'version': fields[0], 'name': fields[1], 'port': int(fields[2]) if fields[2].isdigit() else None,
+                             'status': fields[3], 'data_directory': fields[5]})
+    return clusters
+
+
+def postgresql_configuration():
+    """The PostgreSQL clusters on the router with the settings of POSTGRESQL_SETTINGS of each one that is online (None
+    for one that is not), or None when pg_lsclusters fails."""
+    clusters = postgresql_clusters_of()
+    if clusters is None:
+        return None
+    names = ', '.join(f"'{name}'" for name in POSTGRESQL_SETTINGS)
+    for cluster in clusters:
+        cluster['settings'] = postgresql_query(
+            f"{cluster['version']}/{cluster['name']}",
+            f'SELECT json_object_agg(name, current_setting(name) ORDER BY name) FROM pg_settings WHERE name IN ({names})',
+        ) if cluster['status'].startswith('online') else None
+    return {'clusters': clusters}
+
+
+def postgresql_databases():
+    """What /postgresql shows: the PostgreSQL clusters on the router, each with its databases (name, owner, encoding and
+    size in bytes) and their total size, None for a cluster that is not online; clusters is None when pg_lsclusters fails."""
+    clusters = postgresql_clusters_of()
+    if clusters is None:
+        return {'clusters': None}
+    for cluster in clusters:
+        databases = postgresql_query(
+            f"{cluster['version']}/{cluster['name']}",
+            "SELECT coalesce(json_agg(json_build_object('name', datname, 'owner', pg_get_userbyid(datdba), "
+            "'encoding', pg_encoding_to_char(encoding), 'size', pg_database_size(oid)) ORDER BY datname), '[]') "
+            'FROM pg_database',
+        ) if cluster['status'].startswith('online') else None
+        cluster['databases'] = databases
+        cluster['size'] = sum(database['size'] for database in databases) if isinstance(databases, list) else None
+    return {'clusters': clusters}
+
+
 def details(state):
     """What /config shows: the configuration without its secrets, and what the router resolved and runs."""
     configuration = state.get('configuration', {})
@@ -337,16 +346,25 @@ def details(state):
             'exposed_ports': configuration.get('docker', {}).get('exposed_ports', []),
             'rules': lines_of('iptables', '-S', 'VPN-ROUTER-DOCKER'),
         },
+        'postgresql': postgresql_configuration(),
     }
 
 
 def text_of(status):
-    """The status for people, as plain text."""
+    """The status document of shown for people, as plain text: a header, a blank line, then name: value lines."""
-    lines = ['VPN Router', f'Status: {status["status"]}', f'Version: {status["version"] or "unknown"}', '']
+    lines = ['VPN Router', f'Status: {status["status"]}']
-    lines += [f'{name}: {value}' for name, value in status['units'].items()]
+    if 'version' in status:
-    docker = status['docker']
+        lines.append(f'Version: {status["version"] or "unknown"}')
-    lines.append('Docker: not installed' if not docker['installed']
+    lines += ['', f'Client IP: {status["client_ip"]}']
-                 else f'Docker: installed, service {docker["service"]}, rules {docker["rules"]}')
+    if 'manager_version' in status:
+        lines.append(f'Router Manager: {status["manager_version"] or "not installed"}')
+    if 'services' in status:
+        lines.append(f'Services: {status["services"]["active"]} active, {status["services"]["failed"]} failed')
+    lines += [f'{name}: {value}' for name, value in status.get('units', {}).items()]
+    if 'docker' in status:
+        docker = status['docker']
+        lines.append('Docker: not installed' if not docker['installed']
+                     else f'Docker: installed, service {docker["service"]}, rules {docker["rules"]}')
     if 'site_to_site' in status:
         value = status['site_to_site']['established']
         lines.append(f'Site to site: {"unknown" if value is None else "established" if value else "not established"}')
@@ -370,11 +388,12 @@ def text_of(status):
 
 
 def html_of(status):
-    """The status for a browser, as a page styled with Tailwind CSS."""
+    """The status document of shown for a browser, as a page styled with Tailwind CSS."""
     colour = 'bg-emerald-600 dark:bg-emerald-700' if status['status'] == 'ok' else 'bg-amber-600 dark:bg-amber-700'
-    # Each "name: value" line of the text is a row; indented lines, such as WireGuard peers, are indented here too.
+    version = f', version {status["version"] or "unknown"}' if 'version' in status else ''
+    # Each "name: value" line after the header of the text is a row; indented lines, such as WireGuard peers, are indented here too.
     rows = []
-    for line in text_of(status).splitlines()[4:]:
+    for line in text_of(status).split('\n\n', 1)[1].splitlines():
         if not line:
             continue
         name, _, value = line.partition(': ')
@@ -398,7 +417,7 @@ def html_of(status):
       <div class="overflow-hidden rounded-2xl border border-slate-200 bg-white shadow-lg dark:border-slate-700 dark:bg-slate-800">
         <div class="{colour} px-6 py-4 text-white">
           <h1 class="text-xl font-semibold">VPN Router</h1>
-          <p class="mt-1 text-sm">Status: {escape(status['status'])}, version {escape(status['version'] or 'unknown')}</p>
+          <p class="mt-1 text-sm">Status: {escape(status['status'] + version)}</p>
         </div>
         <div class="px-6 py-4">
           <table class="w-full text-sm"><tbody>{body}</tbody></table>
@@ -411,60 +430,249 @@ def html_of(status):
 """
 
 
-def user_tool(*args, password=None):
-    """Run vpn-router-user; the HTTP status for its exit status and its JSON answer."""
-    try:
-        result = subprocess.run([USER_TOOL, *args], input='' if password is None else password + '\n',
-                                capture_output=True, text=True, timeout=USER_TIMEOUT)
-        document = json.loads(result.stdout)
-    except (OSError, subprocess.TimeoutExpired, ValueError) as exc:
-        return 500, {'error': f'vpn-router-user failed: {exc}'}
-    if not isinstance(document, dict):
-        return 500, {'error': 'vpn-router-user gave no object'}
-    return USER_STATUS.get(result.returncode, 500), document
-
-
 def users(state, method, path, body):
-    """Answer a /users request: the HTTP status and the JSON document.
+    """Answer a /users request with vpnrouter_users: the HTTP status and the JSON document.
 
       GET    /users                   the users
-      POST   /users                   add a user: {"name": ..., "password": ...}
+      POST   /users                   add a user: {"name", "password", "first_name", "last_name", "email"};
+                                      without the names a Manager-only account
+      PUT    /users/<name>            change any of {"first_name", "last_name", "email"}
       PUT    /users/<name>/password   set the password: {"password": ...}
       POST   /users/<name>/disable    lock the user out of OpenVPN
       POST   /users/<name>/enable     let the user in again
+      POST   /users/<name>/unlock     end a lockout after failed logins
       DELETE /users/<name>            remove the user and revoke the certificate
       GET    /users/<name>/profile    the OpenVPN client profile
+      POST   /users/<name>/otp        a new TOTP secret and its otpauth URI; nothing is stored
+      PUT    /users/<name>/otp        store a TOTP secret: {"secret", "code"}, code a code of the secret
+      DELETE /users/<name>/otp        remove the TOTP token
     """
-    if not state.get('users'):
-        return 404, {'error': 'no P2S system with users is configured'}
     if body == 'invalid' or (body is not None and not isinstance(body, dict)):
         return 400, {'error': 'the body must be a JSON object'}
     body = body or {}
-    password = body.get('password')
-    if password is not None and (not isinstance(password, str) or '\n' in password or '\r' in password):
-        return 400, {'error': 'the password must be a string of one line'}
-
     parts = path.strip('/').split('/')
     route = (method, len(parts), parts[2] if len(parts) == 3 else None)
-    if route == ('GET', 1, None):
+    name = parts[1] if len(parts) > 1 else None
-        return user_tool('list')
+    actions = {
-    if route == ('POST', 1, None):
+        ('GET', 1, None): lambda connection: vpnrouter_users.list_users(connection),
-        name = body.get('name')
+        ('POST', 1, None): lambda connection: vpnrouter_users.add(connection, body.get('name'), body.get('password'), body),
-        if not isinstance(name, str) or password is None:
+        ('PUT', 2, None): lambda connection: vpnrouter_users.update(connection, name, body),
-            return 400, {'error': 'name and password are required strings'}
+        ('PUT', 3, 'password'): lambda connection: vpnrouter_users.passwd(connection, name, body.get('password')),
-        code, document = user_tool('add', name, password=password)
+        ('POST', 3, 'disable'): lambda connection: vpnrouter_users.disable(connection, name),
-        return (201 if code == 200 else code), document
+        ('POST', 3, 'enable'): lambda connection: vpnrouter_users.enable(connection, name),
-    if route == ('PUT', 3, 'password'):
+        ('POST', 3, 'unlock'): lambda connection: vpnrouter_users.unlock(connection, name),
-        if password is None:
+        ('DELETE', 2, None): lambda connection: vpnrouter_users.remove(connection, name),
-            return 400, {'error': 'password is a required string'}
+        ('GET', 3, 'profile'): lambda connection: vpnrouter_users.profile(connection, name),
-        return user_tool('passwd', parts[1], password=password)
+        ('POST', 3, 'otp'): lambda connection: vpnrouter_users.otp_new(connection, name),
-    if route in (('POST', 3, 'disable'), ('POST', 3, 'enable')):
+        ('PUT', 3, 'otp'): lambda connection: vpnrouter_users.otp_enrol(connection, name, body.get('secret'), body.get('code')),
-        return user_tool(parts[2], parts[1])
+        ('DELETE', 3, 'otp'): lambda connection: vpnrouter_users.otp_remove(connection, name),
-    if route == ('DELETE', 2, None):
+    }
-        return user_tool('remove', parts[1])
+    # The vpnrouter_users operation of each route that may revoke refresh tokens.
-    if route == ('GET', 3, 'profile'):
+    operations = {('PUT', 3, 'password'): 'passwd', ('POST', 3, 'disable'): 'disable', ('POST', 3, 'unlock'): 'unlock',
-        return user_tool('profile', parts[1])
+                  ('DELETE', 2, None): 'remove'}
-    return 404, {'error': f'no route {method} {path}'}
+    if route not in actions:
+        return 404, {'error': f'no route {method} {path}'}
+    try:
+        with vpnrouter_users.session() as connection:
+            document = actions[route](connection)
+        if operations.get(route) in vpnrouter_auth.REVOKING:
+            vpnrouter_auth.revoke(vpnrouter.load_config(), name)
+    except vpnrouter_users.Failure as failure:
+        return USER_STATUS.get(failure.status, 500), {'error': str(failure)}
+    except (ldap.LDAPError, OSError) as exc:
+        return 500, {'error': f'the user directory failed: {exc}'}
+    except vpnrouter_auth.DatabaseError as exc:
+        return 500, {'error': str(exc)}
+    except SystemExit as exc:
+        # vpnrouter ends with SystemExit, carrying the reason, on a configuration it cannot read.
+        return 500, {'error': str(exc.code)}
+    return (201 if route == ('POST', 1, None) else 200), document
+
+
+def groups(state, method, path, body):
+    """Answer a /groups request with vpnrouter_users: the HTTP status and the JSON document.
+
+      GET    /groups                          the groups with their members
+      POST   /groups                          add a group: {"name", "members": [...], "description"?}
+      PUT    /groups/<name>                   rename it or change the description: {"name"?, "description"?}
+      DELETE /groups/<name>                   remove the group; its members are kept
+      POST   /groups/<name>/members           add a member: {"user": ...}
+      DELETE /groups/<name>/members/<user>    take a member out; the last member cannot leave
+    """
+    if body == 'invalid' or (body is not None and not isinstance(body, dict)):
+        return 400, {'error': 'the body must be a JSON object'}
+    body = body or {}
+    parts = path.strip('/').split('/')
+    name = parts[1] if len(parts) > 1 else None
+    route = (method, len(parts), parts[2] if len(parts) > 2 else None)
+    actions = {
+        ('GET', 1, None): lambda connection: vpnrouter_users.list_groups(connection),
+        ('POST', 1, None): lambda connection: vpnrouter_users.group_add(
+            connection, body.get('name'), body.get('description'), body.get('members')),
+        ('PUT', 2, None): lambda connection: vpnrouter_users.group_update(
+            connection, name, body.get('name'), body.get('description')),
+        ('DELETE', 2, None): lambda connection: vpnrouter_users.group_remove(connection, name),
+        ('POST', 3, 'members'): lambda connection: vpnrouter_users.group_member_add(connection, name, body.get('user')),
+        ('DELETE', 4, 'members'): lambda connection: vpnrouter_users.group_member_remove(connection, name, parts[3]),
+    }
+    if route not in actions:
+        return 404, {'error': f'no route {method} {path}'}
+    try:
+        with vpnrouter_users.session() as connection:
+            document = actions[route](connection)
+    except vpnrouter_users.Failure as failure:
+        return USER_STATUS.get(failure.status, 500), {'error': str(failure)}
+    except (ldap.LDAPError, OSError) as exc:
+        return 500, {'error': f'the user directory failed: {exc}'}
+    return (201 if route == ('POST', 1, None) else 200), document
+
+
+def account(name, method, path, body):
+    """Answer an /account request of the user named by the token with vpnrouter_users: the HTTP status and the JSON document.
+
+      GET  /account            the user, with admin, whether the user is a member of ADMIN_GROUP
+      PUT  /account/password   set the password: {"current_password", "current_code"?, "password"}
+      POST /account/otp        a new TOTP secret and its otpauth URI; nothing is stored
+      PUT  /account/otp        store a TOTP secret: {"current_password", "current_code"?, "secret", "code"},
+                               code a code of the secret
+
+    A change needs the current password and, for an enrolled user, a current code; a wrong one is 403 and counts as
+    a failed login. A password change revokes the user's refresh tokens.
+    """
+    if body == 'invalid' or (body is not None and not isinstance(body, dict)):
+        return 400, {'error': 'the body must be a JSON object'}
+    body = body or {}
+    route = (method, path.rstrip('/'))
+    if route not in (('GET', '/account'), ('PUT', '/account/password'), ('POST', '/account/otp'), ('PUT', '/account/otp')):
+        return 404, {'error': f'no route {method} {path}'}
+    try:
+        if route == ('PUT', '/account/password'):
+            vpnrouter_users.check_password(body.get('password'))
+        if method == 'PUT':
+            vpnrouter_users.login(name, body.get('current_password'), body.get('current_code'))
+        with vpnrouter_users.session() as connection:
+            if route == ('GET', '/account'):
+                document = {**vpnrouter_users.summary(vpnrouter_users.entry(connection, name)),
+                            'admin': vpnrouter_users.is_admin(connection, name)}
+            elif route == ('PUT', '/account/password'):
+                document = vpnrouter_users.passwd(connection, name, body['password'])
+            elif route == ('POST', '/account/otp'):
+                document = vpnrouter_users.otp_new(connection, name)
+            else:
+                document = vpnrouter_users.otp_enrol(connection, name, body.get('secret'), body.get('code'))
+        if route == ('PUT', '/account/password'):
+            vpnrouter_auth.revoke(vpnrouter.load_config(), name)
+    except vpnrouter_users.Failure as failure:
+        if failure.status == 5:
+            return 403, {'error': 'the current password or code is wrong'}
+        return USER_STATUS.get(failure.status, 500), {'error': str(failure)}
+    except (ldap.LDAPError, OSError) as exc:
+        return 500, {'error': f'the user directory failed: {exc}'}
+    except vpnrouter_auth.DatabaseError as exc:
+        return 500, {'error': str(exc)}
+    except SystemExit as exc:
+        return 500, {'error': str(exc.code)}
+    return 200, document
+
+
+def metrics_history(view):
+    """Answer GET /metrics/history?range=<view> with vpnrouter_metrics.history: the HTTP status and the JSON document.
+
+    view is hour, the default, day, month or year (vpnrouter_metrics.RANGES).
+    """
+    if view not in vpnrouter_metrics.RANGES:
+        return 400, {'error': f'range must be one of {", ".join(vpnrouter_metrics.RANGES)}'}
+    try:
+        with vpnrouter_metrics.database(vpnrouter.load_config()) as connection:
+            return 200, vpnrouter_metrics.history(connection, view)
+    except vpnrouter_metrics.DatabaseError as exc:
+        return 500, {'error': str(exc)}
+    except SystemExit as exc:
+        return 500, {'error': str(exc.code)}
+
+
+def changes_admins(connection, method, parts, body):
+    """Whether a /users or /groups request creates, renames to, changes or removes ADMIN_GROUP, or removes a member of it."""
+    admins = vpnrouter_users.ADMIN_GROUP
+    if parts[0] == 'groups':
+        if method == 'POST' and len(parts) == 1:
+            return body.get('name') == admins
+        if method != 'GET' and len(parts) > 1 and parts[1] == admins:
+            return True
+        return method == 'PUT' and len(parts) == 2 and body.get('name') == admins
+    if parts[0] == 'users' and method == 'DELETE' and len(parts) == 2:
+        return admins in vpnrouter_users.groups_of(connection, parts[1])
+    return False
+
+
+def refusal(claims, method, path, body):
+    """Why the holder of the claims may not make the request to a protected path, or '' when it may."""
+    name = claims.get('sub')
+    if path == '/account' or path.startswith('/account/'):
+        return '' if isinstance(name, str) else 'the token names no user'
+    if claims.get('admin') is True:
+        return ''
+    if not isinstance(name, str):
+        return 'the token names no user'
+    admins = vpnrouter_users.ADMIN_GROUP
+    with vpnrouter_users.session() as connection:
+        if not vpnrouter_users.is_admin(connection, name):
+            return f'user {name} is not a member of {admins}'
+        if changes_admins(connection, method, path.strip('/').split('/'), body if isinstance(body, dict) else {}):
+            return f'only a token of vpn-router-token may change the group {admins} or remove its members'
+    return ''
+
+
+def auth(path, body):
+    """Answer a POST to /auth with vpnrouter_users and vpnrouter_auth: the HTTP status and the JSON document.
+
+      POST /auth           sign in: {"name", "password", "code"?}; the tokens
+      POST /auth/refresh   {"refresh_token"}; new tokens, the refresh token used up
+      POST /auth/logout    {"refresh_token"}; the refresh token revoked
+    """
+    if not isinstance(body, dict):
+        return 400, {'error': 'the body must be a JSON object'}
+    try:
+        cfg = vpnrouter.load_config()
+        if path == '/auth':
+            vpnrouter_users.login(body.get('name'), body.get('password'), body.get('code'))
+            with vpnrouter_users.session() as connection:
+                if vpnrouter_users.service(vpnrouter_users.entry(connection, body['name'])):
+                    raise vpnrouter_users.Failure('the name, the password or the code is wrong', 5)
+            return 200, vpnrouter_auth.issue(cfg, body['name'])
+        name = vpnrouter_auth.take(cfg, body.get('refresh_token'))
+        if path == '/auth/logout':
+            return 200, {'revoked': name is not None}
+        if name is None:
+            return 401, {'error': 'the refresh token is not valid'}
+        with vpnrouter_users.session() as connection:
+            user = vpnrouter_users.summary(vpnrouter_users.entry(connection, name))
+        if not user['enabled'] or user['locked']:
+            return 401, {'error': f'user {name} is disabled or locked'}
+        return 200, vpnrouter_auth.issue(cfg, name)
+    except vpnrouter_users.Failure as failure:
+        return (401 if failure.status == 3 else USER_STATUS.get(failure.status, 500)), {'error': str(failure)}
+    except (ldap.LDAPError, OSError) as exc:
+        return 500, {'error': f'the user directory failed: {exc}'}
+    except vpnrouter_auth.DatabaseError as exc:
+        return 500, {'error': str(exc)}
+    except SystemExit as exc:
+        return 500, {'error': str(exc.code)}
+
+
+def auth_wait(address):
+    """Seconds until the client address may call /auth again, 0 when it may now; a call that may is counted."""
+    now = time.monotonic()
+    with _auth_lock:
+        for idle in [key for key, times in _auth_calls.items() if times[-1] <= now - AUTH_WINDOW]:
+            del _auth_calls[idle]
+        calls = _auth_calls[address]
+        while calls and calls[0] <= now - AUTH_WINDOW:
+            calls.popleft()
+        if len(calls) >= AUTH_LIMIT:
+            return int(calls[0] + AUTH_WINDOW - now) + 1
+        calls.append(now)
+        return 0
 
 
 class Handler(BaseHTTPRequestHandler):
@@ -477,39 +685,86 @@ class Handler(BaseHTTPRequestHandler):
                 _cache['at'] = time.monotonic()
             return _cache['data']
 
-    def authorized(self, secret):
+    def claims(self, secret):
-        """Whether the request carries a token signed with the secret; False also while no secret is set."""
+        """The claims of the request's token signed with the secret, or None without an acceptable one."""
         header = self.headers.get('Authorization', '')
-        return bool(secret) and header.startswith('Bearer ') and verify(header[7:].strip(), secret) is not None
+        return vpnrouter_auth.verify(header[7:].strip(), secret) if header.startswith('Bearer ') else None
+
+    def client(self):
+        """The client address: the last X-Forwarded-For entry when the peer is a trusted proxy, else the peer."""
+        peer = self.client_address[0]
+        trusted = section(read_json(CONFIG_FILE), 'web').get('trusted_proxies') or [self.server.server_address[0]]
+        forwarded = self.headers.get('X-Forwarded-For', '')
+        if isinstance(trusted, list) and peer in trusted and forwarded.strip():
+            return forwarded.split(',')[-1].strip()
+        return peer
+
+    def refusal(self, claims, path, payload):
+        """The HTTP status and the message that refuse a request to a protected path, or None when it may be made."""
+        try:
+            reason = refusal(claims, self.command, path, payload)
+        except (ldap.LDAPError, OSError) as exc:
+            return 500, f'the user directory failed: {exc}'
+        return (403, reason) if reason else None
 
     def answer(self, send_body):
         path = self.path.split('?', 1)[0]
         extra = []
+        self.principal = None
         # A body that is not read must not be taken for the next request.
         if self.command not in ('GET', 'HEAD'):
             self.close_connection = True
-        protected = path == '/config' or path == '/users' or path.startswith('/users/')
+        directory = next((name for name in ('users', 'groups', 'account') if path == f'/{name}' or path.startswith(f'/{name}/')), None)
-        state = current() if protected else {}
+        protected = path in ('/config', '/metrics/history', '/postgresql') or directory is not None
-        if protected and not state['jwt_secret']:
+        signing = path in ('/auth', '/auth/refresh', '/auth/logout')
+        state = current() if protected or signing else {}
+        claims = self.claims(state['key']) if protected and state['key'] else None
+        payload = self.request_body() if (protected or signing) and self.command not in ('GET', 'HEAD') else None
+        if claims is not None:
+            self.principal = 'admin' if claims.get('admin') is True else claims.get('sub')
+        if (protected or signing) and not state['key']:
             body, kind, code = b'Not found\n', 'text/plain; charset=utf-8', 404
-        elif protected and not self.authorized(state['jwt_secret']):
+        elif signing and self.command != 'POST':
+            body, kind, code = b'Method not allowed\n', 'text/plain; charset=utf-8', 405
+        elif signing:
+            wait = auth_wait(self.client()) if path != '/auth/logout' else 0
+            if wait:
+                code, document = 429, {'error': f'too many sign-in calls; retry in {wait} seconds'}
+                extra.append(('Retry-After', str(wait)))
+            else:
+                code, document = auth(path, payload)
+            body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
+        elif protected and claims is None:
             body, kind, code = b'Unauthorized\n', 'text/plain; charset=utf-8', 401
             extra.append(('WWW-Authenticate', 'Bearer'))
+        elif protected and (refused := self.refusal(claims, path, payload)):
+            code, kind = refused[0], 'application/json'
+            body = json.dumps({'error': refused[1]}, indent=2).encode() + b'\n'
         elif path == '/config' and self.command in ('GET', 'HEAD'):
             body, kind, code = json.dumps(details(state), indent=2).encode() + b'\n', 'application/json', 200
-        elif protected and path != '/config':
+        elif path == '/postgresql' and self.command in ('GET', 'HEAD'):
-            code, document = users(state, self.command, path, self.request_body())
+            body, kind, code = json.dumps(postgresql_databases(), indent=2).encode() + b'\n', 'application/json', 200
+        elif path == '/metrics/history' and self.command in ('GET', 'HEAD'):
+            query = urllib.parse.parse_qs(urllib.parse.urlsplit(self.path).query)
+            code, document = metrics_history(query.get('range', ['hour'])[0])
+            body, kind = json.dumps(document).encode() + b'\n', 'application/json'
+        elif directory == 'account':
+            code, document = account(claims['sub'], self.command, path, payload)
+            body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
+        elif directory is not None:
+            code, document = (users if directory == 'users' else groups)(state, self.command, path, payload)
             body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
         elif self.command not in ('GET', 'HEAD'):
             body, kind, code = b'Method not allowed\n', 'text/plain; charset=utf-8', 405
-        elif path == '/status':
+        elif path in ('/status', '/'):
-            body, kind, code = json.dumps(self.status(), indent=2).encode() + b'\n', 'application/json', 200
+            document = shown(self.status(), section(read_json(CONFIG_FILE), 'web').get('status_content'), self.client())
-        elif path == '/':
             agent = self.headers.get('User-Agent', '').lower()
-            if 'curl' in agent or 'wget' in agent:
+            if path == '/status':
-                body, kind = text_of(self.status()).encode(), 'text/plain; charset=utf-8'
+                body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
+            elif 'curl' in agent or 'wget' in agent:
+                body, kind = text_of(document).encode(), 'text/plain; charset=utf-8'
             else:
-                body, kind = html_of(self.status()).encode(), 'text/html; charset=utf-8'
+                body, kind = html_of(document).encode(), 'text/html; charset=utf-8'
             code = 200
         else:
             body, kind, code = b'Not found\n', 'text/plain; charset=utf-8', 404
@@ -552,66 +807,45 @@ class Handler(BaseHTTPRequestHandler):
         self.answer(True)
 
     def log_message(self, format, *args):
-        sys.stderr.write('%s - - [%s] %s\n' % (self.client_address[0], self.log_date_time_string(), format % args))
+        principal = getattr(self, 'principal', None) or '-'
-
+        sys.stderr.write('%s - %s [%s] %s\n' % (self.client_address[0], principal, self.log_date_time_string(), format % args))
-
-def wanted():
-    """Where the service should listen now, as (address, port), or the reason it should not listen."""
-    configuration = read_json(CONFIG_FILE)
-    web = section(configuration, 'web')
-    if web.get('enabled', True) is False:
-        return None, 'the web service is disabled in the configuration'
-    port = web.get('port', DEFAULT_PORT)
-    if isinstance(port, bool) or not isinstance(port, int) or not 1 <= port <= 65535:
-        port = DEFAULT_PORT
-    general, interfaces, local = (section(configuration, name) for name in ('general', 'interfaces', 'local'))
-    address = vpnrouter_net.internal_address(str(general.get('mode') or 'manual'), str(interfaces.get('internal') or ''),
-                                             str(local.get('int_addr') or ''))
-    if not address:
-        return None, 'the internal address is not known yet'
-    return (address, port), ''
 
 
 def log(message):
     print(f'vpn-router-web: {message}', file=sys.stderr, flush=True)
 
 
-def discover():
+def initialize():
-    """Work out where to listen every DISCOVERY_SECONDS, and run the server thread there.
+    """Make the signing key and the refresh tokens' database, retrying every INIT_RETRY_SECONDS until both are ready."""
-
+    reported = None
-    When the address or the port changes, the server thread is stopped and a new one listens at
-    the new place; while the service is disabled, the address is not known or it cannot be bound,
-    no server runs and the next round tries again.
-    """
-    server, thread, listening, reported = None, None, None, None
     while True:
-        target, reason = wanted()
+        phase = 'reading the configuration'
-        if target != listening and server is not None:
+        try:
-            server.shutdown()
+            cfg = vpnrouter.load_config()
-            server.server_close()
+            phase = 'making the signing key'
-            thread.join()
+            vpnrouter_auth.ensure_key()
-            server, listening = None, None
+            phase = 'creating the token database'
-        if target is not None and server is None:
+            vpnrouter_auth.provision(cfg)
-            try:
+            phase = 'connecting to the token database'
-                server = ThreadingHTTPServer(target, Handler)
+            with vpnrouter_auth.database(cfg):
-            except OSError as exc:
+                pass
-                reason = f'cannot listen on {target[0]}:{target[1]}: {exc}'
+        except (OSError, SystemExit, vpnrouter_auth.DatabaseError) as exc:
-            else:
+            error = f'{phase} failed: {exc.code if isinstance(exc, SystemExit) else exc}'
-                thread = threading.Thread(target=server.serve_forever, name='server', daemon=True)
+            if error != reported:
-                thread.start()
+                log(f'{error}; retrying every {INIT_RETRY_SECONDS} seconds')
-                listening, reason = target, f'serving on http://{target[0]}:{target[1]}'
+                reported = error
-        if reason and reason != reported:
+            time.sleep(INIT_RETRY_SECONDS)
-            log(reason)
+            continue
-            reported = reason
+        log('ready')
-        time.sleep(DISCOVERY_SECONDS)
+        return
 
 
 def main():
     signal.signal(signal.SIGTERM, lambda *_: sys.exit(0))
-    discovery = threading.Thread(target=discover, name='discovery', daemon=True)
+    initialize()
-    discovery.start()
     try:
-        discovery.join()
+        vpnrouter_serve.serve(lambda: vpnrouter_net.web_listen(read_json(CONFIG_FILE), DEFAULT_PORT), Handler, log,
+                              DISCOVERY_SECONDS)
     except KeyboardInterrupt:
         pass
     return 0

debian-package/src/usr/sbin/vpn-config-edit

@@ -0,0 +1,129 @@
+#!/usr/bin/python3
+"""Show and change the router's configuration, /etc/vpn-router/vpn-router.json.
+
+show prints the file as indented JSON; the file is not changed. set takes a setting as
+section.option, such as web.port, and a value, which it converts to the type the package's
+schema (vpnrouter.SCHEMA) gives the setting: a whole number, true or false, a list of strings
+separated by commas (an empty value is an empty list), or a string. The configuration with the
+new value is checked as vpn-router-setup checks it, and only then written, indented, with mode
+0600; vpn-router-setup applies it at its next run, which set reminds of. A failure is a message
+on standard error; the exit status is 2 for an invalid request and 1 for any other failure.
+-h or --help shows the options.
+
+  vpn-config-edit show
+  vpn-config-edit set SECTION.OPTION VALUE
+"""
+
+import argparse
+import json
+import os
+import sys
+import tempfile
+
+sys.path.insert(0, '/usr/lib/vpn-router')
+
+import vpnrouter
+
+
+class Failure(Exception):
+    """A request that cannot be done; status is 2 invalid, 1 other."""
+
+    def __init__(self, message, status=1):
+        super().__init__(message)
+        self.status = status
+
+
+class Parser(argparse.ArgumentParser):
+    """An argument parser whose errors are Failures with status 2."""
+
+    def error(self, message):
+        raise Failure(f'{message}; see {self.prog} --help', 2)
+
+
+def read():
+    """The settings in the configuration file, without the defaults; {} when the file does not exist."""
+    try:
+        data = json.loads(vpnrouter.CONFIG_FILE.read_text(encoding='utf-8'))
+    except FileNotFoundError:
+        return {}
+    except ValueError as exc:
+        raise Failure(f'cannot parse {vpnrouter.CONFIG_FILE}: {exc}')
+    if not isinstance(data, dict) or not all(isinstance(options, dict) for options in data.values()):
+        raise Failure(f'cannot parse {vpnrouter.CONFIG_FILE}: it is not an object of sections')
+    return data
+
+
+def converted(key, text):
+    """The value of text for the setting key, section.option, of the type the schema gives it."""
+    section, _, option = key.partition('.')
+    if section not in vpnrouter.SCHEMA or option not in vpnrouter.SCHEMA[section]:
+        raise Failure(f'{key}: unknown setting; settings are section.option, such as web.port', 2)
+    kind = type(vpnrouter.SCHEMA[section][option])
+    if kind is bool:
+        if text not in ('true', 'false'):
+            raise Failure(f'{key} takes true or false', 2)
+        return text == 'true'
+    if kind is int:
+        try:
+            return int(text)
+        except ValueError:
+            raise Failure(f'{key} takes a whole number', 2)
+    if kind is list:
+        return [item.strip() for item in text.split(',') if item.strip()]
+    return text
+
+
+def write(data):
+    """Write the settings, indented, with mode 0600, replacing the file in one step."""
+    directory = vpnrouter.CONFIG_FILE.parent
+    descriptor, name = tempfile.mkstemp(dir=directory, prefix='.vpn-router.json.')
+    try:
+        with os.fdopen(descriptor, 'w', encoding='utf-8') as handle:
+            handle.write(json.dumps(data, indent=2) + '\n')
+        os.chmod(name, 0o600)
+        os.replace(name, vpnrouter.CONFIG_FILE)
+    except OSError:
+        os.unlink(name)
+        raise
+
+
+def main(argv):
+    try:
+        parser = Parser(prog='vpn-config-edit', description='Show and change the router configuration.')
+        commands = parser.add_subparsers(dest='command', required=True, parser_class=Parser)
+        commands.add_parser('show', help='print the configuration as indented JSON')
+        setter = commands.add_parser('set', help='set a setting and write the configuration')
+        setter.add_argument('key', metavar='SECTION.OPTION', help='the setting, such as web.port')
+        setter.add_argument('value', metavar='VALUE', help='its value; a list separated by commas')
+        options = parser.parse_args(argv)
+
+        data = read()
+        if options.command == 'show':
+            print(json.dumps(data, indent=2))
+            return 0
+
+        section, _, option = options.key.partition('.')
+        value = converted(options.key, options.value)
+        data.setdefault(section, {})[option] = value
+        # The configuration as vpn-router-setup reads it: the defaults with the file's settings.
+        candidate = {name: dict(defaults) for name, defaults in vpnrouter.SCHEMA.items()}
+        for name, settings in data.items():
+            candidate.setdefault(name, {}).update(settings)
+        try:
+            vpnrouter.validate(candidate)
+        except vpnrouter.ConfigError as exc:
+            raise Failure(str(exc), 2)
+        write(data)
+    except Failure as failure:
+        print(f'vpn-config-edit: {failure}', file=sys.stderr)
+        return failure.status
+    except OSError as exc:
+        print(f'vpn-config-edit: {exc}', file=sys.stderr)
+        return 1
+    print(f'{options.key} set to {json.dumps(value)} in {vpnrouter.CONFIG_FILE}.')
+    print('Apply it with: systemctl restart vpn-router-setup')
+    return 0
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))

debian-package/src/usr/sbin/vpn-router-group

@@ -0,0 +1,120 @@
+#!/usr/bin/python3
+"""Manage the groups of the router's directory from the command line, with vpnrouter_users.
+
+A group has a name, an optional description and at least one member, each a user of the
+directory. The answer is text for people on standard output, and a failure is a message on
+standard error. With --json the answer is JSON on standard output, and a failure is
+{"error": ...} there too. The exit status says what failed: 2 the request is invalid, 3 the
+group or the user does not exist, 4 the group exists, 1 anything else. -h or --help shows the
+commands and their options.
+
+  vpn-router-group [--json] list
+  vpn-router-group [--json] add <name> --member U [--member U ...] [--description D]
+  vpn-router-group [--json] update <name> [--name N] [--description D]
+  vpn-router-group [--json] remove <name>
+  vpn-router-group [--json] member-add <name> <user>
+  vpn-router-group [--json] member-remove <name> <user>
+"""
+
+import argparse
+import json
+import sys
+
+sys.path.insert(0, '/usr/lib/vpn-router')
+
+import ldap
+
+import vpnrouter_users
+from vpnrouter_users import Failure
+
+#: The help line of each command.
+HELP = {
+    'list': 'list the groups with their members',
+    'add': 'add a group with at least one member',
+    'update': 'rename the group or change its description',
+    'remove': 'remove the group; its members are kept',
+    'member-add': 'add a user to the group',
+    'member-remove': 'take a user out of the group; the last member cannot leave',
+}
+
+
+class Parser(argparse.ArgumentParser):
+    """An argument parser whose errors are Failures with status 2, so that --json answers them as JSON."""
+
+    def error(self, message):
+        raise Failure(f'{message}; see {self.prog} --help', 2)
+
+
+def parse(argv):
+    parser = Parser(prog='vpn-router-group', description='Manage the groups of the router.')
+    parser.add_argument('--json', action='store_true', help='answer in JSON')
+    commands = parser.add_subparsers(dest='command', required=True, metavar='command', parser_class=Parser)
+    for command, text in HELP.items():
+        sub = commands.add_parser(command, help=text, description=text[0].upper() + text[1:] + '.')
+        if command != 'list':
+            sub.add_argument('name', help='the group name')
+        if command == 'add':
+            sub.add_argument('--member', dest='members', action='append', required=True, metavar='USER',
+                             help='a member; give it once for each member')
+            sub.add_argument('--description', metavar='TEXT', help='the description')
+        if command == 'update':
+            sub.add_argument('--name', dest='new_name', metavar='NAME', help='the new group name')
+            sub.add_argument('--description', metavar='TEXT', help="the description; '' removes it")
+        if command in ('member-add', 'member-remove'):
+            sub.add_argument('user', help='the user name')
+    return parser.parse_args(argv)
+
+
+def run(connection, options):
+    """The answer of the command the options name."""
+    command = options.command
+    if command == 'list':
+        return vpnrouter_users.list_groups(connection)
+    if command == 'add':
+        return vpnrouter_users.group_add(connection, options.name, options.description, options.members)
+    if command == 'update':
+        return vpnrouter_users.group_update(connection, options.name, options.new_name, options.description)
+    if command == 'remove':
+        return vpnrouter_users.group_remove(connection, options.name)
+    if command == 'member-add':
+        return vpnrouter_users.group_member_add(connection, options.name, options.user)
+    return vpnrouter_users.group_member_remove(connection, options.name, options.user)
+
+
+def text_of(answer):
+    """The answer as text for people."""
+    if answer.get('removed'):
+        return f'group {answer["name"]} removed'
+    groups = answer['groups'] if 'groups' in answer else [answer]
+    if not groups:
+        return 'no groups'
+    head = ('GROUP', 'MEMBERS', 'DESCRIPTION')
+    rows = [(group['name'], ', '.join(group['members']), group['description'] or '-') for group in groups]
+    widths = [max(len(row[column]) for row in (head, *rows)) for column in range(len(head))]
+    return '\n'.join('  '.join(cell.ljust(width) for cell, width in zip(row, widths)).rstrip() for row in (head, *rows))
+
+
+def main(argv):
+    # A request that cannot be parsed is answered in JSON when --json is among the arguments.
+    as_json = '--json' in argv
+    try:
+        options = parse(argv)
+        as_json = options.json
+        with vpnrouter_users.session() as connection:
+            answer = run(connection, options)
+    except Failure as failure:
+        message, status = str(failure), failure.status
+    except (ldap.LDAPError, OSError) as exc:
+        message, status = f'the user directory failed: {exc}', 1
+    else:
+        print(json.dumps(answer) if as_json else text_of(answer))
+        return 0
+    if as_json:
+        print(json.dumps({'error': message}))
+    else:
+        print(f'vpn-router-group: {message}', file=sys.stderr)
+    return status
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))

debian-package/src/usr/sbin/vpn-router-token

@@ -0,0 +1,79 @@
+#!/usr/bin/python3
+"""Make a token with the admin claim for the router's web service, with vpnrouter_auth.
+
+The token is signed with the router's key and is valid for the lifetime given, without a
+maximum. It may call every protected path, and it is the only token that may change the
+admins group. The answer is the token on standard output, and a failure is a message on
+standard error. With --json the answer is {"token", "expires"} on standard output, and a
+failure is {"error": ...} there too. The exit status is 2 for an invalid request and 1 for
+any other failure. -h or --help shows the options.
+
+  vpn-router-token [--json] (--minutes N | --hours N | --days N)
+"""
+
+import argparse
+import json
+import sys
+
+sys.path.insert(0, '/usr/lib/vpn-router')
+
+import vpnrouter_auth
+
+#: The seconds of each unit of the lifetime.
+UNITS = {'minutes': 60, 'hours': 3600, 'days': 86400}
+
+
+class Failure(Exception):
+    """A request that cannot be done; status is 2 invalid, 1 other."""
+
+    def __init__(self, message, status=1):
+        super().__init__(message)
+        self.status = status
+
+
+class Parser(argparse.ArgumentParser):
+    """An argument parser whose errors are Failures with status 2, so that --json answers them as JSON."""
+
+    def error(self, message):
+        raise Failure(f'{message}; see {self.prog} --help', 2)
+
+
+def positive(value):
+    try:
+        number = int(value)
+    except ValueError:
+        number = 0
+    if number < 1:
+        raise argparse.ArgumentTypeError(f'{value!r} is not a positive whole number')
+    return number
+
+
+def main(argv):
+    # A request that cannot be parsed is answered in JSON when --json is among the arguments.
+    as_json = '--json' in argv
+    try:
+        parser = Parser(prog='vpn-router-token', description='Make a token with the admin claim for the web service.')
+        parser.add_argument('--json', action='store_true', help='answer in JSON')
+        lifetime = parser.add_mutually_exclusive_group(required=True)
+        for unit in UNITS:
+            lifetime.add_argument(f'--{unit}', type=positive, metavar='N', help=f'valid for N {unit}')
+        options = parser.parse_args(argv)
+        as_json = options.json
+        seconds = next(getattr(options, unit) * factor for unit, factor in UNITS.items() if getattr(options, unit))
+        answer = vpnrouter_auth.admin_token(seconds)
+    except Failure as failure:
+        message, status = str(failure), failure.status
+    except OSError as exc:
+        message, status = str(exc), 1
+    else:
+        print(json.dumps(answer) if as_json else answer['token'])
+        return 0
+    if as_json:
+        print(json.dumps({'error': message}))
+    else:
+        print(f'vpn-router-token: {message}', file=sys.stderr)
+    return status
+
+
+if __name__ == '__main__':
+    sys.exit(main(sys.argv[1:]))

debian-package/src/usr/sbin/vpn-router-user

@@ -1,224 +1,156 @@
 #!/usr/bin/python3
-"""Manage the users of the router's P2S systems.
+"""Manage the users of the router's directory from the command line, with vpnrouter_users.
 
-A user is an entry in the router's directory, whose password OpenVPN checks, and a client
+Passwords and TOTP secrets are read from the first line of standard input, never from the
-certificate of the router CA with the user name as its common name, which OpenVPN and
+command line. The
-IKEv2 accept. Passwords are read from the first line of standard input, never from the
+answer is text for people on standard output, and a failure is a message on standard error.
-command line. The answer is JSON on standard output; on failure it is {"error": ...} and
+With --json the answer is JSON on standard output, and a failure is {"error": ...} there too.
-the exit status says why: 2 the request is invalid, 3 the user does not exist, 4 the user
+The exit status says what failed: 2 the request is invalid, 3 the user does not exist, 4 the
-exists, 1 anything else.
+user exists, 1 anything else. -h or --help shows the commands and their options.
 
-  vpn-router-user list
+  vpn-router-user [--json] list
-  vpn-router-user add <name>          (password on stdin)
+  vpn-router-user [--json] add <name> --first-name F --last-name L --email E   (password on stdin)
-  vpn-router-user passwd <name>       (password on stdin)
+  vpn-router-user [--json] add <name> [--email E]   (a Manager-only account; password on stdin)
-  vpn-router-user disable <name>
+  vpn-router-user [--json] update <name> [--first-name F] [--last-name L] [--email E]
-  vpn-router-user enable <name>
+  vpn-router-user [--json] passwd <name>       (password on stdin)
-  vpn-router-user remove <name>
+  vpn-router-user [--json] disable <name>
-  vpn-router-user profile <name>
+  vpn-router-user [--json] enable <name>
+  vpn-router-user [--json] unlock <name>
+  vpn-router-user [--json] remove <name>
+  vpn-router-user [--json] profile <name>
+  vpn-router-user [--json] otp-new <name>
+  vpn-router-user [--json] otp-enrol <name> <code>   (secret on stdin)
+  vpn-router-user [--json] otp-remove <name>
 """
 
-import fcntl
+import argparse
 import json
-import pathlib
-import re
-import subprocess
 import sys
 
 sys.path.insert(0, '/usr/lib/vpn-router')
 
 import ldap
-import ldap.modlist
 
 import vpnrouter
-import vpnrouter_ldap
+import vpnrouter_auth
+import vpnrouter_users
+from vpnrouter_users import Failure
 
-#: A user name is one lowercase DNS label, as it is also the common name of the certificate.
+#: The help line of each command.
-NAME = re.compile(r'^[a-z][a-z0-9-]{0,30}[a-z0-9]$|^[a-z]$')
+HELP = {
-PASSWORD_MIN = 8
+    'list': 'list the users',
-PASSWORD_MAX = 128
+    'add': 'add a VPN user with the three person options and issue the certificate, or without the names a '
-#: The value of pwdAccountLockedTime that locks an account until it is unlocked.
+           'Manager-only account, with an optional e-mail address; the password is read from standard input',
-LOCKED = b'000001010000Z'
+    'update': "change the first name, the last name or the e-mail address; a Manager-only account has only the "
-LOCK_FILE = pathlib.Path('/run/vpn-router/users.lock')
+              "e-mail address, which '' removes",
+    'passwd': 'set the password, read from standard input',
+    'disable': 'lock the user out of OpenVPN',
+    'enable': 'let the user in again',
+    'unlock': 'end a lockout after failed logins',
+    'remove': 'revoke the certificate and remove the user',
+    'profile': 'print the OpenVPN client profile',
+    'otp-new': 'make a TOTP secret and its otpauth URI; nothing is stored',
+    'otp-enrol': 'store the TOTP secret read from standard input, when the code is a code of it',
+    'otp-remove': 'remove the TOTP token',
+}
+
+#: The help of the person options.
+PERSON_HELP = {'first_name': 'the first name', 'last_name': 'the last name', 'email': 'the e-mail address'}
 
 
-class Failure(Exception):
+class Parser(argparse.ArgumentParser):
-    """A request that cannot be done; status is the exit status."""
+    """An argument parser whose errors are Failures with status 2, so that --json answers them as JSON."""
 
-    def __init__(self, message, status=1):
+    def error(self, message):
-        super().__init__(message)
+        raise Failure(f'{message}; see {self.prog} --help', 2)
-        self.status = status
 
 
-def dn_of(name):
+def parse(argv):
-    return f'uid={name},{vpnrouter_ldap.USERS}'
+    parser = Parser(prog='vpn-router-user', description='Manage the users of the router.')
-
+    parser.add_argument('--json', action='store_true', help='answer in JSON')
-
+    commands = parser.add_subparsers(dest='command', required=True, metavar='command', parser_class=Parser)
-def certificate(name):
+    for command, text in HELP.items():
-    return vpnrouter.USER_CERT_DIR / f'{name}_cert.pem', vpnrouter.USER_CERT_DIR / f'{name}_key.pem'
+        sub = commands.add_parser(command, help=text, description=text[0].upper() + text[1:] + '.')
-
+        if command != 'list':
-
+            sub.add_argument('name', help='the user name')
-def entry(connection, name):
+        if command in ('add', 'update'):
-    """The attributes of the user's entry; Failure with status 3 when there is none."""
+            for key, _ in vpnrouter_users.PERSON:
-    try:
+                sub.add_argument(f'--{key.replace("_", "-")}', dest=key, metavar='TEXT', help=PERSON_HELP[key])
-        return connection.search_s(dn_of(name), ldap.SCOPE_BASE, attrlist=['uid', 'pwdAccountLockedTime'])[0][1]
+        if command == 'otp-enrol':
-    except ldap.NO_SUCH_OBJECT:
+            sub.add_argument('code', help='a code of the secret from the authenticator')
-        raise Failure(f'user {name} does not exist', 3)
+    return parser.parse_args(argv)
 
 
 def password():
-    """The password from the first line of standard input."""
+    """The password or the TOTP secret from the first line of standard input."""
-    value = sys.stdin.readline().rstrip('\n')
+    return sys.stdin.readline().rstrip('\n')
-    if not PASSWORD_MIN <= len(value) <= PASSWORD_MAX:
-        raise Failure(f'the password must have {PASSWORD_MIN} to {PASSWORD_MAX} characters', 2)
-    return value
 
 
-def simple_ca(*args):
+def run(connection, options, secret):
-    result = subprocess.run([sys.executable, str(vpnrouter.SIMPLE_CA), *args], capture_output=True, text=True)
+    """The answer of the command the options name; secret is the password of add and passwd, the TOTP secret of otp-enrol."""
-    if result.returncode != 0:
+    command = options.command
-        raise Failure(f'simple-ca {args[0]} failed: {result.stderr.strip() or result.stdout.strip()}')
+    if command == 'list':
+        return vpnrouter_users.list_users(connection)
+    values = {key: getattr(options, key, None) for key, _ in vpnrouter_users.PERSON}
+    if command == 'add':
+        return vpnrouter_users.add(connection, options.name, secret, values)
+    if command == 'update':
+        return vpnrouter_users.update(connection, options.name, values)
+    if command == 'passwd':
+        return vpnrouter_users.passwd(connection, options.name, secret)
+    if command == 'otp-enrol':
+        return vpnrouter_users.otp_enrol(connection, options.name, secret, options.code)
+    return getattr(vpnrouter_users, command.replace('-', '_'))(connection, options.name)
 
 
-def publish_crl():
+def text_of(answer):
-    """Give strongSwan the CRL after a revocation; OpenVPN reads it at every connection."""
+    """The answer as text for people."""
-    if vpnrouter.install_file(vpnrouter.PKI_DIR / 'crl.pem', vpnrouter.CRL_DEST, 0o644):
+    if 'profile' in answer:
-        subprocess.run(['swanctl', '--load-creds'], capture_output=True, text=True)
+        return answer['profile'].rstrip('\n')
-
+    if 'secret' in answer:
-
+        return f'secret  {answer["secret"]}\nuri     {answer["uri"]}'
-def summary(attributes):
+    if answer.get('removed'):
-    name = attributes['uid'][0].decode()
+        return f'user {answer["name"]} removed'
-    return {
+    users = answer['users'] if 'users' in answer else [answer]
-        'name': name,
+    if not users:
-        'enabled': 'pwdAccountLockedTime' not in attributes,
+        return 'no users'
-        'certificate': certificate(name)[0].is_file(),
+    head = ('USER', 'KIND', 'STATE', 'CERTIFICATE', 'OTP', 'FIRST NAME', 'LAST NAME', 'E-MAIL')
-    }
+    rows = [(user['name'], 'service' if user['service'] else 'manager' if user['manager_only'] else 'vpn',
-
+             'disabled' if not user['enabled'] else 'locked' if user['locked'] else 'enabled',
-
+             'issued' if user['certificate'] else 'missing',
-def list_users(connection):
+             'enrolled' if user['otp'] else '-', user['first_name'] or '-', user['last_name'] or '-', user['email'] or '-')
-    found = connection.search_s(vpnrouter_ldap.USERS, ldap.SCOPE_ONELEVEL, '(objectClass=inetOrgPerson)',
+            for user in users]
-                                ['uid', 'pwdAccountLockedTime'])
+    widths = [max(len(row[column]) for row in (head, *rows)) for column in range(len(head))]
-    return {'users': sorted((summary(attributes) for _, attributes in found), key=lambda user: user['name'])}
+    return '\n'.join('  '.join(cell.ljust(width) for cell, width in zip(row, widths)).rstrip() for row in (head, *rows))
-
-
-def add(connection, name):
-    secret = password()
-    if not (vpnrouter.PKI_DIR / 'ca_key.pem').is_file():
-        raise Failure('the router CA does not exist; enable OpenVPN or IKEv2 road-warrior access first')
-    try:
-        connection.add_s(dn_of(name), ldap.modlist.addModlist({
-            'objectClass': [b'inetOrgPerson'], 'uid': [name.encode()], 'cn': [name.encode()], 'sn': [name.encode()],
-        }))
-    except ldap.ALREADY_EXISTS:
-        raise Failure(f'user {name} exists', 4)
-    connection.passwd_s(dn_of(name), None, secret)
-    vpnrouter.USER_CERT_DIR.mkdir(mode=0o700, parents=True, exist_ok=True)
-    for path in certificate(name):
-        path.unlink(missing_ok=True)
-    simple_ca('make-cert', '--ca-dir', str(vpnrouter.PKI_DIR), '--cert-dir', str(vpnrouter.USER_CERT_DIR), name)
-    return summary(entry(connection, name))
-
-
-def passwd(connection, name):
-    secret = password()
-    entry(connection, name)
-    connection.passwd_s(dn_of(name), None, secret)
-    return summary(entry(connection, name))
-
-
-def disable(connection, name):
-    if 'pwdAccountLockedTime' not in entry(connection, name):
-        connection.modify_s(dn_of(name), [(ldap.MOD_REPLACE, 'pwdAccountLockedTime', [LOCKED])])
-    return summary(entry(connection, name))
-
-
-def enable(connection, name):
-    if 'pwdAccountLockedTime' in entry(connection, name):
-        connection.modify_s(dn_of(name), [(ldap.MOD_DELETE, 'pwdAccountLockedTime', None)])
-    return summary(entry(connection, name))
-
-
-def remove(connection, name):
-    entry(connection, name)
-    cert, key = certificate(name)
-    if cert.is_file():
-        simple_ca('revoke-cert', '--ca-dir', str(vpnrouter.PKI_DIR), str(cert))
-        simple_ca('make-crl', '--ca-dir', str(vpnrouter.PKI_DIR), '--days', str(vpnrouter.CRL_DAYS))
-        publish_crl()
-    cert.unlink(missing_ok=True)
-    key.unlink(missing_ok=True)
-    connection.delete_s(dn_of(name))
-    return {'name': name, 'removed': True}
-
-
-def profile(connection, name):
-    """The OpenVPN client profile of the user, with the CA, the certificate and the key inline."""
-    entry(connection, name)
-    cfg = vpnrouter.load_config()
-    if not cfg['openvpn']['enabled']:
-        raise Failure('OpenVPN is not enabled', 2)
-    cert, key = certificate(name)
-    if not cert.is_file() or not key.is_file():
-        raise Failure(f'user {name} has no certificate')
-    fqdn = cfg['wan']['local_fqdn'].strip()
-    text = '\n'.join([
-        'client',
-        'dev tun',
-        f'proto {cfg["openvpn"]["protocol"]}',
-        f'remote {fqdn} {cfg["openvpn"]["port"]}',
-        'nobind',
-        'persist-key',
-        'persist-tun',
-        'remote-cert-tls server',
-        f'verify-x509-name {fqdn} name',
-        'auth-user-pass',
-        'auth-nocache',
-        'verb 3',
-        '<ca>', extract_pem(vpnrouter.CA_CERT_DEST.read_text(encoding='utf-8')), '</ca>',
-        '<cert>', extract_pem(cert.read_text(encoding='utf-8')), '</cert>',
-        '<key>', key.read_text(encoding='utf-8').strip(), '</key>',
-        '',
-    ])
-    return {'name': name, 'profile': text}
-
-
-def extract_pem(text):
-    """The PEM blocks of the certificates in a file that also holds their text form."""
-    begin, end = '-----BEGIN CERTIFICATE-----', '-----END CERTIFICATE-----'
-    blocks = []
-    start = text.find(begin)
-    while start != -1:
-        stop = text.index(end, start) + len(end)
-        blocks.append(text[start:stop])
-        start = text.find(begin, stop)
-    return '\n'.join(blocks)
-
-
-COMMANDS = {'list': list_users, 'add': add, 'passwd': passwd, 'disable': disable, 'enable': enable,
-            'remove': remove, 'profile': profile}
 
 
 def main(argv):
+    # A request that cannot be parsed is answered in JSON when --json is among the arguments.
+    as_json = '--json' in argv
     try:
-        if not argv or argv[0] not in COMMANDS or len(argv) != (1 if argv[0] == 'list' else 2):
+        options = parse(argv)
-            raise Failure('usage: vpn-router-user list | add|passwd|disable|enable|remove|profile <name>', 2)
+        as_json = options.json
-        if len(argv) == 2 and not NAME.match(argv[1]):
+        if options.command != 'list':
-            raise Failure('a user name is 1 to 32 lowercase letters, digits and hyphens, starting with a letter', 2)
+            vpnrouter_users.check_name(options.name)
-        LOCK_FILE.parent.mkdir(parents=True, exist_ok=True)
+        secret = password() if options.command in ('add', 'passwd', 'otp-enrol') else None
-        with open(LOCK_FILE, 'w') as lock:
+        with vpnrouter_users.session() as connection:
-            fcntl.flock(lock, fcntl.LOCK_EX)
+            answer = run(connection, options, secret)
-            connection = vpnrouter_ldap.manager()
+        if options.command in vpnrouter_auth.REVOKING:
-            try:
+            vpnrouter_auth.revoke(vpnrouter.load_config(), options.name)
-                answer = COMMANDS[argv[0]](connection, *argv[1:])
-            finally:
-                connection.unbind_s()
     except Failure as failure:
-        print(json.dumps({'error': str(failure)}))
+        message, status = str(failure), failure.status
-        return failure.status
     except (ldap.LDAPError, OSError) as exc:
-        print(json.dumps({'error': f'the user directory failed: {exc}'}))
+        message, status = f'the user directory failed: {exc}', 1
-        return 1
+    except vpnrouter_auth.DatabaseError as exc:
-    print(json.dumps(answer))
+        message, status = str(exc), 1
-    return 0
+    else:
+        print(json.dumps(answer) if as_json else text_of(answer))
+        return 0
+    if as_json:
+        print(json.dumps({'error': message}))
+    else:
+        print(f'vpn-router-user: {message}', file=sys.stderr)
+    return status
 
 
 if __name__ == '__main__':

debian-package/src/usr/share/doc/vpn-router/vpn-router.json.example

@@ -45,7 +45,31 @@
   },
   "web": {
     "enabled": true,
-    "port": 8080,
+    "listen": "both",
-    "jwt_secret": ""
+    "port": 7081,
+    "database": "",
+    "trusted_proxies": [],
+    "status_content": "basic"
+  },
+  "https": {
+    "enabled": true,
+    "email": "",
+    "web_port": 7444,
+    "manager_port": 7443
+  },
+  "metrics": {
+    "enabled": true,
+    "listen": "internal",
+    "port": 9464,
+    "https_port": 9465,
+    "service_account": "metrics",
+    "otlp_endpoint": "",
+    "otlp_username": "",
+    "otlp_password": "",
+    "keep_details": "24 hours",
+    "keep_hourly": "1 month",
+    "keep_daily": "3 months",
+    "keep_monthly": "1 year",
+    "keep_yearly": "10 years"
   }
 }

debian-package/src/usr/share/vpn-router/templates/Caddyfile.j2

@@ -0,0 +1,22 @@
+# Written by vpn-router-setup from /etc/vpn-router/vpn-router.json; edits are overwritten.
+{
+	auto_https disable_redirects
+{%- if https_email %}
+	email {{ https_email }}
+{%- endif %}
+}
+{% if web_enabled %}
+{{ local_fqdn }}:{{ https_web_port }} {
+	reverse_proxy {{ loopback }}:{{ web_port }}
+}
+{% endif %}
+{%- if manager_port %}
+{{ local_fqdn }}:{{ https_manager_port }} {
+	reverse_proxy {{ loopback }}:{{ manager_port }}
+}
+{% endif %}
+{%- if metrics_enabled %}
+{{ local_fqdn }}:{{ metrics_https_port }} {
+	reverse_proxy {{ loopback }}:{{ metrics_port }}
+}
+{% endif %}