fix: the web service starts once libpq can be found.

fix: the web service waits for the PostgreSQL driver instead of failing at start.
feature: bootstrap diagnostics on port 17081 during the first setup.
2026-10-24 11:07:45 +02:00 · 2026-10-24 10:43:51 +02:00 · 2026-10-24 10:07:22 +02:00 · 2026-10-24 09:07:28 +02:00 · 2026-10-24 08:55:25 +02:00 · 2026-10-24 04:19:01 +02:00
29 changed files with 2494 additions and 600 deletions
@@ -27,67 +27,67 @@ port is reachable from the external interface only when it is in `exposed_ports`
 in `DOCKER-USER` are not touched, because the package owns only its own chain. Restarting Docker
 restores the rules. The rules cover IPv4 only.
-## ADR-2: A status web service of the standard library, bound to the internal address
+## ADR-2: A small status web service, bound to the internal address
-Status: Accepted; amended by ADR-7
+Status: Accepted
 Context: The router has no way to see its state without a shell. A check from the protected
 network, by a person or by a monitor, needs a page or an endpoint. The router is an
 internet-facing host, so what the service shows and where it listens matter.
-Decision: The `vpn-router-web` unit runs one Python script that uses only the standard library, in
+Decision: The `vpn-router-web` unit runs one Python script on the standard library's
-the way of a small test server. It binds to the address of the internal interface and to
+`http.server`, with the package's own modules and the Python packages of the Ubuntu archive that
-`web.port`, on by default at 8080. `/` is for people, in plain text for `curl` and `wget` and as
+the package depends on, and no web framework. It listens on the address of the internal interface and on
-a Tailwind CSS page for a browser, and `/status` is the same status as JSON. It shows the
+`web.port`, 7081 by default, and is on unless `web.enabled` is `false` (ADR-7). `/` is for
-state of the services, the package version and the state of the site-to-site, road-warrior and
+people, in plain text for `curl` and `wget` and as a Tailwind CSS page for a browser, and
-WireGuard features, and nothing from the configuration. `vpn-router-setup` resolves the
+`/status` is the same status as JSON. It shows the state of the services, the package version
-interface address, writes it with the port and the features to `/run/vpn-router/web.json`,
+and the state of the site-to-site, road-warrior, OpenVPN and WireGuard features, and nothing from
-opens the port on the internal interface in UFW and restarts the service, so the service does
+the configuration. `vpn-router-setup` opens the port on the internal interface in UFW and writes
-not need the package's configuration code, which needs Jinja2.
+the features and the facts it resolves to `/run/vpn-router/web.json`.
 Consequences: Nothing outside the internal network can reach it, since it listens on the internal
-address only and the firewall opens the port on the internal interface only. It has no
+address only and the firewall opens the port on the internal interface only. `/` and `/status`
-authentication, which is acceptable because it is read only and shows no configuration. It runs
+have no authentication, which is acceptable because they are read only and show no
-as root, because `swanctl` and `wg` need it, with `NoNewPrivileges`, `ProtectSystem=strict`,
+configuration. It runs as root, because `swanctl` and `wg` need it, with `NoNewPrivileges`,
-`ProtectHome` and `PrivateTmp`. The Tailwind style comes from a CDN, so a browser on a network
+`ProtectSystem=strict`, `ProtectHome` and `PrivateTmp`. The Tailwind style comes from a CDN, so a
-without internet shows the unstyled page, and `curl` is unaffected. A change of the internal
+browser on a network without internet shows the unstyled page, and `curl` is unaffected.
 address is picked up at the next run of `vpn-router-setup`.
-## ADR-3: /config is authenticated with a short-lived JWT signed with a shared secret
+## ADR-3: The protected paths take JWTs that only the router signs
-Status: Accepted; amended by ADR-7 and ADR-8
+Status: Accepted
-Context: A client such as manage-ui needs the configuration and the resolved state of the
+Context: Clients such as the Router Manager, and people with `curl`, need the configuration,
-router, which `/` and `/status` must not show. The service has only the standard library, listens
+the users and the groups of the router, which `/` and `/status` must not show. Both people and
-on plain HTTP on the internal network, and its configuration file is readable by root only.
+programs sign in with a user of the directory. A secret shared with the platform that deploys
 the router would let whoever holds it, and every copy of the deployment's state, make tokens.
 The service listens on plain HTTP on the internal network.
-Decision: `/config` needs a JWT in an `Authorization: Bearer` header. The service verifies HS256
+Decision: `/config`, `/users` and `/groups` need a JWT in an `Authorization: Bearer` header,
-tokens with `hmac` and `hashlib`, with the secret `web.jwt_secret`, and accepts no other
+HS256, verified with `hmac` and `hashlib`; no other algorithm is accepted. The key is
-algorithm, because verifying RS256 or ES256 needs a library outside the standard library. The
+`/etc/vpn-router/web.key`, random, made by the web service on its first run and readable by root
-token must carry `exp`, not more than ten minutes ahead, so that the client makes a fresh token for
+only; it is never configured or output. The router itself issues
-each use and a leaked one expires. Without a secret the page is not served. The secrets in the
+the tokens: `POST /auth` checks a user's password and TOTP code with a bind (ADR-14) and answers
-configuration shown are masked by name (`psk`, and options with `secret`, `password`, `token` or a
+an access token naming the user in `sub`, valid for 5 minutes, and a refresh token valid for 8
-`_key` ending in the name).
+hours (ADR-15); `vpn-router-token` makes tokens with the `admin` claim (ADR-13). A token without
 `admin` must expire within ten minutes, so a leaked one is of little use. Without the key the
 paths and `/auth` are not served. The configuration shown has no secret options (ADR-8).
-Consequences: The client and the router share one secret, and whoever holds it can make tokens,
+Consequences: Nothing outside the router can make a token, so a token is either issued to a
-so it is long and random; it is checked to have at least 32 characters. The secret is in the
+user who signed in or made by root on the router. A client keeps the refresh token to stay
-configuration file and in `/run/vpn-router/web.json`, both readable by root only. The token and
+signed in. A token of `vpn-router-token` can be revoked only by replacing the key, which ends
-the answer cross the network unencrypted, so the page is for a trusted internal network; HTTPS
+every token. Tokens, passwords and answers cross the network unencrypted, so the API is for a
-would need a certificate that the package does not have. The masking is by option name, so a new
+trusted internal network; HTTPS would need a certificate that the package does not have.
 secret option needs a name that matches.
 ## ADR-4: Users are kept in a local OpenLDAP directory
 Status: Accepted
 Context: OpenVPN needs user accounts with passwords, and the users are meant to cover every P2S
-system of the router. The options weighed were an external OIDC provider (Entra ID, Keycloak)
+system of the router, including IKEv2 EAP-TLS and WireGuard, which an OIDC provider cannot
-with openvpn-auth-oauth2, an OIDC server embedded on the router, a file of password hashes, and
+authenticate. Everything the router runs comes from the Ubuntu archive. slapd, its Python module
-OpenLDAP on the router. OIDC for OpenVPN needs a browser on the client and a program from outside
+`python3-ldap` and the OpenVPN plugin `openvpn-auth-ldap` are all in the archive; the plugin
-the Ubuntu archive, and it cannot authenticate IKEv2 EAP-TLS or WireGuard. slapd, its Python
+checks a password by binding as the user, so the hash scheme is slapd's alone. An OIDC provider
-module `python3-ldap` and the OpenVPN plugin `openvpn-auth-ldap` are all in the archive; the
+can read the directory through its LDAP connector.
 plugin checks a password by binding as the user, so the hash scheme is slapd's alone. A directory
 can also be read later by an OIDC provider through its LDAP connector.
 Decision: The users are entries in slapd. The package keeps its own database,
 `dc=vpn-router,dc=local` in `/var/lib/ldap/vpn-router` (the path the slapd AppArmor profile
@@ -101,19 +101,18 @@ slapd listens on the local socket only. Provisioning checks every step before ch
 runs at every start of `vpn-router-setup`.
 Consequences: The package depends on `slapd`, `python3-ldap`, `openvpn` and
-`openvpn-auth-ldap`, and `vpn-router-setup` and `vpn-router-user` import `python3-ldap`, the first
+`openvpn-auth-ldap`, and `vpn-router-setup` and `vpn-router-user` import `python3-ldap`. The
-library outside the standard library besides `jinja2`. The service account's password is a
+service account's password is a root-only file. The directory is user data: it is kept on purge,
-root-only file. The directory is user data: it is kept when the features are switched off and
+and it is backed up with `slapcat`, not by copying files.
 on purge, and it is backed up with `slapcat`, not by copying files.
 ## ADR-5: OpenVPN clients need a certificate and the password of the user it names
 Status: Accepted
-Context: The router CA already issues client certificates for IKEv2. OpenVPN can check a
+Context: The router CA issues client certificates for IKEv2. OpenVPN can check a certificate, a
-certificate, a password, or both. With both, a certificate alone and a password alone are not
+password, or both. With both, a certificate alone and a password alone are not enough, but
-enough, but `openvpn-auth-ldap` checks only the password of the user name given at login, which
+`openvpn-auth-ldap` checks only the password of the user name given at login, which need not be
-need not be the user of the certificate.
+the user of the certificate.
 Decision: OpenVPN checks the certificate against the CA and the CRL, the password with
 `openvpn-auth-ldap`, and runs `openvpn-same-user` as `auth-user-pass-verify` with `via-env`,
@@ -130,62 +129,275 @@ checks certificates only; removing a user revokes the certificate for both. A us
 certificate expires after the CA's default of a year and is renewed by removing and adding the
 user again.
-## ADR-6: The web service manages users by running vpn-router-user
+## ADR-6: The web service and vpn-router-user share one module for users
 Status: Accepted
-Context: The web service has only the standard library (ADR-2), and managing users needs
+Context: Users are managed from the command line, by an operator at a terminal, and through
-`python3-ldap` and `simple-ca`. Its unit is sandboxed with `ProtectSystem=strict`.
+`/users` of the web service, by the Router Manager. Both must apply the same rules to names,
 passwords and person attributes, and both change the directory and the CA. `simple-ca` keeps
 the CA's state in one file, so two changes must not run at the same time. The web unit is
 sandboxed with `ProtectSystem=strict`.
-Decision: `/users` runs `vpn-router-user` and returns its JSON answer, mapping its exit status
+Decision: `vpnrouter_users.py` holds the user operations, the checks and a session that takes a
-to the HTTP status. Passwords go to the program on standard input, so they never appear in a
+lock and the connection of the user manager account. The web service imports it and maps a
-command line. The unit allows writing the paths the program needs: `/run/vpn-router` for its
+`Failure` to the HTTP status; `vpn-router-user` imports it and adds the argument parsing and
-lock, `/etc/vpn-router/pki` for the certificates and the CA's state, and `/etc/swanctl/x509crl`
+the output, text for people and JSON with `--json`. The web unit allows writing the paths the
-for the published CRL. A lock serializes the program's runs, because `simple-ca` keeps its state
+operations need: `/run/vpn-router` for the lock, `/etc/vpn-router/pki` for the certificates and
-in one file.
+the CA's state, and `/etc/swanctl/x509crl` for the published CRL.
-Consequences: The web service still imports nothing outside the standard library, and the same
+Consequences: The rules exist once. A request starts no process, and a password never leaves
-program serves the command line and the API. Each request starts a process, which is fine for
+the process that received it. The web service imports `python3-ldap` and the package's
-user administration. The API is protected by the token of `/config`, so whoever has the secret
+configuration module. The API is protected by the tokens of ADR-3, so only members of `admins`
-can manage users.
+and the holders of an admin token manage users.
 ## ADR-7: The web service starts on its own and reads its inputs at each request
 Status: Accepted
-Context: The web service took the address to listen on, the port, the token secret and the
+Context: The status page is the tool to see a problem of `vpn-router-setup`, so it must run
-features from `/run/vpn-router/web.json`, written by `vpn-router-setup`, and ended with a failure
+whatever state setup is in, also before setup has run and while it fails. The internal address,
-while the file or the internal address was missing. It was ordered after `vpn-router-setup`, so a
+`web.port` and `web.enabled` can change at any time.
 problem in the setup kept the status page, the tool to see that problem, from running.
 Decision: The service depends on nothing to start and does not end on its own. A discovery
 thread works out, every 30 seconds, the internal address from `vpn-router.json` as
-`general.mode` directs, with the discovery code in `vpnrouter_net.py`, which uses only the
+`general.mode` directs, with the discovery code in `vpnrouter_net.py`, which `vpnrouter.py` uses
-standard library and which `vpnrouter.py` uses too, and `web.port` and `web.enabled` from the
+too, and `web.port` and `web.enabled` from the same file. It runs the HTTP server in a thread of
-same file. It runs the HTTP server in a thread of its own at that address and port, stops it and
+its own at that address and port, stops it and starts a new one when they change, and runs none
-starts a new one when they change, and runs none while the service is disabled, the address is
+while the service is disabled, the address is not known or it cannot be bound. Each request
-not known or it cannot be bound. Each request reads the token secret from `vpn-router.json` and
+reads the signing key and `vpn-router.json`, and the features and resolved facts from
-the features and resolved facts from `web.json`, which setup still writes, because they need the
+`web.json`, which setup writes when it applies the configuration, so the service shows what setup
-configuration code; while that file is missing the router shows no features. The unit is not
+applied; while that file is missing the router shows no features. The unit is not
 ordered after `vpn-router-setup`, and setup does not restart it.
 Consequences: The status page answers as soon as the internal address exists, whatever state
 setup is in, and follows a changed address, port or `web.enabled` within 30 seconds without a
-restart. `web.json` no longer holds the address, the port or the secret, so the secret is only in
+restart. A new signing key applies to the next request. The interfaces are read every 30
-`vpn-router.json`. A change of the secret applies to the next request. The interfaces are read
+seconds, which costs a few `ip` commands.
 every 30 seconds, which costs a few `ip` commands.
 ## ADR-8: /config leaves secret options out
 Status: Accepted
-Context: `/config` showed the secret options of the configuration with their values replaced by
+Context: A secret has no use outside the router, so a client never needs it, and showing even
-`********`. That still tells a reader which secrets are set, and a secret has no use outside the
+that a secret is set tells a reader more than it needs.
 router, so a client never needs it.
 Decision: The configuration that `vpn-router-setup` writes for the web service has no secret
-options at all. They are recognised by name as before: `psk`, and options with `secret`,
+options at all. They are recognised by name: `psk`, `database`, whose connection string may hold
-`password`, `token` or a `_key` ending in the name.
+a password, and options with `secret`, `password`, `token` or a `_key` ending in the name.
 Consequences: No secret reaches the web service's state file or any answer of the API. A client
-cannot tell whether a secret is set. A new secret option still needs a name that matches.
+cannot tell whether a secret is set. A new secret option needs a name that matches.
 ## ADR-9: The Router Manager is a separate package with a small Python server
 Status: Accepted
 Context: The router needs a web application to manage it, which may run on the router or on
 another Linux system. The router's web service issues tokens through `/auth` (ADR-3) and sends
 no CORS headers, so a browser cannot call it directly, and tokens kept in the page could be read
 by its scripts.
 Decision: `vpn-router-manager` is a Debian package of its own, built in the repository
 `linux-cloud-router-manager`. A React and MUI application in the style of manage-ui is built
 with Vite when the package is built and shipped as static files. A Python server, using the
 Ubuntu packages `python3-requests` and `python3-psycopg`, serves them, signs users in through the
 router's `/auth`, keeps their tokens in a server-side session in PostgreSQL behind an
 `HttpOnly` cookie, and passes every other `/api/<path>` call to the router's `/<path>` with the
 session's access token, refreshing it as needed; it holds no logic of the router's own and no
 key of it. It reads its settings from `/etc/vpn-router-manager/manager.json` at each request; on
 the router the settings not given come from `vpn-router.json`, and it listens on the internal
 address like the status web service, with the same discovery thread. `vpn-router-setup` opens
 the manager's port on the internal interface while the manager is installed, so the firewall
 stays in one place.
 Consequences: The router needs no Node.js; the package needs only `python3` and packages of the
 Ubuntu archive. Tokens never reach the browser, and sessions survive a restart of the manager.
 The router decides what a user may do; the manager only signs in. The application checks the
 shape of every answer of the router. The router's firewall depends on whether the manager is
 installed, and a changed manager port needs `vpn-router-setup` to run again.
 ## ADR-10: Users and their certificates do not depend on a P2S system being enabled
 Status: Accepted
 Context: Users are prepared before a P2S system is switched on, and the Router Manager manages
 them on any configured router, with or without one.
 Decision: Once a platform is chosen, `vpn-router-setup` always provisions the user directory in
 slapd and the CA with its CRL; the server certificate follows once `wan.local_fqdn` is set.
 `/users` and `vpn-router-user` rely only on the directory and the CA. Only the OpenVPN client
 profile needs OpenVPN to be enabled.
 Consequences: slapd always holds the directory and the CA always exists on a configured router,
 even when no P2S system uses them. An operator who wants a CA of their own must place it before
 choosing the platform.
 ## ADR-11: Groups are groupOfNames entries that never become empty
 Status: Accepted
 Context: Users are to be organised in groups. Groups record membership only; nothing on the
 router reads them. LDAP clients that read groups, such as `openvpn-auth-ldap` and the LDAP
 connectors of OIDC providers, expect `groupOfNames` with member DNs, and OpenLDAP's core schema
 requires a `groupOfNames` to have at least one `member`.
 Decision: A group is `cn=<name>,ou=groups,dc=vpn-router,dc=local`, a `groupOfNames` with the
 DNs of its members and an optional `description`. A group is created with at least one member,
 and taking out its last member is refused; such a group is removed instead. Removing a user
 takes it out of its groups and is refused while the user is the last member of one. The group
 operations are in `vpnrouter_users.py` with the user operations, so they share the session and
 its lock; `vpn-router-group` is a tool of its own beside `vpn-router-user`, and the web service
 serves them under `/groups`. Group names follow the rules of user names.
 Consequences: A group always satisfies the schema without a placeholder member, and an empty
 group cannot exist. Removing a user or a member can need another step first. A later use of
 groups, such as allowing OpenVPN only to some, can read them as they are.
 ## ADR-12: TOTP secrets are kept with the OATH-LDAP schema of slapd's otp module
 Status: Accepted
 Context: Users are enrolled for TOTP one-time passwords, and enrolled users must give a code
 whenever they log in, to OpenVPN and to `/auth`. Ubuntu's slapd ships the `otp` module, which
 holds the OATH-LDAP schema and an overlay that checks a code appended to the password at bind
 time; both OpenVPN (`openvpn-auth-ldap`) and `/auth` check passwords with a bind. Ubuntu
 packages `python3-pyotp`. An authenticator app is enrolled from a QR code of an `otpauth://` URI.
 Decision: The `otp` module is loaded and its overlay is configured on the package's database,
 after ppolicy, so every bind of an enrolled user takes the password followed by a current code
 and slapd records the time step used. The TOTP token is the user entry itself: enrolment adds `oathTOTPUser` and `oathTOTPToken` to it, with
 `oathTOTPToken` naming the entry, the secret in `oathSecret` and `oathTOTPParams` naming one
 shared parameter entry, `cn=totp,ou=policies`, with the parameters authenticator apps assume
 (6 digits, HMAC-SHA1, 30 seconds). An ACL lets the user manager account set `oathSecret` and
 no one read it. Enrolment is confirmed: `otp_new` makes a secret with `pyotp` and returns it
 with its URI, storing nothing, and `otp_enrol` stores the secret it is given back only when a
 code of it verifies, so a secret is never stored that the user's app does not produce. The
 router answers the URI; the client renders the QR code.
 Consequences: Removing a user removes its token. One check covers OpenVPN and `/auth`, and
 an enrolled user types the code after the password in the OpenVPN client. The secret passes
 through the client between `otp_new` and `otp_enrol`, so the router keeps no pending state;
 enrolling again replaces the secret only once a code of the new one verifies.
 ## ADR-13: Members of admins manage the router; only CLI tokens change admins
 Status: Accepted
 Context: Signing in through `/auth` (ADR-3) is open to every user of the directory, most of
 whom only use OpenVPN. Finer permissions per operation are not needed. Somebody must be able to
 manage the router before any administrator exists and after every administrator lost access,
 and administrators must not be able to grant themselves or others the administrator role over
 the API.
 Decision: A token naming a user in `sub` may call the protected paths only while the user is a
 member of the group `admins`, read at each request; other users get 403. `vpn-router-token`, run
 as root on the router, makes tokens with the claim `admin: true` and a lifetime given at
 creation, without a maximum, signed with the router's key; `/auth` never issues the claim. An
 admin token may call everything, and only an admin token may change `admins`: create it, rename a
 group to it, change or remove it, add or take out its members, or remove a user who is a member
 of it. The first member is added with `vpn-router-group` on the router, which acts as root.
 Consequences: Removing a user from `admins` takes effect at the next request, without waiting
 for the user's tokens to expire. Membership of `admins` changes only through root on the router.
 An admin token is as powerful as root over the API and is revoked only by replacing the key, so
 it is given a short lifetime unless it is meant for automation.
 ## ADR-14: Failed logins lock the account in slapd; /auth is rate limited per client
 Status: Accepted
 Context: `/auth` is open to the internal network and checks passwords, so it invites guessing;
 OpenVPN checks passwords from the internet. Both check with a bind, so slapd sees every attempt.
 ppolicy locks an account after a number of failed binds for a fixed time; it has no growing
 lockout. A Router Manager signs all its users in from one address.
 Decision: The default password policy locks an account for 15 minutes after 5 failed binds
 within 15 minutes; a successful bind clears the count; `vpn-router-user unlock` and
 `POST /users/<name>/unlock` end a lockout and clear the count, and leave a disabled user disabled.
 `/auth` answers the same 401 for a wrong name, password or code and for a locked or disabled
 user. `/auth` and `/auth/refresh` allow 10 calls a minute per client address, counted in memory,
 and answer 429 with `Retry-After` beyond that. The client address is the peer, or the last
 `X-Forwarded-For` entry for a peer in `web.trusted_proxies`, by default the address the service
 listens on, where a manager on the router calls from.
 Consequences: One lockout covers OpenVPN and `/auth`, kept by slapd alone. Someone who knows a
 user name can lock the user out for 15 minutes at a time. The rate limit restarts with the
 service. A manager elsewhere than on the router has its address added to `web.trusted_proxies`,
 or its users share one limit.
 ## ADR-15: Refresh tokens are kept in PostgreSQL
 Status: Accepted
 Context: An access token lives 5 minutes, and a client stays signed in with a refresh token for
 8 hours; refresh tokens must be revocable when a user signs out, is disabled, changes the
 password or is removed, and must survive a restart of the service. The Router Manager keeps its
 sessions in PostgreSQL too.
 Decision: Refresh tokens are random strings, stored as their SHA-256 with the user name and the
 expiry in the table `refresh_tokens`, through psycopg 3 (`python3-psycopg`). The database is
 `web.database`, or by default the database and role `vpn-router` on the router, which the web
 service creates on its first run (ADR-16) with `runuser -u postgres -- psql` and a random password in
 `/etc/vpn-router/database.secret`, reached over TCP on `localhost`, as Ubuntu's `pg_hba.conf`
 takes peer authentication on the local socket. A refresh token is used once: refreshing deletes
 it and issues a new one, after checking that the user is neither disabled nor locked.
 Consequences: The package depends on `postgresql` and `python3-psycopg`. A stolen database holds
 no usable token. Changing a user's password, disabling, unlocking or removing the user, and
 signing out, end the user's sessions at once; an access token already issued lives out its
 5 minutes.
 ## ADR-16: Each service creates what it needs on its first run
 Status: Accepted
 Context: The package manager configures packages in an order of its own; on a new VM the Router
 Manager was configured before the PostgreSQL server, so a database created at installation was
 missing. PostgreSQL may also start after the services at boot.
 Decision: The web service makes its signing key and creates the refresh tokens' database itself,
 and the Router Manager its sessions database; neither `vpn-router-setup` nor a maintainer script
 creates them. Each service does it before it listens, retrying every 15 seconds and logging each
 failure, then listens. `vpnrouter_auth` turns every failure of the database into
 `DatabaseError`, the only database exception the web service and `vpn-router-user` handle.
 psycopg (Ubuntu ships only its pure-Python implementation) finds libpq through the dynamic linker
 cache, which dpkg refreshes with its deferred `ldconfig` only at the end of an installation, so a
 service started during the installation cannot import it yet; a failed import cannot be repeated
 in the same process either, as psycopg caches the lookup. The services therefore import psycopg
 plainly, and their units wait instead: `ExecStartPre=wait-for-libpq`, a shell script of each
 package, waits until `ldconfig -p` lists `libpq.so.5`, within `TimeoutStartSec=5min`. The
 maintainer scripts start the units with `systemctl --no-block` (debhelper's start is turned off),
 because a start that waited would hold dpkg before its `ldconfig`.
 Consequences: The order of installation and of the start of PostgreSQL and of the linker cache
 does not matter; a database that cannot be created is retried until the cause is gone, and the
 reason is in the service's log. Outside an installation the wait ends at once. The web unit may
 write `/etc/vpn-router` for the key and the database password.
 ## ADR-17: Bootstrap diagnostics run only until the first setup succeeds
 Status: Accepted
 Context: A new VM is reachable only once its setup has worked, and the reason of a failure must
 be readable from the internal network before an SSH session is possible. A service that reports
 the setup cannot be one of the services being set up: those import packages that may not load yet
 (psycopg could not find libpq before `ldconfig` ran), and they would fail before reporting
 anything. Once the router has been set up, SSH is the way to diagnose it. The more a diagnostic
 page does, the more ways it has to fail itself.
 Decision: Each package ships a bootstrap service of the Python standard library alone, a script
 that answers every request with the plain-text output of a fixed list of commands: the setup
 state, the services, the failed units, the addresses and the end of the logs. The router's
 listens on 17081 and the Router Manager's on 17080, beside the ports of the services, 7081 and
 7080, on every address. The units are never enabled and not started at installation;
 `vpn-router-setup` starts them while `/var/lib/vpn-router/setup.done` does not exist, and opens
 their ports on the internal interface. The first successful run writes that file, closes the ports
 and stops them; a failed run leaves them running, so the reason stays readable. Setup records its
 state, step and failure in `/run/vpn-router/setup.json` at every step; `vpnrouter.fail` ends with
 a `SystemExit` carrying its message, so setup records it.
 Consequences: The diagnostics need nothing the setup provides and cannot collide with the real
 services. After the first success they never run again, also not after an upgrade or a reboot.
 The page shows log lines to whoever reaches the internal network during the first setup.
@@ -85,8 +85,10 @@ that deploy a router on Azure or GCP.
 ## Configuration
 `/etc/vpn-router/vpn-router.json` is JSON, mode 0600, and is created on first install. It
-is managed with `ucf`, so it is yours to edit: a later `dpkg-reconfigure` applies what
+is managed with `ucf`, so it is yours to edit: a later `dpkg-reconfigure` applies the answers
-changed and asks before touching anything you altered by hand. A complete example is
+given and asks before touching anything you altered by hand, and an upgrade asks no questions
 and leaves the file as it is. Settings the file leaves out take their defaults when it is read, so a
 file written by cloud-init or by hand needs only the settings that differ. A complete example is
 installed at `/usr/share/doc/vpn-router/vpn-router.json.example`.
 The file is an object of sections, each an object of settings. A section or setting that
@@ -122,8 +124,9 @@ must have the type shown below; an unknown section or setting is an error.
 | `docker` | `mode` | string | `auto` | `auto` secures Docker when its service is installed, `on` requires it, `off` removes the rules |
 | `docker` | `exposed_ports` | list | | TCP ports and port ranges (`8000:8100`) reachable from the external interface; at most 15, a range counting as two |
 | `web` | `enabled` | boolean | `true` | The status web service on the internal address |
-| `web` | `port` | integer | `8080` | TCP port of the status web service |
+| `web` | `port` | integer | `7081` | TCP port of the status web service |
-| `web` | `jwt_secret` | string | | Secret, at least 32 characters, that signs the tokens for `/config`; without it `/config` is not served |
+| `web` | `database` | string | | A PostgreSQL connection string for the refresh tokens; without it, the `vpn-router` database on this host that the web service creates |
 | `web` | `trusted_proxies` | list | the listening address | Addresses whose `X-Forwarded-For` header names the client for the `/auth` rate limit |
 A minimal site-to-site configuration in `auto` mode:
@@ -272,12 +275,12 @@ ca_cert.pem  ca_key.pem  ca_bundle.pem  <label>_cert.pem  <label>_key.pem  crl.p
 `<label>` is the first component of the router FQDN, so `router.example.com` gives
 `router_cert.pem`.
-If road-warrior or OpenVPN access is enabled and this directory is empty, a certificate
+Once a platform is chosen and while this directory is empty, a certificate authority and a CRL
-authority, a server certificate and a CRL valid for ten years are created there
+valid for ten years are created there automatically, whether or not road-warrior or OpenVPN
-automatically; a CA without a CRL gets one. User certificates are issued by
+access is enabled. A CA of the router's own (with `ca_key.pem`) without a CRL gets one, and
-`vpn-router-user` into `users/`. To use your own instead, place the
+without a server certificate gets one once `wan.local_fqdn` is set. User certificates are issued
-files in that layout before enabling the feature; nothing is generated or overwritten when
+by `vpn-router-user` into `users/`. To use your own CA instead, place the files in that layout
-material is already present.
+before choosing the platform; nothing present is overwritten.
 Either way, the package copies what each consumer needs into place: the CA into
 `/etc/swanctl/x509ca/`, the server certificate and key into `/etc/swanctl/x509/` and
@@ -308,60 +311,121 @@ certificate names:
 - the certificate is checked against the CA and the CRL (`crl-verify`, read at every
  connection);
 - the password is checked by `openvpn-auth-ldap`, which binds to the user directory as the
-  user;
+  user; a user enrolled for TOTP types the password followed by the current 6-digit code;
 - `/usr/lib/vpn-router/openvpn-same-user` refuses a login whose user name is not the common
  name of the certificate.
 ### The user directory
-While OpenVPN or road-warrior access is configured, `vpn-router-setup` keeps a directory of
+Once a platform is chosen, `vpn-router-setup` keeps a directory of users in the local slapd,
-users in the local slapd:
+whether or not OpenVPN or road-warrior access is enabled, so users can be managed before either
 is switched on:
 - its own database, `dc=vpn-router,dc=local` in `/var/lib/ldap/vpn-router`, next to the one the
  slapd package creates; root manages it through the local socket, with no administrator
  password;
 - `ou=users` holds one `inetOrgPerson` per user, named by `uid`;
- `cn=user-manager,ou=services` is the account that changes users; it may write `ou=users` and
+- `ou=groups` holds one `groupOfNames` per group, named by `cn`, with its members' DNs in
-  set passwords, never read them, and its password is in
+  `member` and an optional `description`;
 - `cn=user-manager,ou=services` is the account that changes users and groups; it may write
  `ou=users` and `ou=groups` and set passwords, never read them, and its password is in
  `/etc/vpn-router/ldap/user-manager.secret` (root only);
 - passwords are stored as ARGON2 hashes, and a disabled user is locked with the password policy
  (`pwdAccountLockedTime`);
 - the password policy locks an account for 15 minutes after 5 failed binds within 15 minutes,
  whether they come from OpenVPN or from `/auth`; a successful bind clears the count, and
  `vpn-router-user unlock` ends a lockout;
 - slapd's `otp` overlay is configured on the database; a user enrolled for TOTP is also an
  `oathTOTPUser` and `oathTOTPToken` whose `oathTOTPToken` names the entry itself, with the
  secret in `oathSecret`, which the user manager account may set and no one can read, and
  `oathTOTPParams` naming `cn=totp,ou=policies` (6 digits, HMAC-SHA1, 30 second steps, one step
  of tolerance). A bind of an enrolled user takes the password followed by a current code;
 - slapd listens on its local socket only (`SLAPD_SERVICES` in `/etc/default/slapd`).
-The directory is kept when OpenVPN and road-warrior access are switched off, and on purge.
+The directory is kept on purge.
 ### Managing users
-`vpn-router-user` manages the users. A user is the directory entry and a client certificate
+`vpn-router-user` manages the users. A user is the directory entry, with a first name, a last
-with the user name as its common name, in `/etc/vpn-router/pki/users`. Passwords are read from
+name and an e-mail address (`givenName`, `sn` and `mail`; `cn` is the first and the last name),
-the first line of standard input. The answer is JSON.
+and a client certificate with the user name as its common name, in `/etc/vpn-router/pki/users`.
 Passwords and TOTP secrets are read from the first line of standard input. The answer is text: a
 table of the users, the client profile, a TOTP secret or a short message, and a failure is a
 message on standard error.
 `--json`, before the command, gives the answer as JSON on standard output, and a failure as
 `{"error": ...}` there too; the web service uses it. `-h` or `--help` shows the commands, and
 after a command its options.
 ```sh
-echo 's3cr3t-pass' | vpn-router-user add alice
+echo 's3cr3t-pass' | vpn-router-user add alice --first-name Alice --last-name Smith --email alice@example.com
 vpn-router-user update alice --email a.smith@example.com
 vpn-router-user list
-vpn-router-user profile alice
+vpn-router-user profile alice > alice.ovpn
 vpn-router-user --json list
 echo 'n3w-pass' | vpn-router-user passwd alice
 vpn-router-user disable alice
 vpn-router-user enable alice
 vpn-router-user unlock alice
 vpn-router-user remove alice
 vpn-router-user otp-new alice
 echo 'JBSWY3DPEHPK3PXPJBSWY3DPEHPK3PXP' | vpn-router-user otp-enrol alice 123456
 vpn-router-user otp-remove alice
 ```
 | Command | Effect |
 | --- | --- |
-| `list` | The users, whether each is enabled and has a certificate |
+| `list` | The users with their names and e-mail addresses, whether each is enabled or locked, has a certificate and is enrolled for TOTP |
-| `add <name>` | Adds the user and issues the certificate |
+| `add <name> --first-name F --last-name L --email E` | Adds the user and issues the certificate; all three are required |
 | `update <name> [--first-name F] [--last-name L] [--email E]` | Changes the names or the e-mail address |
 | `passwd <name>` | Sets the password |
-| `disable <name>`, `enable <name>` | Locks the user out of OpenVPN, and lets the user in again |
+| `disable <name>`, `enable <name>` | Locks the user out of OpenVPN and `/auth`, and lets the user in again |
-| `remove <name>` | Revokes the certificate, renews the CRL and removes the user |
+| `unlock <name>` | Ends a lockout after failed logins and clears the failure count; a disabled user stays disabled |
 | `remove <name>` | Takes the user out of its groups, revokes the certificate, renews the CRL and removes the user; refused while the user is the last member of a group |
 | `profile <name>` | The OpenVPN client profile, with the CA, the certificate and the key inline |
 | `otp-new <name>` | A new TOTP secret and its `otpauth://` URI, issued by `wan.local_fqdn`, for an authenticator app; nothing is stored |
 | `otp-enrol <name> <code>` | Stores the secret read from standard input when the code is a current code of it, replacing a secret the user has |
 | `otp-remove <name>` | Removes the user's TOTP secret |
 A user name is 1 to 32 lowercase letters, digits and hyphens, starting with a letter; a
-password has 8 to 128 characters. The exit status is 0 on success, 2 for an invalid request,
+password has 8 to 128 characters; a first or last name has 1 to 64 printable characters; an
 e-mail address has one `@`, no spaces and a dot in its domain; a TOTP secret is 32 base32
 characters, as `otp-new` makes it, and a code 6 digits. Once enrolled, a user logs in to OpenVPN
 and `/auth` with the password followed by a current code. In the answers a user added
 without the names or the e-mail address has them as `null`. The exit status is 0 on success, 2 for an invalid request,
 3 when the user does not exist, 4 when the user exists and 1 for any other failure.
 The certificate is also accepted by road-warrior (IKEv2 EAP-TLS) access, which checks
 certificates only, so `disable` does not affect it; `remove` revokes the certificate for both.
 ### Managing groups
 `vpn-router-group` manages the groups. A group has a name, with the rules of a user name, an
 optional description of at most 256 printable characters, and at least one member, each a user
 of the directory. Groups record membership only; nothing on the router reads them. The output,
 `--json`, `-h` and `--help` work as for `vpn-router-user`.
 ```sh
 vpn-router-group add ops --member alice --member bob --description 'Operations team'
 vpn-router-group list
 vpn-router-group update ops --name netops
 vpn-router-group update netops --description ''
 vpn-router-group member-add netops carol
 vpn-router-group member-remove netops bob
 vpn-router-group remove netops
 ```
 | Command | Effect |
 | --- | --- |
 | `list` | The groups with their members and descriptions |
 | `add <name> --member U [--member U ...] [--description D]` | Adds the group with its members |
 | `update <name> [--name N] [--description D]` | Renames the group or sets its description; `''` removes it |
 | `remove <name>` | Removes the group; its members are kept |
 | `member-add <name> <user>` | Adds the user to the group |
 | `member-remove <name> <user>` | Takes the user out of the group; refused for the last member |
 The exit status is 0 on success, 2 for an invalid request, 3 when the group or the user does
 not exist, 4 when the group exists and 1 for any other failure.
 ## Platform modules
 Platform-specific values do not belong in the core, so they are Python modules in
@@ -406,15 +470,38 @@ modules shipped with the package; the configuration file accepts any module inst
 ## Status web service
 The `vpn-router-web` service shows the state of the router on its internal address and
-`web.port` (8080 by default). It uses only the Python standard library, serves HTTP without
+`web.port` (7081 by default). Its pages `/` and `/status` need no authentication and show no
-authentication, and shows no configuration: no key, identity, address or subnet.
+configuration: no key, identity, address or subnet.
 The service first makes its signing key and the refresh tokens' database, retrying every 15
 seconds, for example while PostgreSQL is not running yet, and logs each failure; then it listens.
 ### Bootstrap diagnostics
 Until a run of `vpn-router-setup` has succeeded, every run starts `vpn-router-bootstrap`, which
 serves diagnostics as plain text on port 17081 of every address, and the firewall opens 17081 and
 17080 (the Router Manager's bootstrap) on the internal interface. The first successful run stops
 the bootstrap services and closes their ports, and writes `/var/lib/vpn-router/setup.done`, after
 which they never run again; a failed run leaves them running, so the reason stays readable until
 a later run succeeds. The unit is never enabled.
 The page, read again every 10 seconds, shows the state, the step and the failure of the last
 setup run (`/run/vpn-router/setup.json`, which setup writes at every step), the state of the
 router's services, the failed units, the addresses and routes, `cloud-init status`, and the end of
 the logs of `vpn-router-setup` and `vpn-router-web`:
 ```bash
 curl http://10.0.0.4:17081/
 ```
 | Page | Content |
 | --- | --- |
 | `/` | The status for people: plain text for `curl` and `wget`, a page styled with Tailwind CSS for a browser, in the browser's light or dark scheme, that reloads itself every minute (the style sheet is loaded from a CDN, so the browser needs internet access; the text is readable without it) |
 | `/status` | The same status as JSON |
 | `/auth` | Signing in with a user's password and TOTP code, refreshing and signing out |
 | `/config` | The configuration without its secret options, and what the router resolved and runs, as JSON; needs a token |
 | `/users` | The users of OpenVPN and road-warrior access; needs a token |
 | `/groups` | The groups of users; needs a token |
 The status holds the state of `vpn-router-setup` and `slapd`, of `strongswan` when site-to-site or
 road-warrior access is configured, of `wg-quick@wg0` when WireGuard is, and of
@@ -427,8 +514,60 @@ the WireGuard peers with the age of their last handshake and their traffic. `sta
 all these services are active, and `degraded` otherwise.
 ```bash
-curl http://10.0.0.4:8080/
+curl http://10.0.0.4:7081/
-curl http://10.0.0.4:8080/status
+curl http://10.0.0.4:7081/status
 ```
 ### Tokens and signing in
 `/config`, `/users` and `/groups` need a JWT in an `Authorization: Bearer` header, signed with
 HS256 by the key the web service makes on its first run, `/etc/vpn-router/web.key` (root only).
 The key never leaves the router; while it does not exist these paths and `/auth` are not served
 (404). A missing, expired or wrongly signed token, or one of another algorithm, gets 401 with
 `WWW-Authenticate: Bearer`. There are two kinds of tokens:
 - an access token of `/auth` names a user in `sub` and is valid for 5 minutes; the user must
  be a member of the group `admins`, otherwise the request gets 403;
 - a token of `vpn-router-token`, run as root on the router, has the claim `admin` and is valid
  for the lifetime given, without a maximum. It may call everything, and only such a token may
  change the group `admins`: create it, rename a group to it, change or remove it, add or take
  out its members, or remove a user who is a member of it.
 The first member of `admins` is added on the router:
 ```sh
 vpn-router-group add admins --member alice --description 'Router administrators'
 vpn-router-token --hours 1
 vpn-router-token --json --days 30
 ```
 `POST /auth` signs a user in. Request and answer bodies are JSON:
 | Request | Body | Answer |
 | --- | --- | --- |
 | `POST /auth` | `{"name", "password", "code"}`; `code` only for a user enrolled for TOTP | the tokens |
 | `POST /auth/refresh` | `{"refresh_token"}` | new tokens; the refresh token given is used up |
 | `POST /auth/logout` | `{"refresh_token"}` | `{"revoked": true}`, or `false` for a token not known |
 The tokens are `{"access_token", "token_type": "Bearer", "expires_in": 300, "refresh_token",
 "refresh_expires_in": 28800}`. A wrong name, password or code, and a locked or disabled user,
 get 401 with the same message; a refresh token is refused once its user is disabled, locked or
 removed. Changing a user's password, disabling, unlocking or removing the user revokes the
 user's refresh tokens. Five failed logins lock the account for 15 minutes (see the user
 directory). `/auth` and `/auth/refresh` take at most 10 calls a minute from one client address,
 and answer 429 with `Retry-After` beyond that; for a call from one of `web.trusted_proxies`, by
 default the address the service listens on, the client address is the last entry of
 `X-Forwarded-For`.
 Refresh tokens are kept, as their SHA-256, in PostgreSQL: by default in the database and role
 `vpn-router` on the router, which the web service creates on its first run with a password in
 `/etc/vpn-router/database.secret` (root only), reached over TCP on `localhost`; `web.database`
 names another database. Tokens and passwords travel over plain HTTP on the internal network.
 ```bash
 curl -H 'Content-Type: application/json' -d '{"name": "alice", "password": "s3cr3t-pass", "code": "123456"}' \
     http://10.0.0.4:7081/auth
 curl -H "Authorization: Bearer $TOKEN" http://10.0.0.4:7081/config
 ```
 ### The /config page
@@ -437,71 +576,80 @@ curl http://10.0.0.4:8080/status
 | Key | Content |
 | --- | --- |
-| `configuration` | `vpn-router.json` with the defaults applied, without its secret options (`psk`, `jwt_secret`, and any option named like a password, token or key) |
+| `configuration` | `vpn-router.json` with the defaults applied, without its secret options (`psk`, `database`, and any option named like a secret, password, token or key) |
 | `network` | The external and internal interface, their addresses, the internal gateway, the local and remote IKE identities and subnets, and the IPv4 routes as `ip -j -4 route` prints them |
 | `ipsec` | `connections` and `sas`: one object per connection and per IKE SA, read from `swanctl --list-conns --pretty` and `swanctl --list-sas --pretty`, with the connection's name under `name` and swanctl's keys and string values |
 | `wireguard` | The public key and port of `wg0`, and each peer's public key, endpoint, allowed IPs, handshake age and traffic; no private or preshared key |
 | `docker` | `docker.exposed_ports` and the rules of the `VPN-ROUTER-DOCKER` chain |
 The page needs a JWT in an `Authorization: Bearer` header, signed with HS256 by `web.jwt_secret`.
 The token must have an `exp` claim, not more than ten minutes from now (a minute of clock
 difference is allowed); `nbf` is checked when present. Other algorithms are refused, with 401 and
 `WWW-Authenticate: Bearer`. While `web.jwt_secret` is not set the page is not served (404). The
 token and the page travel over plain HTTP on the internal network, and the token can be used
 until it expires, so keep the lifetime short. A client makes a token from the secret with only
 HMAC-SHA256; with Python:
 ```python
 import base64, hashlib, hmac, json, time
 def b64(data):
    return base64.urlsafe_b64encode(data).rstrip(b'=').decode()
 secret = 'the web.jwt_secret value'
 head = b64(json.dumps({'alg': 'HS256', 'typ': 'JWT'}).encode())
 body = b64(json.dumps({'exp': int(time.time()) + 300}).encode())
 sign = b64(hmac.new(secret.encode(), f'{head}.{body}'.encode(), hashlib.sha256).digest())
 print(f'{head}.{body}.{sign}')
 ```
 ```bash
 curl -H "Authorization: Bearer $TOKEN" http://10.0.0.4:8080/config
 ```
 ### The /users API
-`/users` manages users with the same token as `/config`, by running `vpn-router-user`. Request
+`/users` manages users with the same token as `/config`, with the module `vpn-router-user` uses
 too, `/usr/lib/vpn-router/vpnrouter_users.py`, so both apply the same rules. Request
 and answer bodies are JSON; a request body is at most 4096 bytes.
 | Request | Body | Answer |
 | --- | --- | --- |
-| `GET /users` | | `{"users": [{"name", "enabled", "certificate"}, ...]}` |
+| `GET /users` | | `{"users": [{"name", "enabled", "locked", "certificate", "otp", "first_name", "last_name", "email"}, ...]}` |
-| `POST /users` | `{"name": ..., "password": ...}` | 201 and the user |
+| `POST /users` | `{"name", "password", "first_name", "last_name", "email"}`, all required | 201 and the user |
 | `PUT /users/<name>` | any of `{"first_name", "last_name", "email"}` | the user |
 | `PUT /users/<name>/password` | `{"password": ...}` | the user |
 | `POST /users/<name>/disable` | | the user |
 | `POST /users/<name>/enable` | | the user |
 | `POST /users/<name>/unlock` | | the user |
 | `DELETE /users/<name>` | | `{"name", "removed": true}` |
 | `GET /users/<name>/profile` | | `{"name", "profile"}`, the OpenVPN client profile |
 | `POST /users/<name>/otp` | | `{"name", "secret", "uri"}`, a new TOTP secret; nothing is stored |
 | `PUT /users/<name>/otp` | `{"secret", "code"}` | the user, once the code matches the secret |
 | `DELETE /users/<name>/otp` | | the user |
 A failure is `{"error": ...}` with 400 for an invalid request, 404 for a user that does not
-exist, 409 for a user that exists and 500 otherwise. While neither OpenVPN nor road-warrior
+exist, 409 for a user that exists and 500 otherwise. `/users` works whether or not OpenVPN or
-access is configured, `/users` answers 404. Passwords and private keys cross the internal
+road-warrior access is enabled; the profile needs OpenVPN. Passwords, private keys and TOTP
-network in plain HTTP, like the token.
+secrets cross the internal network in plain HTTP, like the token.
 ```bash
 curl -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
-     -d '{"name": "alice", "password": "s3cr3t-pass"}' http://10.0.0.4:8080/users
+     -d '{"name": "alice", "password": "s3cr3t-pass", "first_name": "Alice", "last_name": "Smith", "email": "alice@example.com"}' \
     http://10.0.0.4:7081/users
 ```
 ### The /groups API
 `/groups` manages groups with the same token and the same module as `/users`.
 | Request | Body | Answer |
 | --- | --- | --- |
 | `GET /groups` | | `{"groups": [{"name", "description", "members": [...]}, ...]}` |
 | `POST /groups` | `{"name", "members": [...]}`, and `"description"` | 201 and the group |
 | `PUT /groups/<name>` | any of `{"name", "description"}`; `""` removes the description | the group |
 | `DELETE /groups/<name>` | | `{"name", "removed": true}` |
 | `POST /groups/<name>/members` | `{"user": ...}` | the group |
 | `DELETE /groups/<name>/members/<user>` | | the group |
 A failure is `{"error": ...}` with 400 for an invalid request, among them taking out the last
 member, 404 for a group or user that does not exist, 409 for a group that exists and 500
 otherwise. `DELETE /users/<name>` is refused with 400 while the user is the last member of a
 group.
 The service starts on its own, without waiting for `vpn-router-setup`, and keeps running. Every
 30 seconds it works out the internal address from `vpn-router.json` as `general.mode` directs,
 and reads `web.port` and `web.enabled`; it listens there, moves when the address or the port
 changes, and stops listening while `web.enabled` is `false` or the address is not known. Each
-request reads the token secret from `vpn-router.json`, and the features and the facts for
+request reads the signing key and `vpn-router.json`, and the features and the facts for
 `/config` from `/run/vpn-router/web.json` (readable by root only), which `vpn-router-setup` writes;
 before that file exists the page shows no features. `vpn-router-setup` opens `web.port` on the
 internal interface in the firewall.
 ## Router Manager
 The `vpn-router-manager` package, built in the repository `linux-cloud-router-manager`, is a web
 application that shows the state of the router and manages its users through the status web
 service. It can be installed on the router or on another Linux system. While it is installed on
 the router, `vpn-router-setup` opens its port (`port` in `/etc/vpn-router-manager/manager.json`,
 7080 by default, HTTP; 7443 is kept for it with TLS) on the internal interface; restart
 `vpn-router-setup` after changing it.
 ## Removal
 `apt-get remove` stops `vpn-router-docker`, which removes the Docker rules. It terminates the
@@ -511,8 +659,9 @@ may be in use by something else.
 `apt-get purge` additionally removes the generated files and the firewall rules, restoring
 `/etc/ufw/before.rules` to its original content, and hands `vpn-router.json` back to `ucf`
-before deleting it so a later reinstall starts clean. Key material and users are never
+before deleting it so a later reinstall starts clean. It also removes the web service's
-deleted: `/etc/vpn-router/pki`, `/etc/vpn-router/ldap`, the user directory in slapd, the
+signing key, and the refresh tokens' database, its role and their password file when the package
 created them. Key material and users are never deleted: `/etc/vpn-router/pki`, `/etc/vpn-router/ldap`, the user directory in slapd, the
 WireGuard key pair and everything under `/etc/swanctl` are left alone.
 ## Building
@@ -1,3 +1,192 @@
 vpn-router (0.9.2-1) unstable; urgency=medium
  fix: the web service starts once libpq can be found.
  * The web service unit waits, with wait-for-libpq, until libpq is in the
    linker cache before it starts the service, for at most 5 minutes; the
    service imports psycopg at start again.
  * The package starts the web service without waiting for it.
 -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 06:00:00 +0200
 vpn-router (0.9.1-1) unstable; urgency=medium
  fix: the web service waits for the PostgreSQL driver instead of failing at start.
  * The web service imports psycopg as the first step of its
    initialization, retrying every 15 seconds and logging the failure while
    libpq cannot be found yet during an installation.
  * vpn-router-user and vpn-router-token load without psycopg; a command
    that needs the token database then reports that the driver cannot be
    loaded, as it reports every failure of that database.
 -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 05:00:00 +0200
 vpn-router (0.9.0-1) unstable; urgency=medium
  feature: bootstrap diagnostics on port 17081 during the first setup.
  * vpn-router-bootstrap serves, on port 17081, the setup state, the
    services, the failed units, the addresses, cloud-init status and the end
    of the logs of setup and the web service, as plain text.
  * vpn-router-setup starts it, and the Router Manager's bootstrap on port
    17080, until a run has succeeded, and opens both ports on the internal
    interface; the first successful run writes
    /var/lib/vpn-router/setup.done, closes the ports and stops them. A
    failed run leaves them running.
  * The web service no longer has a starting phase: it creates its key and
    database, then listens.
 -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 04:00:00 +0200
 vpn-router (0.8.0-1) unstable; urgency=medium
  feature: the web service creates its key and database itself and shows its starting phase.
  * vpn-router-web makes its signing key and creates the refresh tokens'
    database on its first run, retrying every 15 seconds until PostgreSQL
    answers; vpn-router-setup no longer creates them.
  * Until then / and /status show the web service's phase and its failure,
    with the state, the step and the failure of the last run of
    vpn-router-setup, and every page answers 503.
  * vpn-router-setup records its state, current step and failure in
    /run/vpn-router/setup.json.
 -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 03:00:00 +0200
 vpn-router (0.7.1-1) unstable; urgency=medium
  fix: a failing user directory or token database no longer leaves the tunnels down.
  * vpn-router-setup brings up strongSwan and WireGuard before it provisions
    the user directory and the token database. A failure of either no
    longer stops the other steps; setup reports it and fails at the end.
 -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 02:00:00 +0200
 vpn-router (0.7.0-1) unstable; urgency=medium
  feature: users sign in to the web service, and only members of admins manage the router.
  * POST /auth signs a user in with the password, and the TOTP code for an
    enrolled user, and answers a 5 minute access token and an 8 hour refresh
    token; /auth/refresh renews them and /auth/logout revokes the refresh
    token.
  * /config, /users and /groups take tokens of users in the group admins,
    and tokens of the new vpn-router-token, which has an admin claim and a
    chosen lifetime. Only these may change the group admins.
  * The tokens are signed with /etc/vpn-router/web.key, which
    vpn-router-setup makes on the router; web.jwt_secret is ignored.
  * Five failed logins lock an account for 15 minutes, for OpenVPN and
    /auth. vpn-router-user unlock and POST /users/<name>/unlock end the
    lockout, and the users show locked.
  * /auth takes at most 10 calls a minute per client address; for calls from
    web.trusted_proxies the address is read from X-Forwarded-For.
  * Users enrolled for TOTP log in to OpenVPN with the password followed by
    the code.
  * Refresh tokens are kept in PostgreSQL, in the database web.database
    names or in the vpn-router database vpn-router-setup creates. The
    package depends on postgresql and python3-psycopg.
 -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sun, 25 Oct 2026 01:00:00 +0200
 vpn-router (0.6.0-1) unstable; urgency=medium
  feature: users can be enrolled for TOTP one-time passwords.
  * vpn-router-user otp-new makes a TOTP secret and its otpauth URI,
    otp-enrol stores the secret, read from standard input, given a code of
    it, and otp-remove removes it. list and every answer show whether a user
    is enrolled, as otp.
  * The web service enrols with POST, PUT and DELETE /users/<name>/otp.
  * The secret is kept in the user's entry with the OATH-LDAP schema of
    slapd's otp module; only the user manager account may set it, and no
    one can read it. Logins do not ask for the codes.
  * The package depends on python3-pyotp.
 -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 24 Oct 2026 23:30:00 +0200
 vpn-router (0.5.9-1) unstable; urgency=medium
  fix: package upgrades no longer ask the configuration questions.
  * The configuration questions are asked only on a fresh installation and
    by dpkg-reconfigure.
 -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 24 Oct 2026 21:00:00 +0200
 vpn-router (0.5.8-1) unstable; urgency=medium
  fix: an upgrade keeps the configuration and asks nothing.
  * postinst passes on only the debconf answers that were given. The
    defaults of questions never asked, such as platform none, no longer
    replace settings of vpn-router.json.
  * Without an answer that changes it, the configuration ucf is given is
    vpn-router.json byte for byte, so an upgrade has no conflict to ask
    about; settings the file leaves out keep their defaults when it is read.
 -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 24 Oct 2026 20:00:00 +0200
 vpn-router (0.5.7-1) unstable; urgency=medium
  feature: users have a first name, a last name and an e-mail address.
  * vpn-router-user add needs --first-name, --last-name and --email, kept as
    givenName, sn and mail, with cn set to the first and the last name.
    vpn-router-user update changes them, and list and every answer show
    them as first_name, last_name and email.
  * POST /users needs first_name, last_name and email, and PUT
    /users/<name> changes them.
  * vpn-router-user answers in text: a table of the users, the client
    profile or a short message, with failures on standard error. --json
    gives the JSON answers. -h and --help show the commands and their
    options.
  * The web service manages users with vpnrouter_users, the module
    vpn-router-user uses, instead of running vpn-router-user for each
    request.
  * Groups of users, kept as groupOfNames entries in ou=groups with a
    description and at least one member. vpn-router-group lists, adds,
    renames, describes and removes them and adds and takes out members, and
    the web service serves the same under /groups. Removing a user takes it
    out of its groups and is refused while it is the last member of one.
 -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 24 Oct 2026 19:00:00 +0200
 vpn-router (0.5.6-1) unstable; urgency=medium
  update: the status web service listens on port 7081 by default.
  * web.port defaults to 7081.
  * vpn-router-setup opens the Router Manager's port, 7080 by default, on
    the internal interface.
 -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 24 Oct 2026 18:00:00 +0200
 vpn-router (0.5.5-1) unstable; urgency=medium
  feature: users and their certificates no longer need a P2S system enabled.
  * vpn-router-setup provisions the user directory, the CA and its CRL once
    a platform is chosen, whether or not OpenVPN or road-warrior access is
    enabled. The server certificate is issued once wan.local_fqdn is set,
    also for an existing CA that has none.
  * /users and vpn-router-user manage users and issue and revoke their
    certificates without a P2S system; only the OpenVPN client profile
    needs OpenVPN.
 -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 24 Oct 2026 16:00:00 +0200
 vpn-router (0.5.4-1) unstable; urgency=medium
  feature: the firewall lets the Router Manager in on the internal interface.
  * While vpn-router-manager is installed, vpn-router-setup opens its port,
    port in /etc/vpn-router-manager/manager.json or 8090, on the internal
    interface. Purging the package removes the rule.
 -- Slawomir Koszewski <slawek@koszewscy.waw.pl>  Sat, 24 Oct 2026 13:00:00 +0200
 vpn-router (0.5.3-1) unstable; urgency=medium
  fix: the status always shows the state of slapd.
@@ -2,6 +2,12 @@
 set -e
 . /usr/share/debconf/confmodule
 # An upgrade (configure with the old version) asks nothing; vpn-router.json holds the
 # configuration, and dpkg-reconfigure asks the questions again.
 if [ "$1" = "configure" ] && [ -n "$2" ]; then
    exit 0
 fi
 # Every question has a default, so a non-interactive install with nothing
 # preseeded completes without prompting. The defaults are platform "none" and
 # mode "manual", which together mean: install the files, configure nothing.
@@ -21,6 +21,9 @@ Depends: ${misc:Depends},
         python3,
         python3-jinja2,
         python3-ldap,
         python3-pyotp,
         python3-psycopg,
         postgresql,
         slapd,
         openvpn,
         openvpn-auth-ldap,
@@ -2,13 +2,20 @@ src/etc/sysctl.d/99-vpn-router.conf                   etc/sysctl.d/
 src/usr/lib/vpn-router/vpnrouter.py                   usr/lib/vpn-router/
 src/usr/lib/vpn-router/vpnrouter_ldap.py              usr/lib/vpn-router/
 src/usr/lib/vpn-router/vpnrouter_net.py               usr/lib/vpn-router/
 src/usr/lib/vpn-router/vpnrouter_serve.py             usr/lib/vpn-router/
 src/usr/lib/vpn-router/vpnrouter_users.py             usr/lib/vpn-router/
 src/usr/lib/vpn-router/vpnrouter_auth.py              usr/lib/vpn-router/
 src/usr/lib/vpn-router/openvpn-same-user              usr/lib/vpn-router/
 src/usr/sbin/vpn-router-user                          usr/sbin/
 src/usr/sbin/vpn-router-group                         usr/sbin/
 src/usr/sbin/vpn-router-token                         usr/sbin/
 src/usr/lib/vpn-router/simple-ca                      usr/lib/vpn-router/
 src/usr/lib/vpn-router/configure                      usr/lib/vpn-router/
 src/usr/lib/vpn-router/docker-rules                   usr/lib/vpn-router/
 src/usr/lib/vpn-router/setup                          usr/lib/vpn-router/
 src/usr/lib/vpn-router/web                            usr/lib/vpn-router/
 src/usr/lib/vpn-router/bootstrap                      usr/lib/vpn-router/
 src/usr/lib/vpn-router/wait-for-libpq                 usr/lib/vpn-router/
 src/usr/lib/vpn-router/generate-config                usr/lib/vpn-router/
 src/usr/lib/vpn-router/vpnrouter_platforms/*.py       usr/lib/vpn-router/vpnrouter_platforms/
 src/usr/share/vpn-router/templates/*                  usr/share/vpn-router/templates/
@@ -2,28 +2,39 @@
 set -e
 . /usr/share/debconf/confmodule
 # The answer to a question in ANSWER, or "" when the question has not been seen; the
 # default debconf holds for an unseen question is not an answer.
 answer() {
    ANSWER=""
    db_fget "$1" seen
    if [ "$RET" = "true" ]; then
        db_get "$1"
        ANSWER="$RET"
    fi
 }
 case "$1" in
    configure)
        # --- Read the installer's answers ---
-        db_get vpn-router/platform;         VPN_ROUTER_PLATFORM="$RET"
+        answer vpn-router/platform;         VPN_ROUTER_PLATFORM="$ANSWER"
-        db_get vpn-router/mode;             VPN_ROUTER_MODE="$RET"
+        answer vpn-router/mode;             VPN_ROUTER_MODE="$ANSWER"
-        db_get vpn-router/int_addr;         VPN_ROUTER_INT_ADDR="$RET"
+        answer vpn-router/int_addr;         VPN_ROUTER_INT_ADDR="$ANSWER"
-        db_get vpn-router/external_interface; VPN_ROUTER_EXTERNAL_INTERFACE="$RET"
+        answer vpn-router/external_interface; VPN_ROUTER_EXTERNAL_INTERFACE="$ANSWER"
-        db_get vpn-router/internal_interface; VPN_ROUTER_INTERNAL_INTERFACE="$RET"
+        answer vpn-router/internal_interface; VPN_ROUTER_INTERNAL_INTERFACE="$ANSWER"
-        db_get vpn-router/local_fqdn;       VPN_ROUTER_LOCAL_FQDN="$RET"
+        answer vpn-router/local_fqdn;       VPN_ROUTER_LOCAL_FQDN="$ANSWER"
-        db_get vpn-router/local_id_mode;    VPN_ROUTER_LOCAL_ID_MODE="$RET"
+        answer vpn-router/local_id_mode;    VPN_ROUTER_LOCAL_ID_MODE="$ANSWER"
-        db_get vpn-router/local_cidrs;      VPN_ROUTER_LOCAL_CIDRS="$RET"
+        answer vpn-router/local_cidrs;      VPN_ROUTER_LOCAL_CIDRS="$ANSWER"
-        db_get vpn-router/int_gateway_ip;   VPN_ROUTER_INT_GATEWAY_IP="$RET"
+        answer vpn-router/int_gateway_ip;   VPN_ROUTER_INT_GATEWAY_IP="$ANSWER"
-        db_get vpn-router/remote_addrs;     VPN_ROUTER_REMOTE_ADDRS="$RET"
+        answer vpn-router/remote_addrs;     VPN_ROUTER_REMOTE_ADDRS="$ANSWER"
-        db_get vpn-router/remote_id;        VPN_ROUTER_REMOTE_ID="$RET"
+        answer vpn-router/remote_id;        VPN_ROUTER_REMOTE_ID="$ANSWER"
-        db_get vpn-router/remote_cidrs;     VPN_ROUTER_REMOTE_CIDRS="$RET"
+        answer vpn-router/remote_cidrs;     VPN_ROUTER_REMOTE_CIDRS="$ANSWER"
-        db_get vpn-router/psk;              VPN_ROUTER_PSK="$RET"
+        answer vpn-router/psk;              VPN_ROUTER_PSK="$ANSWER"
-        db_get vpn-router/p2s_enabled;      VPN_ROUTER_P2S_ENABLED="$RET"
+        answer vpn-router/p2s_enabled;      VPN_ROUTER_P2S_ENABLED="$ANSWER"
-        db_get vpn-router/p2s_address_pool; VPN_ROUTER_P2S_ADDRESS_POOL="$RET"
+        answer vpn-router/p2s_address_pool; VPN_ROUTER_P2S_ADDRESS_POOL="$ANSWER"
-        db_get vpn-router/p2s_ca_name;      VPN_ROUTER_P2S_CA_NAME="$RET"
+        answer vpn-router/p2s_ca_name;      VPN_ROUTER_P2S_CA_NAME="$ANSWER"
-        db_get vpn-router/wg_enabled;       VPN_ROUTER_WG_ENABLED="$RET"
+        answer vpn-router/wg_enabled;       VPN_ROUTER_WG_ENABLED="$ANSWER"
-        db_get vpn-router/wg_address;       VPN_ROUTER_WG_ADDRESS="$RET"
+        answer vpn-router/wg_address;       VPN_ROUTER_WG_ADDRESS="$ANSWER"
-        db_get vpn-router/wg_listen_port;   VPN_ROUTER_WG_LISTEN_PORT="$RET"
+        answer vpn-router/wg_listen_port;   VPN_ROUTER_WG_LISTEN_PORT="$ANSWER"
        export VPN_ROUTER_PLATFORM VPN_ROUTER_MODE VPN_ROUTER_INT_ADDR \
               VPN_ROUTER_EXTERNAL_INTERFACE \
@@ -59,6 +70,12 @@ case "$1" in
        # The firewall is not touched here. Enabling it, allowing SSH and
        # setting the forward policy all belong to vpn-router-setup, which does
        # them only once the configuration says this host is a router.
        # The web service waits for libpq in the linker cache, which dpkg
        # refreshes after this script, so it is started without waiting for it.
        if [ -d /run/systemd/system ]; then
            systemctl --no-block restart vpn-router-web.service >/dev/null 2>&1 || true
        fi
        ;;
 esac
@@ -11,10 +11,10 @@ strip_ufw_blocks() {
    tmp="$(mktemp)"
    awk '
-        /^# (IPSEC|WIREGUARD|OPENVPN|P2S DNS|WEB|ROUTER FORWARD|ROUTER NAT|ROUTER MSS) RULES START$/ {
+        /^# (IPSEC|WIREGUARD|OPENVPN|P2S DNS|WEB|BOOTSTRAP|MANAGER|ROUTER FORWARD|ROUTER NAT|ROUTER MSS) RULES START$/ {
            skip = 1; pending = 0; next
        }
-        /^# (IPSEC|WIREGUARD|OPENVPN|P2S DNS|WEB|ROUTER FORWARD|ROUTER NAT|ROUTER MSS) RULES END$/ {
+        /^# (IPSEC|WIREGUARD|OPENVPN|P2S DNS|WEB|BOOTSTRAP|MANAGER|ROUTER FORWARD|ROUTER NAT|ROUTER MSS) RULES END$/ {
            skip = 0; next
        }
        skip { next }
@@ -49,7 +49,17 @@ case "$1" in
              /etc/wireguard/wg0.conf \
              /etc/openvpn/server/vpn-router.conf \
              /etc/openvpn/server/vpn-router-ldap.conf \
-              /etc/vpn-router/vpn-router.json
+              /etc/vpn-router/vpn-router.json \
              /etc/vpn-router/web.key \
              /var/lib/vpn-router/setup.done
        rmdir --ignore-fail-on-non-empty /var/lib/vpn-router 2>/dev/null || true
        # The refresh tokens' database and role, when the package created them.
        if [ -f /etc/vpn-router/database.secret ] && command -v psql >/dev/null 2>&1; then
            runuser -u postgres -- psql -X -q -c 'DROP DATABASE IF EXISTS "vpn-router"' || true
            runuser -u postgres -- psql -X -q -c 'DROP ROLE IF EXISTS "vpn-router"' || true
        fi
        rm -f /etc/vpn-router/database.secret
        # Key material and users are not package state: /etc/vpn-router/pki,
        # /etc/vpn-router/ldap, the user directory in slapd, the WireGuard key
@@ -5,4 +5,5 @@
 override_dh_installsystemd:
 	dh_installsystemd --name=vpn-router-setup
 	dh_installsystemd --name=vpn-router-docker
-	dh_installsystemd --name=vpn-router-web
+	dh_installsystemd --name=vpn-router-web --no-start
 	dh_installsystemd --name=vpn-router-bootstrap --no-enable --no-start
@@ -0,0 +1,14 @@
 [Unit]
 Description=VPN Router bootstrap diagnostics page
 Documentation=file:/usr/share/doc/vpn-router/README.md
 # Never enabled: vpn-router-setup starts it until a setup has succeeded, and stops it after that.
 [Service]
 Type=simple
 ExecStart=/usr/bin/python3 /usr/lib/vpn-router/bootstrap
 Restart=on-failure
 RestartSec=5
 NoNewPrivileges=yes
 ProtectSystem=strict
 ProtectHome=yes
 PrivateTmp=yes
@@ -1,8 +1,8 @@
 [Unit]
 Description=VPN Router Setup
 Documentation=file:/usr/share/doc/vpn-router/README.md
-Wants=network-online.target
+Wants=network-online.target postgresql.service
-After=network-online.target docker.service
+After=network-online.target docker.service postgresql.service
 [Service]
 Type=oneshot
@@ -1,17 +1,19 @@
 [Unit]
 Description=VPN Router status web service
 Documentation=file:/usr/share/doc/vpn-router/README.md
-After=network-online.target
+After=network-online.target postgresql.service
 Wants=network-online.target
 [Service]
 Type=simple
 ExecStartPre=/usr/lib/vpn-router/wait-for-libpq
 ExecStart=/usr/bin/python3 /usr/lib/vpn-router/web
 TimeoutStartSec=5min
 Restart=on-failure
 RestartSec=15
 NoNewPrivileges=yes
 ProtectSystem=strict
-ReadWritePaths=-/run/vpn-router -/etc/vpn-router/pki -/etc/swanctl/x509crl
+ReadWritePaths=-/run/vpn-router -/etc/vpn-router -/etc/swanctl/x509crl
 ProtectHome=yes
 PrivateTmp=yes
@@ -0,0 +1,55 @@
 #!/usr/bin/python3
 """Serve diagnostics of the router's first setup on PORT, with the Python standard library alone.
 Run by vpn-router-bootstrap.service, which vpn-router-setup starts until a setup has succeeded and
 then stops. Every request on any address answers, as plain text, the output of COMMANDS run at that
 moment: the state of the last setup run, the state of the services, the addresses, and the end of
 the logs.
 """
 import subprocess
 from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
 PORT = 17081
 #: The title of each section and its command.
 COMMANDS = (
    ('Setup state', ['cat', '/run/vpn-router/setup.json']),
    ('Services', ['systemctl', 'is-active', 'vpn-router-setup', 'vpn-router-web', 'slapd', 'postgresql',
                  'strongswan', 'openvpn-server@vpn-router', 'wg-quick@wg0']),
    ('Failed units', ['systemctl', '--failed', '--no-legend', '--plain']),
    ('Addresses', ['ip', '-brief', 'address']),
    ('Routes', ['ip', 'route']),
    ('cloud-init', ['cloud-init', 'status', '--long']),
    ('vpn-router-setup log', ['journalctl', '--unit', 'vpn-router-setup', '--lines', '60', '--no-pager', '--output', 'short-iso']),
    ('vpn-router-web log', ['journalctl', '--unit', 'vpn-router-web', '--lines', '30', '--no-pager', '--output', 'short-iso']),
 )
 def report():
    """The output of every command of COMMANDS under its title."""
    sections = []
    for title, command in COMMANDS:
        try:
            result = subprocess.run(command, capture_output=True, text=True, timeout=10)
            text = (result.stdout + result.stderr).rstrip()
        except (OSError, subprocess.TimeoutExpired) as exc:
            text = f'cannot run {command[0]}: {exc}'
        sections.append(f'== {title} ==\n{text}\n')
    return '\n'.join(sections)
 class Handler(BaseHTTPRequestHandler):
    def do_GET(self):
        body = report().encode()
        self.send_response(200)
        self.send_header('Content-Type', 'text/plain; charset=utf-8')
        self.send_header('Content-Length', str(len(body)))
        self.send_header('Cache-Control', 'no-store')
        self.send_header('Refresh', '10')
        self.end_headers()
        self.wfile.write(body)
 if __name__ == '__main__':
    ThreadingHTTPServer(('', PORT), Handler).serve_forever()
@@ -6,18 +6,15 @@ Called from postinst, which hands the result to ucf(1). ucf compares it with
 this script never writes to /etc and never has to worry about destroying an
 administrator's edits.
-The candidate is built in three layers, each overriding the one before:
+The candidate is the current /etc/vpn-router/vpn-router.json as it is, or the
-
+shipped defaults when there is none, with every debconf answer that was given
-  1. the shipped defaults
+written over it. Without an answer that changes it, the candidate is the current
-  2. the current contents of /etc/vpn-router/vpn-router.json, if it exists
+file byte for byte, so
-  3. any debconf answer that was actually given
+an upgrade or a reconfiguration that changes nothing gives ucf nothing to ask;
-
+defaults the file leaves out are applied by vpnrouter.load_config() when the
-Layer 2 is what makes reconfiguration safe: a setting the administrator edited
+file is read. An empty answer means "leave alone", which is why clearing the
-by hand, and that debconf has nothing to say about, is carried through
+pre-shared key from the debconf database after installation does not blank it
-unchanged, so ucf only sees a diff where something really changed. An empty
+on the next dpkg-reconfigure.
 answer means "leave alone", which is why clearing the pre-shared key from the
 debconf database after installation does not blank it on the next
 dpkg-reconfigure.
 Answers arrive as VPN_ROUTER_* environment variables and are converted to the
 type of the setting they fill: comma-separated lists become arrays, "true" and
@@ -62,8 +59,21 @@ def main():
        return 2
    output = sys.argv[1]
-    cfg = vpnrouter.load_config()
+    try:
        current = vpnrouter.CONFIG_FILE.read_text(encoding='utf-8')
    except FileNotFoundError:
        current = None
    if current is None:
        cfg = vpnrouter.load_config()
    else:
        try:
            cfg = json.loads(current)
        except ValueError as exc:
            vpnrouter.fail(f'cannot parse {vpnrouter.CONFIG_FILE}: {exc}')
        if not isinstance(cfg, dict):
            vpnrouter.fail(f'cannot parse {vpnrouter.CONFIG_FILE}: the top level is not an object')
    answered = False
    for variable, (section, option) in ANSWERS.items():
        value = os.environ.get(variable)
        if not value:
@@ -76,11 +86,19 @@ def main():
            value = int(value) if value.isdigit() else value
        elif isinstance(default, list):
            value = [item.strip() for item in value.split(',') if item.strip()]
-        cfg[section][option] = value
+        if not isinstance(cfg.get(section, {}), dict):
            vpnrouter.fail(f'cannot parse {vpnrouter.CONFIG_FILE}: {section} is not an object')
        if cfg.setdefault(section, {}).get(option) != value:
            cfg[section][option] = value
            answered = True
    # Without a change the candidate is the current file byte for byte, as ucf compares checksums.
    with open(output, 'w', encoding='utf-8') as handle:
-        json.dump(cfg, handle, indent=2)
+        if current is not None and not answered:
-        handle.write('\n')
+            handle.write(current)
        else:
            json.dump(cfg, handle, indent=2)
            handle.write('\n')
    os.chmod(output, 0o600)
    return 0
@@ -5,6 +5,12 @@ Regenerates the configuration files first, then does everything that needs
 the live system: routes, DNS-derived identity, WireGuard keys, firewall rules
 and service reloads. Idempotent - a second run changes nothing.
 Until a run has succeeded (SETUP_DONE), every run starts the bootstrap
 services, BOOTSTRAP_UNITS, which show the progress and the failure recorded in
 SETUP_STATE on BOOTSTRAP_PORTS, and the firewall opens those ports on the
 internal interface. The first successful run writes SETUP_DONE, closes the
 ports and stops the bootstrap services; a failed run leaves them running.
 Interface names come from the configuration and are never guessed. The
 firewall and NAT rules are written against them, so choosing the wrong one
 would be a silent and security-relevant mistake.
@@ -17,11 +23,10 @@ import re
 import socket
 import subprocess
 import sys
 import time
 sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
 import ldap
 import vpnrouter
 import vpnrouter_ldap
 from vpnrouter import fail
@@ -31,10 +36,25 @@ UFW_DEFAULTS = pathlib.Path('/etc/default/ufw')
 WG_DIR       = pathlib.Path('/etc/wireguard')
 WEB_STATE    = pathlib.Path('/run/vpn-router/web.json')
 #: The state, the current step and the failure of the last run, which the bootstrap services show.
 SETUP_STATE  = pathlib.Path('/run/vpn-router/setup.json')
 #: Written once a run has succeeded; until then setup runs the bootstrap services and opens their ports.
 SETUP_DONE   = pathlib.Path('/var/lib/vpn-router/setup.done')
 #: The units of the bootstrap services, and their ports: the Router Manager's and the router's.
 BOOTSTRAP_UNITS = ('vpn-router-bootstrap.service', 'vpn-router-manager-bootstrap.service')
 BOOTSTRAP_PORTS = (17080, 17081)
 #: The Router Manager, vpn-router-manager, when it is installed, and its settings.
 MANAGER        = pathlib.Path('/usr/lib/vpn-router-manager/manager')
 MANAGER_CONFIG = pathlib.Path('/etc/vpn-router-manager/manager.json')
 MANAGER_PORT   = 7080
 FILTER_ANCHOR = '# End required lines'
 #: Configuration options left out wherever the configuration is shown.
-SECRET_OPTION = re.compile(r'(^psk$)|secret|password|token|_key$')
+SECRET_OPTION = re.compile(r'(^psk$)|secret|password|token|_key$|^database$')
 BLOCKS = (
    ('# IPSEC RULES START',         '# IPSEC RULES END'),
@@ -42,6 +62,8 @@ BLOCKS = (
    ('# OPENVPN RULES START',       '# OPENVPN RULES END'),
    ('# P2S DNS RULES START',       '# P2S DNS RULES END'),
    ('# WEB RULES START',           '# WEB RULES END'),
    ('# BOOTSTRAP RULES START',     '# BOOTSTRAP RULES END'),
    ('# MANAGER RULES START',       '# MANAGER RULES END'),
    ('# ROUTER FORWARD RULES START', '# ROUTER FORWARD RULES END'),
    ('# ROUTER NAT RULES START',    '# ROUTER NAT RULES END'),
    ('# ROUTER MSS RULES START',    '# ROUTER MSS RULES END'),
@@ -129,6 +151,15 @@ def is_configured(ctx):
                vpnrouter.openvpn_ready(ctx)))
 def manager_port():
    """The port of the Router Manager: port in its settings, or MANAGER_PORT when it is not set or not a port."""
    try:
        port = json.loads(MANAGER_CONFIG.read_text(encoding='utf-8')).get('port')
    except (OSError, ValueError, AttributeError):
        return MANAGER_PORT
    return port if isinstance(port, int) and not isinstance(port, bool) and 1 <= port <= 65535 else MANAGER_PORT
 def filter_block(ctx):
    lines = [
        '',
@@ -185,6 +216,23 @@ def filter_block(ctx):
            '# WEB RULES END',
        ]
    if not SETUP_DONE.exists() and ctx['int_iface']:
        lines += [
            '',
            '# BOOTSTRAP RULES START',
            *(f'-A ufw-before-input -i {ctx["int_iface"]} -p tcp --dport {port} -j ACCEPT' for port in BOOTSTRAP_PORTS),
            '# BOOTSTRAP RULES END',
        ]
    if MANAGER.exists() and ctx['int_iface']:
        port = manager_port()
        lines += [
            '',
            '# MANAGER RULES START',
            f'-A ufw-before-input -i {ctx["int_iface"]} -p tcp --dport {port} -j ACCEPT',
            '# MANAGER RULES END',
        ]
    if vpnrouter.routing_ready(ctx):
        lines += ['', '# ROUTER FORWARD RULES START']
        for subnet in ctx['local_cidr_list']:
@@ -323,19 +371,17 @@ def manage_wireguard(ctx, changed):
        run(['systemctl', 'restart', unit], check=False)
-def manage_directory(ctx):
+def manage_directory():
-    """Provision the user directory in slapd while a P2S system that has users is configured.
+    """Provision the user directory in slapd, whether or not a P2S system is configured; the problem, or ''."""
    The directory is left in place when they are switched off, so the users are kept.
    """
    if not (vpnrouter.p2s_ready(ctx) or vpnrouter.openvpn_ready(ctx)):
        return
    if vpnrouter_ldap.ensure_services() or not unit_is_active('slapd'):
-        run(['systemctl', 'restart', 'slapd'])
+        result = run(['systemctl', 'restart', 'slapd'], check=False)
        if result.returncode != 0:
            return f'slapd cannot be restarted: {result.stderr.strip()}'
    try:
        vpnrouter_ldap.provision()
-    except ldap.LDAPError as exc:
+    except Exception as exc:
-        fail(f'the user directory cannot be provisioned: {exc}')
+        return f'the user directory cannot be provisioned: {exc!r}'
    return ''
 def manage_openvpn(ctx, changed):
@@ -398,7 +444,6 @@ def refresh_web(cfg, ctx):
        'p2s': bool(vpnrouter.p2s_ready(ctx)),
        'wireguard': bool(vpnrouter.wg_ready(ctx)),
        'openvpn': bool(vpnrouter.openvpn_ready(ctx)),
        'users': bool(vpnrouter.p2s_ready(ctx) or vpnrouter.openvpn_ready(ctx)),
        'configuration': public,
        'network': {
            'external_interface': ctx['wan_iface'],
@@ -414,25 +459,62 @@ def refresh_web(cfg, ctx):
    }) + '\n', encoding='utf-8')
 def progress(state, step, error=None):
    """Record where setup is in SETUP_STATE: running, deferred, done or failed, the step, and the error of a failure."""
    SETUP_STATE.parent.mkdir(parents=True, exist_ok=True)
    SETUP_STATE.write_text(json.dumps({
        'state': state,
        'step': step,
        'error': error,
        'time': time.strftime('%Y-%m-%dT%H:%M:%S%z'),
    }) + '\n', encoding='utf-8')
 def start_bootstrap():
    """Start the bootstrap services while no run has succeeded; a unit that is not installed is skipped."""
    if not SETUP_DONE.exists():
        for unit in BOOTSTRAP_UNITS:
            run(['systemctl', 'start', '--no-block', unit], check=False)
 def end_bootstrap(ctx):
    """After the first successful run: write SETUP_DONE, close the bootstrap ports and stop the bootstrap services."""
    if SETUP_DONE.exists():
        return
    SETUP_DONE.parent.mkdir(parents=True, exist_ok=True)
    SETUP_DONE.touch()
    if setup_ufw(ctx):
        run(['ufw', 'reload'], check=False)
    for unit in BOOTSTRAP_UNITS:
        run(['systemctl', 'stop', unit], check=False)
 def main():
    start_bootstrap()
    progress('running', 'configuration')
    cfg = vpnrouter.load_config()
    ctx, changed = vpnrouter.configure(cfg)
    if ctx is None:
        print('vpn-router: platform is none, configuration deferred; '
              f'set general.platform in {vpnrouter.CONFIG_FILE}')
        progress('deferred', 'configuration', f'general.platform is none; set it in {vpnrouter.CONFIG_FILE}')
        return 0
    progress('running', 'routes')
    if vpnrouter.routing_ready(ctx):
        apply_routes(ctx)
    if ctx['local_id_mode'] == 'public_ip' and ctx['local_fqdn']:
        progress('running', 'public address')
        address = resolve_public_ip(ctx['local_fqdn'])
        ctx, more = vpnrouter.configure(cfg, local_id_override=address)
        changed |= more
    progress('running', 'WireGuard keys')
    setup_wireguard(ctx)
    progress('running', 'firewall')
    rules_changed = setup_ufw(ctx)
    if is_configured(ctx):
        status = run(['ufw', 'status'], check=False)
@@ -445,19 +527,53 @@ def main():
    elif rules_changed:
        run(['ufw', 'reload'], check=False)
-    manage_directory(ctx)
+    progress('running', 'strongSwan')
    manage_strongswan(ctx, changed)
    progress('running', 'WireGuard')
    manage_wireguard(ctx, changed)
    # The tunnels are up before the directory, whose problem ends setup only after every other step.
    progress('running', 'user directory')
    problem = manage_directory()
    progress('running', 'OpenVPN')
    manage_openvpn(ctx, changed)
    if 'resolved' in changed:
        progress('running', 'resolver')
        run(['systemctl', 'restart', 'systemd-resolved'], check=False)
    progress('running', 'platform module')
    run_platform_setup(cfg, ctx)
    progress('running', 'Docker rules')
    refresh_docker_rules()
    progress('running', 'web service state')
    refresh_web(cfg, ctx)
    if problem:
        progress('failed', 'user directory', problem)
        print(f'vpn-router: error: {problem}', file=sys.stderr)
        return 1
    progress('done', 'done')
    end_bootstrap(ctx)
    return 0
 def record_failure(error):
    """Mark the run failed in the step SETUP_STATE holds."""
    try:
        step = json.loads(SETUP_STATE.read_text(encoding='utf-8')).get('step')
    except (OSError, ValueError):
        step = None
    progress('failed', step, error)
 if __name__ == '__main__':
-    sys.exit(main())
+    # A failing step ends with SystemExit carrying its message (vpnrouter.fail).
    try:
        code = main()
    except SystemExit as exc:
        record_failure(str(exc.code))
        raise
    except Exception as exc:
        record_failure(repr(exc))
        raise
    sys.exit(code)
@@ -54,12 +54,9 @@ SCHEMA = {
    'wireguard':  {'enabled': False, 'address': '', 'listen_port': 51820},
    'openvpn':    {'enabled': False, 'address_pool': '', 'port': 1194, 'protocol': 'udp'},
    'docker':     {'mode': 'auto', 'exposed_ports': []},
-    'web':        {'enabled': True, 'port': 8080, 'jwt_secret': ''},
+    'web':        {'enabled': True, 'port': 7081, 'database': '', 'trusted_proxies': []},
 }
 #: The shortest secret that signs the tokens of the status web service.
 WEB_SECRET_MIN = 32
 DOCKER_MODES = ('auto', 'on', 'off')
 OPENVPN_PROTOCOLS = ('udp', 'tcp')
@@ -98,8 +95,8 @@ def warn(message):
 def fail(message):
-    print(f'vpn-router: error: {message}', file=sys.stderr)
+    """End with status 1; SystemExit carries the message, which Python prints to standard error."""
-    sys.exit(1)
+    sys.exit(f'vpn-router: error: {message}')
 def load_config(path=None):
@@ -262,9 +259,8 @@ def validate(cfg):
    if isinstance(web_port, bool) or not isinstance(web_port, int) or not 1 <= web_port <= 65535:
        raise ConfigError(f'web.port: {web_port!r} is not a valid port')
-    web_secret = _get(cfg, 'web', 'jwt_secret')
+    for item in _get_list(cfg, 'web', 'trusted_proxies'):
-    if web_secret and len(web_secret) < WEB_SECRET_MIN:
+        _check_address(item, 'web.trusted_proxies')
        raise ConfigError(f'web.jwt_secret: use at least {WEB_SECRET_MIN} characters')
    docker_mode = _get(cfg, 'docker', 'mode')
    if docker_mode not in DOCKER_MODES:
@@ -565,24 +561,28 @@ def _simple_ca(*args):
 def provision_pki(ctx):
-    """Create the CA, server certificate and CRL when the PKI directory is empty; a CA without a CRL gets one."""
+    """Create the CA and CRL when the PKI directory is empty, and the server certificate once wan.local_fqdn is set.
    A CA of the router's own (ca_key.pem present) without a CRL gets one, and without a
    certificate for wan.local_fqdn gets one too; nothing present is overwritten.
    """
    PKI_DIR.mkdir(parents=True, exist_ok=True)
    os.chmod(PKI_DIR, 0o700)
    if any(PKI_DIR.iterdir()):
        if (PKI_DIR / 'ca_key.pem').is_file() and not (PKI_DIR / 'crl.pem').is_file():
            _simple_ca('make-crl', '--ca-dir', str(PKI_DIR), '--days', str(CRL_DAYS))
        return
    if not ctx['p2s_ca_name']:
        warn('p2s.ca_name is empty, not creating a local CA')
        return
    if not SIMPLE_CA.is_file():
        fail(f'{SIMPLE_CA} is missing')
-    _simple_ca('make-ca', '--ca-dir', str(PKI_DIR), ctx['p2s_ca_name'])
+    if not any(PKI_DIR.iterdir()):
-    _simple_ca('make-cert', '--ca-dir', str(PKI_DIR),
+        if not ctx['p2s_ca_name']:
-               ctx['local_fqdn'], ctx['local_fqdn'])
+            warn('p2s.ca_name is empty, not creating a local CA')
-    _simple_ca('make-crl', '--ca-dir', str(PKI_DIR), '--days', str(CRL_DAYS))
+            return
        _simple_ca('make-ca', '--ca-dir', str(PKI_DIR), ctx['p2s_ca_name'])
    if not (PKI_DIR / 'ca_key.pem').is_file():
        return
    if not (PKI_DIR / 'crl.pem').is_file():
        _simple_ca('make-crl', '--ca-dir', str(PKI_DIR), '--days', str(CRL_DAYS))
    label = ctx['local_fqdn'].split('.', 1)[0] if ctx['local_fqdn'] else ''
    if label and not (PKI_DIR / f'{label}_cert.pem').is_file():
        _simple_ca('make-cert', '--ca-dir', str(PKI_DIR), ctx['local_fqdn'], ctx['local_fqdn'])
 def distribute_pki(ctx):
@@ -628,8 +628,7 @@ def configure(cfg=None, local_id_override=None):
    changed = set()
-    if p2s_ready(ctx) or openvpn_ready(ctx):
+    provision_pki(ctx)
        provision_pki(ctx)
    if distribute_pki(ctx):
        changed.add('swanctl')
@@ -0,0 +1,204 @@
 """The tokens of the router's web service, for the web service and vpn-router-token.
 Tokens are HS256 JWTs signed with WEB_KEY, a key the web service makes on its first run on the
 router and that never leaves it. /auth issues an access token naming the user in sub, valid ACCESS_SECONDS, and
 a refresh token valid REFRESH_SECONDS; vpn-router-token issues tokens with the admin claim and
 any lifetime. A token without the admin claim must expire within MAX_LIFETIME seconds.
 Every failure of the database raises DatabaseError, the only database exception callers see.
 Refresh tokens are random strings kept in PostgreSQL as their SHA-256, in the database of
 web.database, by default the vpn-router database on this host that provision() creates on the
 web service's first run, with the password in DATABASE_SECRET. A refresh token is used once: refreshing replaces it.
 """
 import base64
 import contextlib
 import hashlib
 import hmac
 import json
 import pathlib
 import secrets
 import subprocess
 import time
 import psycopg
 WEB_KEY = pathlib.Path('/etc/vpn-router/web.key')
 DATABASE_SECRET = pathlib.Path('/etc/vpn-router/database.secret')
 #: The role and the database provision() creates when web.database is not set.
 DATABASE = 'vpn-router'
 #: Seconds an access token of /auth and a refresh token are valid.
 ACCESS_SECONDS = 300
 REFRESH_SECONDS = 8 * 3600
 #: The longest a token without the admin claim may be valid, from now, in seconds.
 MAX_LIFETIME = 600
 #: Seconds of clock difference accepted between a token and this router.
 LEEWAY = 60
 SCHEMA = 'CREATE TABLE IF NOT EXISTS refresh_tokens (hash text PRIMARY KEY, name text NOT NULL, expires timestamptz NOT NULL)'
 class DatabaseError(Exception):
    """The database of the refresh tokens failed."""
 def key():
    """The signing key, or '' while the web service has not made it."""
    try:
        return WEB_KEY.read_text(encoding='utf-8').strip()
    except OSError:
        return ''
 def ensure_key():
    """Make the signing key when it is missing."""
    if not key():
        WEB_KEY.parent.mkdir(parents=True, exist_ok=True)
        WEB_KEY.touch(mode=0o600)
        WEB_KEY.chmod(0o600)
        WEB_KEY.write_text(secrets.token_urlsafe(36) + '\n', encoding='utf-8')
 def b64encode(data):
    return base64.urlsafe_b64encode(data).rstrip(b'=').decode()
 def b64decode(part):
    return base64.urlsafe_b64decode(part + '=' * (-len(part) % 4))
 def sign(claims, secret):
    head = b64encode(json.dumps({'alg': 'HS256', 'typ': 'JWT'}).encode())
    body = b64encode(json.dumps(claims).encode())
    signature = hmac.new(secret.encode(), f'{head}.{body}'.encode(), hashlib.sha256).digest()
    return f'{head}.{body}.{b64encode(signature)}'
 def verify(token, secret):
    """The claims of a token signed with HS256 by the secret, or None when it is not acceptable.
    Only HS256 is accepted, so a token cannot choose a weaker algorithm. The token must expire
    (exp); without the admin claim not later than MAX_LIFETIME seconds from now, so that a stolen
    token is of little use. LEEWAY seconds of clock difference are allowed.
    """
    try:
        header, payload, signature = token.split('.')
        if json.loads(b64decode(header)).get('alg') != 'HS256':
            return None
        expected = hmac.new(secret.encode(), f'{header}.{payload}'.encode(), hashlib.sha256).digest()
        if not hmac.compare_digest(expected, b64decode(signature)):
            return None
        claims = json.loads(b64decode(payload))
        now = time.time()
        expires = claims['exp']
        if isinstance(expires, bool) or not isinstance(expires, (int, float)) or expires < now - LEEWAY:
            return None
        if claims.get('admin') is not True and expires > now + MAX_LIFETIME + LEEWAY:
            return None
        not_before = claims.get('nbf')
        if not_before is not None and (isinstance(not_before, bool) or not isinstance(not_before, (int, float)) or not_before > now + LEEWAY):
            return None
    except (ValueError, KeyError, TypeError, AttributeError):
        return None
    return claims
 def admin_token(seconds):
    """A token with the admin claim, valid for the seconds given."""
    secret = key()
    if not secret:
        raise OSError(f'{WEB_KEY} does not exist; vpn-router-web makes it when it starts')
    now = int(time.time())
    return {'token': sign({'admin': True, 'iat': now, 'exp': now + seconds}, secret), 'expires': now + seconds}
 def connection_string(cfg):
    """web.database, or the database provision() creates; its password, of URL-safe characters, is written unquoted."""
    configured = cfg['web']['database'].strip()
    if configured:
        return configured
    password = DATABASE_SECRET.read_text(encoding='utf-8').strip()
    return f'host=localhost dbname={DATABASE} user={DATABASE} password={password}'
@contextlib.contextmanager
 def database(cfg):
    """A connection to the database of the refresh tokens, with their table, closed at the end; DatabaseError on any failure."""
    try:
        with psycopg.connect(connection_string(cfg), autocommit=True) as connection:
            connection.execute(SCHEMA)
            yield connection
    except psycopg.Error as exc:
        raise DatabaseError(f'the token database failed: {exc}') from exc
 def digest(token):
    return hashlib.sha256(token.encode()).hexdigest()
 def issue(cfg, name):
    """An access token and a refresh token for the user, as /auth answers them."""
    secret = key()
    now = int(time.time())
    refresh = secrets.token_urlsafe(32)
    with database(cfg) as connection:
        connection.execute('DELETE FROM refresh_tokens WHERE expires < now()')
        connection.execute("INSERT INTO refresh_tokens VALUES (%s, %s, now() + make_interval(secs => %s))",
                           (digest(refresh), name, REFRESH_SECONDS))
    return {
        'access_token': sign({'sub': name, 'iat': now, 'exp': now + ACCESS_SECONDS}, secret),
        'token_type': 'Bearer',
        'expires_in': ACCESS_SECONDS,
        'refresh_token': refresh,
        'refresh_expires_in': REFRESH_SECONDS,
    }
 def take(cfg, token):
    """Remove an unexpired refresh token and return the user it was issued to, or None."""
    if not isinstance(token, str) or not token:
        return None
    with database(cfg) as connection:
        row = connection.execute('DELETE FROM refresh_tokens WHERE hash = %s RETURNING name, expires > now()',
                                 (digest(token),)).fetchone()
    return row[0] if row and row[1] else None
 def revoke(cfg, name):
    """Remove all refresh tokens of the user."""
    with database(cfg) as connection:
        connection.execute('DELETE FROM refresh_tokens WHERE name = %s', (name,))
 def psql(sql, name):
    """The output of an SQL statement, read from standard input, run by psql as postgres with name as the variable v1."""
    args = ['runuser', '-u', 'postgres', '--', 'psql', '-X', '-q', '-t', '-A', '-v', 'ON_ERROR_STOP=1', '-v', f'v1={name}']
    result = subprocess.run(args, input=sql, capture_output=True, text=True)
    if result.returncode != 0:
        raise OSError(f'psql failed: {result.stderr.strip()}')
    return result.stdout.strip()
 def provision(cfg):
    """Create the role, its password file and the database when web.database is not set and they are missing.
    The password, of URL-safe characters, reaches psql on standard input, not on its command line.
    """
    if cfg['web']['database'].strip():
        return
    if not psql("SELECT 1 FROM pg_roles WHERE rolname = :'v1'", DATABASE):
        DATABASE_SECRET.unlink(missing_ok=True)
        psql("CREATE ROLE :\"v1\" LOGIN", DATABASE)
    if not DATABASE_SECRET.is_file():
        password = secrets.token_urlsafe(32)
        psql(f"ALTER ROLE :\"v1\" PASSWORD '{password}'", DATABASE)
        DATABASE_SECRET.touch(mode=0o600)
        DATABASE_SECRET.chmod(0o600)
        DATABASE_SECRET.write_text(password + '\n', encoding='utf-8')
    if not psql("SELECT 1 FROM pg_database WHERE datname = :'v1'", DATABASE):
        psql("CREATE DATABASE :\"v1\" OWNER :\"v1\"", DATABASE)
@@ -3,8 +3,14 @@
 The package keeps its own database, dc=vpn-router,dc=local in /var/lib/ldap/vpn-router,
 next to the one slapd's installation creates. Root manages it through the local socket
 with SASL EXTERNAL: the database's root DN is root's peer-credential DN, so there is no
-administrator password. Users are changed by the user-manager service account, which the
+administrator password. Users and groups are changed by the user-manager service account,
-ACLs allow to write ou=users and to set, never read, passwords.
+which the ACLs allow to write ou=users and ou=groups and to set, never read, passwords and
 TOTP secrets.
 The password policy locks an account for LOCKOUT_SECONDS after LOCKOUT_FAILURES failed binds;
 a successful bind clears the count. slapd's otp overlay is configured on the database, so a
 bind of a user with a TOTP token takes the password followed by a current code.
 cn=totp,ou=policies holds the TOTP parameters the token of every user points to.
 """
 import os
@@ -20,9 +26,11 @@ import ldap.sasl
 URI = 'ldapi:///'
 BASE = 'dc=vpn-router,dc=local'
 USERS = f'ou=users,{BASE}'
 GROUPS = f'ou=groups,{BASE}'
 SERVICES = f'ou=services,{BASE}'
 POLICIES = f'ou=policies,{BASE}'
 DEFAULT_POLICY = f'cn=default,{POLICIES}'
 TOTP_PARAMS = f'cn=totp,{POLICIES}'
 MANAGER = f'cn=user-manager,{SERVICES}'
 ROOT = 'gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth'
@@ -32,13 +40,28 @@ MANAGER_SECRET = SECRET_DIR / 'user-manager.secret'
 SLAPD_DEFAULTS = pathlib.Path('/etc/default/slapd')
 SLAPD_SERVICES = 'SLAPD_SERVICES="ldapi:///"'
-MODULES = ('argon2', 'ppolicy')
+MODULES = ('argon2', 'ppolicy', 'otp')
 PASSWORD_HASH = '{ARGON2}'
 #: Failed binds that lock an account, within LOCKOUT_SECONDS, and the seconds it stays locked.
 LOCKOUT_FAILURES = 5
 LOCKOUT_SECONDS = 900
 #: The overlays of the database, in the order they are configured.
 OVERLAYS = ('ppolicy', 'otp')
 #: The TOTP parameters: digits, the OID of HMAC-SHA1, seconds per time step, and the time steps accepted before and after now.
 TOTP_DIGITS = 6
 TOTP_ALGORITHM = '1.2.840.113549.2.7'
 TOTP_PERIOD = 30
 TOTP_WINDOW = 1
 #: The ACLs of the database, in order.
 ACCESS = (
    f'to attrs=userPassword by dn.exact="{MANAGER}" =wx by self =wx by anonymous auth by * none',
    f'to attrs=oathSecret by dn.exact="{MANAGER}" =w by * none',
    f'to dn.subtree="{USERS}" by dn.exact="{MANAGER}" manage by * read',
    f'to dn.subtree="{GROUPS}" by dn.exact="{MANAGER}" manage by * read',
    'to * by * read',
 )
@@ -116,13 +139,18 @@ def _ensure_database(connection):
    return dn
-def _ensure_overlay(connection, database):
+def _ensure_overlays(connection, database):
-    if not connection.search_s(database, ldap.SCOPE_ONELEVEL, '(objectClass=olcPPolicyConfig)', ['1.1']):
+    """Add the overlays of OVERLAYS the database lacks; the otp overlay has no settings of its own."""
-        connection.add_s(f'olcOverlay=ppolicy,{database}', ldap.modlist.addModlist({
+    found = connection.search_s(database, ldap.SCOPE_ONELEVEL, '(objectClass=olcOverlayConfig)', ['olcOverlay'])
-            'objectClass': [b'olcOverlayConfig', b'olcPPolicyConfig'],
+    present = {ORDER.sub(b'', attributes['olcOverlay'][0]).decode() for _, attributes in found}
-            'olcOverlay': [b'ppolicy'],
+    for name in OVERLAYS:
-            'olcPPolicyDefault': [DEFAULT_POLICY.encode()],
+        if name in present:
-        }))
+            continue
        record = {'objectClass': [b'olcOverlayConfig'], 'olcOverlay': [name.encode()]}
        if name == 'ppolicy':
            record = {'objectClass': [b'olcOverlayConfig', b'olcPPolicyConfig'], 'olcOverlay': [b'ppolicy'],
                      'olcPPolicyDefault': [DEFAULT_POLICY.encode()]}
        connection.add_s(f'olcOverlay={name},{database}', ldap.modlist.addModlist(record))
 def _ensure_entry(connection, dn, attributes):
@@ -166,16 +194,26 @@ def provision():
        _ensure_modules(connection)
        _ensure_password_hash(connection)
        database = _ensure_database(connection)
-        _ensure_overlay(connection, database)
+        _ensure_overlays(connection, database)
        _ensure_entry(connection, BASE, {'objectClass': [b'dcObject', b'organization'], 'dc': [b'vpn-router'], 'o': [b'vpn-router']})
-        for unit in (USERS, SERVICES, POLICIES):
+        for unit in (USERS, GROUPS, SERVICES, POLICIES):
            _ensure_entry(connection, unit, {'objectClass': [b'organizationalUnit'], 'ou': [unit.split(',')[0][3:].encode()]})
        _ensure_entry(connection, DEFAULT_POLICY, {
            'objectClass': [b'organizationalRole', b'pwdPolicy'],
            'cn': [b'default'],
            'pwdAttribute': [b'userPassword'],
            'pwdLockout': [b'TRUE'],
-            'pwdMaxFailure': [b'0'],
+            'pwdMaxFailure': [str(LOCKOUT_FAILURES).encode()],
            'pwdLockoutDuration': [str(LOCKOUT_SECONDS).encode()],
            'pwdFailureCountInterval': [str(LOCKOUT_SECONDS).encode()],
        })
        _ensure_entry(connection, TOTP_PARAMS, {
            'objectClass': [b'organizationalRole', b'oathTOTPParams'],
            'cn': [b'totp'],
            'oathOTPLength': [str(TOTP_DIGITS).encode()],
            'oathHMACAlgorithm': [TOTP_ALGORITHM.encode()],
            'oathTOTPTimeStepPeriod': [str(TOTP_PERIOD).encode()],
            'oathTOTPTimeStepWindow': [str(TOTP_WINDOW).encode()],
        })
        _ensure_manager(connection)
    finally:
@@ -1,6 +1,7 @@
 """Interfaces and addresses of the router, read from the system.
-Only the Python standard library is used, so the web service can use this module too.
+Only the Python standard library is used, so the web service and its bootstrap server can use
 this module too.
 """
 import pathlib
@@ -107,3 +108,28 @@ def internal_address(mode, internal, int_addr):
    except ConfigError:
        return ''
    return addresses[0] if addresses else ''
 def web_listen(configuration, default_port):
    """Where the web service listens, from vpn-router.json read as configuration, as ((address, port), '').
    The address is the internal address as general.mode directs, the port web.port, else
    default_port. While web.enabled is false or the address is not known, the answer is (None,
    the reason).
    """
    def section(name):
        value = configuration.get(name)
        return value if isinstance(value, dict) else {}
    web = section('web')
    if web.get('enabled', True) is False:
        return None, 'the web service is disabled in the configuration'
    port = web.get('port', default_port)
    if isinstance(port, bool) or not isinstance(port, int) or not 1 <= port <= 65535:
        port = default_port
    general, interfaces, local = section('general'), section('interfaces'), section('local')
    address = internal_address(str(general.get('mode') or 'manual'), str(interfaces.get('internal') or ''),
                               str(local.get('int_addr') or ''))
    if not address:
        return None, 'the internal address is not known yet'
    return (address, port), ''
@@ -0,0 +1,40 @@
 """The listening loop of the web service and of its bootstrap server.
 Only the Python standard library is used, so the bootstrap server, which must run whatever else
 is broken, can use it.
 """
 import threading
 import time
 from http.server import ThreadingHTTPServer
 def serve(wanted, handler, log, interval):
    """Work out where to listen every interval seconds with wanted, and run an HTTP server of handler there.
    wanted answers ((address, port), '') or (None, the reason not to listen). When the address or
    the port changes, the server is stopped and a new one listens at the new place; while there is
    no place or it cannot be bound, no server runs and the next round tries again. Every new
    reason is logged with log. Does not return.
    """
    server, thread, listening, reported = None, None, None, None
    while True:
        target, reason = wanted()
        if target != listening and server is not None:
            server.shutdown()
            server.server_close()
            thread.join()
            server, listening = None, None
        if target is not None and server is None:
            try:
                server = ThreadingHTTPServer(target, handler)
            except OSError as exc:
                reason = f'cannot listen on {target[0]}:{target[1]}: {exc}'
            else:
                thread = threading.Thread(target=server.serve_forever, name='server', daemon=True)
                thread.start()
                listening, reason = target, f'serving on http://{target[0]}:{target[1]}'
        if reason and reason != reported:
            log(reason)
            reported = reason
        time.sleep(interval)
@@ -0,0 +1,534 @@
 """The users and groups of the router's directory, for vpn-router-user, vpn-router-group and the web service.
 A user is an entry in the router's directory, with a first name, a last name and an e-mail
 address, whose password OpenVPN checks, and a client certificate of the router CA with the
 user name as its common name, which OpenVPN and IKEv2 accept. Users exist whether or not a
 P2S system is enabled.
 A group is a groupOfNames in ou=groups with a name, which follows the rules of user names, an
 optional description and at least one member; a group cannot be empty, so the last member
 cannot leave it and a user who is the last member of a group cannot be removed. Groups only
 record membership; nothing on the router reads them.
 A user may have a TOTP token: the user entry itself is an oathTOTPUser and oathTOTPToken whose
 oathTOTPToken points to itself, with the secret in oathSecret and the parameters of
 vpnrouter_ldap.TOTP_PARAMS. Enrolment stores nothing until a code of the new secret is
 verified: otp_new makes a secret and its otpauth URI, and otp_enrol stores the secret given a
 code of it. slapd's otp overlay then takes the password followed by a code at every bind.
 A user is disabled when pwdAccountLockedTime holds LOCKED, and locked when the password
 policy has set it after failed binds, for vpnrouter_ldap.LOCKOUT_SECONDS; unlock ends that.
 login() checks a user's password and code with a bind. Members of ADMIN_GROUP manage the
 router through the web service. Changing the password, disabling, unlocking and removing a
 user revoke its refresh tokens.
 Every operation runs in a session(), which holds the lock that serializes changes and the
 connection of the user manager account. A request that cannot be done raises Failure, whose
 status says why: 2 the request is invalid, 3 the user does not exist, 4 the user exists, 5 a
 login failed, 1 anything else. Answers are dicts with the keys of the JSON answers.
 """
 import base64
 import calendar
 import contextlib
 import fcntl
 import pathlib
 import re
 import subprocess
 import sys
 import time
 import ldap
 import ldap.filter
 import ldap.modlist
 import pyotp
 import vpnrouter
 import vpnrouter_auth
 import vpnrouter_ldap
 #: A user name is one lowercase DNS label, as it is also the common name of the certificate.
 NAME = re.compile(r'^[a-z][a-z0-9-]{0,30}[a-z0-9]$|^[a-z]$')
 PASSWORD_MIN = 8
 PASSWORD_MAX = 128
 #: The longest first or last name, in characters.
 PERSON_NAME_MAX = 64
 #: An e-mail address: one @, no spaces, and a dot in the domain.
 EMAIL = re.compile(r'^[^@\s]+@[^@\s]+\.[^@\s]+$')
 EMAIL_MAX = 254
 #: The keys of the person attributes and their LDAP attributes.
 PERSON = (('first_name', 'givenName'), ('last_name', 'sn'), ('email', 'mail'))
 #: The attributes read for a user.
 ATTRIBUTES = ['uid', 'pwdAccountLockedTime', 'givenName', 'sn', 'mail', 'oathTOTPToken']
 #: The readable attributes of a TOTP token; oathSecret is never readable.
 TOKEN_ATTRIBUTES = ['oathTOTPToken', 'oathTOTPParams', 'oathSecretTime', 'oathTOTPLastTimeStep', 'oathTOTPTimeStepDrift',
                    'oathLastLogin', 'oathLastFailure', 'oathFailureCount']
 #: A TOTP secret as otp_new makes it: 32 base32 characters, 160 bits.
 TOTP_SECRET = re.compile(r'^[A-Z2-7]{32}$')
 #: The attributes read for a group.
 GROUP_ATTRIBUTES = ['cn', 'description', 'member']
 #: The longest group description, in characters.
 DESCRIPTION_MAX = 256
 #: The value of pwdAccountLockedTime that disables an account until it is enabled.
 LOCKED = b'000001010000Z'
 #: The group whose members manage the router through the web service.
 ADMIN_GROUP = 'admins'
 #: The digits of a TOTP code given to login().
 CODE = re.compile(r'^\d{6}$')
 LOCK_FILE = pathlib.Path('/run/vpn-router/users.lock')
 class Failure(Exception):
    """A request that cannot be done; status is 2 invalid, 3 no such user, 4 the user exists, 5 failed login, 1 other."""
    def __init__(self, message, status=1):
        super().__init__(message)
        self.status = status
@contextlib.contextmanager
 def session():
    """The connection of the user manager account, used under the lock that serializes changes."""
    LOCK_FILE.parent.mkdir(parents=True, exist_ok=True)
    with open(LOCK_FILE, 'w') as lock:
        fcntl.flock(lock, fcntl.LOCK_EX)
        connection = vpnrouter_ldap.manager()
        try:
            yield connection
        finally:
            connection.unbind_s()
 def check_name(name):
    if not isinstance(name, str) or not NAME.match(name):
        raise Failure('a user name is 1 to 32 lowercase letters, digits and hyphens, starting with a letter', 2)
 def check_password(value):
    if not isinstance(value, str) or '\n' in value or '\r' in value:
        raise Failure('the password must be a string of one line', 2)
    if not PASSWORD_MIN <= len(value) <= PASSWORD_MAX:
        raise Failure(f'the password must have {PASSWORD_MIN} to {PASSWORD_MAX} characters', 2)
 def person(values, required):
    """The person attributes of values ({first_name, last_name, email}), checked, as {LDAP attribute: value}.
    A key whose value is None is not given; with required, all three must be given.
    """
    found = {}
    for key, attribute in PERSON:
        value = values.get(key)
        if value is None:
            if required:
                raise Failure(f'{key} is required', 2)
            continue
        if not isinstance(value, str):
            raise Failure(f'{key} must be a string', 2)
        value = value.strip()
        if key == 'email':
            if len(value) > EMAIL_MAX or not EMAIL.match(value):
                raise Failure(f'{value!r} is not an e-mail address', 2)
        elif not value or len(value) > PERSON_NAME_MAX or any(not character.isprintable() for character in value):
            raise Failure(f'the {key.replace("_", " ")} must have 1 to {PERSON_NAME_MAX} printable characters', 2)
        found[attribute] = value
    return found
 def dn_of(name):
    return f'uid={name},{vpnrouter_ldap.USERS}'
 def certificate(name):
    return vpnrouter.USER_CERT_DIR / f'{name}_cert.pem', vpnrouter.USER_CERT_DIR / f'{name}_key.pem'
 def entry(connection, name):
    """The attributes of the user's entry; Failure with status 3 when there is none."""
    check_name(name)
    try:
        return connection.search_s(dn_of(name), ldap.SCOPE_BASE, attrlist=ATTRIBUTES)[0][1]
    except ldap.NO_SUCH_OBJECT:
        raise Failure(f'user {name} does not exist', 3)
 def simple_ca(*args):
    result = subprocess.run([sys.executable, str(vpnrouter.SIMPLE_CA), *args], capture_output=True, text=True)
    if result.returncode != 0:
        raise Failure(f'simple-ca {args[0]} failed: {result.stderr.strip() or result.stdout.strip()}')
 def publish_crl():
    """Give strongSwan the CRL after a revocation; OpenVPN reads it at every connection."""
    if vpnrouter.install_file(vpnrouter.PKI_DIR / 'crl.pem', vpnrouter.CRL_DEST, 0o644):
        subprocess.run(['swanctl', '--load-creds'], capture_output=True, text=True)
 def locked_since(value):
    """The time a pwdAccountLockedTime value names, in seconds since the epoch, or 0 when it names none."""
    try:
        return calendar.timegm(time.strptime(value[:14].decode(), '%Y%m%d%H%M%S'))
    except ValueError:
        return 0
 def summary(attributes):
    """A user as the answers show it; a person attribute the entry does not have is None."""
    name = attributes['uid'][0].decode()
    lock = attributes.get('pwdAccountLockedTime', [None])[0]
    user = {
        'name': name,
        'enabled': lock != LOCKED,
        'locked': lock not in (None, LOCKED) and locked_since(lock) > time.time() - vpnrouter_ldap.LOCKOUT_SECONDS,
        'certificate': certificate(name)[0].is_file(),
        'otp': 'oathTOTPToken' in attributes,
    }
    for key, attribute in PERSON:
        values = attributes.get(attribute)
        user[key] = values[0].decode() if values else None
    return user
 def common_name(attributes, name):
    """The cn of an entry: the first and the last name, or the user name without them."""
    full = ' '.join(value for value in (attributes.get('givenName'), attributes.get('sn')) if value)
    return full or name
 def list_users(connection):
    found = connection.search_s(vpnrouter_ldap.USERS, ldap.SCOPE_ONELEVEL, '(objectClass=inetOrgPerson)', ATTRIBUTES)
    return {'users': sorted((summary(attributes) for _, attributes in found), key=lambda user: user['name'])}
 def add(connection, name, password, values):
    """Add a user with the person attributes of values, all three required, and issue the certificate."""
    check_name(name)
    details = person(values, required=True)
    check_password(password)
    if not (vpnrouter.PKI_DIR / 'ca_key.pem').is_file():
        raise Failure('the router CA does not exist; vpn-router-setup creates it once a platform is chosen')
    record = {'objectClass': [b'inetOrgPerson'], 'uid': [name.encode()],
              'cn': [common_name(details, name).encode()]}
    record.update({attribute: [value.encode()] for attribute, value in details.items()})
    try:
        connection.add_s(dn_of(name), ldap.modlist.addModlist(record))
    except ldap.ALREADY_EXISTS:
        raise Failure(f'user {name} exists', 4)
    connection.passwd_s(dn_of(name), None, password)
    vpnrouter.USER_CERT_DIR.mkdir(mode=0o700, parents=True, exist_ok=True)
    for path in certificate(name):
        path.unlink(missing_ok=True)
    simple_ca('make-cert', '--ca-dir', str(vpnrouter.PKI_DIR), '--cert-dir', str(vpnrouter.USER_CERT_DIR), name)
    return summary(entry(connection, name))
 def update(connection, name, values):
    """Change the person attributes values gives; the cn follows the first and the last name."""
    details = person(values, required=False)
    if not details:
        raise Failure('give at least one of first_name, last_name and email', 2)
    current = entry(connection, name)
    names = {attribute: current[attribute][0].decode() for attribute in ('givenName', 'sn') if current.get(attribute)}
    names.update({attribute: details[attribute] for attribute in ('givenName', 'sn') if attribute in details})
    changes = [(ldap.MOD_REPLACE, attribute, [value.encode()]) for attribute, value in details.items()]
    changes.append((ldap.MOD_REPLACE, 'cn', [common_name(names, name).encode()]))
    connection.modify_s(dn_of(name), changes)
    return summary(entry(connection, name))
 def passwd(connection, name, password):
    check_password(password)
    entry(connection, name)
    connection.passwd_s(dn_of(name), None, password)
    vpnrouter_auth.revoke(vpnrouter.load_config(), name)
    return summary(entry(connection, name))
 def disable(connection, name):
    if entry(connection, name).get('pwdAccountLockedTime') != [LOCKED]:
        connection.modify_s(dn_of(name), [(ldap.MOD_REPLACE, 'pwdAccountLockedTime', [LOCKED])])
    vpnrouter_auth.revoke(vpnrouter.load_config(), name)
    return summary(entry(connection, name))
 def enable(connection, name):
    if entry(connection, name).get('pwdAccountLockedTime') == [LOCKED]:
        connection.modify_s(dn_of(name), [(ldap.MOD_DELETE, 'pwdAccountLockedTime', None)])
    return summary(entry(connection, name))
 def unlock(connection, name):
    """End a lockout after failed binds and clear the failure count; a disabled user stays disabled."""
    entry(connection, name)
    current = connection.search_s(dn_of(name), ldap.SCOPE_BASE, attrlist=['pwdAccountLockedTime', 'pwdFailureTime'])[0][1]
    changes = [(ldap.MOD_DELETE, 'pwdFailureTime', None)] if 'pwdFailureTime' in current else []
    if 'pwdAccountLockedTime' in current and current['pwdAccountLockedTime'] != [LOCKED]:
        changes.append((ldap.MOD_DELETE, 'pwdAccountLockedTime', None))
    if changes:
        connection.modify_s(dn_of(name), changes)
    vpnrouter_auth.revoke(vpnrouter.load_config(), name)
    return summary(entry(connection, name))
 def login(name, password, code):
    """Check the user's password, and the TOTP code of an enrolled user, with a bind as the user.
    A wrong name, password or code, a locked or a disabled user give the same Failure, status 5.
    """
    refused = Failure('the name, the password or the code is wrong', 5)
    if not isinstance(name, str) or not NAME.match(name) or not isinstance(password, str) or not password:
        raise refused
    if code is not None and code != '' and (not isinstance(code, str) or not CODE.match(code)):
        raise refused
    connection = ldap.initialize(vpnrouter_ldap.URI)
    try:
        connection.simple_bind_s(dn_of(name), password + (code or ''))
    except ldap.INVALID_CREDENTIALS:
        raise refused
    finally:
        connection.unbind_s()
 def is_admin(connection, name):
    """Whether the user exists and is a member of ADMIN_GROUP."""
    try:
        entry(connection, name)
    except Failure:
        return False
    return ADMIN_GROUP in groups_of(connection, name)
 def totp(secret):
    return pyotp.TOTP(secret, digits=vpnrouter_ldap.TOTP_DIGITS, interval=vpnrouter_ldap.TOTP_PERIOD)
 def otp_new(connection, name):
    """A new TOTP secret for the user and its otpauth URI, issued by the router's wan.local_fqdn; nothing is stored."""
    entry(connection, name)
    secret = pyotp.random_base32()
    issuer = vpnrouter.load_config()['wan']['local_fqdn'].strip()
    return {'name': name, 'secret': secret, 'uri': totp(secret).provisioning_uri(name=name, issuer_name=issuer or None)}
 def otp_enrol(connection, name, secret, code):
    """Store the secret of otp_new as the user's TOTP token, replacing one the user has, when code is a code of it."""
    current = entry(connection, name)
    if not isinstance(secret, str) or not TOTP_SECRET.match(secret):
        raise Failure('the secret must be 32 base32 characters, as a new secret has them', 2)
    if not isinstance(code, str) or not re.fullmatch(rf'\d{{{vpnrouter_ldap.TOTP_DIGITS}}}', code):
        raise Failure(f'the code must be {vpnrouter_ldap.TOTP_DIGITS} digits', 2)
    if not totp(secret).verify(code, valid_window=vpnrouter_ldap.TOTP_WINDOW):
        raise Failure('the code does not match the secret; check the clock of the authenticator', 2)
    changes = [] if 'oathTOTPToken' in current else [(ldap.MOD_ADD, 'objectClass', [b'oathTOTPUser', b'oathTOTPToken'])]
    changes += [
        (ldap.MOD_REPLACE, 'oathTOTPToken', [dn_of(name).encode()]),
        (ldap.MOD_REPLACE, 'oathTOTPParams', [vpnrouter_ldap.TOTP_PARAMS.encode()]),
        (ldap.MOD_REPLACE, 'oathSecret', [base64.b32decode(secret)]),
        (ldap.MOD_REPLACE, 'oathSecretTime', [time.strftime('%Y%m%d%H%M%SZ', time.gmtime()).encode()]),
    ]
    connection.modify_s(dn_of(name), changes)
    return summary(entry(connection, name))
 def otp_remove(connection, name):
    """Remove the user's TOTP token, when the user has one."""
    if 'oathTOTPToken' in entry(connection, name):
        token = connection.search_s(dn_of(name), ldap.SCOPE_BASE, attrlist=TOKEN_ATTRIBUTES)[0][1]
        changes = [(ldap.MOD_DELETE, 'objectClass', [b'oathTOTPUser', b'oathTOTPToken']),
                   (ldap.MOD_DELETE, 'oathSecret', None)]
        changes += [(ldap.MOD_DELETE, attribute, None) for attribute in token]
        connection.modify_s(dn_of(name), changes)
    return summary(entry(connection, name))
 def remove(connection, name):
    """Remove the user from its groups and the directory, and revoke the certificate.
    A user who is the last member of a group is not removed.
    """
    entry(connection, name)
    groups = groups_of(connection, name)
    alone = sorted(group for group, members in groups.items() if members == [name])
    if alone:
        raise Failure(f'user {name} is the last member of {", ".join(alone)}; remove the group first', 2)
    for group in groups:
        connection.modify_s(group_dn(group), [(ldap.MOD_DELETE, 'member', [dn_of(name).encode()])])
    cert, key = certificate(name)
    if cert.is_file():
        simple_ca('revoke-cert', '--ca-dir', str(vpnrouter.PKI_DIR), str(cert))
        simple_ca('make-crl', '--ca-dir', str(vpnrouter.PKI_DIR), '--days', str(vpnrouter.CRL_DAYS))
        publish_crl()
    cert.unlink(missing_ok=True)
    key.unlink(missing_ok=True)
    connection.delete_s(dn_of(name))
    vpnrouter_auth.revoke(vpnrouter.load_config(), name)
    return {'name': name, 'removed': True}
 def profile(connection, name):
    """The OpenVPN client profile of the user, with the CA, the certificate and the key inline."""
    entry(connection, name)
    cfg = vpnrouter.load_config()
    if not cfg['openvpn']['enabled']:
        raise Failure('OpenVPN is not enabled', 2)
    cert, key = certificate(name)
    if not cert.is_file() or not key.is_file():
        raise Failure(f'user {name} has no certificate')
    fqdn = cfg['wan']['local_fqdn'].strip()
    text = '\n'.join([
        'client',
        'dev tun',
        f'proto {cfg["openvpn"]["protocol"]}',
        f'remote {fqdn} {cfg["openvpn"]["port"]}',
        'nobind',
        'persist-key',
        'persist-tun',
        'remote-cert-tls server',
        f'verify-x509-name {fqdn} name',
        'auth-user-pass',
        'auth-nocache',
        'verb 3',
        '<ca>', extract_pem(vpnrouter.CA_CERT_DEST.read_text(encoding='utf-8')), '</ca>',
        '<cert>', extract_pem(cert.read_text(encoding='utf-8')), '</cert>',
        '<key>', key.read_text(encoding='utf-8').strip(), '</key>',
        '',
    ])
    return {'name': name, 'profile': text}
 def group_dn(name):
    return f'cn={name},{vpnrouter_ldap.GROUPS}'
 def check_group_name(name):
    if not isinstance(name, str) or not NAME.match(name):
        raise Failure('a group name is 1 to 32 lowercase letters, digits and hyphens, starting with a letter', 2)
 def check_description(value):
    """The description, stripped; '' means none."""
    if not isinstance(value, str):
        raise Failure('the description must be a string', 2)
    value = value.strip()
    if len(value) > DESCRIPTION_MAX or any(not character.isprintable() for character in value):
        raise Failure(f'the description must have at most {DESCRIPTION_MAX} printable characters', 2)
    return value
 def member_names(attributes):
    """The user names of a group's member DNs, sorted."""
    names = []
    for value in attributes.get('member', []):
        dn = value.decode()
        if dn.startswith('uid=') and dn.endswith(f',{vpnrouter_ldap.USERS}'):
            names.append(dn[4:-len(f',{vpnrouter_ldap.USERS}')])
    return sorted(names)
 def group_summary(attributes):
    """A group as the answers show it."""
    values = attributes.get('description')
    return {
        'name': attributes['cn'][0].decode(),
        'description': values[0].decode() if values else '',
        'members': member_names(attributes),
    }
 def group_entry(connection, name):
    """The attributes of the group's entry; Failure with status 3 when there is none."""
    check_group_name(name)
    try:
        return connection.search_s(group_dn(name), ldap.SCOPE_BASE, attrlist=GROUP_ATTRIBUTES)[0][1]
    except ldap.NO_SUCH_OBJECT:
        raise Failure(f'group {name} does not exist', 3)
 def groups_of(connection, name):
    """The groups the user is a member of, as {group name: its member names}."""
    found = connection.search_s(vpnrouter_ldap.GROUPS, ldap.SCOPE_ONELEVEL,
                                f'(&(objectClass=groupOfNames)(member={ldap.filter.escape_filter_chars(dn_of(name))}))',
                                GROUP_ATTRIBUTES)
    return {attributes['cn'][0].decode(): member_names(attributes) for _, attributes in found}
 def list_groups(connection):
    found = connection.search_s(vpnrouter_ldap.GROUPS, ldap.SCOPE_ONELEVEL, '(objectClass=groupOfNames)', GROUP_ATTRIBUTES)
    return {'groups': sorted((group_summary(attributes) for _, attributes in found), key=lambda group: group['name'])}
 def group_add(connection, name, description, members):
    """Add a group with a description ('' for none) and at least one member, each an existing user."""
    check_group_name(name)
    description = check_description(description if description is not None else '')
    if not isinstance(members, list) or not members:
        raise Failure('a group needs at least one member', 2)
    for member in members:
        entry(connection, member)
    record = {'objectClass': [b'groupOfNames'], 'cn': [name.encode()],
              'member': [dn_of(member).encode() for member in sorted(set(members))]}
    if description:
        record['description'] = [description.encode()]
    try:
        connection.add_s(group_dn(name), ldap.modlist.addModlist(record))
    except ldap.ALREADY_EXISTS:
        raise Failure(f'group {name} exists', 4)
    return group_summary(group_entry(connection, name))
 def group_update(connection, name, new_name=None, description=None):
    """Rename the group or change its description ('' removes it); at least one must be given."""
    if new_name is None and description is None:
        raise Failure('give a new name or a description', 2)
    group_entry(connection, name)
    if description is not None:
        description = check_description(description)
        change = (ldap.MOD_REPLACE, 'description', [description.encode()]) if description else \
            (ldap.MOD_REPLACE, 'description', [])
        connection.modify_s(group_dn(name), [change])
    if new_name is not None and new_name != name:
        check_group_name(new_name)
        try:
            connection.rename_s(group_dn(name), f'cn={new_name}', delold=1)
        except ldap.ALREADY_EXISTS:
            raise Failure(f'group {new_name} exists', 4)
        name = new_name
    return group_summary(group_entry(connection, name))
 def group_remove(connection, name):
    group_entry(connection, name)
    connection.delete_s(group_dn(name))
    return {'name': name, 'removed': True}
 def group_member_add(connection, name, user):
    attributes = group_entry(connection, name)
    entry(connection, user)
    if user not in member_names(attributes):
        connection.modify_s(group_dn(name), [(ldap.MOD_ADD, 'member', [dn_of(user).encode()])])
    return group_summary(group_entry(connection, name))
 def group_member_remove(connection, name, user):
    """Take the user out of the group; the last member cannot leave."""
    check_name(user)
    members = member_names(group_entry(connection, name))
    if user not in members:
        raise Failure(f'user {user} is not a member of group {name}', 3)
    if members == [user]:
        raise Failure(f'user {user} is the last member of group {name}; remove the group instead', 2)
    connection.modify_s(group_dn(name), [(ldap.MOD_DELETE, 'member', [dn_of(user).encode()])])
    return group_summary(group_entry(connection, name))
 def extract_pem(text):
    """The PEM blocks of the certificates in a file that also holds their text form."""
    begin, end = '-----BEGIN CERTIFICATE-----', '-----END CERTIFICATE-----'
    blocks = []
    start = text.find(begin)
    while start != -1:
        stop = text.index(end, start) + len(end)
        blocks.append(text[start:stop])
        start = text.find(begin, stop)
    return '\n'.join(blocks)
@@ -0,0 +1,12 @@
 #!/bin/sh
 # Wait until libpq is in the dynamic linker cache, where psycopg looks it up.
 # ExecStartPre= of vpn-router-web.service; TimeoutStartSec= of the unit bounds the wait.
 reported=
 until ldconfig -p | grep -q '^[[:space:]]*libpq\.so\.5 '; do
    if [ -z "$reported" ]; then
        echo 'waiting for libpq.so.5 in the linker cache'
        reported=yes
    fi
    sleep 2
 done
@@ -1,8 +1,8 @@
 #!/usr/bin/python3
 """Serve the status of the router on its internal address.
-Run by vpn-router-web.service. It needs nothing but the Python standard library; users are
+Run by vpn-router-web.service. Users are managed with vpnrouter_users, the module that
-managed by running vpn-router-user.
+vpn-router-user uses too.
 Pages:
  /        the status for people: plain text for curl and wget, a styled page
@@ -10,30 +10,42 @@ Pages:
  /status  the status as JSON
  /config  the configuration without its secrets, and what the router resolved and
           runs, as JSON
-  /users   the users of the P2S systems: list, add, set the password, disable, enable,
+  /users   the users in the router's directory, whether or not a P2S system is enabled: list,
-           remove, and the OpenVPN client profile (see users())
+           add, change, set the password, disable, enable, remove, the OpenVPN client
           profile, and TOTP enrolment (see users())
  /groups  the groups of the router's directory: list, add, rename or describe, remove, and
           add and take out members (see groups())
  /auth    sign-in with a user's password and TOTP code, refresh and sign-out (see auth())
-/config and /users need an HS256 JWT in an Authorization: Bearer header that is signed
+/config, /users and /groups need an HS256 JWT in an Authorization: Bearer header signed with
-with web.jwt_secret and expires within ten minutes; they are not served (404) while
+vpnrouter_auth.WEB_KEY (see vpnrouter_auth.verify). A token of /auth names a user in sub, who
-web.jwt_secret is not set.
+must be a member of vpnrouter_users.ADMIN_GROUP; a token of vpn-router-token has the admin
 claim, and only such a token may change ADMIN_GROUP: create, rename to it, change, remove it,
 add or take out its members, and remove one of its members. These paths and /auth are not
 served (404) while the key does not exist. /auth is limited to AUTH_LIMIT calls per client
 address in AUTH_WINDOW seconds; the address of a call from one of web.trusted_proxies, by
 default the address the service listens on, is the last entry of its X-Forwarded-For header.
 / and /status show only the status: whether the services run, the package version, whether
 Docker is installed, and the state of site-to-site, road-warrior (IPsec P2S), OpenVPN and
 WireGuard. No configuration data, key or address is shown.
-The service needs nothing else to start and does not end on its own. A discovery thread works
+The service first initializes, without listening: it makes the signing key and the refresh
-out the internal address from /etc/vpn-router/vpn-router.json as general.mode directs, with
+tokens' database (vpnrouter_auth), retrying every INIT_RETRY_SECONDS until both are ready and
-web.port and web.enabled, every DISCOVERY_SECONDS, and runs the server thread there: it starts
+logging each failure. Then it listens. Its unit starts it only once wait-for-libpq has found libpq
-it, moves it when the address or the port changes, and stops it while the service is disabled
+in the linker cache, where psycopg looks it up.
-or the address is not known. Every request reads what it needs when it is made: the token secret
+
-from the configuration, and the features of the router and the facts shown by /config from
+The service needs nothing else to start and does not end on its own. Every DISCOVERY_SECONDS it
 works out the internal address from /etc/vpn-router/vpn-router.json as general.mode directs,
 with web.port and web.enabled (vpnrouter_net.web_listen), and runs the server there
 (vpnrouter_serve.serve): it starts it, moves it when the address or the port changes, and stops
 it while the service is disabled or the address is not known. Every request reads what it needs when it is made: the signing
 key, the configuration, and the features of the router and the facts shown by /config from
 /run/vpn-router/web.json, which vpn-router-setup writes. Before setup has written that file
 the router has no features, and /config shows no configuration or interfaces.
 """
-import base64
+import collections
 import hashlib
 import hmac
 import json
 import os
 import signal
@@ -42,49 +54,56 @@ import sys
 import threading
 import time
 from html import escape
-from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+from http.server import BaseHTTPRequestHandler
 sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
 import ldap
 import vpnrouter
 import vpnrouter_auth
 import vpnrouter_net
 import vpnrouter_serve
 import vpnrouter_users
 STATE_FILE = '/run/vpn-router/web.json'
 CONFIG_FILE = '/etc/vpn-router/vpn-router.json'
 #: Seconds between two attempts of the initialization.
 INIT_RETRY_SECONDS = 15
 #: The port when web.port is not set.
-DEFAULT_PORT = 8080
+DEFAULT_PORT = 7081
 #: Seconds between two discoveries of where the service should listen.
 DISCOVERY_SECONDS = 30
 COMMAND_TIMEOUT = 5
 CACHE_SECONDS = 2
-#: Seconds of clock difference accepted between a token and this router.
+#: The calls of /auth allowed per client address in AUTH_WINDOW seconds.
-LEEWAY = 60
+AUTH_LIMIT = 10
-
+AUTH_WINDOW = 60
 #: The longest a token may be valid, from now, in seconds.
 MAX_LIFETIME = 600
 TAILWIND = 'https://cdn.jsdelivr.net/npm/@tailwindcss/browser@4'
 #: Seconds after which the status page in a browser reloads itself.
 REFRESH_SECONDS = 60
 USER_TOOL = '/usr/sbin/vpn-router-user'
 OPENVPN_STATUS = '/run/openvpn-server/status-vpn-router.log'
 #: Seconds a user command may take; adding a user creates an RSA key.
 USER_TIMEOUT = 120
 #: The largest request body accepted, in bytes.
 BODY_LIMIT = 4096
-#: The HTTP status for each exit status of vpn-router-user; any other is 500.
+#: The HTTP status for each status of a vpnrouter_users.Failure; any other is 500.
-USER_STATUS = {0: 200, 2: 400, 3: 404, 4: 409}
+USER_STATUS = {2: 400, 3: 404, 4: 409, 5: 401}
 _cache = {'at': 0.0, 'data': None}
 _lock = threading.Lock()
 #: The times of the recent /auth calls of each client address.
 _auth_calls = collections.defaultdict(collections.deque)
 _auth_lock = threading.Lock()
 def read_json(path):
    """The object in a JSON file, or {} when the file is missing or holds no object."""
@@ -103,14 +122,12 @@ def section(configuration, name):
 def current():
-    """What a request needs, read now: the features and facts setup resolved, and the token secret.
+    """What a request needs, read now: the features and facts setup resolved, and the signing key.
-    Before setup has run the router has no features, and without a secret /config and /users are
+    Before setup has run the router has no features, and without a key the protected paths and
-    not served.
+    /auth are not served.
    """
-    state = read_json(STATE_FILE)
+    return dict(read_json(STATE_FILE), key=vpnrouter_auth.key())
    secret = section(read_json(CONFIG_FILE), 'web').get('jwt_secret')
    return dict(state, jwt_secret=secret if isinstance(secret, str) else '')
 def run(*args):
@@ -251,37 +268,6 @@ def collect(state):
    return status
 def b64decode(part):
    return base64.urlsafe_b64decode(part + '=' * (-len(part) % 4))
 def verify(token, secret):
    """The claims of a token signed with HS256 by the secret, or None when it is not acceptable.
    Only HS256 is accepted, so a token cannot choose a weaker algorithm. The token
    must expire (exp), and not later than MAX_LIFETIME seconds from now, so that a
    stolen token is of little use; LEEWAY seconds of clock difference are allowed.
    """
    try:
        header, payload, signature = token.split('.')
        if json.loads(b64decode(header)).get('alg') != 'HS256':
            return None
        expected = hmac.new(secret.encode(), f'{header}.{payload}'.encode(), hashlib.sha256).digest()
        if not hmac.compare_digest(expected, b64decode(signature)):
            return None
        claims = json.loads(b64decode(payload))
        now = time.time()
        expires = claims['exp']
        if isinstance(expires, bool) or not isinstance(expires, (int, float)) or not now - LEEWAY <= expires <= now + MAX_LIFETIME + LEEWAY:
            return None
        not_before = claims.get('nbf')
        if not_before is not None and (isinstance(not_before, bool) or not isinstance(not_before, (int, float)) or not_before > now + LEEWAY):
            return None
    except (ValueError, KeyError, TypeError, AttributeError):
        return None
    return claims
 def lines_of(*args):
    """The output lines of a command, or None when it fails."""
    result = run(*args)
@@ -409,60 +395,174 @@ def html_of(status):
 """
 def user_tool(*args, password=None):
    """Run vpn-router-user; the HTTP status for its exit status and its JSON answer."""
    try:
        result = subprocess.run([USER_TOOL, *args], input='' if password is None else password + '\n',
                                capture_output=True, text=True, timeout=USER_TIMEOUT)
        document = json.loads(result.stdout)
    except (OSError, subprocess.TimeoutExpired, ValueError) as exc:
        return 500, {'error': f'vpn-router-user failed: {exc}'}
    if not isinstance(document, dict):
        return 500, {'error': 'vpn-router-user gave no object'}
    return USER_STATUS.get(result.returncode, 500), document
 def users(state, method, path, body):
-    """Answer a /users request: the HTTP status and the JSON document.
+    """Answer a /users request with vpnrouter_users: the HTTP status and the JSON document.
      GET    /users                   the users
-      POST   /users                   add a user: {"name": ..., "password": ...}
+      POST   /users                   add a user: {"name", "password", "first_name", "last_name", "email"}
      PUT    /users/<name>            change any of {"first_name", "last_name", "email"}
      PUT    /users/<name>/password   set the password: {"password": ...}
      POST   /users/<name>/disable    lock the user out of OpenVPN
      POST   /users/<name>/enable     let the user in again
      POST   /users/<name>/unlock     end a lockout after failed logins
      DELETE /users/<name>            remove the user and revoke the certificate
      GET    /users/<name>/profile    the OpenVPN client profile
      POST   /users/<name>/otp        a new TOTP secret and its otpauth URI; nothing is stored
      PUT    /users/<name>/otp        store a TOTP secret: {"secret", "code"}, code a code of the secret
      DELETE /users/<name>/otp        remove the TOTP token
    """
    if not state.get('users'):
        return 404, {'error': 'no P2S system with users is configured'}
    if body == 'invalid' or (body is not None and not isinstance(body, dict)):
        return 400, {'error': 'the body must be a JSON object'}
    body = body or {}
    password = body.get('password')
    if password is not None and (not isinstance(password, str) or '\n' in password or '\r' in password):
        return 400, {'error': 'the password must be a string of one line'}
    parts = path.strip('/').split('/')
    route = (method, len(parts), parts[2] if len(parts) == 3 else None)
-    if route == ('GET', 1, None):
+    name = parts[1] if len(parts) > 1 else None
-        return user_tool('list')
+    actions = {
-    if route == ('POST', 1, None):
+        ('GET', 1, None): lambda connection: vpnrouter_users.list_users(connection),
-        name = body.get('name')
+        ('POST', 1, None): lambda connection: vpnrouter_users.add(connection, body.get('name'), body.get('password'), body),
-        if not isinstance(name, str) or password is None:
+        ('PUT', 2, None): lambda connection: vpnrouter_users.update(connection, name, body),
-            return 400, {'error': 'name and password are required strings'}
+        ('PUT', 3, 'password'): lambda connection: vpnrouter_users.passwd(connection, name, body.get('password')),
-        code, document = user_tool('add', name, password=password)
+        ('POST', 3, 'disable'): lambda connection: vpnrouter_users.disable(connection, name),
-        return (201 if code == 200 else code), document
+        ('POST', 3, 'enable'): lambda connection: vpnrouter_users.enable(connection, name),
-    if route == ('PUT', 3, 'password'):
+        ('POST', 3, 'unlock'): lambda connection: vpnrouter_users.unlock(connection, name),
-        if password is None:
+        ('DELETE', 2, None): lambda connection: vpnrouter_users.remove(connection, name),
-            return 400, {'error': 'password is a required string'}
+        ('GET', 3, 'profile'): lambda connection: vpnrouter_users.profile(connection, name),
-        return user_tool('passwd', parts[1], password=password)
+        ('POST', 3, 'otp'): lambda connection: vpnrouter_users.otp_new(connection, name),
-    if route in (('POST', 3, 'disable'), ('POST', 3, 'enable')):
+        ('PUT', 3, 'otp'): lambda connection: vpnrouter_users.otp_enrol(connection, name, body.get('secret'), body.get('code')),
-        return user_tool(parts[2], parts[1])
+        ('DELETE', 3, 'otp'): lambda connection: vpnrouter_users.otp_remove(connection, name),
-    if route == ('DELETE', 2, None):
+    }
-        return user_tool('remove', parts[1])
+    if route not in actions:
-    if route == ('GET', 3, 'profile'):
+        return 404, {'error': f'no route {method} {path}'}
-        return user_tool('profile', parts[1])
+    try:
-    return 404, {'error': f'no route {method} {path}'}
+        with vpnrouter_users.session() as connection:
            document = actions[route](connection)
    except vpnrouter_users.Failure as failure:
        return USER_STATUS.get(failure.status, 500), {'error': str(failure)}
    except (ldap.LDAPError, OSError) as exc:
        return 500, {'error': f'the user directory failed: {exc}'}
    except vpnrouter_auth.DatabaseError as exc:
        return 500, {'error': str(exc)}
    except SystemExit as exc:
        # vpnrouter ends with SystemExit, carrying the reason, on a configuration it cannot read.
        return 500, {'error': str(exc.code)}
    return (201 if route == ('POST', 1, None) else 200), document
 def groups(state, method, path, body):
    """Answer a /groups request with vpnrouter_users: the HTTP status and the JSON document.
      GET    /groups                          the groups with their members
      POST   /groups                          add a group: {"name", "members": [...], "description"?}
      PUT    /groups/<name>                   rename it or change the description: {"name"?, "description"?}
      DELETE /groups/<name>                   remove the group; its members are kept
      POST   /groups/<name>/members           add a member: {"user": ...}
      DELETE /groups/<name>/members/<user>    take a member out; the last member cannot leave
    """
    if body == 'invalid' or (body is not None and not isinstance(body, dict)):
        return 400, {'error': 'the body must be a JSON object'}
    body = body or {}
    parts = path.strip('/').split('/')
    name = parts[1] if len(parts) > 1 else None
    route = (method, len(parts), parts[2] if len(parts) > 2 else None)
    actions = {
        ('GET', 1, None): lambda connection: vpnrouter_users.list_groups(connection),
        ('POST', 1, None): lambda connection: vpnrouter_users.group_add(
            connection, body.get('name'), body.get('description'), body.get('members')),
        ('PUT', 2, None): lambda connection: vpnrouter_users.group_update(
            connection, name, body.get('name'), body.get('description')),
        ('DELETE', 2, None): lambda connection: vpnrouter_users.group_remove(connection, name),
        ('POST', 3, 'members'): lambda connection: vpnrouter_users.group_member_add(connection, name, body.get('user')),
        ('DELETE', 4, 'members'): lambda connection: vpnrouter_users.group_member_remove(connection, name, parts[3]),
    }
    if route not in actions:
        return 404, {'error': f'no route {method} {path}'}
    try:
        with vpnrouter_users.session() as connection:
            document = actions[route](connection)
    except vpnrouter_users.Failure as failure:
        return USER_STATUS.get(failure.status, 500), {'error': str(failure)}
    except (ldap.LDAPError, OSError) as exc:
        return 500, {'error': f'the user directory failed: {exc}'}
    return (201 if route == ('POST', 1, None) else 200), document
 def changes_admins(connection, method, parts, body):
    """Whether a /users or /groups request creates, renames to, changes or removes ADMIN_GROUP, or removes a member of it."""
    admins = vpnrouter_users.ADMIN_GROUP
    if parts[0] == 'groups':
        if method == 'POST' and len(parts) == 1:
            return body.get('name') == admins
        if method != 'GET' and len(parts) > 1 and parts[1] == admins:
            return True
        return method == 'PUT' and len(parts) == 2 and body.get('name') == admins
    if parts[0] == 'users' and method == 'DELETE' and len(parts) == 2:
        return admins in vpnrouter_users.groups_of(connection, parts[1])
    return False
 def refusal(claims, method, path, body):
    """Why the holder of the claims may not make the request to a protected path, or '' when it may."""
    if claims.get('admin') is True:
        return ''
    name = claims.get('sub')
    if not isinstance(name, str):
        return 'the token names no user'
    admins = vpnrouter_users.ADMIN_GROUP
    with vpnrouter_users.session() as connection:
        if not vpnrouter_users.is_admin(connection, name):
            return f'user {name} is not a member of {admins}'
        if changes_admins(connection, method, path.strip('/').split('/'), body if isinstance(body, dict) else {}):
            return f'only a token of vpn-router-token may change the group {admins} or remove its members'
    return ''
 def auth(path, body):
    """Answer a POST to /auth with vpnrouter_users and vpnrouter_auth: the HTTP status and the JSON document.
      POST /auth           sign in: {"name", "password", "code"?}; the tokens
      POST /auth/refresh   {"refresh_token"}; new tokens, the refresh token used up
      POST /auth/logout    {"refresh_token"}; the refresh token revoked
    """
    if not isinstance(body, dict):
        return 400, {'error': 'the body must be a JSON object'}
    try:
        cfg = vpnrouter.load_config()
        if path == '/auth':
            vpnrouter_users.login(body.get('name'), body.get('password'), body.get('code'))
            return 200, vpnrouter_auth.issue(cfg, body['name'])
        name = vpnrouter_auth.take(cfg, body.get('refresh_token'))
        if path == '/auth/logout':
            return 200, {'revoked': name is not None}
        if name is None:
            return 401, {'error': 'the refresh token is not valid'}
        with vpnrouter_users.session() as connection:
            user = vpnrouter_users.summary(vpnrouter_users.entry(connection, name))
        if not user['enabled'] or user['locked']:
            return 401, {'error': f'user {name} is disabled or locked'}
        return 200, vpnrouter_auth.issue(cfg, name)
    except vpnrouter_users.Failure as failure:
        return (401 if failure.status == 3 else USER_STATUS.get(failure.status, 500)), {'error': str(failure)}
    except (ldap.LDAPError, OSError) as exc:
        return 500, {'error': f'the user directory failed: {exc}'}
    except vpnrouter_auth.DatabaseError as exc:
        return 500, {'error': str(exc)}
    except SystemExit as exc:
        return 500, {'error': str(exc.code)}
 def auth_wait(address):
    """Seconds until the client address may call /auth again, 0 when it may now; a call that may is counted."""
    now = time.monotonic()
    with _auth_lock:
        for idle in [key for key, times in _auth_calls.items() if times[-1] <= now - AUTH_WINDOW]:
            del _auth_calls[idle]
        calls = _auth_calls[address]
        while calls and calls[0] <= now - AUTH_WINDOW:
            calls.popleft()
        if len(calls) >= AUTH_LIMIT:
            return int(calls[0] + AUTH_WINDOW - now) + 1
        calls.append(now)
        return 0
 class Handler(BaseHTTPRequestHandler):
@@ -475,28 +575,65 @@ class Handler(BaseHTTPRequestHandler):
                _cache['at'] = time.monotonic()
            return _cache['data']
-    def authorized(self, secret):
+    def claims(self, secret):
-        """Whether the request carries a token signed with the secret; False also while no secret is set."""
+        """The claims of the request's token signed with the secret, or None without an acceptable one."""
        header = self.headers.get('Authorization', '')
-        return bool(secret) and header.startswith('Bearer ') and verify(header[7:].strip(), secret) is not None
+        return vpnrouter_auth.verify(header[7:].strip(), secret) if header.startswith('Bearer ') else None
    def client(self):
        """The client address: the last X-Forwarded-For entry when the peer is a trusted proxy, else the peer."""
        peer = self.client_address[0]
        trusted = section(read_json(CONFIG_FILE), 'web').get('trusted_proxies') or [self.server.server_address[0]]
        forwarded = self.headers.get('X-Forwarded-For', '')
        if isinstance(trusted, list) and peer in trusted and forwarded.strip():
            return forwarded.split(',')[-1].strip()
        return peer
    def refusal(self, claims, path, payload):
        """The HTTP status and the message that refuse a request to a protected path, or None when it may be made."""
        try:
            reason = refusal(claims, self.command, path, payload)
        except (ldap.LDAPError, OSError) as exc:
            return 500, f'the user directory failed: {exc}'
        return (403, reason) if reason else None
    def answer(self, send_body):
        path = self.path.split('?', 1)[0]
        extra = []
        self.principal = None
        # A body that is not read must not be taken for the next request.
        if self.command not in ('GET', 'HEAD'):
            self.close_connection = True
-        protected = path == '/config' or path == '/users' or path.startswith('/users/')
+        directory = next((name for name in ('users', 'groups') if path == f'/{name}' or path.startswith(f'/{name}/')), None)
-        state = current() if protected else {}
+        protected = path == '/config' or directory is not None
-        if protected and not state['jwt_secret']:
+        signing = path in ('/auth', '/auth/refresh', '/auth/logout')
        state = current() if protected or signing else {}
        claims = self.claims(state['key']) if protected and state['key'] else None
        payload = self.request_body() if (protected or signing) and self.command not in ('GET', 'HEAD') else None
        if claims is not None:
            self.principal = 'admin' if claims.get('admin') is True else claims.get('sub')
        if (protected or signing) and not state['key']:
            body, kind, code = b'Not found\n', 'text/plain; charset=utf-8', 404
-        elif protected and not self.authorized(state['jwt_secret']):
+        elif signing and self.command != 'POST':
            body, kind, code = b'Method not allowed\n', 'text/plain; charset=utf-8', 405
        elif signing:
            wait = auth_wait(self.client()) if path != '/auth/logout' else 0
            if wait:
                code, document = 429, {'error': f'too many sign-in calls; retry in {wait} seconds'}
                extra.append(('Retry-After', str(wait)))
            else:
                code, document = auth(path, payload)
            body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
        elif protected and claims is None:
            body, kind, code = b'Unauthorized\n', 'text/plain; charset=utf-8', 401
            extra.append(('WWW-Authenticate', 'Bearer'))
        elif protected and (refused := self.refusal(claims, path, payload)):
            code, kind = refused[0], 'application/json'
            body = json.dumps({'error': refused[1]}, indent=2).encode() + b'\n'
        elif path == '/config' and self.command in ('GET', 'HEAD'):
            body, kind, code = json.dumps(details(state), indent=2).encode() + b'\n', 'application/json', 200
        elif protected and path != '/config':
-            code, document = users(state, self.command, path, self.request_body())
+            code, document = (users if directory == 'users' else groups)(state, self.command, path, payload)
            body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
        elif self.command not in ('GET', 'HEAD'):
            body, kind, code = b'Method not allowed\n', 'text/plain; charset=utf-8', 405
@@ -550,66 +687,45 @@ class Handler(BaseHTTPRequestHandler):
        self.answer(True)
    def log_message(self, format, *args):
-        sys.stderr.write('%s - - [%s] %s\n' % (self.client_address[0], self.log_date_time_string(), format % args))
+        principal = getattr(self, 'principal', None) or '-'
-
+        sys.stderr.write('%s - %s [%s] %s\n' % (self.client_address[0], principal, self.log_date_time_string(), format % args))
 def wanted():
    """Where the service should listen now, as (address, port), or the reason it should not listen."""
    configuration = read_json(CONFIG_FILE)
    web = section(configuration, 'web')
    if web.get('enabled', True) is False:
        return None, 'the web service is disabled in the configuration'
    port = web.get('port', DEFAULT_PORT)
    if isinstance(port, bool) or not isinstance(port, int) or not 1 <= port <= 65535:
        port = DEFAULT_PORT
    general, interfaces, local = (section(configuration, name) for name in ('general', 'interfaces', 'local'))
    address = vpnrouter_net.internal_address(str(general.get('mode') or 'manual'), str(interfaces.get('internal') or ''),
                                             str(local.get('int_addr') or ''))
    if not address:
        return None, 'the internal address is not known yet'
    return (address, port), ''
 def log(message):
    print(f'vpn-router-web: {message}', file=sys.stderr, flush=True)
-def discover():
+def initialize():
-    """Work out where to listen every DISCOVERY_SECONDS, and run the server thread there.
+    """Make the signing key and the refresh tokens' database, retrying every INIT_RETRY_SECONDS until both are ready."""
-
+    reported = None
    When the address or the port changes, the server thread is stopped and a new one listens at
    the new place; while the service is disabled, the address is not known or it cannot be bound,
    no server runs and the next round tries again.
    """
    server, thread, listening, reported = None, None, None, None
    while True:
-        target, reason = wanted()
+        phase = 'reading the configuration'
-        if target != listening and server is not None:
+        try:
-            server.shutdown()
+            cfg = vpnrouter.load_config()
-            server.server_close()
+            phase = 'making the signing key'
-            thread.join()
+            vpnrouter_auth.ensure_key()
-            server, listening = None, None
+            phase = 'creating the token database'
-        if target is not None and server is None:
+            vpnrouter_auth.provision(cfg)
-            try:
+            phase = 'connecting to the token database'
-                server = ThreadingHTTPServer(target, Handler)
+            with vpnrouter_auth.database(cfg):
-            except OSError as exc:
+                pass
-                reason = f'cannot listen on {target[0]}:{target[1]}: {exc}'
+        except (OSError, SystemExit, vpnrouter_auth.DatabaseError) as exc:
-            else:
+            error = f'{phase} failed: {exc.code if isinstance(exc, SystemExit) else exc}'
-                thread = threading.Thread(target=server.serve_forever, name='server', daemon=True)
+            if error != reported:
-                thread.start()
+                log(f'{error}; retrying every {INIT_RETRY_SECONDS} seconds')
-                listening, reason = target, f'serving on http://{target[0]}:{target[1]}'
+                reported = error
-        if reason and reason != reported:
+            time.sleep(INIT_RETRY_SECONDS)
-            log(reason)
+            continue
-            reported = reason
+        log('ready')
-        time.sleep(DISCOVERY_SECONDS)
+        return
 def main():
    signal.signal(signal.SIGTERM, lambda *_: sys.exit(0))
-    discovery = threading.Thread(target=discover, name='discovery', daemon=True)
+    initialize()
    discovery.start()
    try:
-        discovery.join()
+        vpnrouter_serve.serve(lambda: vpnrouter_net.web_listen(read_json(CONFIG_FILE), DEFAULT_PORT), Handler, log,
                              DISCOVERY_SECONDS)
    except KeyboardInterrupt:
        pass
    return 0
@@ -0,0 +1,120 @@
 #!/usr/bin/python3
 """Manage the groups of the router's directory from the command line, with vpnrouter_users.
 A group has a name, an optional description and at least one member, each a user of the
 directory. The answer is text for people on standard output, and a failure is a message on
 standard error. With --json the answer is JSON on standard output, and a failure is
 {"error": ...} there too. The exit status says what failed: 2 the request is invalid, 3 the
 group or the user does not exist, 4 the group exists, 1 anything else. -h or --help shows the
 commands and their options.
  vpn-router-group [--json] list
  vpn-router-group [--json] add <name> --member U [--member U ...] [--description D]
  vpn-router-group [--json] update <name> [--name N] [--description D]
  vpn-router-group [--json] remove <name>
  vpn-router-group [--json] member-add <name> <user>
  vpn-router-group [--json] member-remove <name> <user>
 """
 import argparse
 import json
 import sys
 sys.path.insert(0, '/usr/lib/vpn-router')
 import ldap
 import vpnrouter_users
 from vpnrouter_users import Failure
 #: The help line of each command.
 HELP = {
    'list': 'list the groups with their members',
    'add': 'add a group with at least one member',
    'update': 'rename the group or change its description',
    'remove': 'remove the group; its members are kept',
    'member-add': 'add a user to the group',
    'member-remove': 'take a user out of the group; the last member cannot leave',
 }
 class Parser(argparse.ArgumentParser):
    """An argument parser whose errors are Failures with status 2, so that --json answers them as JSON."""
    def error(self, message):
        raise Failure(f'{message}; see {self.prog} --help', 2)
 def parse(argv):
    parser = Parser(prog='vpn-router-group', description='Manage the groups of the router.')
    parser.add_argument('--json', action='store_true', help='answer in JSON')
    commands = parser.add_subparsers(dest='command', required=True, metavar='command', parser_class=Parser)
    for command, text in HELP.items():
        sub = commands.add_parser(command, help=text, description=text[0].upper() + text[1:] + '.')
        if command != 'list':
            sub.add_argument('name', help='the group name')
        if command == 'add':
            sub.add_argument('--member', dest='members', action='append', required=True, metavar='USER',
                             help='a member; give it once for each member')
            sub.add_argument('--description', metavar='TEXT', help='the description')
        if command == 'update':
            sub.add_argument('--name', dest='new_name', metavar='NAME', help='the new group name')
            sub.add_argument('--description', metavar='TEXT', help="the description; '' removes it")
        if command in ('member-add', 'member-remove'):
            sub.add_argument('user', help='the user name')
    return parser.parse_args(argv)
 def run(connection, options):
    """The answer of the command the options name."""
    command = options.command
    if command == 'list':
        return vpnrouter_users.list_groups(connection)
    if command == 'add':
        return vpnrouter_users.group_add(connection, options.name, options.description, options.members)
    if command == 'update':
        return vpnrouter_users.group_update(connection, options.name, options.new_name, options.description)
    if command == 'remove':
        return vpnrouter_users.group_remove(connection, options.name)
    if command == 'member-add':
        return vpnrouter_users.group_member_add(connection, options.name, options.user)
    return vpnrouter_users.group_member_remove(connection, options.name, options.user)
 def text_of(answer):
    """The answer as text for people."""
    if answer.get('removed'):
        return f'group {answer["name"]} removed'
    groups = answer['groups'] if 'groups' in answer else [answer]
    if not groups:
        return 'no groups'
    head = ('GROUP', 'MEMBERS', 'DESCRIPTION')
    rows = [(group['name'], ', '.join(group['members']), group['description'] or '-') for group in groups]
    widths = [max(len(row[column]) for row in (head, *rows)) for column in range(len(head))]
    return '\n'.join('  '.join(cell.ljust(width) for cell, width in zip(row, widths)).rstrip() for row in (head, *rows))
 def main(argv):
    # A request that cannot be parsed is answered in JSON when --json is among the arguments.
    as_json = '--json' in argv
    try:
        options = parse(argv)
        as_json = options.json
        with vpnrouter_users.session() as connection:
            answer = run(connection, options)
    except Failure as failure:
        message, status = str(failure), failure.status
    except (ldap.LDAPError, OSError) as exc:
        message, status = f'the user directory failed: {exc}', 1
    else:
        print(json.dumps(answer) if as_json else text_of(answer))
        return 0
    if as_json:
        print(json.dumps({'error': message}))
    else:
        print(f'vpn-router-group: {message}', file=sys.stderr)
    return status
 if __name__ == '__main__':
    sys.exit(main(sys.argv[1:]))
@@ -0,0 +1,79 @@
 #!/usr/bin/python3
 """Make a token with the admin claim for the router's web service, with vpnrouter_auth.
 The token is signed with the router's key and is valid for the lifetime given, without a
 maximum. It may call every protected path, and it is the only token that may change the
 admins group. The answer is the token on standard output, and a failure is a message on
 standard error. With --json the answer is {"token", "expires"} on standard output, and a
 failure is {"error": ...} there too. The exit status is 2 for an invalid request and 1 for
 any other failure. -h or --help shows the options.
  vpn-router-token [--json] (--minutes N | --hours N | --days N)
 """
 import argparse
 import json
 import sys
 sys.path.insert(0, '/usr/lib/vpn-router')
 import vpnrouter_auth
 #: The seconds of each unit of the lifetime.
 UNITS = {'minutes': 60, 'hours': 3600, 'days': 86400}
 class Failure(Exception):
    """A request that cannot be done; status is 2 invalid, 1 other."""
    def __init__(self, message, status=1):
        super().__init__(message)
        self.status = status
 class Parser(argparse.ArgumentParser):
    """An argument parser whose errors are Failures with status 2, so that --json answers them as JSON."""
    def error(self, message):
        raise Failure(f'{message}; see {self.prog} --help', 2)
 def positive(value):
    try:
        number = int(value)
    except ValueError:
        number = 0
    if number < 1:
        raise argparse.ArgumentTypeError(f'{value!r} is not a positive whole number')
    return number
 def main(argv):
    # A request that cannot be parsed is answered in JSON when --json is among the arguments.
    as_json = '--json' in argv
    try:
        parser = Parser(prog='vpn-router-token', description='Make a token with the admin claim for the web service.')
        parser.add_argument('--json', action='store_true', help='answer in JSON')
        lifetime = parser.add_mutually_exclusive_group(required=True)
        for unit in UNITS:
            lifetime.add_argument(f'--{unit}', type=positive, metavar='N', help=f'valid for N {unit}')
        options = parser.parse_args(argv)
        as_json = options.json
        seconds = next(getattr(options, unit) * factor for unit, factor in UNITS.items() if getattr(options, unit))
        answer = vpnrouter_auth.admin_token(seconds)
    except Failure as failure:
        message, status = str(failure), failure.status
    except OSError as exc:
        message, status = str(exc), 1
    else:
        print(json.dumps(answer) if as_json else answer['token'])
        return 0
    if as_json:
        print(json.dumps({'error': message}))
    else:
        print(f'vpn-router-token: {message}', file=sys.stderr)
    return status
 if __name__ == '__main__':
    sys.exit(main(sys.argv[1:]))
@@ -1,224 +1,150 @@
 #!/usr/bin/python3
-"""Manage the users of the router's P2S systems.
+"""Manage the users of the router's directory from the command line, with vpnrouter_users.
-A user is an entry in the router's directory, whose password OpenVPN checks, and a client
+Passwords and TOTP secrets are read from the first line of standard input, never from the
-certificate of the router CA with the user name as its common name, which OpenVPN and
+command line. The
-IKEv2 accept. Passwords are read from the first line of standard input, never from the
+answer is text for people on standard output, and a failure is a message on standard error.
-command line. The answer is JSON on standard output; on failure it is {"error": ...} and
+With --json the answer is JSON on standard output, and a failure is {"error": ...} there too.
-the exit status says why: 2 the request is invalid, 3 the user does not exist, 4 the user
+The exit status says what failed: 2 the request is invalid, 3 the user does not exist, 4 the
-exists, 1 anything else.
+user exists, 1 anything else. -h or --help shows the commands and their options.
-  vpn-router-user list
+  vpn-router-user [--json] list
-  vpn-router-user add <name>          (password on stdin)
+  vpn-router-user [--json] add <name> --first-name F --last-name L --email E   (password on stdin)
-  vpn-router-user passwd <name>       (password on stdin)
+  vpn-router-user [--json] update <name> [--first-name F] [--last-name L] [--email E]
-  vpn-router-user disable <name>
+  vpn-router-user [--json] passwd <name>       (password on stdin)
-  vpn-router-user enable <name>
+  vpn-router-user [--json] disable <name>
-  vpn-router-user remove <name>
+  vpn-router-user [--json] enable <name>
-  vpn-router-user profile <name>
+  vpn-router-user [--json] unlock <name>
  vpn-router-user [--json] remove <name>
  vpn-router-user [--json] profile <name>
  vpn-router-user [--json] otp-new <name>
  vpn-router-user [--json] otp-enrol <name> <code>   (secret on stdin)
  vpn-router-user [--json] otp-remove <name>
 """
-import fcntl
+import argparse
 import json
 import pathlib
 import re
 import subprocess
 import sys
 sys.path.insert(0, '/usr/lib/vpn-router')
 import ldap
 import ldap.modlist
-import vpnrouter
+import vpnrouter_auth
-import vpnrouter_ldap
+import vpnrouter_users
 from vpnrouter_users import Failure
-#: A user name is one lowercase DNS label, as it is also the common name of the certificate.
+#: The help line of each command.
-NAME = re.compile(r'^[a-z][a-z0-9-]{0,30}[a-z0-9]$|^[a-z]$')
+HELP = {
-PASSWORD_MIN = 8
+    'list': 'list the users',
-PASSWORD_MAX = 128
+    'add': 'add a user and issue the certificate; the password is read from standard input',
-#: The value of pwdAccountLockedTime that locks an account until it is unlocked.
+    'update': 'change the first name, the last name or the e-mail address',
-LOCKED = b'000001010000Z'
+    'passwd': 'set the password, read from standard input',
-LOCK_FILE = pathlib.Path('/run/vpn-router/users.lock')
+    'disable': 'lock the user out of OpenVPN',
    'enable': 'let the user in again',
    'unlock': 'end a lockout after failed logins',
    'remove': 'revoke the certificate and remove the user',
    'profile': 'print the OpenVPN client profile',
    'otp-new': 'make a TOTP secret and its otpauth URI; nothing is stored',
    'otp-enrol': 'store the TOTP secret read from standard input, when the code is a code of it',
    'otp-remove': 'remove the TOTP token',
 }
 #: The help of the person options.
 PERSON_HELP = {'first_name': 'the first name', 'last_name': 'the last name', 'email': 'the e-mail address'}
-class Failure(Exception):
+class Parser(argparse.ArgumentParser):
-    """A request that cannot be done; status is the exit status."""
+    """An argument parser whose errors are Failures with status 2, so that --json answers them as JSON."""
-    def __init__(self, message, status=1):
+    def error(self, message):
-        super().__init__(message)
+        raise Failure(f'{message}; see {self.prog} --help', 2)
        self.status = status
-def dn_of(name):
+def parse(argv):
-    return f'uid={name},{vpnrouter_ldap.USERS}'
+    parser = Parser(prog='vpn-router-user', description='Manage the users of the router.')
-
+    parser.add_argument('--json', action='store_true', help='answer in JSON')
-
+    commands = parser.add_subparsers(dest='command', required=True, metavar='command', parser_class=Parser)
-def certificate(name):
+    for command, text in HELP.items():
-    return vpnrouter.USER_CERT_DIR / f'{name}_cert.pem', vpnrouter.USER_CERT_DIR / f'{name}_key.pem'
+        sub = commands.add_parser(command, help=text, description=text[0].upper() + text[1:] + '.')
-
+        if command != 'list':
-
+            sub.add_argument('name', help='the user name')
-def entry(connection, name):
+        if command in ('add', 'update'):
-    """The attributes of the user's entry; Failure with status 3 when there is none."""
+            for key, _ in vpnrouter_users.PERSON:
-    try:
+                sub.add_argument(f'--{key.replace("_", "-")}', dest=key, metavar='TEXT', required=command == 'add',
-        return connection.search_s(dn_of(name), ldap.SCOPE_BASE, attrlist=['uid', 'pwdAccountLockedTime'])[0][1]
+                                 help=PERSON_HELP[key])
-    except ldap.NO_SUCH_OBJECT:
+        if command == 'otp-enrol':
-        raise Failure(f'user {name} does not exist', 3)
+            sub.add_argument('code', help='a code of the secret from the authenticator')
    return parser.parse_args(argv)
 def password():
-    """The password from the first line of standard input."""
+    """The password or the TOTP secret from the first line of standard input."""
-    value = sys.stdin.readline().rstrip('\n')
+    return sys.stdin.readline().rstrip('\n')
    if not PASSWORD_MIN <= len(value) <= PASSWORD_MAX:
        raise Failure(f'the password must have {PASSWORD_MIN} to {PASSWORD_MAX} characters', 2)
    return value
-def simple_ca(*args):
+def run(connection, options, secret):
-    result = subprocess.run([sys.executable, str(vpnrouter.SIMPLE_CA), *args], capture_output=True, text=True)
+    """The answer of the command the options name; secret is the password of add and passwd, the TOTP secret of otp-enrol."""
-    if result.returncode != 0:
+    command = options.command
-        raise Failure(f'simple-ca {args[0]} failed: {result.stderr.strip() or result.stdout.strip()}')
+    if command == 'list':
        return vpnrouter_users.list_users(connection)
    values = {key: getattr(options, key, None) for key, _ in vpnrouter_users.PERSON}
    if command == 'add':
        return vpnrouter_users.add(connection, options.name, secret, values)
    if command == 'update':
        return vpnrouter_users.update(connection, options.name, values)
    if command == 'passwd':
        return vpnrouter_users.passwd(connection, options.name, secret)
    if command == 'otp-enrol':
        return vpnrouter_users.otp_enrol(connection, options.name, secret, options.code)
    return getattr(vpnrouter_users, command.replace('-', '_'))(connection, options.name)
-def publish_crl():
+def text_of(answer):
-    """Give strongSwan the CRL after a revocation; OpenVPN reads it at every connection."""
+    """The answer as text for people."""
-    if vpnrouter.install_file(vpnrouter.PKI_DIR / 'crl.pem', vpnrouter.CRL_DEST, 0o644):
+    if 'profile' in answer:
-        subprocess.run(['swanctl', '--load-creds'], capture_output=True, text=True)
+        return answer['profile'].rstrip('\n')
-
+    if 'secret' in answer:
-
+        return f'secret  {answer["secret"]}\nuri     {answer["uri"]}'
-def summary(attributes):
+    if answer.get('removed'):
-    name = attributes['uid'][0].decode()
+        return f'user {answer["name"]} removed'
-    return {
+    users = answer['users'] if 'users' in answer else [answer]
-        'name': name,
+    if not users:
-        'enabled': 'pwdAccountLockedTime' not in attributes,
+        return 'no users'
-        'certificate': certificate(name)[0].is_file(),
+    head = ('USER', 'STATE', 'CERTIFICATE', 'OTP', 'FIRST NAME', 'LAST NAME', 'E-MAIL')
-    }
+    rows = [(user['name'], 'disabled' if not user['enabled'] else 'locked' if user['locked'] else 'enabled',
-
+             'issued' if user['certificate'] else 'missing',
-
+             'enrolled' if user['otp'] else '-', user['first_name'] or '-', user['last_name'] or '-', user['email'] or '-')
-def list_users(connection):
+            for user in users]
-    found = connection.search_s(vpnrouter_ldap.USERS, ldap.SCOPE_ONELEVEL, '(objectClass=inetOrgPerson)',
+    widths = [max(len(row[column]) for row in (head, *rows)) for column in range(len(head))]
-                                ['uid', 'pwdAccountLockedTime'])
+    return '\n'.join('  '.join(cell.ljust(width) for cell, width in zip(row, widths)).rstrip() for row in (head, *rows))
    return {'users': sorted((summary(attributes) for _, attributes in found), key=lambda user: user['name'])}
 def add(connection, name):
    secret = password()
    if not (vpnrouter.PKI_DIR / 'ca_key.pem').is_file():
        raise Failure('the router CA does not exist; enable OpenVPN or IKEv2 road-warrior access first')
    try:
        connection.add_s(dn_of(name), ldap.modlist.addModlist({
            'objectClass': [b'inetOrgPerson'], 'uid': [name.encode()], 'cn': [name.encode()], 'sn': [name.encode()],
        }))
    except ldap.ALREADY_EXISTS:
        raise Failure(f'user {name} exists', 4)
    connection.passwd_s(dn_of(name), None, secret)
    vpnrouter.USER_CERT_DIR.mkdir(mode=0o700, parents=True, exist_ok=True)
    for path in certificate(name):
        path.unlink(missing_ok=True)
    simple_ca('make-cert', '--ca-dir', str(vpnrouter.PKI_DIR), '--cert-dir', str(vpnrouter.USER_CERT_DIR), name)
    return summary(entry(connection, name))
 def passwd(connection, name):
    secret = password()
    entry(connection, name)
    connection.passwd_s(dn_of(name), None, secret)
    return summary(entry(connection, name))
 def disable(connection, name):
    if 'pwdAccountLockedTime' not in entry(connection, name):
        connection.modify_s(dn_of(name), [(ldap.MOD_REPLACE, 'pwdAccountLockedTime', [LOCKED])])
    return summary(entry(connection, name))
 def enable(connection, name):
    if 'pwdAccountLockedTime' in entry(connection, name):
        connection.modify_s(dn_of(name), [(ldap.MOD_DELETE, 'pwdAccountLockedTime', None)])
    return summary(entry(connection, name))
 def remove(connection, name):
    entry(connection, name)
    cert, key = certificate(name)
    if cert.is_file():
        simple_ca('revoke-cert', '--ca-dir', str(vpnrouter.PKI_DIR), str(cert))
        simple_ca('make-crl', '--ca-dir', str(vpnrouter.PKI_DIR), '--days', str(vpnrouter.CRL_DAYS))
        publish_crl()
    cert.unlink(missing_ok=True)
    key.unlink(missing_ok=True)
    connection.delete_s(dn_of(name))
    return {'name': name, 'removed': True}
 def profile(connection, name):
    """The OpenVPN client profile of the user, with the CA, the certificate and the key inline."""
    entry(connection, name)
    cfg = vpnrouter.load_config()
    if not cfg['openvpn']['enabled']:
        raise Failure('OpenVPN is not enabled', 2)
    cert, key = certificate(name)
    if not cert.is_file() or not key.is_file():
        raise Failure(f'user {name} has no certificate')
    fqdn = cfg['wan']['local_fqdn'].strip()
    text = '\n'.join([
        'client',
        'dev tun',
        f'proto {cfg["openvpn"]["protocol"]}',
        f'remote {fqdn} {cfg["openvpn"]["port"]}',
        'nobind',
        'persist-key',
        'persist-tun',
        'remote-cert-tls server',
        f'verify-x509-name {fqdn} name',
        'auth-user-pass',
        'auth-nocache',
        'verb 3',
        '<ca>', extract_pem(vpnrouter.CA_CERT_DEST.read_text(encoding='utf-8')), '</ca>',
        '<cert>', extract_pem(cert.read_text(encoding='utf-8')), '</cert>',
        '<key>', key.read_text(encoding='utf-8').strip(), '</key>',
        '',
    ])
    return {'name': name, 'profile': text}
 def extract_pem(text):
    """The PEM blocks of the certificates in a file that also holds their text form."""
    begin, end = '-----BEGIN CERTIFICATE-----', '-----END CERTIFICATE-----'
    blocks = []
    start = text.find(begin)
    while start != -1:
        stop = text.index(end, start) + len(end)
        blocks.append(text[start:stop])
        start = text.find(begin, stop)
    return '\n'.join(blocks)
 COMMANDS = {'list': list_users, 'add': add, 'passwd': passwd, 'disable': disable, 'enable': enable,
            'remove': remove, 'profile': profile}
 def main(argv):
    # A request that cannot be parsed is answered in JSON when --json is among the arguments.
    as_json = '--json' in argv
    try:
-        if not argv or argv[0] not in COMMANDS or len(argv) != (1 if argv[0] == 'list' else 2):
+        options = parse(argv)
-            raise Failure('usage: vpn-router-user list | add|passwd|disable|enable|remove|profile <name>', 2)
+        as_json = options.json
-        if len(argv) == 2 and not NAME.match(argv[1]):
+        if options.command != 'list':
-            raise Failure('a user name is 1 to 32 lowercase letters, digits and hyphens, starting with a letter', 2)
+            vpnrouter_users.check_name(options.name)
-        LOCK_FILE.parent.mkdir(parents=True, exist_ok=True)
+        secret = password() if options.command in ('add', 'passwd', 'otp-enrol') else None
-        with open(LOCK_FILE, 'w') as lock:
+        with vpnrouter_users.session() as connection:
-            fcntl.flock(lock, fcntl.LOCK_EX)
+            answer = run(connection, options, secret)
            connection = vpnrouter_ldap.manager()
            try:
                answer = COMMANDS[argv[0]](connection, *argv[1:])
            finally:
                connection.unbind_s()
    except Failure as failure:
-        print(json.dumps({'error': str(failure)}))
+        message, status = str(failure), failure.status
        return failure.status
    except (ldap.LDAPError, OSError) as exc:
-        print(json.dumps({'error': f'the user directory failed: {exc}'}))
+        message, status = f'the user directory failed: {exc}', 1
-        return 1
+    except vpnrouter_auth.DatabaseError as exc:
-    print(json.dumps(answer))
+        message, status = str(exc), 1
-    return 0
+    else:
        print(json.dumps(answer) if as_json else text_of(answer))
        return 0
    if as_json:
        print(json.dumps({'error': message}))
    else:
        print(f'vpn-router-user: {message}', file=sys.stderr)
    return status
 if __name__ == '__main__':
@@ -45,7 +45,8 @@
  },
  "web": {
    "enabled": true,
-    "port": 8080,
+    "port": 7081,
-    "jwt_secret": ""
+    "database": "",
    "trusted_proxies": []
  }
 }
Author	SHA1	Message	Date
slawek	b1e6d9cd0b	fix: the web service starts once libpq can be found. Release / release (push) Successful in 12s Details	2026-10-24 11:07:45 +02:00
slawek	732cc8cb54	fix: the web service waits for the PostgreSQL driver instead of failing at start. Release / release (push) Successful in 25s Details	2026-10-24 10:43:51 +02:00
slawek	c0cd323a0c	feature: bootstrap diagnostics on port 17081 during the first setup. Release / release (push) Successful in 13s Details	2026-10-24 10:07:22 +02:00
slawek	2e2bacee7d	feature: the web service creates its key and database itself and shows its starting phase. Release / release (push) Successful in 29s Details	2026-10-24 09:07:28 +02:00
slawek	4f1aa282f6	fix: a failing user directory or token database no longer leaves the tunnels down. Release / release (push) Failing after 15m34s Details	2026-10-24 08:55:25 +02:00
slawek	caf0e334f0	feature: users sign in to the web service, and only members of admins manage the router. Release / release (push) Failing after 16m57s Details	2026-10-24 04:19:01 +02:00
slawek	791fc7fddd	feature: users can be enrolled for TOTP one-time passwords. Release / release (push) Failing after 15m23s Details	2026-10-24 03:00:35 +02:00
slawek	f135d776f8	fix: package upgrades no longer ask the configuration questions. Release / release (push) Successful in 18s Details	2026-10-24 02:16:19 +02:00
slawek	cda33fc4c7	fix: an upgrade keeps the configuration and asks nothing. Release / release (push) Successful in 18s Details	2026-10-24 02:08:23 +02:00
slawek	08693f7e68	feature: users have a first name, a last name and an e-mail address. Release / release (push) Successful in 11s Details	2026-10-24 01:47:50 +02:00
slawek	b9003bacb9	update: the status web service listens on port 7081 by default. Release / release (push) Successful in 11s Details	2026-10-24 01:14:21 +02:00
slawek	0bf78b16b1	feature: users and their certificates no longer need a P2S system enabled. Release / release (push) Successful in 21s Details	2026-10-24 00:00:44 +02:00
slawek	0ff74cc8c9	feature: the firewall lets the Router Manager in on the internal interface. Release / release (push) Successful in 11s Details	2026-10-23 23:24:59 +02:00