linux-cloud-router/ADR.md

# Architecture Decision Records

## ADR-1: Docker's published ports are filtered in DOCKER-USER by the package

Status: Accepted

Context: Docker adds its own iptables rules for published ports. They are evaluated before
UFW's, so a published port is reachable from the external interface whatever UFW says. Docker
reserves the `DOCKER-USER` chain, evaluated first in `FORWARD`, for rules of the operator. The
router is an internet-facing host, and Docker is installed on it for services run alongside the
router.

Decision: The package secures Docker itself. A `docker` configuration section has `mode`
(`auto`, `on`, `off`) and `exposed_ports`. In `auto` mode the rules are applied when
`docker.service` is installed, which is what the package looks up; `on` requires Docker and
`off` removes the rules. The rules are in a chain of their own, `VPN-ROUTER-DOCKER`, entered by
one jump at the top of `DOCKER-USER`. For traffic from the external interface they accept IPsec
protected traffic, TCP to `exposed_ports` and established connections, and drop the rest. The
`vpn-router-docker` unit applies them after `docker.service` starts and removes them when it
stops; `vpn-router-setup` restarts it while Docker runs.

Consequences: Installing Docker first is recommended; `auto` mode then finds it as soon as the
package is configured. When Docker is installed after the package, the rules are applied by
restarting `vpn-router-setup`, a reboot or a restart of Docker, not by Docker's first start,
because `vpn-router-docker` is already active. A published
port is reachable from the external interface only when it is in `exposed_ports`. Other rules
in `DOCKER-USER` are not touched, because the package owns only its own chain. Restarting Docker
restores the rules. The rules cover IPv4 only.

## ADR-2: A small status web service, bound to the internal address

Status: Accepted

Context: The router has no way to see its state without a shell. A check from the protected
network, by a person or by a monitor, needs a page or an endpoint. The router is an
internet-facing host, so what the service shows and where it listens matter.

Decision: The `vpn-router-web` unit runs one Python script on the standard library's
`http.server`, with the package's own modules and the Python packages of the Ubuntu archive that
the package depends on, and no web framework. It listens on the address of the internal interface and on
`web.port`, 7081 by default, and is on unless `web.enabled` is `false` (ADR-7). `/` is for
people, in plain text for `curl` and `wget` and as a Tailwind CSS page for a browser, and
`/status` is the same status as JSON. It shows the state of the services, the package version
and the state of the site-to-site, road-warrior, OpenVPN and WireGuard features, and nothing from
the configuration. `vpn-router-setup` opens the port on the internal interface in UFW and writes
the features and the facts it resolves to `/run/vpn-router/web.json`.

Consequences: Nothing outside the internal network can reach it, since it listens on the internal
address only and the firewall opens the port on the internal interface only. `/` and `/status`
have no authentication, which is acceptable because they are read only and show no
configuration. It runs as root, because `swanctl` and `wg` need it, with `NoNewPrivileges`,
`ProtectSystem=strict`, `ProtectHome` and `PrivateTmp`. The Tailwind style comes from a CDN, so a
browser on a network without internet shows the unstyled page, and `curl` is unaffected.

## ADR-3: The protected paths take JWTs that only the router signs

Status: Accepted

Context: Clients such as the Router Manager, and people with `curl`, need the configuration,
the users and the groups of the router, which `/` and `/status` must not show. Both people and
programs sign in with a user of the directory. A secret shared with the platform that deploys
the router would let whoever holds it, and every copy of the deployment's state, make tokens.
The service listens on plain HTTP on the internal network.

Decision: `/config`, `/users` and `/groups` need a JWT in an `Authorization: Bearer` header,
HS256, verified with `hmac` and `hashlib`; no other algorithm is accepted. The key is
`/etc/vpn-router/web.key`, random, made by the web service on its first run and readable by root
only; it is never configured or output. The router itself issues
the tokens: `POST /auth` checks a user's password and TOTP code with a bind (ADR-14) and answers
an access token naming the user in `sub`, valid for 5 minutes, and a refresh token valid for 8
hours (ADR-15); `vpn-router-token` makes tokens with the `admin` claim (ADR-13). A token without
`admin` must expire within ten minutes, so a leaked one is of little use. Without the key the
paths and `/auth` are not served. The configuration shown has no secret options (ADR-8).

Consequences: Nothing outside the router can make a token, so a token is either issued to a
user who signed in or made by root on the router. A client keeps the refresh token to stay
signed in. A token of `vpn-router-token` can be revoked only by replacing the key, which ends
every token. Tokens, passwords and answers cross the network unencrypted, so the API is for a
trusted internal network; HTTPS would need a certificate that the package does not have.

## ADR-4: Users are kept in a local OpenLDAP directory

Status: Accepted

Context: OpenVPN needs user accounts with passwords, and the users are meant to cover every P2S
system of the router, including IKEv2 EAP-TLS and WireGuard, which an OIDC provider cannot
authenticate. Everything the router runs comes from the Ubuntu archive. slapd, its Python module
`python3-ldap` and the OpenVPN plugin `openvpn-auth-ldap` are all in the archive; the plugin
checks a password by binding as the user, so the hash scheme is slapd's alone. An OIDC provider
can read the directory through its LDAP connector.

Decision: The users are entries in slapd. The package keeps its own database,
`dc=vpn-router,dc=local` in `/var/lib/ldap/vpn-router` (the path the slapd AppArmor profile
allows), next to the database the slapd package creates from the host's domain, so the package
does not depend on the debconf answers of slapd. Its root DN is root's peer-credential DN on the
local socket, so `vpn-router-setup` manages it with SASL EXTERNAL and no administrator password
exists. Users are changed by `cn=user-manager,ou=services`, whose ACLs allow writing `ou=users`
and setting passwords (`=wx`) but not reading them. Passwords are hashed with ARGON2, and the
ppolicy overlay locks a disabled user (`pwdAccountLockedTime`, effective with `pwdLockout`).
slapd listens on the local socket only. Provisioning checks every step before changing it, so it
runs at every start of `vpn-router-setup`.

Consequences: The package depends on `slapd`, `python3-ldap`, `openvpn` and
`openvpn-auth-ldap`, and `vpn-router-setup` and `vpn-router-user` import `python3-ldap`. The
service account's password is a root-only file. The directory is user data: it is kept on purge,
and it is backed up with `slapcat`, not by copying files.

## ADR-5: OpenVPN clients need a certificate and the password of the user it names

Status: Accepted

Context: The router CA issues client certificates for IKEv2. OpenVPN can check a certificate, a
password, or both. With both, a certificate alone and a password alone are not enough, but
`openvpn-auth-ldap` checks only the password of the user name given at login, which need not be
the user of the certificate.

Decision: OpenVPN checks the certificate against the CA and the CRL, the password with
`openvpn-auth-ldap`, and runs `openvpn-same-user` as `auth-user-pass-verify` with `via-env`,
which accepts the login only when the user name equals the certificate's common name. OpenVPN
calls both and needs both to pass. A user's certificate has the user name as its common name, is
issued when the user is added and revoked when the user is removed. The CRL is created with the
CA, so that `crl-verify` has a file from the start, and is valid for ten years, because the router
renews it only when it revokes a certificate. The server uses the router's own certificate, whose
common name is `wan.local_fqdn`; the client profile checks that name (`verify-x509-name`), since
the CA's certificates carry both the server and the client purpose.

Consequences: Disabling a user locks the password, which stops OpenVPN but not IKEv2, which
checks certificates only; removing a user revokes the certificate for both. A user's
certificate expires after the CA's default of a year and is renewed by removing and adding the
user again.

## ADR-6: The web service and vpn-router-user share one module for users

Status: Accepted

Context: Users are managed from the command line, by an operator at a terminal, and through
`/users` of the web service, by the Router Manager. Both must apply the same rules to names,
passwords and person attributes, and both change the directory and the CA. `simple-ca` keeps
the CA's state in one file, so two changes must not run at the same time. The web unit is
sandboxed with `ProtectSystem=strict`.

Decision: `vpnrouter_users.py` holds the user operations, the checks and a session that takes a
lock and the connection of the user manager account. The web service imports it and maps a
`Failure` to the HTTP status; `vpn-router-user` imports it and adds the argument parsing and
the output, text for people and JSON with `--json`. The web unit allows writing the paths the
operations need: `/run/vpn-router` for the lock, `/etc/vpn-router/pki` for the certificates and
the CA's state, and `/etc/swanctl/x509crl` for the published CRL.

Consequences: The rules exist once. A request starts no process, and a password never leaves
the process that received it. The web service imports `python3-ldap` and the package's
configuration module. The API is protected by the tokens of ADR-3, so only members of `admins`
and the holders of an admin token manage users.

## ADR-7: The web service starts on its own and reads its inputs at each request

Status: Accepted

Context: The status page is the tool to see a problem of `vpn-router-setup`, so it must run
whatever state setup is in, also before setup has run and while it fails. The internal address,
`web.port` and `web.enabled` can change at any time.

Decision: The service depends on nothing to start and does not end on its own. A discovery
thread works out, every 30 seconds, the internal address from `vpn-router.json` as
`general.mode` directs, with the discovery code in `vpnrouter_net.py`, which `vpnrouter.py` uses
too, and `web.port` and `web.enabled` from the same file. It runs the HTTP server in a thread of
its own at that address and port, stops it and starts a new one when they change, and runs none
while the service is disabled, the address is not known or it cannot be bound. Each request
reads the signing key and `vpn-router.json`, and the features and resolved facts from
`web.json`, which setup writes when it applies the configuration, so the service shows what setup
applied; while that file is missing the router shows no features. The unit is not
ordered after `vpn-router-setup`, and setup does not restart it.

Consequences: The status page answers as soon as the internal address exists, whatever state
setup is in, and follows a changed address, port or `web.enabled` within 30 seconds without a
restart. A new signing key applies to the next request. The interfaces are read every 30
seconds, which costs a few `ip` commands.

## ADR-8: /config leaves secret options out

Status: Accepted

Context: A secret has no use outside the router, so a client never needs it, and showing even
that a secret is set tells a reader more than it needs.

Decision: The configuration that `vpn-router-setup` writes for the web service has no secret
options at all. They are recognised by name: `psk`, `database`, whose connection string may hold
a password, and options with `secret`, `password`, `token` or a `_key` ending in the name.

Consequences: No secret reaches the web service's state file or any answer of the API. A client
cannot tell whether a secret is set. A new secret option needs a name that matches.

## ADR-9: The Router Manager is a separate package with a small Python server

Status: Accepted

Context: The router needs a web application to manage it, which may run on the router or on
another Linux system. The router's web service issues tokens through `/auth` (ADR-3) and sends
no CORS headers, so a browser cannot call it directly, and tokens kept in the page could be read
by its scripts.

Decision: `vpn-router-manager` is a Debian package of its own, built in the repository
`linux-cloud-router-manager`. A React and MUI application in the style of manage-ui is built
with Vite when the package is built and shipped as static files. A Python server, using the
Ubuntu packages `python3-requests` and `python3-psycopg`, serves them, signs users in through the
router's `/auth`, keeps their tokens in a server-side session in PostgreSQL behind an
`HttpOnly` cookie, and passes every other `/api/<path>` call to the router's `/<path>` with the
session's access token, refreshing it as needed; it holds no logic of the router's own and no
key of it. It reads its settings from `/etc/vpn-router-manager/manager.json` at each request; on
the router the settings not given come from `vpn-router.json`, and it listens on the internal
address like the status web service, with the same discovery thread. `vpn-router-setup` opens
the manager's port on the internal interface while the manager is installed, so the firewall
stays in one place.

Consequences: The router needs no Node.js; the package needs only `python3` and packages of the
Ubuntu archive. Tokens never reach the browser, and sessions survive a restart of the manager.
The router decides what a user may do; the manager only signs in. The application checks the
shape of every answer of the router. The router's firewall depends on whether the manager is
installed, and a changed manager port needs `vpn-router-setup` to run again.

## ADR-10: Users and their certificates do not depend on a P2S system being enabled

Status: Accepted

Context: Users are prepared before a P2S system is switched on, and the Router Manager manages
them on any configured router, with or without one.

Decision: Once a platform is chosen, `vpn-router-setup` always provisions the user directory in
slapd and the CA with its CRL; the server certificate follows once `wan.local_fqdn` is set.
`/users` and `vpn-router-user` rely only on the directory and the CA. Only the OpenVPN client
profile needs OpenVPN to be enabled.

Consequences: slapd always holds the directory and the CA always exists on a configured router,
even when no P2S system uses them. An operator who wants a CA of their own must place it before
choosing the platform.

## ADR-11: Groups are groupOfNames entries that never become empty

Status: Accepted

Context: Users are to be organised in groups. Groups record membership only; nothing on the
router reads them. LDAP clients that read groups, such as `openvpn-auth-ldap` and the LDAP
connectors of OIDC providers, expect `groupOfNames` with member DNs, and OpenLDAP's core schema
requires a `groupOfNames` to have at least one `member`.

Decision: A group is `cn=<name>,ou=groups,dc=vpn-router,dc=local`, a `groupOfNames` with the
DNs of its members and an optional `description`. A group is created with at least one member,
and taking out its last member is refused; such a group is removed instead. Removing a user
takes it out of its groups and is refused while the user is the last member of one. The group
operations are in `vpnrouter_users.py` with the user operations, so they share the session and
its lock; `vpn-router-group` is a tool of its own beside `vpn-router-user`, and the web service
serves them under `/groups`. Group names follow the rules of user names.

Consequences: A group always satisfies the schema without a placeholder member, and an empty
group cannot exist. Removing a user or a member can need another step first. A later use of
groups, such as allowing OpenVPN only to some, can read them as they are.

## ADR-12: TOTP secrets are kept with the OATH-LDAP schema of slapd's otp module

Status: Accepted

Context: Users are enrolled for TOTP one-time passwords, and enrolled users must give a code
whenever they log in, to OpenVPN and to `/auth`. Ubuntu's slapd ships the `otp` module, which
holds the OATH-LDAP schema and an overlay that checks a code appended to the password at bind
time; both OpenVPN (`openvpn-auth-ldap`) and `/auth` check passwords with a bind. Ubuntu
packages `python3-pyotp`. An authenticator app is enrolled from a QR code of an `otpauth://` URI.

Decision: The `otp` module is loaded and its overlay is configured on the package's database,
after ppolicy, so every bind of an enrolled user takes the password followed by a current code
and slapd records the time step used. The TOTP token is the user entry itself: enrolment adds `oathTOTPUser` and `oathTOTPToken` to it, with
`oathTOTPToken` naming the entry, the secret in `oathSecret` and `oathTOTPParams` naming one
shared parameter entry, `cn=totp,ou=policies`, with the parameters authenticator apps assume
(6 digits, HMAC-SHA1, 30 seconds). An ACL lets the user manager account set `oathSecret` and
no one read it. Enrolment is confirmed: `otp_new` makes a secret with `pyotp` and returns it
with its URI, storing nothing, and `otp_enrol` stores the secret it is given back only when a
code of it verifies, so a secret is never stored that the user's app does not produce. The
router answers the URI; the client renders the QR code.

Consequences: Removing a user removes its token. One check covers OpenVPN and `/auth`, and
an enrolled user types the code after the password in the OpenVPN client. The secret passes
through the client between `otp_new` and `otp_enrol`, so the router keeps no pending state;
enrolling again replaces the secret only once a code of the new one verifies.

## ADR-13: Members of admins manage the router; only CLI tokens change admins

Status: Accepted

Context: Signing in through `/auth` (ADR-3) is open to every user of the directory, most of
whom only use OpenVPN. Finer permissions per operation are not needed. Somebody must be able to
manage the router before any administrator exists and after every administrator lost access,
and administrators must not be able to grant themselves or others the administrator role over
the API.

Decision: A token naming a user in `sub` may call the protected paths only while the user is a
member of the group `admins`, read at each request; other users get 403. `vpn-router-token`, run
as root on the router, makes tokens with the claim `admin: true` and a lifetime given at
creation, without a maximum, signed with the router's key; `/auth` never issues the claim. An
admin token may call everything, and only an admin token may change `admins`: create it, rename a
group to it, change or remove it, add or take out its members, or remove a user who is a member
of it. The first member is added with `vpn-router-group` on the router, which acts as root.

Consequences: Removing a user from `admins` takes effect at the next request, without waiting
for the user's tokens to expire. Membership of `admins` changes only through root on the router.
An admin token is as powerful as root over the API and is revoked only by replacing the key, so
it is given a short lifetime unless it is meant for automation.

## ADR-14: Failed logins lock the account in slapd; /auth is rate limited per client

Status: Accepted

Context: `/auth` is open to the internal network and checks passwords, so it invites guessing;
OpenVPN checks passwords from the internet. Both check with a bind, so slapd sees every attempt.
ppolicy locks an account after a number of failed binds for a fixed time; it has no growing
lockout. A Router Manager signs all its users in from one address.

Decision: The default password policy locks an account for 15 minutes after 5 failed binds
within 15 minutes; a successful bind clears the count; `vpn-router-user unlock` and
`POST /users/<name>/unlock` end a lockout and clear the count, and leave a disabled user disabled.
`/auth` answers the same 401 for a wrong name, password or code and for a locked or disabled
user. `/auth` and `/auth/refresh` allow 10 calls a minute per client address, counted in memory,
and answer 429 with `Retry-After` beyond that. The client address is the peer, or the last
`X-Forwarded-For` entry for a peer in `web.trusted_proxies`, by default the address the service
listens on, where a manager on the router calls from.

Consequences: One lockout covers OpenVPN and `/auth`, kept by slapd alone. Someone who knows a
user name can lock the user out for 15 minutes at a time. The rate limit restarts with the
service. A manager elsewhere than on the router has its address added to `web.trusted_proxies`,
or its users share one limit.

## ADR-15: Refresh tokens are kept in PostgreSQL

Status: Accepted

Context: An access token lives 5 minutes, and a client stays signed in with a refresh token for
8 hours; refresh tokens must be revocable when a user signs out, is disabled, changes the
password or is removed, and must survive a restart of the service. The Router Manager keeps its
sessions in PostgreSQL too.

Decision: Refresh tokens are random strings, stored as their SHA-256 with the user name and the
expiry in the table `refresh_tokens`, through psycopg 3 (`python3-psycopg`). The database is
`web.database`, or by default the database and role `vpn-router` on the router, which the web
service creates on its first run (ADR-16) with `runuser -u postgres -- psql` and a random password in
`/etc/vpn-router/database.secret`, reached over TCP on `localhost`, as Ubuntu's `pg_hba.conf`
takes peer authentication on the local socket. A refresh token is used once: refreshing deletes
it and issues a new one, after checking that the user is neither disabled nor locked.

Consequences: The package depends on `postgresql` and `python3-psycopg`. A stolen database holds
no usable token. Changing a user's password, disabling, unlocking or removing the user, and
signing out, end the user's sessions at once; an access token already issued lives out its
5 minutes.

## ADR-16: Each service creates what it needs on its first run

Status: Accepted

Context: The package manager configures packages in an order of its own; on a new VM the Router
Manager was configured before the PostgreSQL server, so a database created at installation was
missing. PostgreSQL may also start after the services at boot.

Decision: The web service makes its signing key and creates the refresh tokens' database itself,
and the Router Manager its sessions database; neither `vpn-router-setup` nor a maintainer script
creates them. Each service does it before it listens, retrying every 15 seconds and logging each
failure, then listens. `vpnrouter_auth` turns every failure of the database into
`DatabaseError`, the only database exception the web service and `vpn-router-user` handle.

psycopg (Ubuntu ships only its pure-Python implementation) finds libpq through the dynamic linker
cache, which dpkg refreshes with its deferred `ldconfig` only at the end of an installation, so a
service started during the installation cannot import it yet; a failed import cannot be repeated
in the same process either, as psycopg caches the lookup. The services therefore import psycopg
plainly, and their units wait instead: `ExecStartPre=wait-for-libpq`, a shell script of each
package, waits until `ldconfig -p` lists `libpq.so.5`, within `TimeoutStartSec=5min`. The
maintainer scripts start the units with `systemctl --no-block` (debhelper's start is turned off),
because a start that waited would hold dpkg before its `ldconfig`.

Consequences: The order of installation and of the start of PostgreSQL and of the linker cache
does not matter; a database that cannot be created is retried until the cause is gone, and the
reason is in the service's log. Outside an installation the wait ends at once. The web unit may
write `/etc/vpn-router` for the key and the database password.

## ADR-17: Bootstrap diagnostics run only until the first setup succeeds

Status: Accepted

Context: A new VM is reachable only once its setup has worked, and the reason of a failure must
be readable from the internal network before an SSH session is possible. A service that reports
the setup cannot be one of the services being set up: those import packages that may not load yet
(psycopg could not find libpq before `ldconfig` ran), and they would fail before reporting
anything. Once the router has been set up, SSH is the way to diagnose it. The more a diagnostic
page does, the more ways it has to fail itself.

Decision: Each package ships a bootstrap service of the Python standard library alone, a script
that answers every request with the plain-text output of a fixed list of commands: the setup
state, the services, the failed units, the addresses and the end of the logs. The router's
listens on 17081 and the Router Manager's on 17080, beside the ports of the services, 7081 and
7080, on every address. The units are never enabled and not started at installation;
`vpn-router-setup` starts them while `/var/lib/vpn-router/setup.done` does not exist, and opens
their ports on the internal interface. The first successful run writes that file, closes the ports
and stops them; a failed run leaves them running, so the reason stays readable. Setup records its
state, step and failure in `/run/vpn-router/setup.json` at every step; `vpnrouter.fail` ends with
a `SystemExit` carrying its message, so setup records it.

Consequences: The diagnostics need nothing the setup provides and cannot collide with the real
services. After the first success they never run again, also not after an upgrade or a reboot.
The page shows log lines to whoever reaches the internal network during the first setup.