linux-cloud-router/ADR.md

Architecture Decision Records

ADR-1: Docker's published ports are filtered in DOCKER-USER by the package

Status: Accepted

Context: Docker adds its own iptables rules for published ports. They are evaluated before UFW's, so a published port is reachable from the external interface whatever UFW says. Docker reserves the DOCKER-USER chain, evaluated first in FORWARD, for rules of the operator. The router is an internet-facing host, and Docker is installed on it for services run alongside the router.

Decision: The package secures Docker itself. A docker configuration section has mode (auto, on, off) and exposed_ports. In auto mode the rules are applied when docker.service is installed, which is what the package looks up; on requires Docker and off removes the rules. The rules are in a chain of their own, VPN-ROUTER-DOCKER, entered by one jump at the top of DOCKER-USER. For traffic from the external interface they accept IPsec protected traffic, TCP to exposed_ports and established connections, and drop the rest. The vpn-router-docker unit applies them after docker.service starts and removes them when it stops; vpn-router-setup restarts it while Docker runs.

Consequences: Installing Docker first is recommended; auto mode then finds it as soon as the package is configured. When Docker is installed after the package, the rules are applied by restarting vpn-router-setup, a reboot or a restart of Docker, not by Docker's first start, because vpn-router-docker is already active. A published port is reachable from the external interface only when it is in exposed_ports. Other rules in DOCKER-USER are not touched, because the package owns only its own chain. Restarting Docker restores the rules. The rules cover IPv4 only.

ADR-2: A status web service of the standard library, bound to the internal address

Status: Accepted; amended by ADR-7

Context: The router has no way to see its state without a shell. A check from the protected network, by a person or by a monitor, needs a page or an endpoint. The router is an internet-facing host, so what the service shows and where it listens matter.

Decision: The vpn-router-web unit runs one Python script that uses only the standard library, in the way of a small test server. It binds to the address of the internal interface and to web.port, on by default at 8080. / is for people, in plain text for curl and wget and as a Tailwind CSS page for a browser, and /status is the same status as JSON. It shows the state of the services, the package version and the state of the site-to-site, road-warrior and WireGuard features, and nothing from the configuration. vpn-router-setup resolves the interface address, writes it with the port and the features to /run/vpn-router/web.json, opens the port on the internal interface in UFW and restarts the service, so the service does not need the package's configuration code, which needs Jinja2.

Consequences: Nothing outside the internal network can reach it, since it listens on the internal address only and the firewall opens the port on the internal interface only. It has no authentication, which is acceptable because it is read only and shows no configuration. It runs as root, because swanctl and wg need it, with NoNewPrivileges, ProtectSystem=strict, ProtectHome and PrivateTmp. The Tailwind style comes from a CDN, so a browser on a network without internet shows the unstyled page, and curl is unaffected. A change of the internal address is picked up at the next run of vpn-router-setup.

ADR-3: /config is authenticated with a short-lived JWT signed with a shared secret

Status: Accepted; amended by ADR-7 and ADR-8

Context: A client such as manage-ui needs the configuration and the resolved state of the router, which / and /status must not show. The service has only the standard library, listens on plain HTTP on the internal network, and its configuration file is readable by root only.

Decision: /config needs a JWT in an Authorization: Bearer header. The service verifies HS256 tokens with hmac and hashlib, with the secret web.jwt_secret, and accepts no other algorithm, because verifying RS256 or ES256 needs a library outside the standard library. The token must carry exp, not more than ten minutes ahead, so that the client makes a fresh token for each use and a leaked one expires. Without a secret the page is not served. The secrets in the configuration shown are masked by name (psk, and options with secret, password, token or a _key ending in the name).

Consequences: The client and the router share one secret, and whoever holds it can make tokens, so it is long and random; it is checked to have at least 32 characters. The secret is in the configuration file and in /run/vpn-router/web.json, both readable by root only. The token and the answer cross the network unencrypted, so the page is for a trusted internal network; HTTPS would need a certificate that the package does not have. The masking is by option name, so a new secret option needs a name that matches.

ADR-4: Users are kept in a local OpenLDAP directory

Status: Accepted

Context: OpenVPN needs user accounts with passwords, and the users are meant to cover every P2S system of the router. The options weighed were an external OIDC provider (Entra ID, Keycloak) with openvpn-auth-oauth2, an OIDC server embedded on the router, a file of password hashes, and OpenLDAP on the router. OIDC for OpenVPN needs a browser on the client and a program from outside the Ubuntu archive, and it cannot authenticate IKEv2 EAP-TLS or WireGuard. slapd, its Python module python3-ldap and the OpenVPN plugin openvpn-auth-ldap are all in the archive; the plugin checks a password by binding as the user, so the hash scheme is slapd's alone. A directory can also be read later by an OIDC provider through its LDAP connector.

Decision: The users are entries in slapd. The package keeps its own database, dc=vpn-router,dc=local in /var/lib/ldap/vpn-router (the path the slapd AppArmor profile allows), next to the database the slapd package creates from the host's domain, so the package does not depend on the debconf answers of slapd. Its root DN is root's peer-credential DN on the local socket, so vpn-router-setup manages it with SASL EXTERNAL and no administrator password exists. Users are changed by cn=user-manager,ou=services, whose ACLs allow writing ou=users and setting passwords (=wx) but not reading them. Passwords are hashed with ARGON2, and the ppolicy overlay locks a disabled user (pwdAccountLockedTime, effective with pwdLockout). slapd listens on the local socket only. Provisioning checks every step before changing it, so it runs at every start of vpn-router-setup.

Consequences: The package depends on slapd, python3-ldap, openvpn and openvpn-auth-ldap, and vpn-router-setup and vpn-router-user import python3-ldap, the first library outside the standard library besides jinja2. The service account's password is a root-only file. The directory is user data: it is kept when the features are switched off and on purge, and it is backed up with slapcat, not by copying files.

ADR-5: OpenVPN clients need a certificate and the password of the user it names

Status: Accepted

Context: The router CA already issues client certificates for IKEv2. OpenVPN can check a certificate, a password, or both. With both, a certificate alone and a password alone are not enough, but openvpn-auth-ldap checks only the password of the user name given at login, which need not be the user of the certificate.

Decision: OpenVPN checks the certificate against the CA and the CRL, the password with openvpn-auth-ldap, and runs openvpn-same-user as auth-user-pass-verify with via-env, which accepts the login only when the user name equals the certificate's common name. OpenVPN calls both and needs both to pass. A user's certificate has the user name as its common name, is issued when the user is added and revoked when the user is removed. The CRL is created with the CA, so that crl-verify has a file from the start, and is valid for ten years, because the router renews it only when it revokes a certificate. The server uses the router's own certificate, whose common name is wan.local_fqdn; the client profile checks that name (verify-x509-name), since the CA's certificates carry both the server and the client purpose.

Consequences: Disabling a user locks the password, which stops OpenVPN but not IKEv2, which checks certificates only; removing a user revokes the certificate for both. A user's certificate expires after the CA's default of a year and is renewed by removing and adding the user again.

ADR-6: The web service manages users by running vpn-router-user

Status: Accepted

Context: The web service has only the standard library (ADR-2), and managing users needs python3-ldap and simple-ca. Its unit is sandboxed with ProtectSystem=strict.

Decision: /users runs vpn-router-user and returns its JSON answer, mapping its exit status to the HTTP status. Passwords go to the program on standard input, so they never appear in a command line. The unit allows writing the paths the program needs: /run/vpn-router for its lock, /etc/vpn-router/pki for the certificates and the CA's state, and /etc/swanctl/x509crl for the published CRL. A lock serializes the program's runs, because simple-ca keeps its state in one file.

Consequences: The web service still imports nothing outside the standard library, and the same program serves the command line and the API. Each request starts a process, which is fine for user administration. The API is protected by the token of /config, so whoever has the secret can manage users.

ADR-7: The web service starts on its own and reads its inputs at each request

Status: Accepted

Context: The web service took the address to listen on, the port, the token secret and the features from /run/vpn-router/web.json, written by vpn-router-setup, and ended with a failure while the file or the internal address was missing. It was ordered after vpn-router-setup, so a problem in the setup kept the status page, the tool to see that problem, from running.

Decision: The service depends on nothing to start and does not end on its own. A discovery thread works out, every 30 seconds, the internal address from vpn-router.json as general.mode directs, with the discovery code in vpnrouter_net.py, which uses only the standard library and which vpnrouter.py uses too, and web.port and web.enabled from the same file. It runs the HTTP server in a thread of its own at that address and port, stops it and starts a new one when they change, and runs none while the service is disabled, the address is not known or it cannot be bound. Each request reads the token secret from vpn-router.json and the features and resolved facts from web.json, which setup still writes, because they need the configuration code; while that file is missing the router shows no features. The unit is not ordered after vpn-router-setup, and setup does not restart it.

Consequences: The status page answers as soon as the internal address exists, whatever state setup is in, and follows a changed address, port or web.enabled within 30 seconds without a restart. web.json no longer holds the address, the port or the secret, so the secret is only in vpn-router.json. A change of the secret applies to the next request. The interfaces are read every 30 seconds, which costs a few ip commands.

ADR-8: /config leaves secret options out

Status: Accepted

Context: /config showed the secret options of the configuration with their values replaced by ********. That still tells a reader which secrets are set, and a secret has no use outside the router, so a client never needs it.

Decision: The configuration that vpn-router-setup writes for the web service has no secret options at all. They are recognised by name as before: psk, and options with secret, password, token or a _key ending in the name.

Consequences: No secret reaches the web service's state file or any answer of the API. A client cannot tell whether a secret is set. A new secret option still needs a name that matches.

ADR-9: The Router Manager is a separate package with a small Python server

Status: Accepted

Context: The router needs a web application to manage it, which may run on the router or on another Linux system. The router's web service needs a token signed with web.jwt_secret for /config and /users and sends no CORS headers, so a browser cannot call it with the secret kept away from the page.

Decision: vpn-router-manager is a Debian package of its own, built in the repository linux-cloud-router-manager. A React and MUI application in the style of manage-ui is built with Vite when the package is built and shipped as static files. A Python server, using the Ubuntu packages python3-jwt and python3-requests, serves them and passes every /api/<path> call to the router's /<path>, signing a short-lived token with the router's secret; it holds no logic of the router's own. It reads its settings from /etc/vpn-router-manager/manager.json at each request; on the router the settings not given come from vpn-router.json, and it listens on the internal address like the status web service, with the same discovery thread. vpn-router-setup opens the manager's port on the internal interface while the manager is installed, so the firewall stays in one place. The manager asks for no authentication yet.

Consequences: The router needs no Node.js; the package needs only python3 and packages of the Ubuntu archive. The secret never reaches the browser. The application checks the shape of every answer of the router. Anyone who reaches the manager's port manages the router, so on another system it listens on 127.0.0.1 unless told otherwise. The router's firewall depends on whether the manager is installed, and a changed manager port needs vpn-router-setup to run again.