vpn-router

A Debian package that configures a Linux host or virtual machine as a VPN router:

  • site-to-site IKEv2 IPSec with pre-shared key or certificate authentication (strongSwan swanctl)
  • road-warrior (P2S) access using IKEv2 EAP-TLS
  • a two-level certificate authority: a root, which may be kept offline, and issuing CAs for the VPN clients, for site-to-site and for general use, with certificate templates, published certificates and CRLs, and an optional ACME server
  • OpenVPN (P2S) access with a certificate and a password per user, the users kept in a local LDAP directory and managed with vpn-router-user or the web service
  • optional WireGuard endpoint
  • routing and NAT for a protected subnet, TCP MSS clamping, and UFW firewall rules
  • firewall rules for the ports Docker publishes, when Docker is installed
  • metrics of the router kept in PostgreSQL, served for an OpenTelemetry Collector and pushed over OTLP
  • static web sites served by Caddy, uploaded through the web service or WebDAV, and an optional authoritative name server (PowerDNS) for a zone of the sites

Nothing in the package is specific to a cloud provider. It is built on the external and internal NIC names, and a mode setting says how much of the rest you are supplying and how much is read from the system. Detection happens only in the mode that asks for it, so a router never quietly guesses which interface to NAT out of unless you told it to. Platform-specific additions live in separate modules and are selected explicitly.

Installing the package configures nothing by default. The shipped platform is none, which means the files are laid down and the machine is left alone until you say otherwise.

Installing

The package can be installed by hand or as a step in automated provisioning. Both are supported equally, and both end up with the same two things: a configuration file and a service that applies it.

/etc/vpn-router/vpn-router.json      the configuration
systemctl restart vpn-router-setup   apply it

If the host runs Docker, the recommended order is Docker first and the package second. The package secures the ports Docker publishes when it finds the Docker service; the steps for a host that already runs Docker, and for installing Docker later, are under Docker.

By hand

apt-get install vpn-router          # answer "none" to the platform question
$EDITOR /etc/vpn-router/vpn-router.json
systemctl restart vpn-router-setup

Answering none to the platform question defers everything: no further questions are asked and nothing on the machine is configured. The only change an install makes is the package's own /etc/sysctl.d/ drop-in, which enables IP forwarding and relaxes rp_filter as a router needs. Set platform in the file when you are ready.

Automated

Preseed the answers and the router comes up configured, with no follow-up command. Note that platform must be something other than none, and mode says how much you are supplying:

debconf-set-selections <<'EOF'
vpn-router vpn-router/platform         select generic
vpn-router vpn-router/mode             select manual
vpn-router vpn-router/external_interface string eth0
vpn-router vpn-router/internal_interface string eth1
vpn-router vpn-router/int_addr         string 10.1.1.4
vpn-router vpn-router/int_gateway_ip   string 10.1.1.1
vpn-router vpn-router/local_fqdn       string router.example.com
vpn-router vpn-router/local_cidrs      string 10.0.0.0/24
vpn-router vpn-router/remote_addrs     string peer.example.net
vpn-router vpn-router/remote_id        string peer.example.net
vpn-router vpn-router/remote_cidrs     string 192.168.0.0/24
vpn-router vpn-router/psk              password s3cr3t
EOF
DEBIAN_FRONTEND=noninteractive apt-get install -y vpn-router

With mode interfaces the int_addr and int_gateway_ip lines are dropped and read from the internal NIC instead; with mode auto the interface names go too.

Alternatively write /etc/vpn-router/vpn-router.json directly before or after installing. A configuration-management tool should do that and then restart vpn-router-setup.

See examples/ for a cloud-init template and a shell installer, and examples/azure/ / examples/gcp/ for full Terraform examples that deploy a router on Azure or GCP.

Configuration

/etc/vpn-router/vpn-router.json is JSON, mode 0600, and is created on first install. It is managed with ucf, so it is yours to edit: a later dpkg-reconfigure applies the answers given and asks before touching anything you altered by hand, and an upgrade asks no questions and leaves the file as it is. Settings the file leaves out take their defaults when it is read, so a file written by cloud-init or by hand needs only the settings that differ. A complete example is installed at /usr/share/doc/vpn-router/vpn-router.json.example.

The file is an object of sections, each an object of settings. A section or setting that is left out takes its default, and an empty string or empty list means unset. Every value must have the type shown below; an unknown section or setting is an error.

Section Setting Type Default Meaning
general platform string none none, generic, azure or gcp. none defers configuration entirely
general mode string manual How much you supply: manual, interfaces or auto
interfaces external string Name of the NIC facing the untrusted network
interfaces internal string Name of the NIC facing the protected network
wan local_fqdn string This router's name or IP address: its IKE identity in fqdn mode, the name resolved in public_ip mode, and the name of HTTPS, road-warrior and OpenVPN
local cidrs list Local subnets routed via the internal side, masqueraded to the external one, and offered to road-warrior and OpenVPN clients
local int_addr string This host's address on the internal network
local int_gateway_ip string Next hop on the internal side
site_to_site object The site-to-site connections, each a name and an object of the settings below
p2s enabled boolean false Road-warrior access
p2s address_pool string CIDR assigned to road-warrior clients
p2s dns_listen string internal The addresses of the resolver the road-warrior, OpenVPN and WireGuard clients are given: internal, external or both
wireguard enabled boolean false WireGuard endpoint
wireguard address string The router's own address on wg0 with the prefix of the peers' network, for example 192.168.200.1/24; the network and the broadcast address of a network larger than /31 are refused
wireguard listen_port integer 51820 UDP port
openvpn enabled boolean false OpenVPN access for users
openvpn address_pool string IPv4 CIDR assigned to OpenVPN clients
openvpn port integer 1194 Port of the OpenVPN server on the external interface
openvpn protocol string udp udp or tcp
docker mode string auto auto secures Docker when its service is installed, on requires it, off removes the rules
docker exposed_ports list TCP ports and port ranges (8000:8100) reachable from the external interface; at most 15, a range counting as two
web enabled boolean true The status web service
web listen string both The interfaces the status web service listens on: internal, external or both
web port integer 7081 TCP port of the status web service
web database string A PostgreSQL connection string for the refresh tokens; without it, the vpn-router database on this host that the web service creates
web trusted_proxies list the address the call came in on Addresses whose X-Forwarded-For header names the client for the /auth rate limit and the Client IP of the status
web status_content string basic What / and /status show: basic, standard or full (see Status web service)
web shell boolean false /shell opens a login shell for a member of admins with console login (see The shell)
web shell_idle_minutes integer 15 Minutes without a keystroke after which a shell is closed, 1 to 1440
web log_files list Absolute paths of the log files /logs shows, at most 32 (see The /logs API)
web shell_auth_timeout integer 15 Minutes after the last use of a shell in which a new shell opens without signing in again, 0 (always sign in) to 1440
ssh ldap_login boolean false sshd accepts the directory password of the console accounts in the directory (see Console login and SSH)
https enabled boolean true Caddy serves the status web service and the Router Manager over HTTPS for wan.local_fqdn
https email string The e-mail address of the Let's Encrypt account
https web_port integer 7444 HTTPS port of the status web service
https manager_port integer 7443 HTTPS port of the Router Manager
https caddy_source string ubuntu Where Caddy is installed from: ubuntu, the Ubuntu archive, or upstream, Caddy's own repository (see HTTPS)
oidc enabled boolean false The OpenID Provider at the status web service's HTTPS site, which needs web.enabled, https.enabled and wan.local_fqdn (see OpenID Provider)
sites enabled boolean false Caddy serves the web sites (see Web sites)
sites upload_max_mb integer 512 The largest upload of a site's file or zip archive, in MiB, 1 to 10240
sites webdav boolean false WebDAV of the sites' folders, which needs https.enabled and wan.local_fqdn (see WebDAV)
sites webdav_port integer 7445 HTTPS port of WebDAV
dns enabled boolean false The name server of dns.zone, which needs dns.zone, dns.nameserver and dns.addresses (see Name server)
dns zone string The zone the name server answers for
dns nameserver string The name server's host name
dns addresses list The public addresses of the zone's apex, the name server and the sites' host names
dns listen string external The interfaces the name server answers on: internal, external or both
dns ttl integer 300 TTL of the records, 60 to 86400 seconds
pki root_name string VPN Router Root CA Common name of the root CA, when the router creates it
pki root_days integer 7300 Days the root CA's certificate is valid
pki ca_days integer 3650 Days an issuing CA's certificate is valid
pki key string rsa-4096 Key of the root and the issuing CAs: rsa-2048, rsa-3072, rsa-4096, ec-p256 or ec-p384
pki crl_days integer 7 Days the CRLs of the issuing CAs are valid
pki root_crl_days integer 180 Days the root CRL is valid, when the router signs it
pki publish list http Where the CA certificates and CRLs are published: http, http-80, https, https-default (see Certificates)
pki p2s_name string VPN Router P2S CA Common name of the P2S issuing CA
pki s2s_name string VPN Router Site-to-Site CA Common name of the site-to-site issuing CA
pki issuing_cas object The general-purpose issuing CAs, each a name and {name, templates, acme} (see Certificates)
pki templates object Certificate templates, each a name and the options it changes or defines (see Certificates)
pki extended_key_usages object Custom extended key usages templates may give, each a name and its OID in dotted form (see Certificates)
pki root_key_warning boolean true vpn-router-ca status and /ca warn while the root key is on the router; false silences that warning
metrics enabled boolean true The metrics service: collection, history and /metrics
metrics listen string internal The interfaces /metrics is served on: internal, external or both
metrics port integer 9464 TCP port of /metrics
metrics https_port integer 9465 HTTPS port of /metrics, served by Caddy while https.enabled is true
metrics service_account string metrics The service account /metrics takes, which vpn-router-setup adds and keeps
metrics otlp_endpoint string An OTLP/HTTP URL the metrics are pushed to every minute, for example https://collector.example.com:4318/v1/metrics
metrics otlp_username string The user name of HTTP Basic authentication at the OTLP endpoint
metrics otlp_password string Its password
metrics keep_details string 24 hours How long the per-minute metrics are kept
metrics keep_hourly string 1 month How long the hourly summaries are kept
metrics keep_daily string 3 months How long the daily summaries are kept
metrics keep_monthly string 1 year How long the monthly summaries are kept
metrics keep_yearly string 10 years How long the yearly summaries are kept
backup enabled boolean false Scheduled backups (see Backups)
backup schedule string daily hourly, daily or weekly
backup target string local local, an archive in backup.path, or pbs, a Proxmox Backup Server
backup path string /var/backups/vpn-router The folder of local archives, an absolute path
backup keep_hourly integer 24 Hourly local archives kept, with the hourly schedule
backup keep_daily integer 7 Daily local archives kept, with the hourly and the daily schedule
backup keep_weekly integer 4 Weekly local archives kept
backup keep_monthly integer 12 Monthly local archives kept
backup keep_yearly integer 0 Yearly local archives kept
backup metrics boolean false The backup holds the database of the metrics and the refresh tokens
backup rclone_remote string remote:path of /etc/vpn-router/rclone.conf the local archives are copied to
backup rclone_mode string sync sync mirrors the archives, copy deletes nothing in the cloud
backup pbs_repository string user@realm!token@host[:port]:datastore
backup pbs_password string The secret of the API token
backup pbs_fingerprint string The SHA-256 fingerprint of the server's certificate, when the system's CAs do not verify it
backup pbs_namespace string The namespace of the datastore, such as routers/azure
backup pbs_keyfile string An encryption key of proxmox-backup-client key create, an absolute path

Site-to-site connections

site_to_site maps a connection name to its settings. A name starts with a lowercase letter and has at most 31 lowercase letters, digits and hyphens; it names the strongSwan connection, its child and its secret (ike-psk-<name>). The install answers fill the connection remote-site; more are added in the file, with vpn-config-edit, or through /site-to-site of the status web service, which the Router Manager uses.

Setting Type Default Meaning
addrs list Remote gateway addresses or FQDNs
id string Remote IKE identity, an FQDN or an IPv4 or IPv6 address
cidrs list Remote subnets reachable through the tunnel
local_cidrs list Local subnets advertised into the tunnel
local_id_mode string fqdn The router's IKE identity: fqdn is wan.local_fqdn, public_ip the address it resolves to, internal_ip the internal address
auth string psk How both sides authenticate: psk, a pre-shared key, or certificate
psk string Pre-shared key, with auth psk
psk_file string Path to a file holding the key; wins over psk
remote_ca string With auth certificate, the PEM certificate of the CA that issued the peer's certificate; without it the router's site-to-site CA

A connection is set up once all of addrs, id, cidrs, local_cidrs and its local identity are known, with psk the key, and with certificate the router's site-to-site certificate; all such connections are written to /etc/swanctl/conf.d/site-to-site.conf. Traffic from a connection's local_cidrs to its cidrs is not masqueraded.

With auth certificate, the router authenticates with its site-to-site certificate, which the site-to-site issuing CA signs for wan.local_fqdn and for the public or the internal address that a connection with auth certificate uses as its identity (local_id_mode public_ip or internal_ip); it is issued again when that set of names changes. The peer's certificate must name id and be issued by the router's site-to-site CA, or by the CA of remote_ca. A peer with no PKI of its own gets its certificate from the router (template site-to-site, see Certificates), with the root and the site-to-site CA certificates as its trust anchors.

A minimal site-to-site configuration in auto mode:

{
  "general": { "platform": "generic", "mode": "auto" },
  "wan": { "local_fqdn": "router.example.com" },
  "local": { "cidrs": ["10.0.0.0/24"] },
  "site_to_site": {
    "remote-site": {
      "addrs": ["peer.example.net"],
      "id": "peer.example.net",
      "cidrs": ["192.168.0.0/24"],
      "local_cidrs": ["10.0.0.0/24"],
      "psk": "s3cr3t"
    }
  }
}

The router's IKE identity (wan.local_fqdn in fqdn mode) and the remote one (id) are an FQDN or an IPv4 or IPv6 address. They are written to the strongSwan configuration as given; strongSwan reads an address as an IP identity and a name as an FQDN, so a leading @ is not needed and is refused. Road-warrior and OpenVPN access need a name in wan.local_fqdn.

WireGuard peers are P2S users' devices: a VPN user of the router's directory may have up to 16 peers, each with its public key and its IPs, IPv4 addresses or networks, kept in the user's entry (vpnRouterWireGuardPeer) and set with PUT /users/<name> (wireguard_peers) or on the WireGuard tab of the Router Manager's Remote access management. A key is used by one peer only, the IPs of two peers may not overlap, and no peer's IPs cover the router's own address of wireguard.address. The package owns the [Interface] section of /etc/wireguard/wg0.conf and, between the lines # BEGIN and # END WireGuard peers of the users of the router's directory, a [Peer] section for each peer of a user who is not disabled; setup rewrites them when the settings or the peers change, and a change of a user's peers, disabling, enabling or removing a user with peers restarts setup. Every other [Peer] section, added by hand, is carried across untouched. While the directory cannot be reached, the users' peers in the file stay as they are.

Every feature is independent. Leaving a group empty simply means that feature is not configured, and its generated files are removed.

What mode decides

mode states how much of the network configuration you are supplying, and therefore how much the package reads from the system:

mode You provide The package works out
manual both interface names, int_addr, int_gateway_ip nothing
interfaces both interface names int_addr and int_gateway_ip, from those interfaces
auto nothing the interface names too: the external one from the routing table, the internal one as the only other NIC with an address

manual is the only mode in which no detection code runs at all. auto is the convenient one and can pick the wrong interface, which is a trade you make deliberately by choosing it rather than something the package does behind your back.

Detection ignores virtual interfaces, those the kernel lists under /sys/devices/virtual: loopback, bridges such as docker0, veth pairs and tunnels. Only the NICs of the machine count, so Docker does not change the result. If the host has more than two NICs with an address, auto stops and says so; use mode interfaces and name them.

The addresses strongSwan binds on always come from the external NIC, in every mode, so they are never configured twice and cannot drift out of step with the host. If a named interface does not exist the service stops and says so. In interfaces and auto mode the internal gateway is taken from the routing table, falling back to the first host address of the connected subnet with a warning when the routing table has nothing to say.

Docker

Docker publishes container ports in its own firewall rules, which bypass UFW and can expose ports on the external interface. Docker evaluates the DOCKER-USER chain before its own rules, so the package filters there. With docker.mode set to auto, the package secures Docker when docker.service is installed, and does nothing otherwise.

On a host that already runs Docker

Installing Docker first is the recommended order. After installing the package:

  1. Set general.platform in /etc/vpn-router/vpn-router.json. While it is none, nothing is configured, the Docker rules included.

  2. List the TCP ports the running containers must expose to the internet, for example "docker": { "exposed_ports": ["443"] }. The default is none: once the rules are applied, every published port that is not listed is unreachable from the external interface, and remains reachable through the IPsec tunnel.

  3. Apply the configuration with systemctl restart vpn-router-setup. Docker is running, so this also restarts vpn-router-docker, which installs the rules.

  4. Check the result:

    iptables -S DOCKER-USER        # "-j VPN-ROUTER-DOCKER" is the first rule
    iptables -S VPN-ROUTER-DOCKER  # the rules of the external interface
    

When Docker is installed after the package

The Docker rules are not applied when Docker starts for the first time, so run systemctl restart vpn-router-setup after installing Docker, then check as above. A reboot or a restart of Docker applies them as well.

The rules sit in their own chain, VPN-ROUTER-DOCKER, entered by one jump at the top of DOCKER-USER; other rules in DOCKER-USER are left alone. For traffic that arrives on the external interface, in this order:

  1. IPsec protected traffic is accepted.
  2. TCP to the ports in docker.exposed_ports is accepted; none by default.
  3. Packets of established and related connections are accepted.
  4. Everything else is dropped.

Traffic from every other interface, including the internal one and the Docker bridges, is not affected. The rules cover IPv4 only.

vpn-router-setup starts after docker.service, so Docker is up when it looks for it; the ordering has no effect on a host without Docker. The vpn-router-docker service applies the rules after docker.service starts and removes them when Docker stops, so restarting Docker restores them. systemctl restart vpn-router-setup applies a changed configuration while Docker runs. With docker.mode set to on, the service fails when Docker is not installed or its daemon is not running; with off it removes the rules.

Editing the configuration

vpn-config-edit, run as root, shows and changes /etc/vpn-router/vpn-router.json:

vpn-config-edit show
vpn-config-edit set web.status_content standard
vpn-config-edit set web.port 7082
vpn-config-edit set web.trusted_proxies 127.0.0.1,10.0.0.4
vpn-config-edit set site_to_site.branch.addrs branch.example.net
vpn-config-edit remove site_to_site.branch
vpn-config-edit set --restart oidc.enabled true

show prints the file as indented JSON and does not change it. set takes a setting as section.option, or as site_to_site.NAME.option for a site-to-site connection, which it creates with the defaults when it does not exist, and converts the value to the setting's type: a whole number, true or false, a list separated by commas (an empty value is an empty list), or a string; an unknown setting is refused. remove site_to_site.NAME removes a connection. It checks the whole configuration as vpn-router-setup does, and only then writes the file, indented, with mode 0600, and reminds that vpn-router-setup applies it; with --restart, set and remove restart vpn-router-setup and wait until it has applied the change, and exit with status 1 when it fails. The status web service writes the file the same way, and a lock keeps the two from overwriting each other's change. A refused request exits with status 2, any other failure with 1.

Bash completion

Bash completes the commands, options and arguments of vpn-router-user, vpn-router-group, vpn-router-token, vpn-config-edit, vpn-router-backup, vpn-router-ca, vpn-router-caddy and vpn-router-site with the Tab key. The package installs a file per command in /usr/share/bash-completion/completions, which the bash-completion package, recommended by vpn-router, loads on the first Tab after the command. For root, who can read them, the completions also offer the users and groups of the directory, the settings and site-to-site connections of vpn-router.json with true and false for a boolean setting, the issuing CAs and certificate templates, and the web sites; for other users only the fixed words.

sudo -i
vpn-config-edit set oidc.<Tab>
vpn-router-user passwd <Tab>

Applying changes

systemctl restart vpn-router-setup

The service also runs at boot. With platform none it does nothing and says so. Otherwise it regenerates the configuration files, resolves the interfaces and their addresses as mode directs, applies routes, generates WireGuard keys and injects firewall rules. It provisions the user directory in slapd while OpenVPN or road-warrior access is configured. It then starts strongswan, wg-quick@wg0 and openvpn-server@vpn-router if the corresponding feature is configured, reloading or restarting them only when their inputs changed, and restarts the remote access clients' resolver only when its configuration changed. Turning a feature off in the configuration stops its service and removes its files. Every step is idempotent, so restarting with no changes does nothing.

To regenerate the configuration files and PKI without starting, stopping or reloading anything:

/usr/lib/vpn-router/configure

This does still write to the system: it renders into /etc/swanctl/, /etc/systemd/resolved.conf.d/ and /etc/wireguard/, creates what the PKI is missing and adds the root CA to the host trust store. What it leaves alone is routes, firewall rules and services - that is what systemctl restart vpn-router-setup adds.

Certificates

The router has a two-level certificate authority, kept by vpn-router-setup and managed with vpn-router-ca, vpn-router-user and /ca of the status web service, which the Router Manager uses. It is held in ou=pki,dc=vpn-router,dc=local of the router's directory, next to the users:

cn=root                       the root CA; without vpnRouterPrivateKey while the root is offline
cn=<name>                     an issuing CA; with vpnRouterCertificateRequest while it waits for the offline root
cn=<name>-<time>              an issuing CA taken out of use, with vpnRouterRetired
serialNumber=<serial>,cn=<CA> a certificate the CA issued

A CA entry (class vpnRouterAuthority) has its certificate (cACertificate;binary), key, CRL (certificateRevocationList;binary) and CRL number. A certificate entry (class vpnRouterCertificate) has the certificate (userCertificate;binary), its kind (ca, router, user or external), template, names, expiry and revocation, and for the router's and the users' certificates the key. A user's certificate is also the userCertificate of the user's entry. No account of the directory reads a key (vpnRouterPrivateKey); only root does, through the local socket. slapcat writes the whole PKI into every backup with the users.

ldapsearch -Q -Y EXTERNAL -H ldapi:/// -b ou=pki,dc=vpn-router,dc=local -s one cn

The CAs

The root signs the issuing CAs only (path length 1), and the issuing CAs sign the certificates (path length 0):

  • p2s, named pki.p2s_name, signs the router's VPN server certificate, which IKEv2 and OpenVPN present, and the users' certificates. Road-warrior and OpenVPN accept client certificates of this CA only.
  • s2s, named pki.s2s_name, signs the router's site-to-site certificate and the certificates of its peers.
  • The general-purpose CAs of pki.issuing_cas, each a name and {"name": common name, "templates": [...], "acme": false}. The name starts with a lowercase letter, has at most 31 lowercase letters, digits and hyphens, and is not root, p2s or s2s; templates are the templates the CA issues.

Once a platform is chosen, vpn-router-setup creates what is missing: the root when it has neither certificate nor key, each issuing CA, the router's certificates once wan.local_fqdn is set, and the CRLs. The names, lifetimes and key type of pki apply when something is created. An issuing CA removed from pki.issuing_cas is retired: its entry is renamed cn=<name>-<time>, loses its key, and its certificate is revoked on the root CRL. The router renews its own certificates when less than a third of their lifetime is left, when the names they need change, or when their CA changes; a user's certificate is renewed by removing and adding the user.

The package copies what each consumer needs: the root, p2s and s2s into /etc/swanctl/x509ca/ as root.pem, p2s.pem and s2s.pem (strongSwan reads one certificate per file), the remote CA of each site-to-site connection that has one as s2s-<name>.pem, the router's certificates into /etc/swanctl/x509/ and their keys into /etc/swanctl/private/, the CRLs into /etc/swanctl/x509crl/, the root and p2s with their CRLs into /etc/openvpn/server/vpn-router-ca.pem and vpn-router-crl.pem, and the root into the system trust store.

vpn-router-ca status shows the root, the issuing CAs and what needs attention:

vpn-router-ca status
vpn-router-ca list --ca p2s

Templates

A template says what a certificate holds:

Option Type Default Meaning
key string rsa-3072 rsa-2048, rsa-3072, rsa-4096, ec-p256 or ec-p384, for a key the router makes
days integer 365 Days the certificate is valid, at most until its CA's certificate expires
key_usage list digital_signature Of digital_signature, content_commitment, key_encipherment, data_encipherment, key_agreement
extended_key_usage list Of the built-in server_auth, client_auth, code_signing, email_protection, time_stamping, ocsp_signing, ipsec_ike, ike_intermediate, and the custom usages of pki.extended_key_usages
policies list Certificate policy OIDs in dotted form, written into a Certificate Policies extension
subject string name The common name: dns, a DNS name, which is also a subject alternative name; name, any name; email, an e-mail address
sans list The subject alternative names allowed: of dns, ip, email

The built-in templates:

Template Key usage Extended key usage Subject Names Issued by
vpn-server digital signature, key encipherment server auth, IKE intermediate dns dns, ip p2s, for the router only
vpn-user digital signature client auth name dns, email p2s, for the users only
site-to-site digital signature, key encipherment server auth, client auth dns dns, ip s2s
server digital signature, key encipherment server auth dns dns, ip general-purpose CAs
client digital signature client auth name dns, email general-purpose CAs

pki.templates changes options of the built-in templates, which cannot be removed, and adds templates of its own, named like the issuing CAs:

"pki": {
  "issuing_cas": { "servers": { "name": "Example Server CA", "templates": ["server", "web", "smartcard"] } },
  "extended_key_usages": { "smartcard_logon": "1.3.6.1.4.1.311.20.2.2" },
  "templates": {
    "server": { "days": 397 },
    "web": { "key": "ec-p256", "days": 90, "key_usage": ["digital_signature"],
             "extended_key_usage": ["server_auth"], "subject": "dns", "sans": ["dns"] },
    "smartcard": { "key_usage": ["digital_signature", "key_encipherment"],
                   "extended_key_usage": ["client_auth", "smartcard_logon"], "subject": "name",
                   "sans": ["email"], "policies": ["1.3.6.1.4.1.99999.1.1"] }
  }
}

A custom extended key usage is first added to pki.extended_key_usages, a name of at most 31 lowercase letters, digits, hyphens and underscores that is not a built-in one, and its OID; a template then gives it by its name. A usage a template gives cannot be removed. policies puts the OIDs of the issuance policies a certificate is issued under into its Certificate Policies extension, without qualifiers.

Issuing and revoking

vpn-router-ca issue --ca servers --template server --csr web.csr > web.pem
vpn-router-ca issue --ca servers --template server --subject web.example.com --san 10.0.0.5 --pfx web.pfx
vpn-router-ca revoke 5F0A12C3D4 --reason key_compromise

With a CSR, the certificate is for the CSR's key, common name and subject alternative names, checked against the template, and the certificate and the chain (issuing CA, root) are printed. Without one, the router makes a key as the template says and writes a PKCS#12 file with the key, the certificate and the chain, protected by the password read from the first line of standard input; the router keeps the certificate and not the key. vpn-server and vpn-user are issued by the router only.

revoke takes the serial number list shows and a reason: unspecified, key_compromise, affiliation_changed, superseded, cessation_of_operation or privilege_withdrawn. A user's certificate is revoked by removing the user, and an issuing CA by removing it from pki.issuing_cas; a revoked router certificate is issued again by vpn-router-setup.

Published certificates and CRLs

Every certificate names its issuer's certificate (AIA) and CRL (CRL distribution point) at each endpoint of pki.publish, in that order. They are served, without authentication, as DER at /pki/<name>.crt and /pki/<name>.crl, the root as root, from copies in /var/lib/vpn-router/pki-publish:

Endpoint URL Served by
http http://<wan.local_fqdn>:<web.port>/pki/ the status web service; needs web.enabled
http-80 http://<wan.local_fqdn>/pki/ Caddy, on port 80, open on both interfaces; needs https.enabled
https https://<wan.local_fqdn>:<https.web_port>/pki/ Caddy; needs https.enabled
https-default https://<wan.local_fqdn>/pki/ Caddy, on port 443, open on both interfaces; needs https.enabled

The URLs are written into a certificate when it is signed, so a change of pki.publish or of the names applies to the certificates signed afterwards. An empty pki.publish leaves the extensions out.

The CRLs of the issuing CAs are valid for pki.crl_days, and the root CRL, when the router signs it, for pki.root_crl_days. A CRL is signed again at every revocation and, once half of its validity has passed, by vpn-router-pki.timer, which runs vpn-router-ca maintain daily; it also has vpn-router-setup renew the router's certificates when they are due. OpenVPN checks the CRL of every CA of a client's chain, the root's included, and refuses every client while one has expired.

Offline root

Keeping the root key off the router is recommended, not required; while it is on the router, vpn-router-ca status says so, unless pki.root_key_warning is false. Export it, encrypted with a passphrase read from the first line of standard input, keep the file in a safe place, and remove the key from the router:

vpn-router-ca root export-key --out /root/vpn-router-root-key.pem
vpn-router-ca root remove-key

While the root key is not on the router, a new issuing CA gets its key and a CSR and waits; vpn-router-ca status says so. Print the CSR and the extensions to sign it with, sign it with the root on the offline machine, and import the certificate:

vpn-router-ca ca csr servers          # the CSR, then the [ issuing_ca ] extensions
# on the offline machine, with the CSR in servers.csr and the extensions in issuing_ca.cnf:
openssl x509 -req -in servers.csr -CA root-cert.pem -CAkey vpn-router-root-key.pem -CAcreateserial \
    -days 3650 -sha256 -extfile issuing_ca.cnf -extensions issuing_ca -out servers.pem
vpn-router-ca ca import servers servers.pem

The router cannot sign the root CRL without the key. Sign it where the key is, or put the key back for a moment, and take the CRL before the current one expires:

vpn-router-ca root import-key /root/vpn-router-root-key.pem   # the passphrase on standard input
vpn-router-ca maintain
vpn-router-ca root remove-key
# or, with a CRL signed elsewhere:
vpn-router-ca root import-crl root.crl

An existing root CA of your own is used by placing its certificate, and its key if the router may use it, as /etc/vpn-router/pki/root/cert.pem and key.pem before vpn-router-setup first runs with a platform, as cloud-init can. Setup imports them into the directory and removes the files. Without the key, the issuing CAs wait for their certificates as above.

ACME server

A general-purpose CA with "acme": true backs an ACME server of Caddy, while https.enabled, at https://<wan.local_fqdn>:<https.web_port>/acme/<name>/directory, so servers get and renew certificates with any ACME client:

certbot certonly --standalone -d web.example.com \
    --server https://router.example.com:7444/acme/servers/directory

Caddy proves the client's control of the name as Let's Encrypt does, so it must reach the client. The certificates are valid for 12 hours, carry Caddy's extensions, are not listed by vpn-router-ca list, cannot be revoked, and no template applies to them. Caddy reads the CA's certificate and key from /etc/caddy/vpn-router-pki/<name>/, readable by its group.

OpenVPN and users

With openvpn.enabled, an address_pool and wan.local_fqdn set, vpn-router-setup runs openvpn-server@vpn-router with /etc/openvpn/server/vpn-router.conf. The server uses the router's VPN server certificate, the root and the P2S CA and their CRLs (see Certificates), pushes routes to local.cidrs and the router's addresses of p2s.dns_listen as DNS servers, and opens openvpn.port on the external interface.

A client needs both a certificate of the P2S CA and the password of the user the certificate names:

  • the certificate is checked against the root and the P2S CA and their CRLs (crl-verify, read again when the file changes), and /usr/lib/vpn-router/openvpn-issuer (tls-verify) refuses a certificate another issuing CA signed;
  • the password is checked by openvpn-auth-ldap, which binds to the user directory as the user; a user enrolled for TOTP types the password followed by the current 6-digit code;
  • /usr/lib/vpn-router/openvpn-same-user refuses a login whose user name is not the common name of the certificate.

The user directory

Once a platform is chosen, vpn-router-setup keeps a directory of users in the local slapd, whether or not OpenVPN or road-warrior access is enabled, so users can be managed before either is switched on:

  • its own database, dc=vpn-router,dc=local in /var/lib/ldap/vpn-router, next to the one the slapd package creates; root manages it through the local socket, with no administrator password;
  • ou=users holds one entry per user, named by uid: an inetOrgPerson for a VPN user, an account for a Manager-only account (see Managing users);
  • the package's own schema, cn=vpn-router,cn=schema,cn=config, under the OID 2.25.98477352595735834532404256531620013001 of the UUID arc: the auxiliary class vpnRouterAccount, which Manager-only and service accounts have, and the attribute vpnRouterScope, authorization scopes, of which service marks a service account; displayName and description are the display name and the description of a service account, and vpnRouterRedirectUri, vpnRouterPostLogoutRedirectUri and vpnRouterAllowedGroup the settings of a service account that is a client of the OpenID Provider;
  • ou=groups holds one groupOfNames per group, named by cn, with its members' DNs in member and an optional description;
  • cn=user-manager,ou=services is the account that changes users and groups; it may write ou=users and ou=groups and set passwords, never read them, and its password is in /etc/vpn-router/ldap/user-manager.secret (root only);
  • passwords are stored as ARGON2 hashes, and a disabled user is locked with the password policy (pwdAccountLockedTime);
  • the password policy locks an account for 15 minutes after 5 failed binds within 15 minutes, whether they come from OpenVPN or from /auth; a successful bind clears the count, and vpn-router-user unlock ends a lockout;
  • slapd's otp overlay is configured on the database; a user enrolled for TOTP is also an oathTOTPUser and oathTOTPToken whose oathTOTPToken names the entry itself, with the secret in oathSecret, which the user manager account may set and no one can read, and oathTOTPParams naming cn=totp,ou=policies (6 digits, HMAC-SHA1, 30 second steps, one step of tolerance). A bind of an enrolled user takes the password followed by a current code;
  • slapd listens on its local socket only (SLAPD_SERVICES in /etc/default/slapd).

The directory is kept on purge.

Managing users

vpn-router-user manages the users. A user is a directory entry of one of three kinds, fixed when it is added:

  • a VPN user, an inetOrgPerson with a first name, a last name and an e-mail address (givenName, sn and mail; cn is the first and the last name), and a client certificate with the user name as its common name, the entry's userCertificate, whose key the PKI keeps;
  • a Manager-only account, an account, simpleSecurityObject and vpnRouterAccount with the name, the password and an optional e-mail address, for notifications: no names and no certificate, so no VPN access. It can sign in to /auth and, as a member of admins, manage the router. add without the first and the last name adds one; its names cannot be set later;
  • a service account, a Manager-only account whose vpnRouterScope is service, which another system signs in with to use a router service, such as an OpenTelemetry Collector reading /metrics; it cannot sign in to /auth. Besides its name, it has a display name and a description, and a service account with redirect URIs is a client of the OpenID Provider (see OpenID Provider). add --service adds one, and the web service manages them at /service-accounts, not at /users. Internal service accounts are those the router creates and manages for its own services, display name and description included; at present that is the metrics service's, the account metrics.service_account names while metrics are enabled (see Metrics), which vpn-router-setup names Metrics. The API only lists them: /service-accounts refuses every other request on one, and /groups refuses to put one in or take one out of a group, with 403.

Passwords and TOTP secrets are read from the first line of standard input. The answer is text: a table of the users, the client profile, a TOTP secret or a short message, and a failure is a message on standard error. --json, before the command, gives the answer as JSON on standard output, and a failure as {"error": ...} there too; the web service uses it. -h or --help shows the commands, and after a command its options.

echo 's3cr3t-pass' | vpn-router-user add alice --first-name Alice --last-name Smith --email alice@example.com
echo 'm4nager-pass' | vpn-router-user add ops
echo 's3rvice-pass' | vpn-router-user add backup --service
vpn-router-user update alice --email a.smith@example.com
vpn-router-user list
vpn-router-user profile alice > alice.ovpn
vpn-router-user --json list
echo 'n3w-pass' | vpn-router-user passwd alice
vpn-router-user disable alice
vpn-router-user enable alice
vpn-router-user unlock alice
vpn-router-user remove alice
vpn-router-user otp-new alice
echo 'JBSWY3DPEHPK3PXPJBSWY3DPEHPK3PXP' | vpn-router-user otp-enrol alice 123456
vpn-router-user otp-remove alice
Command Effect
list The users with their kind, names and e-mail addresses, whether each is enabled or locked, has a certificate and is enrolled for TOTP, the number of passkeys and the console login
add <name> --first-name F --last-name L --email E Adds a VPN user and issues the certificate
add <name> [--email E] Adds a Manager-only account, with the e-mail address when given; one name without the other two options is refused
add <name> --service [--display-name N] [--description D] [--email E] [--redirect-uri U ...] [--post-logout-redirect-uri U ...] [--allowed-group G ...] Adds a service account, with its display name, description, e-mail address and OpenID Provider client settings when given; the names are refused
update <name> [--first-name F] [--last-name L] [--email E] Changes the names or the e-mail address
update <name> [--display-name N] [--description D] [--email E] [--redirect-uri U ...] [--post-logout-redirect-uri U ...] [--allowed-group G ...] Changes the display name, the description, the e-mail address or the OpenID Provider client settings of a service account; '' removes one; a repeated client option gives all values of its kind
passwd <name> Sets the password
disable <name>, enable <name> Locks the user out of OpenVPN and /auth, and lets the user in again
unlock <name> Ends a lockout after failed logins and clears the failure count; a disabled user stays disabled
remove <name> Takes the user out of its groups, revokes the certificate, renews the CRL and removes the user; refused while the user is the last member of a group
profile <name> The OpenVPN client profile, with the CA, the certificate and the key inline
otp-new <name> A new TOTP secret and its otpauth:// URI, issued by wan.local_fqdn, for an authenticator app; nothing is stored
otp-enrol <name> <code> Stores the secret read from standard input when the code is a current code of it, replacing a secret the user has
otp-remove <name> Removes the user's TOTP secret and ends the sessions of the user's own POSIX account
console <name> (--posix | --account LOCAL | --off) Switches the console login of a member of admins enrolled for TOTP on, as the user's own POSIX account or a local account, or off (see Console login and SSH)
passkeys-remove <name> Removes all the user's passkeys (see Passkeys)

A user name is 1 to 32 lowercase letters, digits and hyphens, starting with a letter, and a service account may instead be named by a GUID in lowercase, such as 3f2b8c1e-9a4d-4e6f-8b2a-1c5d7e9f0a3b; a password has 8 to 128 characters; a first or last name has 1 to 64 printable characters; an e-mail address has one @, no spaces and a dot in its domain; a TOTP secret is 32 base32 characters, as otp-new makes it, and a code 6 digits. Once enrolled, a user logs in to OpenVPN and /auth with the password followed by a current code. In the answers manager_only, service and internal tell the kind, and a Manager-only account has its names as null; update changes only its e-mail address, and --email '' removes it. Changing a user's password, disabling, unlocking or removing the user also revokes its refresh tokens of /auth and its codes, tokens and sessions of the OpenID Provider. The exit status is 0 on success, 2 for an invalid request, 3 when the user does not exist, 4 when the user exists and 1 for any other failure.

When the Router Manager is installed with an administrator (its debconf questions vpn-router-manager/admin_username and admin_password), its package leaves the answers in /etc/vpn-router-manager/admin.json (root only). vpn-router-setup then creates that user as a Manager-only account, adds it to the group admins, which it creates when missing, and removes the file.

The certificate is also accepted by road-warrior (IKEv2 EAP-TLS) access, which checks certificates only, so disable does not affect it; remove revokes the certificate for both.

Console login and SSH

A member of admins enrolled for TOTP may have console login: the Router Manager's Shell page (see The shell), and, for the user's own POSIX account, SSH and sudo. Only root on the router switches it, with vpn-router-user console or a token of vpn-router-token, as for the group admins. Switching it on is refused without the enrolment, and a user whose enrolment is removed is refused the shell, SSH and sudo until enrolling again; the user's open shells end within a minute, and removing the enrolment ends the sessions of the user's own POSIX account:

vpn-router-user console alice --posix
vpn-router-user console bob --account azureuser
vpn-router-user console alice --off
vpn-config-edit set ssh.ldap_login true
  • Own POSIX account (--posix): the user's entry becomes a posixAccount of the directory with the user's name, a uid from 20000 up, the local group vpn-router-console as its group, /home/<name> and /bin/bash. nslcd makes it known to the system over slapd's local socket and checks its password by binding as the user, so the user types the password followed by a current code, failed logins count towards the lockout, and a disabled user cannot sign in. The home is created at the first login and kept when the user is removed. Members of vpn-router-console may run any command with sudo, with the same password. A local account of the same name is refused.
  • Local account (--account): the console runs as an existing local account with a uid from 1000 to 19999, such as the VM's admin user; its SSH keys and sudo rights stay as they are.
  • Off (--off): the system no longer knows the user's own POSIX account, and its sessions end; its uid is kept for the next time. Leaving admins, or removing or renaming the group, switches the console login of the members off.

While ssh.ldap_login is true, setup lets sshd accept the password of the members of vpn-router-console (/etc/ssh/sshd_config.d/vpn-router.conf); other accounts keep sshd's settings. A password change is made in the Router Manager or with vpn-router-user passwd, not with passwd.

Passkeys

A user who is not a service account may sign in to the Router Manager and to the OpenID Provider with a passkey instead of the password and code. Passkeys are for those sign-ins only: OpenVPN, the shell, SSH and sudo keep the password and code. A passkey belongs to wan.local_fqdn, so the browser offers it on the HTTPS addresses on that name, and passkeys need the name set; a passkey made before the name changes is no longer offered. The authenticator checks the user (a PIN or a fingerprint), and the passkey is kept by it or by the user's password manager.

A user adds a passkey on the Router Manager's Account page with the current password and code, and removes it there. Each passkey is a value of vpnRouterPasskey on the user's entry, with its public key, label and the time it was added; a user has at most 10. An administrator removes all passkeys of a user on the Users page, through /users, or with:

vpn-router-user passkeys-remove alice

A disabled or locked user is refused a passkey sign-in too.

Managing groups

vpn-router-group manages the groups. A group has a name, with the rules of a user name, an optional description of at most 256 printable characters, and at least one member, each a user of the directory. The group admins grants the management of the router, and the allowed groups of a client of the OpenID Provider admit their members to it; other groups record membership only. Renaming a group renames it in the clients that allow it, and a group a client allows cannot be removed. The output, --json, -h and --help work as for vpn-router-user.

vpn-router-group add ops --member alice --member bob --description 'Operations team'
vpn-router-group list
vpn-router-group update ops --name netops
vpn-router-group update netops --description ''
vpn-router-group member-add netops carol
vpn-router-group member-remove netops bob
vpn-router-group remove netops
Command Effect
list The groups with their members and descriptions
add <name> --member U [--member U ...] [--description D] Adds the group with its members
update <name> [--name N] [--description D] Renames the group or sets its description; '' removes it
remove <name> Removes the group; its members are kept; refused while a client of the OpenID Provider allows it
member-add <name> <user> Adds the user to the group
member-remove <name> <user> Takes the user out of the group; refused for the last member

The exit status is 0 on success, 2 for an invalid request, 3 when the group or the user does not exist, 4 when the group exists and 1 for any other failure.

Platform modules

Platform-specific values do not belong in the core, so they are Python modules in /usr/lib/vpn-router/vpnrouter_platforms/, imported at run time and selected by general.platform. The platform is never probed: the caller already knows where it is deploying.

none is not a module. It is the shipped default and means "configure nothing at all", which is how a deferred install is expressed. generic is the module to choose when the router should be configured but the platform adds nothing.

The shipped modules currently add nothing. Road-warrior, OpenVPN and WireGuard clients are given the router's addresses of p2s.dns_listen as their DNS servers, by default the internal address, the WireGuard clients in the configuration the Router Manager makes for a device (DNS =), where vpn-router-resolver.service, dnsmasq with /etc/vpn-router/dnsmasq.conf, answers and the firewall opens port 53 for the client pools and the WireGuard network, so no provider resolver has to be advertised or routed through the tunnel on any platform. dnsmasq forwards to systemd-resolved on 127.0.0.53, which keeps only its loopback stubs, and forwards no name without a dot, so the router's own host name, _gateway and localhost are not answered to the clients. The clients reach the internal address when it is inside local.cidrs, which they route through the tunnel. Port 53 of the other addresses stays free for the name server (see Name server). The modules exist as the documented place for platform specifics if that changes.

A module may define any of the following, and takes the default for anything it omits:

Name Type Effect
EXTRA_P2S_DNS str added to the road-warrior pool dns list
EXTRA_LOCAL_TS str added to local_ts on both connections
apply(ctx) function called once the system is configured, to apply platform-specific state; must be idempotent

ctx is the template context, so a module reads what it needs from it - ctx['wan_iface'], ctx['int_iface'], ctx['local_cidrs'] and the rest.

Adding a platform is adding one file. Modules are discovered by listing the package, so nothing in the core needs editing:

# /usr/lib/vpn-router/vpnrouter_platforms/example.py
"""Example provider."""

EXTRA_P2S_DNS = '192.0.2.53'
EXTRA_LOCAL_TS = '192.0.2.53/32'

Set general.platform to example and restart the service. The debconf question offers only the modules shipped with the package; the configuration file accepts any module installed.

Status web service

The vpn-router-web service shows the state of the router on the addresses of the interfaces web.listen names, the internal, the external or both (the default), and on 127.0.0.1, where Caddy calls it (see HTTPS), with web.port (7081 by default). Its pages / and /status need no authentication and show no configuration: no key, identity, subnet, or address but the client's. They show as much as web.status_content says, read at each request:

status_content / and /status show
basic (default) The overall status (ok or degraded), the time and the Client IP
standard Also the vpn-router version, the vpn-router-manager version or that it is not installed, and the number of active services and of services in any other state, without their names
full Also each service and its state, Docker, how many site-to-site connections are set up and how many of them are established (site_to_site: connections and established, null when swanctl cannot be asked), road-warrior, OpenVPN and WireGuard

The Client IP is the address the request came from: the last entry of X-Forwarded-For for a request from one of web.trusted_proxies, by default the address the request came in on, so behind Caddy or a Router Manager on the router it is the browser's. /status names the content in content and the Client IP in client_ip; / keeps the time and the link to /status.

The service first makes its signing key, the OpenID Provider's key and the refresh tokens' database, retrying every 15 seconds, for example while PostgreSQL is not running yet, and logs each failure; then it listens.

Bootstrap diagnostics

Until a run of vpn-router-setup has succeeded, every run starts vpn-router-bootstrap, which serves diagnostics as plain text on port 17081 of every address, and the firewall opens 17081 and 17080 (the Router Manager's bootstrap) on the internal interface. The first successful run stops the bootstrap services and closes their ports, and writes /var/lib/vpn-router/setup.done, after which they never run again; a failed run leaves them running, so the reason stays readable until a later run succeeds. The unit is never enabled.

The page, read again every 10 seconds, shows the state, the step and the failure of the last setup run (/run/vpn-router/setup.json, which setup writes at every step), the state of the router's services, the failed units, the addresses and routes, cloud-init status, and the end of the logs of vpn-router-setup and vpn-router-web:

curl http://10.0.0.4:17081/
Page Content
/ The status for people: plain text for curl and wget, a page styled with Tailwind CSS for a browser, in the browser's light or dark scheme, that reloads itself every minute (the style sheet is loaded from a CDN, so the browser needs internet access; the text is readable without it)
/status The same status as JSON
/auth Signing in with a user's password and TOTP code, refreshing and signing out
/config The configuration without its secret options, and what the router resolved and runs, as JSON; needs a token
/users The users of OpenVPN and road-warrior access; needs a token
/service-accounts The service accounts; needs a token
/groups The groups of users; needs a token
/site-to-site The site-to-site connections: list, add, change and remove; needs a token
/account The account of the signed-in user: its details, its password and its TOTP enrolment; needs a user's token
/logs The last lines of the journal of the router's units and of the files of web.log_files; needs a token
/sites The web sites and their files; needs a token
/dns The name server's settings and records; needs a token
/.well-known/openid-configuration, /oidc/... The OpenID Provider, on 127.0.0.1 only, where Caddy passes the requests (see OpenID Provider)

The status holds the state of vpn-router-setup, slapd and each PostgreSQL cluster unit (postgresql@<version>-<cluster>, or postgresql as no cluster without one), of vpn-router-manager when that package is installed, of strongswan when site-to-site or road-warrior access is configured, of wg-quick@wg0 when WireGuard is, and of openvpn-server@vpn-router when OpenVPN is; the number of connected OpenVPN clients, from the server's status file, which OpenVPN rewrites every 5 seconds; the package version and, as manager_version, the installed version of vpn-router-manager (null, shown as not installed, without it); whether Docker is installed, and when it is, the state of docker.service and of vpn-router-docker (the Docker rules), which do not change status; as site_to_site, the number of site-to-site connections that are set up (connections) and of those established (established, null when swanctl cannot be asked), without their names; the number of connected road-warrior (IPsec P2S) clients; and the WireGuard peers with the age of their last handshake and their traffic. status is ok when all these services are active, and degraded otherwise.

curl http://10.0.0.4:7081/
curl http://10.0.0.4:7081/status

Tokens and signing in

/config, /users, /service-accounts, /groups, /site-to-site, /backup, /remote-access, /ca, /logs, /sites, /dns, /shell and /account need a JWT in an Authorization: Bearer header, signed with HS256 by the key the web service makes on its first run, /etc/vpn-router/web.key (root only). The key never leaves the router; while it does not exist these paths and /auth are not served (404). A missing, expired or wrongly signed token, or one of another algorithm, gets 401 with WWW-Authenticate: Bearer. There are two kinds of tokens:

  • an access token of /auth names a user in sub and is valid for 5 minutes; every user may call /account, and for the other paths the user must be a member of the group admins, otherwise the request gets 403;
  • a token of vpn-router-token, run as root on the router, has the claim admin and is valid for the lifetime given, without a maximum. It names no user, so /account refuses it with 403; it may call everything else, and only such a token may change the group admins: create it, rename a group to it, change or remove it, add or take out its members, or remove a user who is a member of it.

The first member of admins is added on the router:

vpn-router-group add admins --member alice --description 'Router administrators'
vpn-router-token --hours 1
vpn-router-token --json --days 30

POST /auth signs a user in. Request and answer bodies are JSON:

Request Body Answer
POST /auth {"name", "password", "code"}; code only for a user enrolled for TOTP the tokens and "name"
POST /auth/passkey {} {"id", "options"}: a pending sign-in, valid 2 minutes, and the options of navigator.credentials.get in their JSON form
POST /auth/passkey/complete {"id", "credential"}, credential the browser's answer as its toJSON() gives it the tokens and "name", the user whose passkey signed
POST /auth/refresh {"refresh_token"} new tokens; the refresh token given is used up
POST /auth/logout {"refresh_token"} {"revoked": true}, or false for a token not known

The tokens are {"access_token", "token_type": "Bearer", "expires_in": 300, "refresh_token", "refresh_expires_in": 28800}. A wrong name, password or code, and a locked or disabled user, get 401 with the same message; a refresh token is refused once its user is disabled, locked or removed. Changing a user's password, disabling, unlocking or removing the user revokes the user's refresh tokens. Five failed logins lock the account for 15 minutes (see the user directory). A passkey that does not sign, is not the user's, or of a disabled, locked or service account gets 401 too (see Passkeys). /auth, /auth/passkey/complete and /auth/refresh take at most 10 calls a minute from one client address, and answer 429 with Retry-After beyond that; for a call from one of web.trusted_proxies, by default the address the call came in on, the client address is the last entry of X-Forwarded-For.

Refresh tokens are kept, as their SHA-256, in PostgreSQL: by default in the database and role vpn-router on the router, which the web service creates on its first run with a password in /etc/vpn-router/database.secret (root only), reached over TCP on localhost; web.database names another database. Tokens and passwords travel over plain HTTP on the internal network.

curl -H 'Content-Type: application/json' -d '{"name": "alice", "password": "s3cr3t-pass", "code": "123456"}' \
     http://10.0.0.4:7081/auth
curl -H "Authorization: Bearer $TOKEN" http://10.0.0.4:7081/config

The /status/full page

/status/full, with the same token as /config, shows the whole status document, as /status does with web.status_content full, whatever that setting says, so the Router Manager shows every service and tunnel while the public status page shows less.

The shell

While web.shell is true, /shell gives a member of admins whose console login is on and who is enrolled for TOTP (see Console login and SSH) a login shell on the router, as the user's console account, for the Router Manager's Shell page. It is off by default and switched on and off on the router only:

vpn-router-user console alice --posix
vpn-config-edit set web.shell true
Request Body Answer
POST /shell {"password", "code"}; {} within web.shell_auth_timeout {"ticket", "expires_in": 30}
GET /shell?ticket=<ticket> a WebSocket handshake the terminal

POST /shell needs an access token of /auth, not one of vpn-router-token, and signs its user in again: a wrong password or code gets 403 and counts as a failed login; it gets 403 too while web.shell is false, the user's console login is off or the user is not enrolled for TOTP. Without a password it gives a ticket while the user's last sign-in of POST /shell or keystroke in a shell is less than web.shell_auth_timeout minutes ago, and {"ticket": null} otherwise. The ticket opens one shell for the same user within 30 seconds. Over the WebSocket, binary messages of the client are keystrokes, a text message {"cols", "rows"} sets the window size, and binary messages of the router are the terminal's output. The shell is a PAM login session of the console account, run by systemd-run in a transient unit vpn-router-shell-<id>; sudo works as for SSH. It is closed after web.shell_idle_minutes without a keystroke, when the shell exits, when the connection closes, or within a minute of the user's console login being switched off or the TOTP enrolment removed; the router then sends a close frame with the reason. The journal of vpn-router-web records who opened each shell, from which address, in which unit, when it ended and why, and each refused ticket or shell with the reason.

The /config page

/config shows what / and /status leave out, for a client such as manage-ui:

Key Content
configuration vpn-router.json with the defaults applied, without its secret options at any level (psk, database, and any option named like a secret, password, token or key)
network The external and internal interface, their addresses, the internal gateway, local.cidrs, site_to_site: each connection with name, local_identity, remote_identity, local_cidrs and remote_cidrs, and the IPv4 routes as ip -j -4 route prints them
ipsec connections and sas: one object per connection and per IKE SA, read from swanctl --list-conns --pretty and swanctl --list-sas --pretty, with the connection's name under name and swanctl's keys and string values
wireguard The public key and port of wg0, and each peer's public key, endpoint, allowed IPs, handshake age and traffic; no private or preshared key
docker docker.exposed_ports and the rules of the VPN-ROUTER-DOCKER chain
postgresql clusters: each PostgreSQL cluster of pg_lsclusters with its version, name, port, status and data directory, and for one that is online its settings: server_version, listen_addresses, port, max_connections, shared_buffers, effective_cache_size, work_mem, password_encryption, ssl and timezone; null when pg_lsclusters fails
caddy The installed Caddy: version, source, upstream once Caddy's repository is configured and ubuntu otherwise, and service, the state of caddy.service; null when Caddy is not installed

The /postgresql page

/postgresql, with the same token as /config, shows the databases of the PostgreSQL clusters on the router:

  • Content: clusters lists each cluster with its version, name, port and status, its databases (name, owner, encoding and size in bytes, pg_database_size) and their total size.
  • A cluster that is not online: databases and size are null.
  • Collection: the web service reads it with psql as the postgres user at each request.

The /logs API

/logs, with the same token as /config, shows the last lines of the journal and of log files, read at each request:

Request Answer
GET /logs {"units", "files", "max_lines"}: the units whose journal it shows, the files of web.log_files and the most lines one request answers, 1000
GET /logs/journal?unit=<unit>&lines=<n> {"lines": [...]}: the last lines of the unit's journal, or of the whole journal without unit, with ISO times
GET /logs/file?path=<path>&lines=<n> {"lines": [...]}: the last lines of a file web.log_files lists

The units are the package's own (vpn-router-setup, -web, -metrics, -pki, -backup, -caddy, -resolver, -webdav), vpn-router-manager, caddy, slapd, nslcd, postgresql, strongswan, wg-quick@wg0, openvpn-server@vpn-router, ssh and pdns. lines is 1 to 1000, 200 without it. A file is read from at most its last MiB, so very long lines give fewer lines. A unit or a file not in the lists is 404, as is a listed file that does not exist.

vpn-config-edit set web.log_files /var/log/syslog,/var/log/auth.log

The /sites API

/sites, with the same token as /config, manages the web sites, with the module vpn-router-site uses too, /usr/lib/vpn-router/vpnrouter_sites.py:

Request Answer
GET /sites {"settings", "sites"}: the sites section and the sites, each with its settings but private_key, with private_key_set, modified and directory
PUT /sites Changes any of the settings of the sites section
POST /sites Adds a site: {"name", ...} with any of its settings; an existing name is 409
GET /sites/<name> The site
PUT /sites/<name> Changes any of its settings; an empty or absent private_key keeps the key
DELETE /sites/<name>?content=true Removes the site, with content=true also its folder
GET /sites/<name>/files/<path> A folder: {"path", "entries": [{"name", "folder", "size", "modified"}]}; a file: the file, as an attachment
PUT /sites/<name>/files/<path> Writes the body to the file, creating its folders; with ?unzip=true the body is a zip archive extracted into the folder of the path: {"path", "files"}
POST /sites/<name>/files/<path> {"folder": true} creates the folder; {"rename": "<path>"} moves the file or folder
DELETE /sites/<name>/files/<path> Removes the file or folder

The paths of /files/ are relative to the site's folder and may not leave it; a PUT needs a Content-Length. The files of a site may also be used by a member of its editors group. Invalid settings are 400, a missing site 404.

curl -X PUT -H "Authorization: Bearer $TOKEN" --data-binary @site.zip \
    'https://router.example.com:7444/sites/www/files/?unzip=true'

The /dns API

/dns, with the same token as /config:

Request Answer
GET /dns {"settings", "installed", "running", "records", "manual", "problem"}: the dns section, whether PowerDNS is installed and runs, the records it serves, the manual records, and why they cannot be read, null when they can
PUT /dns Changes any of the settings of the dns section
POST /dns/records Adds a manual record: {"name", "type", "content", "ttl"?}; name is relative to dns.zone, @ for the zone itself, or absolute with a final dot, and ttl is dns.ttl without it
DELETE /dns/records?name=&type=&content= Removes the manual records of the name and type, only the one of content when it is given

A TXT content without quotes is quoted; MX is <preference> <host>, SRV <priority> <weight> <port> <host> and CAA <flags> <tag> "<value>". A CNAME must be its name's only record.

The /users API

/users manages users with the same token as /config, with the module vpn-router-user uses too, /usr/lib/vpn-router/vpnrouter_users.py, so both apply the same rules. Request and answer bodies are JSON; a request body is at most 65536 bytes.

Request Body Answer
GET /users {"users": [{"name", "manager_only", "service", "internal", "enabled", "locked", "certificate", "otp", "console", "wireguard_peers", "passkeys", "first_name", "last_name", "email"}, ...], "wireguard": {"enabled", "address"}}; console is null or {"enabled", "account", "local", "uid"}, wireguard_peers a list of {"public_key", "ips"}, passkeys a list of {"id", "label", "created"}, and wireguard the endpoint's settings, whose network the peers' addresses come from
POST /users {"name", "password", "first_name", "last_name", "email"} for a VPN user; {"name", "password", "email"?} for a Manager-only account 201 and the user
PUT /users/<name> any of {"first_name", "last_name", "email", "wireguard_peers"}; wireguard_peers, [{"public_key", "ips"}], replaces a VPN user's WireGuard peers (see Configuration); for a Manager-only account only {"email"}, "" removing it the user
PUT /users/<name>/password {"password": ...} the user
POST /users/<name>/disable the user
POST /users/<name>/enable the user
POST /users/<name>/unlock the user
DELETE /users/<name> {"name", "removed": true}
GET /users/<name>/profile {"name", "profile"}, the OpenVPN client profile
POST /users/<name>/otp {"name", "secret", "uri"}, a new TOTP secret; nothing is stored
PUT /users/<name>/otp {"secret", "code"} the user, once the code matches the secret
DELETE /users/<name>/otp the user
DELETE /users/<name>/passkeys the user, without passkeys
PUT /users/<name>/console {"enabled", "account"?}; only with a token of vpn-router-token the user

A failure is {"error": ...} with 400 for an invalid request, 404 for a user that does not exist or is a service account, 409 for a user that exists and 500 otherwise. /users does not list service accounts; /service-accounts serves them. /users works whether or not OpenVPN or road-warrior access is enabled; the profile needs OpenVPN. Passwords, private keys and TOTP secrets cross the internal network in plain HTTP, like the token.

curl -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
     -d '{"name": "alice", "password": "s3cr3t-pass", "first_name": "Alice", "last_name": "Smith", "email": "alice@example.com"}' \
     http://10.0.0.4:7081/users

The /service-accounts API

/service-accounts manages the service accounts with the same token and the same module as /users.

Request Body Answer
GET /service-accounts {"service_accounts": [{"name", "display_name", "description", "internal", "enabled", "locked", "email", "redirect_uris", "post_logout_redirect_uris", "allowed_groups", ...}, ...], "oidc": {"enabled", "issuer"}}, each as /users shows a user, and the issuer of the OpenID Provider, null while it is disabled
POST /service-accounts {"name", "password", "display_name"?, "description"?, "email"?, "redirect_uris"?, "post_logout_redirect_uris"?, "allowed_groups"?} 201 and the account
PUT /service-accounts/<name> any of {"display_name", "description", "email", "redirect_uris", "post_logout_redirect_uris", "allowed_groups"}; "" removes one of the first three, [] the values of the others the account
PUT /service-accounts/<name>/password {"password": ...} the account
POST /service-accounts/<name>/disable the account
POST /service-accounts/<name>/enable the account
POST /service-accounts/<name>/unlock the account
DELETE /service-accounts/<name> {"name", "removed": true}

A display name has at most 64 printable characters and a description at most 256. The client settings are lists of at most 16 values: URIs of https, or of http for localhost, 127.0.0.1 or ::1, without a fragment and of at most 1024 characters, and names of existing groups (see OpenID Provider). The internal service accounts, internal true, are only listed: every other request on one gets 403, as the router manages them, their display name and description included. A failure is otherwise as for /users, with 404 also for a name that is not a service account.

The /account API

/account serves the user the access token names, whether or not the user is a member of admins, with the same module as /users:

Request Body Answer
GET /account the user as /users shows it, and "admin", whether the user is a member of admins
PUT /account/password {"current_password", "current_code", "password"} the user
POST /account/otp {"name", "secret", "uri"}, a new TOTP secret; nothing is stored
PUT /account/otp {"current_password", "current_code", "secret", "code"} the user, once the code matches the secret
POST /account/passkeys {"current_password", "current_code"} {"id", "options"}: a pending registration, valid 2 minutes, and the options of navigator.credentials.create in their JSON form
PUT /account/passkeys {"id", "credential", "label"?}, credential the browser's answer as its toJSON() gives it; label at most 64 characters, Passkey without one the user
DELETE /account/passkeys/<id> the user

current_code is needed only for a user enrolled for TOTP; removing a passkey needs neither. A wrong current password or code gets 403 and counts as a failed login; slapd accepts each code once, so a code used to sign in is not accepted again, and a change right after signing in waits for the next code. A user enrols or replaces the enrolment; removing it is left to /users and vpn-router-user. Changing the password revokes the user's refresh tokens. Other failures are those of /users.

The /groups API

/groups manages groups with the same token and the same module as /users.

Request Body Answer
GET /groups {"groups": [{"name", "description", "members": [...]}, ...]}
POST /groups {"name", "members": [...]}, and "description" 201 and the group
PUT /groups/<name> any of {"name", "description"}; "" removes the description the group
DELETE /groups/<name> {"name", "removed": true}
POST /groups/<name>/members {"user": ...} the group
DELETE /groups/<name>/members/<user> the group

A failure is {"error": ...} with 400 for an invalid request, among them taking out the last member, 404 for a group or user that does not exist, 409 for a group that exists and 500 otherwise. DELETE /users/<name> is refused with 400 while the user is the last member of a group.

The /site-to-site API

/site-to-site manages the site-to-site connections of vpn-router.json with the same token as /users.

Request Body Answer
GET /site-to-site {"connections": [{"name", "addrs", "id", "cidrs", "local_cidrs", "local_id_mode", "auth", "psk_file", "remote_ca", "psk_set", "established"}, ...]}
POST /site-to-site {"name", "addrs", "id", "cidrs", "local_cidrs", "local_id_mode", "auth", "psk", "remote_ca"}; psk is required with auth psk, the default 201 and the connection
PUT /site-to-site/<name> any of the settings of POST but name; an empty or absent psk keeps the key, a psk given replaces psk_file the connection
DELETE /site-to-site/<name> {"name", "removed": true}

A connection is shown without its key: psk_set says whether psk holds one, and established whether its tunnel is up, null when swanctl cannot be asked. Each change is made under the lock vpn-config-edit takes, checked as vpn-router-setup checks the configuration, written to vpn-router.json, and applied by restarting vpn-router-setup without waiting for it. A failure is {"error": ...} with 400 for an invalid request or configuration, with the reason, 404 for a connection that does not exist, 409 for a name that exists and 500 otherwise. Keys cross the internal network in plain HTTP, like the token.

The /backup API

/backup shows and changes the backup settings and starts a backup, with the same token as /users.

Request Body Answer
GET /backup {"settings", "state", "archives", "next", "client", "rclone"}
PUT /backup any of the settings of the backup section; an empty or absent pbs_password keeps the secret {"settings"}
POST /backup/run 202 and {"started": true}; 409 while a backup runs

The answer of GET /backup holds:

  • settings: the backup section without pbs_password, with pbs_password_set.
  • state: the last run, null before the first. Its result is running, ok, warning (the backup was made but the cloud copy failed), failed or interrupted (the run ended without recording its result). It also has started, finished, target, archive or snapshot, size in bytes, removed (the archives retention deleted), error and cloud ({"remote", "mode", "result", "error"}).
  • archives: the local archives of this host, newest first, as {"name", "time", "size"}.
  • next: the next run as systemd shows it, null while backups are off.
  • client: whether proxmox-backup-client is installed.
  • rclone: {"installed", "config", "remotes"}, the remotes being the sections of rclone.conf.

A change is made, checked and applied as for /site-to-site.

The /remote-access API

/remote-access shows and changes the remote access systems, IKEv2 road-warrior (p2s), OpenVPN (openvpn) and WireGuard (wireguard), with the same token as /users.

Request Body Answer
GET /remote-access {"p2s", "openvpn", "wireguard", "running", "local_fqdn", "wireguard_server"}; wireguard_server is {"public_key", "endpoint", "allowed_ips", "dns"}: the router's WireGuard public key (null before setup made it), <wan.local_fqdn>:<wireguard.listen_port> (null without a name), the networks a peer reaches through the tunnel, the router's local CIDRs and the WireGuard network, and the addresses of the remote access clients' resolver ([] while it does not run)
PUT /remote-access/<system> any of the settings of the section p2s, openvpn or wireguard the section

running tells for each system whether setup runs it. IKEv2 and OpenVPN run once enabled with an address pool, wan.local_fqdn set and the router's VPN server certificate issued by the P2S CA; WireGuard once enabled with an address. A system enabled without its address pool or address is refused. A change is made, checked and applied as for /site-to-site. The router's firewall follows the settings; a cloud firewall in front of the router, such as an Azure network security group, must allow the OpenVPN and WireGuard ports itself.

The /ca API

/ca manages the PKI (see Certificates) with the same token as /users, with the module vpn-router-ca uses too, /usr/lib/vpn-router/vpnrouter_pki.py.

Request Body Answer
GET /ca {"settings", "templates", "extended_key_usages", "status", "certificates"}
PUT /ca any of the settings of the pki section {"settings", "templates"}
POST /ca/certificates {"ca", "template", "csr"} 201 and {"serial", "certificate", "chain"}, PEM
POST /ca/certificates {"ca", "template", "subject", "sans", "password"} 201 and {"serial", "pfx"}, a PKCS#12 file in base64
POST /ca/certificates/<serial>/revoke {"reason"}, optional the certificate, with reissue for a router certificate
GET /ca/bundle/<name> {"name", "pem"}: the certificates a client trusts the CA with, the root alone for root, the issuing CA followed by the root for an issuing CA
GET /ca/issuing/<name>/csr {"name", "csr", "extensions"} of an issuing CA waiting for the offline root
PUT /ca/issuing/<name>/certificate {"certificate"}, PEM, signed by the root {"name", "subject", "expires"}
PUT /ca/root/crl {"crl"}, PEM, signed by the root {"next_update"}

The answer of GET /ca holds:

  • settings: the pki section.
  • templates: every template, the built-in ones included, with all its options.
  • extended_key_usages: every extended key usage templates may give, name -> {"oid", "builtin"}.
  • status: root (subject, expires, key, whether the root key is on the router, crl_next_update and crl_source, signed or uploaded), authorities, one per issuing CA (name, common_name, state: active, pending or missing, builtin, templates, acme, subject, expires, key, crl_next_update, acme_url), retired and warnings.
  • certificates: the certificates issued, newest first, each {"serial", "ca", "kind", "template", "subject", "sans", "created", "expires", "revoked", "reason", "state"}; kind is ca, router, user or external, and state valid, expired or revoked.

A change of the settings is made, checked and applied as for /site-to-site; an imported issuing CA certificate and a revoked router certificate restart vpn-router-setup too. A failure is {"error": ...} with 400 for an invalid request, 404 for something that does not exist, 409 when the state does not allow it, as for a revoked certificate or an issuing CA that has its certificate, and 500 otherwise. The root key never crosses the API; vpn-router-ca exports, removes and imports it.

The service starts on its own, without waiting for vpn-router-setup, and keeps running. Every 30 seconds it reads web.listen, web.port and web.enabled from vpn-router.json and works out the addresses of the interfaces web.listen names as general.mode directs; it listens on each, moves when an address or the port changes, and stops listening while web.enabled is false or no address is known. A change of web.listen takes effect within 30 seconds, or when the service restarts. Each request reads the signing key and vpn-router.json, and the features and the facts for /config from /run/vpn-router/web.json (readable by root only), which vpn-router-setup writes; before that file exists the page shows no features. vpn-router-setup opens web.port on the internal and the external interface in the firewall.

HTTPS

Caddy serves the status web service on https.web_port (7444 by default), while the Router Manager is installed the manager on https.manager_port (7443), and while metrics are enabled the metrics service on metrics.https_port (9465), all for wan.local_fqdn, while https.enabled is true and wan.local_fqdn is set. It obtains and renews the certificate from Let's Encrypt with the HTTP challenge, so wan.local_fqdn must resolve to the router's public address and port 80 must be reachable from the internet; the certificate and the ACME account are kept in /var/lib/caddy. Port 80 answers the challenge only; it does not redirect to HTTPS.

vpn-router-setup writes Caddy's configuration to /etc/caddy/vpn-router.Caddyfile (root, group caddy, the group Caddy runs as, mode 0640), which the package's drop-in /usr/lib/systemd/system/caddy.service.d/vpn-router.conf makes caddy.service read; caddy.service does not start before the file exists, and /etc/caddy/Caddyfile is not used. Setup starts Caddy, reloads it when the file changes, and stops it and removes the file when HTTPS is off and no web site is served (see Web sites). In the firewall it opens port 80 on the external interface and the HTTPS ports on the internal and the external interface, and ports 80 and 443 on both when pki.publish has http-80 and https-default, on which Caddy serves only the CA files (see Certificates). Caddy calls both services on 127.0.0.1, where they listen whatever web.listen and the manager's listen say, and passes the client's address in X-Forwarded-For, which the manager trusts from 127.0.0.1 and the web service by default, as the address the call came in on; a web.trusted_proxies that is set must list 127.0.0.1. The plain HTTP ports stay as web.listen and the manager's listen set them. docker.exposed_ports may not take port 80 or the HTTPS ports while https.enabled is true.

curl https://router.example.com:7444/status

Installing Caddy

The package does not depend on Caddy; vpn-router-setup has it installed, while HTTPS is on, from the source https.caddy_source names:

  • ubuntu, the default: the caddy package of the Ubuntu archive.
  • upstream: Caddy's own repository, https://dl.cloudsmith.io/public/caddy/stable/deb/debian, whose source the package writes to /etc/apt/sources.list.d/caddy-stable.sources with its key, downloaded over HTTPS, in /etc/apt/keyrings/caddy-stable.asc.

apt runs in vpn-router-caddy.service, which waits for dpkg's lock, so setup run by a package installation does not wait for itself; the unit runs vpn-router-caddy apply and then setup again, which serves with the new Caddy. vpn-router-caddy status shows the configured source, the source in effect and the versions.

A router installed with ubuntu is switched to upstream's Caddy with:

vpn-router-caddy use-upstream

It sets https.caddy_source to upstream, configures the repository and replaces Caddy, printing each step. Upstream versions sort above Ubuntu's, so once the repository is configured apt keeps upstream's Caddy, and ubuntu has no effect; setup says so. Going back is a downgrade nothing does on its own: remove the source and the key, run apt-get update, and install Ubuntu's version with apt-get install --allow-downgrades caddy=<version>, the version apt-cache policy caddy lists for the archive.

OpenID Provider

While oidc.enabled is true, the status web service is an OpenID Provider (OpenID Connect Core 1.0) for the users of the directory. Its issuer is the web service's HTTPS site, https://<wan.local_fqdn>:<https.web_port>, without the port when it is 443; the web service answers it only on 127.0.0.1, where Caddy passes the requests, so the provider is never reached over plain HTTP. The setting needs web.enabled, https.enabled and wan.local_fqdn.

vpn-config-edit set --restart oidc.enabled true
curl https://router.example.com:7444/.well-known/openid-configuration
Endpoint Purpose
GET /.well-known/openid-configuration The discovery document
GET /oidc/jwks The public key ID tokens are signed with
GET, POST /oidc/authorize The authorization code flow; the sign-in page while the browser has no session
POST /oidc/token Exchanges a code or a refresh token for tokens
GET, POST /oidc/userinfo The claims of the user of an access token
GET /oidc/logout Ends the session (RP-initiated logout)

A client is a service account with at least one redirect URI: its name is the client_id and its password the client_secret, sent with HTTP Basic authentication (client_secret_basic) or in the body of the token request (client_secret_post). The password is checked with a bind, so 5 wrong secrets lock the client for 15 minutes. Its settings are:

  • redirect URIs, where the provider sends the browser back with a code, matched exactly; a URI uses https, or http for localhost, 127.0.0.1 or ::1;
  • post-logout redirect URIs, where /oidc/logout may send the browser afterwards;
  • allowed groups: only their members may sign in to the client; without any, every user who is not a service account may.
echo 'cl1ent-s3cret' | vpn-router-user add 3f2b8c1e-9a4d-4e6f-8b2a-1c5d7e9f0a3b --service \
    --display-name Grafana --redirect-uri https://grafana.example.com/login/generic_oauth \
    --post-logout-redirect-uri https://grafana.example.com/login --allowed-group ops
vpn-router-user update 3f2b8c1e-9a4d-4e6f-8b2a-1c5d7e9f0a3b --allowed-group ''

The flow is the authorization code flow with PKCE, whose code_challenge_method must be S256; the implicit and hybrid flows, and the password and client credentials grants, are refused. A registered client is trusted, so a user is not asked for consent. A user signs in on the provider's page with the name, the password and, when enrolled, the TOTP code, or with a passkey (see Passkeys); failed sign-ins count towards the lockout, and POST /oidc/login takes at most 10 calls a minute from one client address, as /auth does. A disabled, locked or removed user, a service account, and a user outside the client's allowed groups cannot sign in, and the codes and tokens already issued to such a user stop working. The sign-in starts a session, a cookie of the issuer's /oidc/ path valid for 8 hours, so later requests of any client go on without the page; prompt=login and max_age ask for a new sign-in, and prompt=none answers login_required without a session.

Scope Claims
openid (required) sub, the entryUUID of the user's entry
profile preferred_username (the user name), name, given_name, family_name
email email
groups groups, the names of the user's groups

The claims of the scopes granted are in the ID token and in the answer of /oidc/userinfo. ID tokens are RS256 JWTs, valid for 5 minutes, with auth_time and amr (pwd, pwd and otp, or hwk and user for a passkey), signed with an RSA key of 3072 bits that the web service makes on its first run, /etc/vpn-router/oidc.key (root only), and named in the JWKS by its RFC 7638 thumbprint. A code is valid for 1 minute and used once, an access token for 5 minutes, and a refresh token for 8 hours and used once: each refresh answers a new one with a new ID token. The codes, tokens and sessions are kept, as their SHA-256, in the database of the refresh tokens of /auth. Changing a user's password, disabling, unlocking or removing the user ends its sessions and revokes its codes and tokens.

Web sites

While sites.enabled is true, Caddy serves static web sites from folders on the router, each /var/lib/vpn-router/sites/<name>. The sites are rows of the database vpn-router-sites, which the web service creates on its first run with its own role and the password in /etc/vpn-router/sites-database.secret; they are managed with vpn-router-site, /sites and the Router Manager, each change applied by restarting vpn-router-setup. A site has:

Setting Meaning
name 1 to 31 lowercase letters, digits and hyphens, starting with a letter; also the name of its folder
enabled Whether Caddy serves it
hosts The host names (or IP addresses) Caddy answers on the port, the names TLS clients send as SNI; none answers any name
port The TCP port, 443 by default
listen internal, external (the default) or both: the interfaces whose firewall opens the port
tls none (plain HTTP), acme (Let's Encrypt, the default), ca (certificates of an issuing CA of pki.issuing_cas, named by ca) or file (certificate and private_key, PEM)
index_file The file served for a folder, index.html by default
spa Whether a path without a file is served the site's index_file, for single-page applications
editors The group whose members may change the site's files besides the admins

A site may not take a port of the router's own services (SSH, web.port, the HTTPS ports, the metrics ports, the Router Manager's, sites.webdav_port, 7446, 53, the bootstrap ports, a TCP openvpn.port and docker.exposed_ports) or wan.local_fqdn. Enabled sites on one port need host names of their own, the same listen and the same kind of TLS (plain HTTP or HTTPS). A site of acme needs host names, without wildcards or addresses, that resolve to the router, and port 80, which setup opens on the external interface. The certificate of file must be the key's and name the hosts; it is written to /etc/caddy/vpn-router-sites/<name> (root, group caddy, 0640). An issuing CA of ca is given to Caddy as the ACME servers' CAs are, in /etc/caddy/vpn-router-pki; its certificates are trusted by clients that trust the router's root. Caddy runs while HTTPS is ready or a site is served; while the sites cannot be read, setup leaves the Caddyfile as it is.

vpn-config-edit set --restart sites.enabled true
vpn-router-site add www --host www.example.com --host example.com --spa
vpn-router-site add intranet --host intranet.example.lan --port 8443 --listen internal --tls ca --ca web \
    --editors web-team
vpn-router-site update www --no-enabled
vpn-router-site remove intranet --content

vpn-router-site remove keeps the folder without --content. Renaming a group renames it in the sites; a group a site has cannot be removed.

Site files

The files of a site are read and written through the web service (see The /sites API) by the admins and the members of the site's editors group, and through WebDAV. An upload is at most sites.upload_max_mb MiB (512 by default, at most 10240); a zip archive is extracted into a folder, and one with an absolute path, a .., a link, more than 10000 members or more content than the limit is refused before anything is written. Files are written with mode 0644 and folders with 0755, so Caddy reads them.

WebDAV

While sites.webdav is true and HTTPS is ready, vpn-router-setup runs vpn-router-webdav.service, rclone serve webdav on 127.0.0.1:7446, which Caddy serves at https://<wan.local_fqdn>:<sites.webdav_port> (7445 by default) and the firewall opens on both interfaces. A user signs in with the name and password of the router's directory, followed by the TOTP code for an enrolled user; an admin sees a folder for every site, any other user the folders of the sites whose editors group the user is a member of, and no folder means no access. rclone writes an upload to its cache in /var/cache/vpn-router-webdav and to the site's folder a few seconds later. rclone is installed from the Ubuntu archive by vpn-router-install@webdav.service when it is missing.

vpn-config-edit set --restart sites.webdav true

Name server

While dns.enabled is true, vpn-router-setup runs PowerDNS (pdns.service) as the authoritative name server of one zone, dns.zone, on port 53 of the addresses of dns.listen (internal, external, the default, or both), which the firewall opens for UDP and TCP. The records are in the database vpn-router-dns (password in /etc/vpn-router/dns-database.secret), read by PowerDNS's gpgsql backend; setup writes the settings /etc/powerdns/pdns.d/vpn-router.conf (root, group pdns, 0640) and installs pdns-server and pdns-backend-pgsql from the Ubuntu archive with vpn-router-install@dns.service when they are missing, without the recommended bind backend.

Setting Meaning
dns.zone The zone, such as sites.example.com
dns.nameserver The name server's host name, the zone's NS and the SOA's primary
dns.addresses The public addresses of the zone's apex, of the name server when it is in the zone, and of every host name of an enabled site in the zone
dns.ttl The TTL of the records and the SOA's negative TTL, 60 to 86400 seconds, 300 by default

Records added with /dns or the Router Manager (A, AAAA, CNAME, TXT, MX, SRV, CAA) are served too, but not those of a name and type the zone's own records have. Setup writes the zone at each run; the SOA's serial grows only when the records change. While P2S, OpenVPN or WireGuard is enabled, dnsmasq answers their clients on port 53 of the addresses of p2s.dns_listen, by default the internal address, so dns.listen must not share an interface with it. The parent zone must delegate dns.zone to dns.nameserver, with glue records for a name server inside the zone.

vpn-config-edit set dns.zone sites.example.com
vpn-config-edit set dns.nameserver ns1.sites.example.com
vpn-config-edit set dns.addresses 203.0.113.10
vpn-config-edit set --restart dns.enabled true

Metrics

While metrics.enabled is true, vpn-router-setup runs vpn-router-metrics, which every minute takes a sample of the router with psutil:

  • the CPU utilization;
  • the memory used and in total;
  • the free and the total space of /;
  • the bytes received and sent per second on the external and the internal interface;
  • the 1, 5 and 15 minute load averages;
  • the VPN clients of each enabled VPN system: road-warrior (IPsec P2S) and OpenVPN clients, and WireGuard peers with a handshake in the last 3 minutes;
  • for each site-to-site connection that is set up: whether it is up (1 or 0), the bytes and the packets per second received and sent through its child SAs since the previous sample, and the seconds its IKE SA has been up. The child SAs' counters start again at a rekey; a counter lower than in the previous sample counts from zero.

History

The samples are kept in the router's PostgreSQL database, the one of the refresh tokens (web.database), in four detail tables, one row per minute, and per interface, kind of client or site-to-site connection, and each is summarized hourly, daily, monthly and yearly. A summary row has the start of its period, in UTC, and the average and the maximum of each value; the average of a connection's up is the part of the period it was up:

Level Tables Kept
Details metrics_system, metrics_network, metrics_vpn, metrics_site_to_site metrics.keep_details
Hourly metrics_system_hourly, metrics_network_hourly, metrics_vpn_hourly, metrics_site_to_site_hourly metrics.keep_hourly
Daily metrics_system_daily, metrics_network_daily, metrics_vpn_daily, metrics_site_to_site_daily metrics.keep_daily
Monthly metrics_system_monthly, metrics_network_monthly, metrics_vpn_monthly, metrics_site_to_site_monthly metrics.keep_monthly
Yearly metrics_system_yearly, metrics_network_yearly, metrics_vpn_yearly, metrics_site_to_site_yearly metrics.keep_yearly

metrics_site_to_site has the columns connection, up, rx, tx, packets_rx, packets_tx and uptime.

How it works:

  • Summaries: a period is summarized once it is complete, from the level below. A stopped service catches up on the periods it missed, as long as their rows are still kept.

  • Garbage collection: runs on start and once a day. It deletes the rows older than their retention, a count and a unit such as 24 hours, 3 months or 10 years.

  • History API: GET /metrics/history?range=hour|day|month|year of the status web service, protected like /config, answers the views of the Router Manager's Metrics tab. Each point has the system values, network by interface, clients by kind and site_to_site by connection, each value as its avg and max:

    Range Bucket Source
    hour 1 minute details
    day 10 minutes details
    month 6 hours hourly
    year 1 month monthly
  • Site-to-site availability: site_to_site_up of the same answer divides the view into slots of its own, slot, and gives for each slot and connection the percentage of the samples in which the connection was up, 0 to 100, as points of {"time", "connections": {name: percent}}:

    Range Slot Source
    hour 5 minutes details
    day 1 hour details
    month 1 day hourly
    year 1 month monthly

    A sample is taken every minute, so an interruption shorter than a minute may be missed; a slot without samples has no value, and the year view, like the others of its level, has the complete months only.

Scraping

GET /metrics of the metrics service serves the last sample in the Prometheus text format, written by prometheus_client.

  • Metric names: the OpenTelemetry semantic convention names in their Prometheus form:
    • system_cpu_utilization;
    • system_memory_usage_bytes and system_memory_limit_bytes;
    • system_filesystem_usage_bytes and system_filesystem_utilization;
    • system_network_io_bytes_total;
    • system_cpu_load_average_1m, _5m and _15m;
    • vpn_router_clients, by kind: p2s, openvpn and wireguard;
    • by connection, for each site-to-site connection: vpn_router_site_to_site_up, vpn_router_site_to_site_uptime_seconds, and, by direction too, vpn_router_site_to_site_io_bytes_per_second and vpn_router_site_to_site_packets_per_second. The rates are gauges, since the child SAs' counters start again at each rekey.
  • Listening: on the interfaces metrics.listen names, internal by default, at metrics.port, and on 127.0.0.1. While HTTPS is on, Caddy serves it at metrics.https_port for wan.local_fqdn. vpn-router-setup opens both ports on the internal and the external interface.
  • Authentication: HTTP Basic as the service account metrics.service_account names, metrics by default. The password is checked with a bind to the user directory, so 5 failed attempts lock the account as for any user.
  • The account: vpn-router-setup owns it while metrics are enabled.
    • It adds the account with a random password, which it writes to /etc/vpn-router/metrics.secret (root only). Give that password to the collector.
    • When the file is missing or no longer holds the account's password, setup sets a new random password and ends any lockout. Delete the file and restart vpn-router-setup to change the password.
    • A disabled account is left as it is.
    • A user of that name who is not a service account is a setup failure.

An OpenTelemetry Collector scrapes it with its prometheus receiver:

receivers:
  prometheus:
    config:
      scrape_configs:
        - job_name: vpn-router
          scrape_interval: 60s
          scheme: https
          basic_auth:
            username: metrics
            password_file: /etc/otelcol/vpn-router-metrics.secret   # a copy of /etc/vpn-router/metrics.secret
          static_configs:
            - targets: ['router.example.com:9465']

Pushing

When metrics.otlp_endpoint is set, every sample is also pushed there as an OTLP/HTTP JSON request.

  • Content: the semantic convention names (system.cpu.utilization, system.network.io and so on), vpn_router.clients, the gauges vpn_router.site_to_site.up, .uptime, .io and .packets with the attribute connection, and the resource attributes service.name vpn-router and host.name.
  • Authentication: HTTP Basic with metrics.otlp_username and metrics.otlp_password when they are set.
  • Failures: a failed push is logged and not repeated; the next sample is pushed a minute later.

Backups

While backup.enabled is true, vpn-router-setup runs vpn-router-backup.timer on backup.schedule. The timer starts vpn-router-backup.service, which runs vpn-router-backup run. The schedule is written to /etc/systemd/system/vpn-router-backup.timer.d/schedule.conf:

  • hourly runs at the start of each hour;
  • daily runs at midnight;
  • weekly runs at midnight between Sunday and Monday.

A run missed while the router was off is made at the next boot.

What a backup holds

A backup holds the state that vpn-router-setup cannot make again:

Item Content
/etc/vpn-router vpn-router.json, the secrets and rclone.conf
/etc/wireguard The WireGuard keys and the peers
/etc/ufw, /etc/default/ufw The firewall rules
/etc/vpn-router-manager The Router Manager's settings, when it is installed
files/<path> Each psk_file outside /etc/vpn-router
ldap/vpn-router.ldif The user directory, written by slapcat: users, groups, service accounts, password hashes and TOTP secrets, and the PKI: the CAs and their keys, the root key while it is on the router, and the certificates issued, with the keys of the router's and the users' certificates
postgresql/vpn-router.dump With backup.metrics, the database of the metrics and the refresh tokens, written by pg_dump -Fc
/var/lib/vpn-router/sites The folders of the web sites
postgresql/vpn-router-sites.dump, postgresql/vpn-router-dns.dump The databases of the web sites and of the name server, once they exist
manifest.json The host, the time, the version of the package, the schedule and the items

The files and the directory are copied while the locks of the configuration and of the users are held, so no change of the API or the command-line tools falls between them; pg_dump reads a snapshot of its own. The copy is staged in /var/lib/vpn-router/backup/staging, which is removed after each run. The backup holds every secret of the router, so keep it as safe as the router itself.

Local archives

With backup.target local, a backup is a tar archive compressed with zstd, vpn-router-<host>-<UTC time>.tar.zst, in backup.path (mode 0600, the folder 0700). The folder may be a mount, such as an NFS share or an Azure Files share, which exports the archives.

After each backup, the archives of this host that the retention settings no longer keep are deleted. The rules are those of Proxmox Backup Server's prune:

  • The settings apply in order: keep_hourly, keep_daily, keep_weekly (ISO weeks, Monday to Sunday), keep_monthly, keep_yearly.
  • Each keeps the newest archive of each of its last N periods that have archives, in local time.
  • A period that has an archive a former setting kept is passed over.
  • A setting finer than the schedule does not apply: keep_hourly only with hourly, keep_daily with hourly and daily. At least one setting that applies must be above 0.

Cloud copy with rclone

With backup.rclone_remote set, each local backup is followed by an rclone run that copies the archives of this host to the remote. rclone comes from the rclone package, which the package suggests:

sudo apt-get install rclone
sudo rclone config --config /etc/vpn-router/rclone.conf
sudo chmod 0600 /etc/vpn-router/rclone.conf
sudo vpn-config-edit set backup.rclone_remote azure:backups/router
sudo systemctl restart vpn-router-setup

The two modes:

  • sync (the default) makes the remote mirror the archives of this host, so an archive retention deletes is deleted remotely too.
  • copy uploads new archives and deletes nothing, so retention in the cloud is left to its own rules, such as a lifecycle policy.

Other files in the remote path are left alone in both modes. A failed cloud copy leaves the local backup in place, and the run's result is warning.

Proxmox Backup Server

With backup.target pbs, the staged copy is sent with proxmox-backup-client as the archive router.pxar of the snapshot host/<host>/<time>, in backup.pbs_namespace when it is set. The server keeps the snapshots by its own prune jobs; the router prunes nothing.

The static client comes from Proxmox's client repository, which the package suggests but does not set up. Following Proxmox's installation guide, as /etc/apt/sources.list.d/pbs-client.sources:

Types: deb
URIs: http://download.proxmox.com/debian/pbs-client
Suites: trixie
Components: main
Signed-By: /usr/share/keyrings/proxmox-archive-keyring.gpg

Get the key https://enterprise.proxmox.com/debian/proxmox-archive-keyring-trixie.gpg into /usr/share/keyrings/proxmox-archive-keyring.gpg, check it against the SHA-256 the installation guide gives, and install proxmox-backup-client-static. Until it is installed, a backup to PBS fails with a message that names the package; nothing else depends on it.

On the server, make an API token for the router and give the token itself the DatastoreBackup role on /datastore/<store>, or on the namespace path. backup.pbs_repository names the token, such as backup@pbs!router@pbs.example.com:store, and backup.pbs_password holds its secret. Set backup.pbs_fingerprint when the server's certificate is self-signed. With backup.pbs_keyfile, the snapshot is encrypted with a key of proxmox-backup-client key create --kdf none <file>. Keep a copy of that key away from the router, since a restore needs it.

Running, listing and restoring

sudo vpn-router-backup run          # a backup now, whether or not backups are enabled
sudo vpn-router-backup list         # the local archives, or the snapshots on the server
sudo vpn-router-backup restore /var/backups/vpn-router/vpn-router-router-20261025T030000Z.tar.zst
sudo vpn-router-backup restore --pbs host/router/2026-10-25T03:00:00Z

Each run is recorded in /var/lib/vpn-router/backup.json, which /backup and the Router Manager show. Only one run or restore takes place at a time.

restore is only on the command line. It shows what the backup holds and asks for yes, which --yes gives. It then:

  1. stops the web service, the metrics service, WebDAV, PowerDNS and slapd;
  2. puts back the files;
  3. loads the directory with slapadd into an empty /var/lib/ldap/vpn-router;
  4. sets the database roles' passwords to the restored password files, and restores each database the backup holds with pg_restore --clean;
  5. restarts vpn-router-setup, which regenerates everything else, followed by WireGuard, OpenVPN, strongSwan and the Router Manager, and reloads ufw.

What it replaces is kept in /var/lib/vpn-router/backup/before-restore/<UTC time>. To restore onto a new VM, install the package first and let vpn-router-setup run once. The restored configuration names the interfaces of the old VM, so change them with vpn-config-edit if they differ.

Router Manager

The vpn-router-manager package, built in the repository linux-cloud-router-manager, is a web application that shows the state of the router and manages its users, site-to-site connections, backups and certificates through the status web service. It can be installed on the router or on another Linux system. While it is installed on the router, vpn-router-setup opens its port (port in /etc/vpn-router-manager/manager.json, 7080 by default, HTTP; HTTPS is Caddy's, on https.manager_port) on the internal and the external interface; restart vpn-router-setup after changing it. The manager's listen chooses the interfaces it listens on.

Removal

apt-get remove stops vpn-router-docker, which removes the Docker rules. It terminates the tunnels and takes wg-quick@wg0, openvpn-server@vpn-router, caddy, vpn-router-metrics, vpn-router-webdav, vpn-router-backup.timer and vpn-router-pki.timer down, but leaves the configuration in place. strongswan is left enabled, since it is a shared service that may be in use by something else.

apt-get purge additionally removes the generated files and the firewall rules, restoring /etc/ufw/before.rules to its original content, and hands vpn-router.json back to ucf before deleting it so a later reinstall starts clean. It also removes the web service's signing key and the OpenID Provider's, the password file of the metrics service account, the database of the refresh tokens and the metrics and those of the web sites and the name server, their roles and their password files when the package created them, Caddy's configuration /etc/caddy/vpn-router.Caddyfile and PowerDNS's /etc/powerdns/pdns.d/vpn-router.conf; the certificate in /var/lib/caddy belongs to the caddy package. It removes the backup timer's schedule, the record of the last backup, the published CA files, the copies of the CAs in /etc/caddy/vpn-router-pki, the sites' certificates in /etc/caddy/vpn-router-sites and OpenVPN's CA and CRL files. Caddy, rclone and PowerDNS stay installed, and with Caddy the source of its repository. Key material, users, content and backups are never deleted: /etc/vpn-router/ldap, /etc/vpn-router/rclone.conf, the user directory in slapd with the PKI, the WireGuard key pair, everything under /etc/swanctl, the folders of the web sites in /var/lib/vpn-router/sites, the archives in backup.path and /var/lib/vpn-router/backup/before-restore are left alone.

Building

cd debian-package
./build.sh

The build runs in a container and writes the package to debian-package/out/. ./publish.sh uploads it to the configured repository.

S
Description
An Ubuntu/Debian package that transforms a generic Ubuntu/Debian Linux into a cloud router with S2S and P2S VPN.
Readme
3 MiB
0 Stars 3 Watchers 0 Forks
Languages
Python 90.6%
Shell 4.9%
PLpgSQL 2.8%
Jinja 1.5%
Makefile 0.1%