Compare commits

..
5 Commits
20 changed files with 1193 additions and 76 deletions

No files matched your search

+65 -12
View File
@@ -465,11 +465,14 @@ Decision: The package defines its schema, `cn=vpn-router,cn=schema,cn=config`, w
`vpnrouter_ldap.provision` adds when missing and keeps as `SCHEMA_ATTRIBUTE_TYPES` and
`SCHEMA_OBJECT_CLASSES`. Its OID is `2.25.98477352595735834532404256531620013001`, made from a
random UUID under the arc 2.25 (ITU-T X.667), which needs no registration; attribute types are
under `.1` and object classes under `.2`. The first version has the attribute `vpnRouterScope`
(multi-valued, case-insensitive strings) and the auxiliary class `vpnRouterAccount`,
`MAY ( mail $ vpnRouterScope )`. Manager-only accounts have the class from their creation, so an
e-mail address can be added later without changing their classes; VPN users get it once scopes are
used.
under `.1` and object classes under `.2`. It has the attribute `vpnRouterScope` (multi-valued,
case-insensitive strings) and the auxiliary class `vpnRouterAccount`,
`MAY ( mail $ vpnRouterScope $ displayName $ description )`; `displayName` and `description`, the
standard attributes, are the display name and the description of a service account, which
`account` cannot hold. Manager-only accounts have the class from their creation, so an e-mail
address can be added later without changing their classes; VPN users get it once scopes are used.
`vpnrouter_ldap.provision` replaces the definitions in `cn=config` that differ, so an extended class
reaches an existing directory at the next setup run.
Consequences: The schema check stays strict. New attributes extend `vpnRouterAccount`, and their
OIDs continue the arcs. The scope `service` marks a service account (ADR-23). The OID must never
@@ -572,9 +575,11 @@ which another system signs in with to use a router service. Administrators add s
with `vpn-router-user add --service` and the web service's `/service-accounts`.
`/service-accounts` is a path of its own, apart from `/users`, which does not list or serve them:
a service account has no names, certificate, profile or TOTP, and is not a person an
administrator manages with the users. Internal service accounts are those the router creates and
manages for its own services (`vpnrouter_users.internal_accounts`), so the system, not an
administrator, owns them; at present that is the metrics service's. `/service-accounts` marks
administrator manages with the users. A service account has a display name and a description
besides its name, so an administrator can tell what each is for. Internal service accounts are
those the router creates and manages for its own services (`vpnrouter_users.internal_accounts`),
with every property, display name and description included, so the system, not an
administrator, owns them; at present that is the metrics service's, which setup names Metrics. `/service-accounts` marks
them as `internal` and only lists them: every other request on one, and putting one in or taking
one out of a group at `/groups`, gets 403, whatever the token. The router changes them through the
directory's user-manager account, not the API, and root may still with `vpn-router-user`. For the
@@ -601,18 +606,18 @@ upgrade restarts the process that starts it: the manager's postinst restarts
`vpn-router-manager.service`, and the router's restarts `vpn-router-web` and `vpn-router-setup`. A
child of the manager would be stopped with the manager's cgroup in the middle of dpkg, leaving
packages half configured, and the manager's unit cannot run apt at all, since `ProtectSystem=strict`
makes `/usr` and `/var/lib/dpkg` read-only. The package lists are updated by a cron job, not by the
manager.
makes `/usr` and `/var/lib/dpkg` read-only. Ubuntu's `apt-daily` timer updates the package lists
twice a day, so a release newer than its last run is not seen until the next one.
Decision: The manager starts the helper `/usr/lib/vpn-router-manager/upgrade` with `systemd-run` as
the transient unit `vpn-router-manager-upgrade`, of its own cgroup and without a sandbox, and does
not wait for it. The helper:
- finishes an interrupted dpkg run with `dpkg --configure -a`;
- runs `apt-get install --only-upgrade` of the two packages, non-interactive, keeping changed configuration files, with `needrestart` suspended, and waiting for the dpkg lock that the cron job may hold;
- runs `apt-get install --only-upgrade` of the two packages, non-interactive, keeping changed configuration files, with `needrestart` suspended, and waiting for the dpkg lock that `apt-daily` or `unattended-upgrades` may hold;
- records the state and the result in `/var/lib/vpn-router-manager/upgrade.json`, replaced by a rename, and the output in `upgrade.log`.
A restarted manager reads them again. A state of running without an active unit is shown as interrupted. The manager reboots the router the same way, as a transient timer `vpn-router-manager-reboot` 5 seconds later, so the answer reaches the browser first. Only members of `admins`, as the router's `/account` answers, may start either.
A restarted manager reads them again. A state of running without an active unit is shown as interrupted. The manager reboots the router the same way, as a transient timer `vpn-router-manager-reboot` 5 seconds later, so the answer reaches the browser first. It checks for updates the same way too: Check update runs `apt-get update` in the transient unit `vpn-router-manager-check`, which touches apt's update stamp when it succeeds, so an admin need not wait for the `apt-daily` timer. Only members of `admins`, as the router's `/account` answers, may start any of them.
Consequences: The fixed unit name allows one upgrade at a time. The page loses the manager and the
router's web service for a while during an upgrade and reads again until they answer. Sessions
@@ -674,3 +679,51 @@ one connection, dropping the others. A change through the API is answered before
applied it, so a configuration that setup cannot apply shows in the setup state, not in the
answer. A connection with a key file keeps it until a key is given through the API, which then
replaces the file.
## ADR-26: Backups are a staged, consistent copy, archived locally with tar and zstd or sent to a Proxmox Backup Server
Status: Accepted
Context: An image backup of the VM, such as Azure Backup, restores the whole disk. It cannot be
exported as a file, read on another system, or restored onto a new VM. The state that matters is
small, but it is spread over several places: `vpn-router.json` and the secrets, the PKI, the
WireGuard keys and peers, the firewall rules, the slapd directory and, optionally, the
PostgreSQL database. Only part of it can be made again: the CA, the password hashes and the TOTP
secrets cannot. The API and the command-line tools change the configuration, the PKI and the
directory while the router runs.
Decision: `vpn-router-backup run`, started by a timer on `backup.schedule`, stages a copy of that
state in one directory and then archives or sends it.
- **Consistency:** the files are copied and the directory is written with `slapcat` while the
configuration lock and the users lock are held, the locks every change of the API and the tools
takes, so a user, their certificate and the CRL are copied together. `slapcat` reads the MDB
database consistently while slapd runs (ADR-4), and `pg_dump` reads a snapshot. Files that
`vpn-router-setup` regenerates are left out.
- **Local target:** a tar archive compressed with zstd, both from the Ubuntu archive, so a local
backup needs nothing else and the archive can be read on any Linux system. A `pxar` archive
would need Proxmox's client and is not compressed.
- **Retention:** local archives are pruned by the rules of Proxmox Backup Server's prune, so the
settings mean the same for both targets and can be checked with Proxmox's prune simulator.
Settings finer than the schedule do not apply.
- **Cloud copy:** rclone, from the Ubuntu archive, copies the archives to any storage it
supports. The remote is defined in `/etc/vpn-router/rclone.conf` with `rclone config`, since
each backend has its own settings. `vpn-router.json` names only the remote, and the
configuration file is part of the backup. `sync` makes retention apply to the remote as well;
`copy` leaves it to the storage's lifecycle rules. Only this host's archives are included, so a
shared remote path is safe.
- **Proxmox Backup Server:** the static `proxmox-backup-client` sends the staged directory as one
`pxar` archive of a host snapshot. Deduplication makes frequent backups cheap. The server's
prune jobs keep the snapshots, so the token needs only `DatastoreBackup`, which cannot delete.
The client is a suggestion, not a dependency, because it comes from Proxmox's repository, which
an Ubuntu system does not know. Without it, only a PBS backup fails.
- **Restore:** only on the command line, as root. It replaces the directory and the database, so
it stops their services, keeps what it replaces and lets `vpn-router-setup` rebuild the rest.
The API offers no restore, so a stolen admin session cannot replace the router's keys.
Consequences: A backup holds every secret of the router, including the CA key, in plain form
unless a PBS key file encrypts it, so where it is kept must be as safe as the router. A restore
onto another VM brings back the old interface names and addresses, which may need changing. The
timer's schedules are systemd's shorthands, so all backups of one schedule start at the same
moment. A change of the users between the end of the copy and the next backup is in no backup
until then.
+174 -20
View File
@@ -143,6 +143,23 @@ must have the type shown below; an unknown section or setting is an error.
| `metrics` | `keep_daily` | string | `3 months` | How long the daily summaries are kept |
| `metrics` | `keep_monthly` | string | `1 year` | How long the monthly summaries are kept |
| `metrics` | `keep_yearly` | string | `10 years` | How long the yearly summaries are kept |
| `backup` | `enabled` | boolean | `false` | Scheduled backups (see Backups) |
| `backup` | `schedule` | string | `daily` | `hourly`, `daily` or `weekly` |
| `backup` | `target` | string | `local` | `local`, an archive in `backup.path`, or `pbs`, a Proxmox Backup Server |
| `backup` | `path` | string | `/var/backups/vpn-router` | The folder of local archives, an absolute path |
| `backup` | `keep_hourly` | integer | `24` | Hourly local archives kept, with the hourly schedule |
| `backup` | `keep_daily` | integer | `7` | Daily local archives kept, with the hourly and the daily schedule |
| `backup` | `keep_weekly` | integer | `4` | Weekly local archives kept |
| `backup` | `keep_monthly` | integer | `12` | Monthly local archives kept |
| `backup` | `keep_yearly` | integer | `0` | Yearly local archives kept |
| `backup` | `metrics` | boolean | `false` | The backup holds the database of the metrics and the refresh tokens |
| `backup` | `rclone_remote` | string | | `remote:path` of `/etc/vpn-router/rclone.conf` the local archives are copied to |
| `backup` | `rclone_mode` | string | `sync` | `sync` mirrors the archives, `copy` deletes nothing in the cloud |
| `backup` | `pbs_repository` | string | | `user@realm!token@host[:port]:datastore` |
| `backup` | `pbs_password` | string | | The secret of the API token |
| `backup` | `pbs_fingerprint` | string | | The SHA-256 fingerprint of the server's certificate, when the system's CAs do not verify it |
| `backup` | `pbs_namespace` | string | | The namespace of the datastore, such as `routers/azure` |
| `backup` | `pbs_keyfile` | string | | An encryption key of `proxmox-backup-client key create`, an absolute path |
### Site-to-site connections
@@ -393,9 +410,10 @@ is switched on:
`account` for a Manager-only account (see Managing users);
- the package's own schema, `cn=vpn-router,cn=schema,cn=config`, under the OID
`2.25.98477352595735834532404256531620013001` of the UUID arc: the auxiliary class
`vpnRouterAccount` with `MAY ( mail $ vpnRouterScope )`, which Manager-only and service accounts
have, and the attribute `vpnRouterScope`, authorization scopes, of which `service` marks a
service account;
`vpnRouterAccount` with `MAY ( mail $ vpnRouterScope $ displayName $ description )`, which
Manager-only and service accounts have, and the attribute `vpnRouterScope`, authorization
scopes, of which `service` marks a service account; `displayName` and `description` are the
display name and the description of a service account;
- `ou=groups` holds one `groupOfNames` per group, named by `cn`, with its members' DNs in
`member` and an optional `description`;
- `cn=user-manager,ou=services` is the account that changes users and groups; it may write
@@ -429,10 +447,12 @@ it is added:
the router. `add` without the first and the last name adds one; its names cannot be set later;
- a service account, a Manager-only account whose `vpnRouterScope` is `service`, which another
system signs in with to use a router service, such as an OpenTelemetry Collector reading
`/metrics`; it cannot sign in to `/auth`. `add --service` adds one, and the web service manages
them at `/service-accounts`, not at `/users`. Internal service accounts are those the router
creates and manages for its own services; at present that is the metrics service's, the account
`metrics.service_account` names while metrics are enabled (see Metrics). The API only lists
`/metrics`; it cannot sign in to `/auth`. Besides its name, it has a display name and a
description. `add --service` adds one, and the web service manages them at `/service-accounts`,
not at `/users`. Internal service accounts are those the router creates and manages for its own
services, display name and description included; at present that is the metrics service's, the
account `metrics.service_account` names while metrics are enabled (see Metrics), which
`vpn-router-setup` names Metrics. The API only lists
them: `/service-accounts` refuses every other request on one, and `/groups` refuses to put one
in or take one out of a group, with 403.
@@ -466,8 +486,9 @@ vpn-router-user otp-remove alice
| `list` | The users with their kind, names and e-mail addresses, whether each is enabled or locked, has a certificate and is enrolled for TOTP |
| `add <name> --first-name F --last-name L --email E` | Adds a VPN user and issues the certificate |
| `add <name> [--email E]` | Adds a Manager-only account, with the e-mail address when given; one name without the other two options is refused |
| `add <name> --service [--email E]` | Adds a service account, with the e-mail address when given; the names are refused |
| `add <name> --service [--display-name N] [--description D] [--email E]` | Adds a service account, with its display name, description and e-mail address when given; the names are refused |
| `update <name> [--first-name F] [--last-name L] [--email E]` | Changes the names or the e-mail address |
| `update <name> [--display-name N] [--description D] [--email E]` | Changes the display name, the description or the e-mail address of a service account; `''` removes one |
| `passwd <name>` | Sets the password |
| `disable <name>`, `enable <name>` | Locks the user out of OpenVPN and `/auth`, and lets the user in again |
| `unlock <name>` | Ends a lockout after failed logins and clears the failure count; a disabled user stays disabled |
@@ -752,18 +773,19 @@ curl -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
| Request | Body | Answer |
| --- | --- | --- |
| `GET /service-accounts` | | `{"service_accounts": [{"name", "internal", "enabled", "locked", "email", ...}, ...]}`, each as `/users` shows a user |
| `POST /service-accounts` | `{"name", "password", "email"?}` | 201 and the account |
| `PUT /service-accounts/<name>` | `{"email"}`; `""` removes it | the account |
| `GET /service-accounts` | | `{"service_accounts": [{"name", "display_name", "description", "internal", "enabled", "locked", "email", ...}, ...]}`, each as `/users` shows a user |
| `POST /service-accounts` | `{"name", "password", "display_name"?, "description"?, "email"?}` | 201 and the account |
| `PUT /service-accounts/<name>` | any of `{"display_name", "description", "email"}`; `""` removes one | the account |
| `PUT /service-accounts/<name>/password` | `{"password": ...}` | the account |
| `POST /service-accounts/<name>/disable` | | the account |
| `POST /service-accounts/<name>/enable` | | the account |
| `POST /service-accounts/<name>/unlock` | | the account |
| `DELETE /service-accounts/<name>` | | `{"name", "removed": true}` |
The internal service accounts, `internal` true, are only listed: every other request on one gets
403, as the router manages them. A failure is otherwise as for `/users`, with 404 also for a
name that is not a service account.
A display name has at most 64 printable characters and a description at most 256. The internal
service accounts, `internal` true, are only listed: every other request on one gets 403, as the
router manages them, their display name and description included. A failure is otherwise as for
`/users`, with 404 also for a name that is not a service account.
### The /account API
@@ -821,6 +843,28 @@ is `{"error": ...}` with 400 for an invalid request or configuration, with the r
connection that does not exist, 409 for a name that exists and 500 otherwise. Keys cross the
internal network in plain HTTP, like the token.
### The /backup API
`/backup` shows and changes the backup settings and starts a backup, with the same token as
`/users`.
| Request | Body | Answer |
| --- | --- | --- |
| `GET /backup` | | `{"settings", "state", "archives", "next", "client", "rclone"}` |
| `PUT /backup` | any of the settings of the `backup` section; an empty or absent `pbs_password` keeps the secret | `{"settings"}` |
| `POST /backup/run` | | 202 and `{"started": true}`; 409 while a backup runs |
The answer of `GET /backup` holds:
- **`settings`:** the `backup` section without `pbs_password`, with `pbs_password_set`.
- **`state`:** the last run, `null` before the first. Its `result` is `running`, `ok`, `warning` (the backup was made but the cloud copy failed), `failed` or `interrupted` (the run ended without recording its result). It also has `started`, `finished`, `target`, `archive` or `snapshot`, `size` in bytes, `removed` (the archives retention deleted), `error` and `cloud` (`{"remote", "mode", "result", "error"}`).
- **`archives`:** the local archives of this host, newest first, as `{"name", "time", "size"}`.
- **`next`:** the next run as systemd shows it, `null` while backups are off.
- **`client`:** whether `proxmox-backup-client` is installed.
- **`rclone`:** `{"installed", "config", "remotes"}`, the remotes being the sections of `rclone.conf`.
A change is made, checked and applied as for `/site-to-site`.
The service starts on its own, without waiting for `vpn-router-setup`, and keeps running. Every
30 seconds it reads `web.listen`, `web.port` and `web.enabled` from `vpn-router.json` and works
out the addresses of the interfaces `web.listen` names as `general.mode` directs; it listens on
@@ -907,6 +951,17 @@ How it works:
| `month` | 6 hours | hourly |
| `year` | 1 month | monthly |
- **Site-to-site availability:** `site_to_site_up` of the same answer divides the view into slots of its own, `slot`, and gives for each slot and connection the percentage of the samples in which the connection was up, 0 to 100, as `points` of `{"time", "connections": {name: percent}}`:
| Range | Slot | Source |
|---|---|---|
| `hour` | 5 minutes | details |
| `day` | 1 hour | details |
| `month` | 1 day | hourly |
| `year` | 1 month | monthly |
A sample is taken every minute, so an interruption shorter than a minute may be missed; a slot without samples has no value, and the year view, like the others of its level, has the complete months only.
### Scraping
`GET /metrics` of the metrics service serves the last sample in the Prometheus text format, written by `prometheus_client`.
@@ -952,11 +1007,107 @@ When `metrics.otlp_endpoint` is set, every sample is also pushed there as an OTL
- **Authentication:** HTTP Basic with `metrics.otlp_username` and `metrics.otlp_password` when they are set.
- **Failures:** a failed push is logged and not repeated; the next sample is pushed a minute later.
## Backups
While `backup.enabled` is `true`, `vpn-router-setup` runs `vpn-router-backup.timer` on `backup.schedule`. The timer starts `vpn-router-backup.service`, which runs `vpn-router-backup run`. The schedule is written to `/etc/systemd/system/vpn-router-backup.timer.d/schedule.conf`:
- `hourly` runs at the start of each hour;
- `daily` runs at midnight;
- `weekly` runs at midnight between Sunday and Monday.
A run missed while the router was off is made at the next boot.
### What a backup holds
A backup holds the state that `vpn-router-setup` cannot make again:
| Item | Content |
|---|---|
| `/etc/vpn-router` | `vpn-router.json`, the secrets, `rclone.conf`, the PKI and the certificates of the users |
| `/etc/wireguard` | The WireGuard keys and the peers |
| `/etc/ufw`, `/etc/default/ufw` | The firewall rules |
| `/etc/vpn-router-manager` | The Router Manager's settings, when it is installed |
| `files/<path>` | Each `psk_file` outside `/etc/vpn-router` |
| `ldap/vpn-router.ldif` | The user directory, written by `slapcat`: users, groups, service accounts, password hashes and TOTP secrets |
| `postgresql/vpn-router.dump` | With `backup.metrics`, the database of the metrics and the refresh tokens, written by `pg_dump -Fc` |
| `manifest.json` | The host, the time, the version of the package, the schedule and the items |
The files and the directory are copied while the locks of the configuration and of the users are held, so no change of the API or the command-line tools falls between them; `pg_dump` reads a snapshot of its own. The copy is staged in `/var/lib/vpn-router/backup/staging`, which is removed after each run. The backup holds every secret of the router, so keep it as safe as the router itself.
### Local archives
With `backup.target` `local`, a backup is a tar archive compressed with zstd, `vpn-router-<host>-<UTC time>.tar.zst`, in `backup.path` (mode 0600, the folder 0700). The folder may be a mount, such as an NFS share or an Azure Files share, which exports the archives.
After each backup, the archives of this host that the retention settings no longer keep are deleted. The rules are those of Proxmox Backup Server's prune:
- The settings apply in order: `keep_hourly`, `keep_daily`, `keep_weekly` (ISO weeks, Monday to Sunday), `keep_monthly`, `keep_yearly`.
- Each keeps the newest archive of each of its last N periods that have archives, in local time.
- A period that has an archive a former setting kept is passed over.
- A setting finer than the schedule does not apply: `keep_hourly` only with `hourly`, `keep_daily` with `hourly` and `daily`. At least one setting that applies must be above 0.
### Cloud copy with rclone
With `backup.rclone_remote` set, each local backup is followed by an rclone run that copies the archives of this host to the remote. rclone comes from the `rclone` package, which the package suggests:
```sh
sudo apt-get install rclone
sudo rclone config --config /etc/vpn-router/rclone.conf
sudo chmod 0600 /etc/vpn-router/rclone.conf
sudo vpn-config-edit set backup.rclone_remote azure:backups/router
sudo systemctl restart vpn-router-setup
```
The two modes:
- `sync` (the default) makes the remote mirror the archives of this host, so an archive retention deletes is deleted remotely too.
- `copy` uploads new archives and deletes nothing, so retention in the cloud is left to its own rules, such as a lifecycle policy.
Other files in the remote path are left alone in both modes. A failed cloud copy leaves the local backup in place, and the run's result is `warning`.
### Proxmox Backup Server
With `backup.target` `pbs`, the staged copy is sent with `proxmox-backup-client` as the archive `router.pxar` of the snapshot `host/<host>/<time>`, in `backup.pbs_namespace` when it is set. The server keeps the snapshots by its own prune jobs; the router prunes nothing.
The static client comes from Proxmox's client repository, which the package suggests but does not set up. Following Proxmox's installation guide, as `/etc/apt/sources.list.d/pbs-client.sources`:
```text
Types: deb
URIs: http://download.proxmox.com/debian/pbs-client
Suites: trixie
Components: main
Signed-By: /usr/share/keyrings/proxmox-archive-keyring.gpg
```
Get the key `https://enterprise.proxmox.com/debian/proxmox-archive-keyring-trixie.gpg` into `/usr/share/keyrings/proxmox-archive-keyring.gpg`, check it against the SHA-256 the [installation guide](https://pbs.proxmox.com/docs/installation.html) gives, and install `proxmox-backup-client-static`. Until it is installed, a backup to PBS fails with a message that names the package; nothing else depends on it.
On the server, make an API token for the router and give the token itself the `DatastoreBackup` role on `/datastore/<store>`, or on the namespace path. `backup.pbs_repository` names the token, such as `backup@pbs!router@pbs.example.com:store`, and `backup.pbs_password` holds its secret. Set `backup.pbs_fingerprint` when the server's certificate is self-signed. With `backup.pbs_keyfile`, the snapshot is encrypted with a key of `proxmox-backup-client key create --kdf none <file>`. Keep a copy of that key away from the router, since a restore needs it.
### Running, listing and restoring
```sh
sudo vpn-router-backup run # a backup now, whether or not backups are enabled
sudo vpn-router-backup list # the local archives, or the snapshots on the server
sudo vpn-router-backup restore /var/backups/vpn-router/vpn-router-router-20261025T030000Z.tar.zst
sudo vpn-router-backup restore --pbs host/router/2026-10-25T03:00:00Z
```
Each run is recorded in `/var/lib/vpn-router/backup.json`, which `/backup` and the Router Manager show. Only one run or restore takes place at a time.
`restore` is only on the command line. It shows what the backup holds and asks for `yes`, which `--yes` gives. It then:
1. stops the web service, the metrics service and slapd;
2. puts back the files;
3. loads the directory with `slapadd` into an empty `/var/lib/ldap/vpn-router`;
4. sets the database role's password to the restored `database.secret`, and restores the database with `pg_restore --clean` when the backup holds it;
5. restarts `vpn-router-setup`, which regenerates everything else, followed by WireGuard, OpenVPN, strongSwan and the Router Manager, and reloads ufw.
What it replaces is kept in `/var/lib/vpn-router/backup/before-restore/<UTC time>`. To restore onto a new VM, install the package first and let `vpn-router-setup` run once. The restored configuration names the interfaces of the old VM, so change them with `vpn-config-edit` if they differ.
## Router Manager
The `vpn-router-manager` package, built in the repository `linux-cloud-router-manager`, is a web
application that shows the state of the router and manages its users and site-to-site
connections through the status web service. It can be installed on the router or on another Linux system. While it is installed on
application that shows the state of the router and manages its users, site-to-site
connections and backups through the status web service. It can be installed on the router or on another Linux system. While it is installed on
the router, `vpn-router-setup` opens its port (`port` in `/etc/vpn-router-manager/manager.json`,
7080 by default, HTTP; HTTPS is Caddy's, on `https.manager_port`) on the internal and the external
interface; restart `vpn-router-setup` after changing it. The manager's `listen` chooses the
@@ -965,8 +1116,8 @@ interfaces it listens on.
## Removal
`apt-get remove` stops `vpn-router-docker`, which removes the Docker rules. It terminates the
tunnels and takes `wg-quick@wg0`, `openvpn-server@vpn-router`, `caddy` and `vpn-router-metrics`
down, but leaves the
tunnels and takes `wg-quick@wg0`, `openvpn-server@vpn-router`, `caddy`, `vpn-router-metrics` and
`vpn-router-backup.timer` down, but leaves the
configuration in place. `strongswan` is left enabled, since it is a shared service that
may be in use by something else.
@@ -976,8 +1127,11 @@ before deleting it so a later reinstall starts clean. It also removes the web se
signing key, the password file of the metrics service account, and the database of the refresh
tokens and the metrics, its role and their password
file when the package created them, and Caddy's configuration `/etc/caddy/vpn-router.Caddyfile`; the certificate in
`/var/lib/caddy` belongs to the `caddy` package. Key material and users are never deleted: `/etc/vpn-router/pki`, `/etc/vpn-router/ldap`, the user directory in slapd, the
WireGuard key pair and everything under `/etc/swanctl` are left alone.
`/var/lib/caddy` belongs to the `caddy` package. It removes the backup timer's schedule and the
record of the last backup. Key material, users and backups are never deleted: `/etc/vpn-router/pki`,
`/etc/vpn-router/ldap`, `/etc/vpn-router/rclone.conf`, the user directory in slapd, the WireGuard
key pair, everything under `/etc/swanctl`, the archives in `backup.path` and
`/var/lib/vpn-router/backup/before-restore` are left alone.
## Building
+81
View File
@@ -1,3 +1,84 @@
vpn-router (0.20.0-1) unstable; urgency=medium
feature: the router makes scheduled backups to a local folder, copied to the cloud with rclone, or to a Proxmox Backup Server, and vpn-router-backup restores them.
* The backup section of vpn-router.json switches on backups and sets
the schedule (hourly, daily or weekly), the target (local or pbs),
the folder, the retention of local archives, whether the metrics
database is included, the rclone remote and mode, and the Proxmox
Backup Server repository, token secret, fingerprint, namespace and
key file.
* A backup holds /etc/vpn-router, /etc/wireguard, the ufw rules, the
Router Manager's settings, PSK files outside /etc/vpn-router, the
user directory as slapcat writes it and, optionally, the database as
pg_dump writes it, copied while the configuration and the users are
locked.
* Local backups are tar archives compressed with zstd. Archives the
retention settings no longer keep are deleted by the rules of Proxmox
Backup Server's prune, and rclone can sync or copy the archives to a
remote of /etc/vpn-router/rclone.conf.
* Backups to a Proxmox Backup Server are made with
proxmox-backup-client-static, which the package suggests; the server
keeps them.
* vpn-router-backup runs a backup, lists the backups and restores a
local archive or a snapshot; vpn-router-backup.timer runs it on the
schedule.
* /backup of the status web service shows and changes the backup
settings, shows the last run, the local archives and the next run,
and starts a backup.
* The package depends on zstd and suggests rclone and
proxmox-backup-client-static.
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Wed, 28 Oct 2026 10:00:00 +0100
vpn-router (0.19.1-3) unstable; urgency=medium
update: ADR-24 names Ubuntu's apt-daily timer, not a cron job, as what updates the package lists.
* ADR-24 says that Ubuntu's apt-daily timer updates the package lists
twice a day, and that apt-daily or unattended-upgrades may hold the
dpkg lock the upgrade waits for.
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Tue, 27 Oct 2026 10:00:00 +0100
vpn-router (0.19.1-2) unstable; urgency=medium
update: ADR-24 says that the Router Manager checks for updates on demand.
* ADR-24 describes Check update of the Router Manager, which updates
apt's package lists in the transient unit vpn-router-manager-check.
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Tue, 27 Oct 2026 09:00:00 +0100
vpn-router (0.19.1-1) unstable; urgency=medium
fix: /metrics/history gives the availability of each site-to-site connection as the percentage of time it was up in slots of the view.
* site_to_site_up of /metrics/history divides the view into slots of 5
minutes for an hour, 1 hour for a day, 1 day for a month and 1 month
for a year, and gives for each slot and connection the percentage of
the samples in which the connection was up.
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Mon, 26 Oct 2026 21:00:00 +0100
vpn-router (0.19.0-1) unstable; urgency=medium
feature: service accounts have a display name and a description, which the router sets for its internal ones.
* vpnRouterAccount of the package's schema also takes displayName and
description; vpn-router-setup updates the schema of an existing
directory.
* /service-accounts shows display_name and description, takes them when
adding a service account and changes them with PUT; "" removes one.
/users shows them as null.
* vpn-router-user add --service and update take --display-name and
--description.
* vpn-router-setup gives the metrics service's internal account the
display name Metrics and its description, and sets them again when
they differ.
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Mon, 26 Oct 2026 20:00:00 +0100
vpn-router (0.18.3-1) unstable; urgency=medium
fix: internal service accounts are the router's own, of which the metrics service's is the one at present, and the documentation says what a service account is.
+7 -2
View File
@@ -31,14 +31,19 @@ Depends: ${misc:Depends},
openvpn-auth-ldap,
iproute2,
systemd-resolved,
caddy
caddy,
zstd
Suggests: rclone,
proxmox-backup-client-static
Description: Linux VPN router with IPSec, road-warrior, OpenVPN and WireGuard support
Configures a Linux host or virtual machine as a VPN router: site-to-site
IKEv2 IPSec (strongSwan swanctl), optional road-warrior access using EAP-TLS,
optional OpenVPN with a certificate and a password per user, kept in a local
LDAP directory, optional WireGuard, and the routing, NAT, MSS clamping and
firewall rules a router needs. It keeps metrics of the router in PostgreSQL
and serves them for an OpenTelemetry Collector. Nothing in the package is
and serves them for an OpenTelemetry Collector, and makes scheduled backups
to a local folder, copied to the cloud with rclone, or to a Proxmox Backup
Server. Nothing in the package is
specific to a cloud provider.
.
Settings live in /etc/vpn-router/vpn-router.json, which is created on first
+2
View File
@@ -7,11 +7,13 @@ src/usr/lib/vpn-router/vpnrouter_users.py usr/lib/vpn-router/
src/usr/lib/vpn-router/vpnrouter_auth.py usr/lib/vpn-router/
src/usr/lib/vpn-router/vpnrouter_metrics.py usr/lib/vpn-router/
src/usr/lib/vpn-router/vpnrouter_status.py usr/lib/vpn-router/
src/usr/lib/vpn-router/vpnrouter_backup.py usr/lib/vpn-router/
src/usr/lib/vpn-router/openvpn-same-user usr/lib/vpn-router/
src/usr/sbin/vpn-router-user usr/sbin/
src/usr/sbin/vpn-router-group usr/sbin/
src/usr/sbin/vpn-router-token usr/sbin/
src/usr/sbin/vpn-config-edit usr/sbin/
src/usr/sbin/vpn-router-backup usr/sbin/
src/usr/lib/vpn-router/simple-ca usr/lib/vpn-router/
src/usr/lib/vpn-router/configure usr/lib/vpn-router/
src/usr/lib/vpn-router/docker-rules usr/lib/vpn-router/
+9 -4
View File
@@ -53,8 +53,12 @@ case "$1" in
/etc/vpn-router/vpn-router.json \
/etc/vpn-router/web.key \
/etc/vpn-router/metrics.secret \
/var/lib/vpn-router/setup.done
rmdir --ignore-fail-on-non-empty /var/lib/vpn-router 2>/dev/null || true
/etc/systemd/system/vpn-router-backup.timer.d/schedule.conf \
/var/lib/vpn-router/setup.done \
/var/lib/vpn-router/backup.json
rmdir --ignore-fail-on-non-empty /etc/systemd/system/vpn-router-backup.timer.d 2>/dev/null || true
rm -rf /var/lib/vpn-router/backup/staging /var/lib/vpn-router/backup/restore
rmdir --ignore-fail-on-non-empty /var/lib/vpn-router/backup /var/lib/vpn-router 2>/dev/null || true
# The refresh tokens' database and role, when the package created them.
if [ -f /etc/vpn-router/database.secret ] && command -v psql >/dev/null 2>&1; then
@@ -64,8 +68,9 @@ case "$1" in
rm -f /etc/vpn-router/database.secret
# Key material and users are not package state: /etc/vpn-router/pki,
# /etc/vpn-router/ldap, the user directory in slapd, the WireGuard key
# pair and everything under /etc/swanctl are left in place.
# /etc/vpn-router/ldap, /etc/vpn-router/rclone.conf, the user directory
# in slapd, the WireGuard key pair, everything under /etc/swanctl, the
# backups and what a restore replaced are left in place.
rmdir --ignore-fail-on-non-empty /etc/vpn-router 2>/dev/null || true
;;
esac
+5 -4
View File
@@ -17,10 +17,11 @@ case "$1" in
fi
# vpn-router-setup enabled wg-quick@wg0, openvpn-server@vpn-router,
# caddy and vpn-router-metrics, so the package takes them down again.
# Use the debhelper wrappers rather than systemctl, so the script
# behaves on a machine without systemd.
for unit in wg-quick@wg0.service openvpn-server@vpn-router.service caddy.service vpn-router-metrics.service; do
# caddy, vpn-router-metrics and the backup timer, so the package takes
# them down again. Use the debhelper wrappers rather than systemctl, so
# the script behaves on a machine without systemd.
for unit in wg-quick@wg0.service openvpn-server@vpn-router.service caddy.service vpn-router-metrics.service \
vpn-router-backup.timer; do
if command -v deb-systemd-invoke >/dev/null 2>&1; then
deb-systemd-invoke stop "$unit" >/dev/null 2>&1 || true
fi
+1
View File
@@ -8,3 +8,4 @@ override_dh_installsystemd:
dh_installsystemd --name=vpn-router-web --no-start
dh_installsystemd --name=vpn-router-metrics --no-enable --no-start
dh_installsystemd --name=vpn-router-bootstrap --no-enable --no-start
dh_installsystemd --name=vpn-router-backup --no-enable --no-start
@@ -0,0 +1,11 @@
[Unit]
Description=VPN Router backup
Documentation=file:/usr/share/doc/vpn-router/README.md
After=network-online.target slapd.service postgresql.service
Wants=network-online.target
[Service]
Type=oneshot
ExecStart=/usr/sbin/vpn-router-backup run
Nice=10
IOSchedulingClass=idle
@@ -0,0 +1,10 @@
[Unit]
Description=VPN Router backup schedule
Documentation=file:/usr/share/doc/vpn-router/README.md
[Timer]
OnCalendar=daily
Persistent=true
[Install]
WantedBy=timers.target
+35 -5
View File
@@ -63,6 +63,9 @@ MANAGER_PORT = 7080
#: The password of the metrics service account, which setup makes and keeps (root only).
METRICS_SECRET = pathlib.Path('/etc/vpn-router/metrics.secret')
#: The schedule of the backup timer, written from backup.schedule.
BACKUP_SCHEDULE = pathlib.Path('/etc/systemd/system/vpn-router-backup.timer.d/schedule.conf')
#: The configuration of Caddy, the HTTPS proxy, which the package's drop-in for caddy.service names;
#: Caddy runs as the user and group CADDY_GROUP, which the caddy package creates.
CADDYFILE = pathlib.Path('/etc/caddy/vpn-router.Caddyfile')
@@ -458,14 +461,16 @@ def manage_admin():
def manage_metrics_account(ctx):
"""Keep the service account metrics.service_account names, whose password is in METRICS_SECRET; the problem, or ''.
While metrics are enabled, setup owns the account: it adds it as a service account with a random password when
it is missing, and gives it a new random password, ending any lockout, when METRICS_SECRET is missing or does not
hold its password. A disabled account is left as it is. An account of that name that is not a service account is
left alone and is a problem.
While metrics are enabled, setup owns the account, an internal service account: it adds it as a service account
with a random password and the labels of vpnrouter_users.internal_accounts when it is missing, sets those labels
when they differ, and gives it a new random password, ending any lockout, when METRICS_SECRET is missing or does
not hold its password. The password of a disabled account is left as it is. An account of that name that is not a
service account is left alone and is a problem.
"""
if not vpnrouter.metrics_ready(ctx):
return ''
name = ctx['metrics_service_account']
wanted = vpnrouter_users.internal_accounts().get(name, {})
try:
vpnrouter_users.check_name(name)
try:
@@ -481,6 +486,8 @@ def manage_metrics_account(ctx):
current = None
if current is not None and not vpnrouter_users.service(current):
return f'metrics.service_account: {name} is a user, not a service account'
if current is not None and any(vpnrouter_users.summary(current)[key] != value for key, value in wanted.items()):
vpnrouter_users.update_service(connection, name, wanted)
# An administrator who disabled the account meant it; its password is left as it is.
if current is not None and not vpnrouter_users.summary(current)['enabled']:
return ''
@@ -497,7 +504,7 @@ def manage_metrics_account(ctx):
METRICS_SECRET.chmod(0o600)
METRICS_SECRET.write_text(password + '\n', encoding='utf-8')
if current is None:
vpnrouter_users.add(connection, name, password, {}, service=True)
vpnrouter_users.add(connection, name, password, wanted, service=True)
else:
vpnrouter_users.passwd(connection, name, password)
vpnrouter_users.unlock(connection, name)
@@ -562,6 +569,27 @@ def manage_metrics(ctx):
run(['systemctl', 'restart', unit], check=False)
def manage_backup(cfg):
"""Run the backup timer on backup.schedule while backups are enabled, and stop it when not.
The schedule is a drop-in of the timer, BACKUP_SCHEDULE, which replaces the OnCalendar of the shipped unit.
"""
unit = 'vpn-router-backup.timer'
if not cfg['backup']['enabled']:
run(['systemctl', 'disable', '--now', unit], check=False)
if BACKUP_SCHEDULE.exists():
BACKUP_SCHEDULE.unlink()
run(['systemctl', 'daemon-reload'], check=False)
return
text = f'[Timer]\nOnCalendar=\nOnCalendar={cfg["backup"]["schedule"]}\n'
if not BACKUP_SCHEDULE.exists() or BACKUP_SCHEDULE.read_text(encoding='utf-8') != text:
BACKUP_SCHEDULE.parent.mkdir(parents=True, exist_ok=True)
BACKUP_SCHEDULE.write_text(text, encoding='utf-8')
run(['systemctl', 'daemon-reload'], check=False)
run(['systemctl', 'restart', unit], check=False)
run(['systemctl', 'enable', '--now', unit], check=False)
def run_platform_setup(cfg, ctx):
"""Let the platform module apply its own state, if it defines any."""
module = vpnrouter.platform_module(cfg)
@@ -730,6 +758,8 @@ def main():
refresh_web(cfg, ctx)
progress('running', 'metrics service')
manage_metrics(ctx)
progress('running', 'backup timer')
manage_backup(cfg)
if problem:
progress('failed', failed_step, problem)
print(f'vpn-router: error: {problem}', file=sys.stderr)
@@ -66,6 +66,11 @@ SCHEMA = {
'otlp_endpoint': '', 'otlp_username': '', 'otlp_password': '',
'keep_details': '24 hours', 'keep_hourly': '1 month', 'keep_daily': '3 months',
'keep_monthly': '1 year', 'keep_yearly': '10 years'},
'backup': {'enabled': False, 'schedule': 'daily', 'target': 'local', 'path': '/var/backups/vpn-router',
'keep_hourly': 24, 'keep_daily': 7, 'keep_weekly': 4, 'keep_monthly': 12, 'keep_yearly': 0,
'metrics': False, 'rclone_remote': '', 'rclone_mode': 'sync',
'pbs_repository': '', 'pbs_password': '', 'pbs_fingerprint': '', 'pbs_namespace': '',
'pbs_keyfile': ''},
}
#: site_to_site maps a connection name to its settings: option -> default, the type of the default being the required type.
@@ -83,6 +88,30 @@ METRICS_KEEP = ('keep_details', 'keep_hourly', 'keep_daily', 'keep_monthly', 'ke
#: A retention period as PostgreSQL reads an interval: a count and a unit.
METRICS_INTERVAL = re.compile(r'^[1-9]\d{0,4} (hour|day|month|year)s?$')
#: How often backups run, as systemd OnCalendar shorthands, from the most to the least often.
BACKUP_SCHEDULES = ('hourly', 'daily', 'weekly')
#: Where backups go: an archive in backup.path, or a Proxmox Backup Server.
BACKUP_TARGETS = ('local', 'pbs')
#: How rclone follows backup.path in the cloud: sync mirrors it, copy deletes nothing remotely.
RCLONE_MODES = ('sync', 'copy')
#: The retention settings of local backups, from the hourly to the yearly ones.
BACKUP_KEEP = ('keep_hourly', 'keep_daily', 'keep_weekly', 'keep_monthly', 'keep_yearly')
#: A remote of rclone.conf and an optional path: remote:path.
RCLONE_REMOTE = re.compile(r'^[\w.][\w. -]*:\S*$')
#: A PBS repository: [[auth-id@]server[:port]:]datastore.
PBS_REPOSITORY = re.compile(r'^(\S+:)?[A-Za-z0-9_][A-Za-z0-9._-]*$')
#: The SHA-256 fingerprint of a server certificate, as PBS shows it.
PBS_FINGERPRINT = re.compile(r'^([0-9A-Fa-f]{2}:){31}[0-9A-Fa-f]{2}$')
#: A PBS namespace: up to seven names joined by /.
PBS_NAMESPACE = re.compile(r'^[A-Za-z0-9_][A-Za-z0-9._-]*(/[A-Za-z0-9_][A-Za-z0-9._-]*){0,6}$')
#: The port of Let's Encrypt's HTTP challenge, which Caddy answers while HTTPS is on.
ACME_HTTP_PORT = 80
@@ -412,6 +441,46 @@ def validate(cfg):
raise ConfigError(
f'metrics.{option}: {keep!r} is not a count and a unit, such as "24 hours" or "3 months"')
schedule = _get(cfg, 'backup', 'schedule')
if schedule not in BACKUP_SCHEDULES:
raise ConfigError(
f'backup.schedule: {schedule!r} is not one of {", ".join(BACKUP_SCHEDULES)}')
target = _get(cfg, 'backup', 'target')
if target not in BACKUP_TARGETS:
raise ConfigError(f'backup.target: {target!r} is not one of {", ".join(BACKUP_TARGETS)}')
for option in ('path', 'pbs_keyfile'):
value = _get(cfg, 'backup', option)
if value and not value.startswith('/'):
raise ConfigError(f'backup.{option}: {value!r} is not an absolute path')
if not _get(cfg, 'backup', 'path'):
raise ConfigError('backup.path is required')
for option in BACKUP_KEEP:
if _get(cfg, 'backup', option) < 0:
raise ConfigError(f'backup.{option} must not be negative')
if target == 'local' and not any(_get(cfg, 'backup', option) for option in backup_keeps(schedule)):
raise ConfigError(
f'backup: with the {schedule} schedule, at least one of {", ".join(backup_keeps(schedule))} '
'is above 0')
remote = _get(cfg, 'backup', 'rclone_remote')
if remote and not RCLONE_REMOTE.match(remote):
raise ConfigError(f'backup.rclone_remote: {remote!r} is not a remote and a path, such as azure:backups/router')
rclone_mode = _get(cfg, 'backup', 'rclone_mode')
if rclone_mode not in RCLONE_MODES:
raise ConfigError(f'backup.rclone_mode: {rclone_mode!r} is not one of {", ".join(RCLONE_MODES)}')
if target == 'pbs':
repository = _get(cfg, 'backup', 'pbs_repository')
if not repository or not _get(cfg, 'backup', 'pbs_password'):
raise ConfigError('backup.pbs_repository and backup.pbs_password are required with the pbs target')
if not PBS_REPOSITORY.match(repository):
raise ConfigError(
f'backup.pbs_repository: {repository!r} is not of the form user@realm!token@host:datastore')
fingerprint = _get(cfg, 'backup', 'pbs_fingerprint')
if fingerprint and not PBS_FINGERPRINT.match(fingerprint):
raise ConfigError(f'backup.pbs_fingerprint: {fingerprint!r} is not 32 hexadecimal pairs joined by colons')
namespace = _get(cfg, 'backup', 'pbs_namespace')
if namespace and not PBS_NAMESPACE.match(namespace):
raise ConfigError(f'backup.pbs_namespace: {namespace!r} is not up to seven names joined by /')
docker_mode = _get(cfg, 'docker', 'mode')
if docker_mode not in DOCKER_MODES:
raise ConfigError(
@@ -438,6 +507,11 @@ def validate(cfg):
'a range counting as two')
def backup_keeps(schedule):
"""The retention settings of local backups that apply to the schedule: none finer than it."""
return BACKUP_KEEP[BACKUP_SCHEDULES.index(schedule):]
def resolve_psk(connection, name):
"""Return the pre-shared key of the site_to_site connection: psk_file wins, else psk; '' when none is configured."""
psk_file = connection['psk_file'].strip()
@@ -0,0 +1,412 @@
"""The backups of the router, for vpn-router-backup and the web service.
A backup is a consistent copy of the state that setup cannot make again, staged in STAGING: the
trees of TREES, the files of site_to_site psk_file outside them under files/, the directory as
slapcat writes it (ldap/vpn-router.ldif), with backup.metrics the database as pg_dump writes it
(postgresql/vpn-router.dump), and manifest.json. The trees and the directory are copied while
the configuration lock and the users lock are held, so the configuration, the PKI and the users
match; pg_dump reads a snapshot of its own.
backup.target local writes the staging directory as a tar archive compressed with zstd,
ARCHIVE, to backup.path, removes the archives of this host that the retention settings of the
schedule no longer keep, and with backup.rclone_remote syncs or copies the archives of this host
to the remote of RCLONE_CONFIG. backup.target pbs sends the staging directory to a Proxmox Backup
Server as the archive router.pxar of the snapshot host/<host>/<time>; the server keeps them.
A run holds RUN_LOCK and records itself in STATE: result running, ok, warning (the backup is
made, the cloud copy failed) or failed, with the times, the target, the archive or snapshot,
its size, the error and the cloud copy. restore() puts a backup back and runs setup.
"""
import contextlib
import datetime
import fcntl
import json
import os
import pathlib
import re
import shutil
import socket
import subprocess
import tempfile
import time
import vpnrouter
import vpnrouter_auth
import vpnrouter_ldap
import vpnrouter_users
STATE = pathlib.Path('/var/lib/vpn-router/backup.json')
WORK_DIR = pathlib.Path('/var/lib/vpn-router/backup')
STAGING = WORK_DIR / 'staging'
RESTORE_DIR = WORK_DIR / 'restore'
#: What a restore replaces is kept here, in a directory named after the time of the restore.
BEFORE_RESTORE = WORK_DIR / 'before-restore'
RUN_LOCK = pathlib.Path('/run/vpn-router/backup.lock')
RCLONE_CONFIG = pathlib.Path('/etc/vpn-router/rclone.conf')
PBS_CLIENT = pathlib.Path('/usr/bin/proxmox-backup-client')
UNIT = 'vpn-router-backup.service'
TIMER = 'vpn-router-backup.timer'
#: The state a backup copies: directories and files, each where it is on the system.
TREES = ('/etc/vpn-router', '/etc/wireguard', '/etc/ufw', '/etc/default/ufw', '/etc/vpn-router-manager')
#: The name of a local archive: the host and the UTC time of the backup.
ARCHIVE = re.compile(r'^vpn-router-(?P<host>[A-Za-z0-9][A-Za-z0-9.-]*)-(?P<stamp>\d{8}T\d{6}Z)\.tar\.zst$')
STAMP = '%Y%m%dT%H%M%SZ'
LDIF = 'ldap/vpn-router.ldif'
DUMP = 'postgresql/vpn-router.dump'
PBS_ARCHIVE = 'router.pxar'
#: The period of each retention setting, as strftime formats of local time; weeks are ISO weeks.
PERIODS = {'keep_hourly': '%Y-%m-%d %H', 'keep_daily': '%Y-%m-%d', 'keep_weekly': '%G-%V', 'keep_monthly': '%Y-%m',
'keep_yearly': '%Y'}
#: The units a restore stops while it replaces the directory and the database, and the ones it restarts after setup.
RESTORE_STOPS = ('vpn-router-web.service', 'vpn-router-metrics.service', 'slapd.service')
RESTORE_RESTARTS = ('wg-quick@wg0.service', 'openvpn-server@vpn-router.service', 'strongswan.service',
'vpn-router-manager.service')
class Failure(Exception):
"""A backup or a restore that cannot be done."""
def now():
"""The local time in ISO 8601, to the second, with its offset."""
return datetime.datetime.now().astimezone().isoformat(timespec='seconds')
def host():
"""The name of this host, which names its archives and its PBS backup group."""
return socket.gethostname().split('.')[0]
def command(args, **kwargs):
"""Run a command; Failure with its standard error when it fails."""
try:
result = subprocess.run(args, capture_output=True, text=True, **kwargs)
except OSError as exc:
raise Failure(f'{args[0]}: {exc.strerror}')
if result.returncode != 0:
raise Failure(f'{" ".join(args[:2])} failed: {result.stderr.strip() or result.stdout.strip()}')
return result.stdout
def write_state(state):
"""Write STATE in one step."""
STATE.parent.mkdir(parents=True, exist_ok=True)
descriptor, name = tempfile.mkstemp(dir=STATE.parent, prefix='.backup.json.')
with os.fdopen(descriptor, 'w', encoding='utf-8') as handle:
handle.write(json.dumps(state, indent=2) + '\n')
os.chmod(name, 0o600)
os.replace(name, STATE)
def running():
"""Whether a run holds RUN_LOCK."""
try:
with open(RUN_LOCK, encoding='utf-8') as handle:
fcntl.flock(handle, fcntl.LOCK_EX | fcntl.LOCK_NB)
except FileNotFoundError:
return False
except BlockingIOError:
return True
return False
def state():
"""The state of the last run, {} when there was none; a run that ended without recording it is interrupted."""
try:
recorded = json.loads(STATE.read_text(encoding='utf-8'))
except (OSError, ValueError):
return {}
if recorded.get('result') == 'running' and not running():
recorded['result'] = 'interrupted'
return recorded
@contextlib.contextmanager
def run_lock():
"""Hold RUN_LOCK; Failure when another run holds it."""
RUN_LOCK.parent.mkdir(parents=True, exist_ok=True)
with open(RUN_LOCK, 'a', encoding='utf-8') as handle:
try:
fcntl.flock(handle, fcntl.LOCK_EX | fcntl.LOCK_NB)
except BlockingIOError:
raise Failure('another backup or restore is running')
yield
def archives(path):
"""The archives of this host in path, newest first, as [{name, time, size}]; time is the UTC time of the backup."""
found = []
try:
entries = list(os.scandir(path))
except OSError:
return found
for entry in entries:
match = ARCHIVE.match(entry.name)
if match and match['host'] == host() and entry.is_file():
when = datetime.datetime.strptime(match['stamp'], STAMP).replace(tzinfo=datetime.timezone.utc)
found.append({'name': entry.name, 'time': when.isoformat(), 'size': entry.stat().st_size})
return sorted(found, key=lambda item: item['time'], reverse=True)
def kept(found, keeps):
"""The names of the archives the retention settings keep, by the rules of Proxmox Backup Server's prune.
found is newest first. The settings apply in the order of PERIODS: each keeps the newest archive of each of its
last N periods that have archives and no archive a former setting kept, and the older archives of those periods
are removed, so no later setting keeps them.
"""
marks = {}
for option, period in PERIODS.items():
count = keeps.get(option, 0)
if not count:
continue
periods = {moment(item).strftime(period) for item in found if marks.get(item['name']) == 'keep'}
included = set()
for item in found:
if item['name'] in marks:
continue
key = moment(item).strftime(period)
if key in periods:
continue
if key not in included:
if len(included) >= count:
break
included.add(key)
marks[item['name']] = 'keep'
else:
marks[item['name']] = 'remove'
return {name for name, mark in marks.items() if mark == 'keep'}
def moment(item):
"""The local time of an archive of archives()."""
return datetime.datetime.fromisoformat(item['time']).astimezone()
def psk_files(cfg):
"""The psk_file of each site_to_site connection outside the trees of TREES."""
files = []
for connection in cfg['site_to_site'].values():
name = connection['psk_file'].strip()
if name and not any(name == tree or name.startswith(tree + '/') for tree in TREES) and os.path.isfile(name):
files.append(name)
return files
def database_args(cfg):
"""The command prefix and the database argument of pg_dump and pg_restore: postgres and vpn-router, or web.database."""
configured = cfg['web']['database'].strip()
if configured:
return [], [f'--dbname={configured}']
return ['runuser', '-u', 'postgres', '--'], [f'--dbname={vpnrouter_auth.DATABASE}']
def stage(cfg):
"""Copy what a backup holds to STAGING, emptied first, and return the manifest."""
shutil.rmtree(STAGING, ignore_errors=True)
STAGING.mkdir(parents=True, mode=0o700)
trees = [tree for tree in TREES if os.path.exists(tree)]
files = psk_files(cfg)
vpnrouter_users.LOCK_FILE.parent.mkdir(parents=True, exist_ok=True)
with vpnrouter.config_lock(), open(vpnrouter_users.LOCK_FILE, 'w') as users:
fcntl.flock(users, fcntl.LOCK_EX)
command(['cp', '-a', '--parents', *trees, str(STAGING)])
if files:
(STAGING / 'files').mkdir()
command(['cp', '-a', '--parents', *files, str(STAGING / 'files')])
(STAGING / LDIF).parent.mkdir()
command(['slapcat', '-b', vpnrouter_ldap.BASE, '-l', str(STAGING / LDIF)])
items = trees + [f'files{name}' for name in files] + [LDIF]
if cfg['backup']['metrics']:
(STAGING / DUMP).parent.mkdir()
prefix, database = database_args(cfg)
with open(STAGING / DUMP, 'wb') as dump:
result = subprocess.run([*prefix, 'pg_dump', '-Fc', *database], stdout=dump, stderr=subprocess.PIPE)
if result.returncode != 0:
raise Failure(f'pg_dump failed: {result.stderr.decode(errors="replace").strip()}')
items.append(DUMP)
version = subprocess.run(['dpkg-query', '-W', '-f=${Version}', 'vpn-router'], capture_output=True, text=True)
manifest = {'host': host(), 'time': now(), 'version': version.stdout.strip() or None,
'schedule': cfg['backup']['schedule'], 'items': items}
(STAGING / 'manifest.json').write_text(json.dumps(manifest, indent=2) + '\n', encoding='utf-8')
return manifest
def pbs_environment(cfg):
"""The environment of proxmox-backup-client: the repository, the token secret and the fingerprint of backup."""
if not PBS_CLIENT.exists():
raise Failure(f'{PBS_CLIENT} does not exist; install proxmox-backup-client-static')
backup = cfg['backup']
environment = dict(os.environ, PBS_REPOSITORY=backup['pbs_repository'], PBS_PASSWORD=backup['pbs_password'])
if backup['pbs_fingerprint']:
environment['PBS_FINGERPRINT'] = backup['pbs_fingerprint']
return environment
def pbs_options(cfg):
"""The namespace and key file options of proxmox-backup-client from backup."""
options = []
if cfg['backup']['pbs_namespace']:
options += ['--ns', cfg['backup']['pbs_namespace']]
if cfg['backup']['pbs_keyfile']:
options += ['--keyfile', cfg['backup']['pbs_keyfile']]
return options
def cloud(cfg):
"""Sync or copy the archives of this host in backup.path to backup.rclone_remote; Failure when it fails."""
backup = cfg['backup']
rclone = shutil.which('rclone')
if not rclone:
raise Failure('rclone is not installed')
if not RCLONE_CONFIG.is_file():
raise Failure(f'{RCLONE_CONFIG} does not exist; make the remote with rclone config --config {RCLONE_CONFIG}')
command([rclone, backup['rclone_mode'], backup['path'], backup['rclone_remote'], '--config', str(RCLONE_CONFIG),
'--include', f'vpn-router-{host()}-*.tar.zst'])
def run():
"""Make a backup to backup.target as the configuration says, recording it in STATE; the recorded state."""
cfg = vpnrouter.load_config()
backup = cfg['backup']
started = time.time()
record = {'result': 'running', 'started': now(), 'finished': None, 'target': backup['target'], 'archive': None,
'snapshot': None, 'size': None, 'removed': [], 'error': None, 'cloud': None}
with run_lock():
write_state(record)
try:
stage(cfg)
if backup['target'] == 'pbs':
epoch = int(started)
command([str(PBS_CLIENT), 'backup', f'{PBS_ARCHIVE}:{STAGING}', '--backup-type', 'host',
'--backup-id', host(), '--backup-time', str(epoch), *pbs_options(cfg)],
env=pbs_environment(cfg))
stamp = datetime.datetime.fromtimestamp(epoch, datetime.timezone.utc).strftime('%Y-%m-%dT%H:%M:%SZ')
record['snapshot'] = f'host/{host()}/{stamp}'
record['size'] = sum(path.stat().st_size for path in STAGING.rglob('*') if path.is_file())
else:
path = pathlib.Path(backup['path'])
path.mkdir(parents=True, mode=0o700, exist_ok=True)
name = f'vpn-router-{host()}-{time.strftime(STAMP, time.gmtime(started))}.tar.zst'
partial = path / f'{name}.partial'
command(['tar', '--zstd', '-cf', str(partial), '-C', str(STAGING), '.'])
partial.chmod(0o600)
partial.rename(path / name)
record['archive'] = name
record['size'] = (path / name).stat().st_size
found = archives(path)
keep = kept(found, {option: backup[option] for option in vpnrouter.backup_keeps(backup['schedule'])})
for item in found:
if item['name'] not in keep:
(path / item['name']).unlink()
record['removed'].append(item['name'])
if backup['rclone_remote']:
record['cloud'] = {'remote': backup['rclone_remote'], 'mode': backup['rclone_mode'],
'result': 'ok', 'error': None}
try:
cloud(cfg)
except Failure as exc:
record['cloud'].update(result='failed', error=str(exc))
record['result'] = 'warning' if record['cloud'] and record['cloud']['result'] != 'ok' else 'ok'
except (Failure, OSError) as exc:
record['result'] = 'failed'
record['error'] = str(exc)
finally:
shutil.rmtree(STAGING, ignore_errors=True)
record['finished'] = now()
write_state(record)
return record
def snapshots(cfg):
"""The PBS snapshots of this host, as proxmox-backup-client lists them."""
namespace = cfg['backup']['pbs_namespace']
return command([str(PBS_CLIENT), 'snapshot', 'list', f'host/{host()}', *(['--ns', namespace] if namespace else [])],
env=pbs_environment(cfg))
def extract(cfg, source, pbs):
"""Unpack a local archive, or with pbs fetch a snapshot, to RESTORE_DIR, emptied first; the manifest."""
shutil.rmtree(RESTORE_DIR, ignore_errors=True)
RESTORE_DIR.mkdir(parents=True, mode=0o700)
if pbs:
command([str(PBS_CLIENT), 'restore', source, PBS_ARCHIVE, str(RESTORE_DIR), *pbs_options(cfg)],
env=pbs_environment(cfg))
else:
command(['tar', '--zstd', '-xf', source, '-C', str(RESTORE_DIR)])
try:
return json.loads((RESTORE_DIR / 'manifest.json').read_text(encoding='utf-8'))
except (OSError, ValueError):
raise Failure(f'{source} is not a backup of vpn-router-backup: it has no manifest.json')
def put_back(relative, target, saved):
"""Move target to saved, keeping its path, and copy the staged relative path of RESTORE_DIR in its place."""
target = pathlib.Path(target)
if target.exists() or target.is_symlink():
keep = saved / target.relative_to('/')
keep.parent.mkdir(parents=True, exist_ok=True)
shutil.move(target, keep)
target.parent.mkdir(parents=True, exist_ok=True)
command(['cp', '-a', str(RESTORE_DIR / relative), str(target)])
def restore(source, pbs, confirm):
"""Put the backup back: a local archive, or with pbs a snapshot; confirm(manifest) is asked before any change.
What is replaced is kept in BEFORE_RESTORE. The services of the directory and the database are stopped while they
are replaced, and setup applies the configuration afterwards. Returns the directory of what was replaced, or None
when confirm declined.
"""
cfg = vpnrouter.load_config()
with run_lock():
try:
manifest = extract(cfg, source, pbs)
if not confirm(manifest):
return None
saved = BEFORE_RESTORE / time.strftime(STAMP, time.gmtime())
saved.mkdir(parents=True, mode=0o700)
for unit in RESTORE_STOPS:
subprocess.run(['systemctl', 'stop', unit], capture_output=True)
for item in manifest['items']:
if item.startswith('/'):
put_back(item.lstrip('/'), item, saved)
elif item.startswith('files/'):
put_back(item, item[len('files'):], saved)
if LDIF in manifest['items']:
directory = vpnrouter_ldap.DB_DIRECTORY
if directory.exists():
shutil.move(directory, saved / 'ldap')
directory.mkdir(mode=0o700)
command(['slapadd', '-b', vpnrouter_ldap.BASE, '-l', str(RESTORE_DIR / LDIF)])
command(['chown', '-R', 'openldap:openldap', str(directory)])
subprocess.run(['systemctl', 'start', 'slapd.service'], capture_output=True)
restored = vpnrouter.load_config()
if not restored['web']['database'].strip():
secret = vpnrouter_auth.DATABASE_SECRET
if secret.is_file() and vpnrouter_auth.psql("SELECT 1 FROM pg_roles WHERE rolname = :'v1'",
vpnrouter_auth.DATABASE):
password = secret.read_text(encoding='utf-8').strip()
vpnrouter_auth.psql(f"ALTER ROLE :\"v1\" PASSWORD '{password}'", vpnrouter_auth.DATABASE)
vpnrouter_auth.provision(restored)
if DUMP in manifest['items']:
prefix, database = database_args(restored)
with open(RESTORE_DIR / DUMP, 'rb') as dump:
result = subprocess.run([*prefix, 'pg_restore', '--clean', '--if-exists', *database], stdin=dump,
stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
if result.returncode != 0:
raise Failure(f'pg_restore failed: {result.stderr.decode(errors="replace").strip()}')
subprocess.run(['systemctl', 'start', 'vpn-router-web.service'], capture_output=True)
command(['systemctl', 'restart', 'vpn-router-setup.service'])
for unit in RESTORE_RESTARTS:
subprocess.run(['systemctl', 'try-restart', unit], capture_output=True)
subprocess.run(['ufw', 'reload'], capture_output=True)
return saved
finally:
shutil.rmtree(RESTORE_DIR, ignore_errors=True)
@@ -52,8 +52,9 @@ OVERLAYS = ('ppolicy', 'otp')
#: The package's own schema, under an OID of the UUID arc 2.25 (ITU-T X.667): attribute types
#: under .1 and object classes under .2. vpnRouterAccount, auxiliary, adds properties of the
#: package to a user of either kind: mail, for a Manager-only account, and vpnRouterScope,
#: authorization scopes, of which service marks a service account.
#: package to a user of either kind: mail, for a Manager-only account, vpnRouterScope,
#: authorization scopes, of which service marks a service account, and displayName and
#: description, the labels of a service account.
SCHEMA_OID = '2.25.98477352595735834532404256531620013001'
SCHEMA_NAME = 'vpn-router'
SCHEMA_ATTRIBUTE_TYPES = (
@@ -62,7 +63,7 @@ SCHEMA_ATTRIBUTE_TYPES = (
)
SCHEMA_OBJECT_CLASSES = (
f"( {SCHEMA_OID}.2.1 NAME 'vpnRouterAccount' DESC 'VPN router: properties of a user or a Manager-only account' "
"SUP top AUXILIARY MAY ( mail $ vpnRouterScope ) )",
"SUP top AUXILIARY MAY ( mail $ vpnRouterScope $ displayName $ description ) )",
)
#: The TOTP parameters: digits, the OID of HMAC-SHA1, seconds per time step, and the time steps accepted before and after now.
@@ -44,6 +44,15 @@ LEVELS = (
#: The kinds of VPN clients of metrics_vpn.
VPN_KINDS = ('p2s', 'openvpn', 'wireguard')
#: The time slots of the site-to-site availability of each view of history(): the summary level read (None: the
#: details), the slot as PostgreSQL starts it from time, and its length.
UP_SLOTS = {
'hour': (None, "date_bin('5 minutes', time, timestamptz '2000-01-01 00:00:00+00')", '5 minutes'),
'day': (None, "date_trunc('hour', time, 'UTC')", '1 hour'),
'month': ('hourly', "date_trunc('day', time, 'UTC')", '1 day'),
'year': ('monthly', "date_trunc('month', time, 'UTC')", '1 month'),
}
#: The views of history(): the summary level read (None: the details), the bucket and the span.
RANGES = {
'hour': (None, '1 minute', '1 hour'),
@@ -148,8 +157,13 @@ def history(connection, view):
each interface to its rx and tx, clients, which maps each kind of VPN client to its clients,
and site_to_site, which maps each site-to-site connection to the values of
metrics_site_to_site; a value without data is None.
site_to_site_up is the availability of each site-to-site connection in the time slots of UP_SLOTS, oldest first:
for each slot its start in Unix seconds and, per connection, the percentage of its samples in which the connection
was up, 0 to 100; the summaries' averages of up give it for the month and the year.
"""
level, bucket, span = RANGES[view]
up_level, up_start, up_slot = UP_SLOTS[view]
def rows(table):
keys, values = TABLES[table]
@@ -182,4 +196,12 @@ def history(connection, view):
for row in rows('metrics_site_to_site'):
point(row[0])['site_to_site'][row[1]] = pairs(row[2:], TABLES['metrics_site_to_site'][1])
empty = {value: {'avg': None, 'max': None} for value in TABLES['metrics_system'][1]}
return {'range': view, 'bucket': bucket, 'points': [{**empty, **points[start]} for start in sorted(points)]}
slots = {}
for start, name, percent in connection.execute(
f"SELECT extract(epoch FROM {up_start})::bigint AS slot, connection, avg({'up' if up_level is None else 'up_avg'}) * 100 "
f"FROM {'metrics_site_to_site' if up_level is None else f'metrics_site_to_site_{up_level}'} "
f"WHERE time >= now() - interval '{span}' GROUP BY slot, connection ORDER BY slot").fetchall():
slots.setdefault(start, {'time': start, 'connections': {}})['connections'][name] = percent
return {'range': view, 'bucket': bucket, 'points': [{**empty, **points[start]} for start in sorted(points)],
'site_to_site_up': {'slot': up_slot, 'points': [slots[start] for start in sorted(slots)]}}
@@ -8,9 +8,10 @@ simpleSecurityObject and vpnRouterAccount (the package's schema) with a name, a
optionally an e-mail address, no names, no certificate and so no VPN access; it is added when
neither the first nor the last name is given, and can sign in to the web service. A service
account is a Manager-only account with the scope SERVICE_SCOPE in vpnRouterScope, which another
system signs in with to use a router service, and which cannot sign in to the web service. The
internal service accounts are those the router creates and manages for its own services
(internal_accounts); vpn-router-setup keeps the metrics service's.
system signs in with to use a router service, and which cannot sign in to the web service; it has
a display name and a description (labels), in displayName and description of vpnRouterAccount. The
internal service accounts are those the router creates and manages for its own services, labels
included (internal_accounts); vpn-router-setup keeps the metrics service's.
Users exist whether or not a P2S system is enabled.
A group is a groupOfNames in ou=groups with a name, which follows the rules of user names, an
@@ -65,8 +66,11 @@ EMAIL = re.compile(r'^[^@\s]+@[^@\s]+\.[^@\s]+$')
EMAIL_MAX = 254
#: The keys of the person attributes and their LDAP attributes.
PERSON = (('first_name', 'givenName'), ('last_name', 'sn'), ('email', 'mail'))
#: The keys of the labels of a service account and their LDAP attributes.
LABELS = (('display_name', 'displayName'), ('description', 'description'))
#: The attributes read for a user.
ATTRIBUTES = ['uid', 'objectClass', 'pwdAccountLockedTime', 'givenName', 'sn', 'mail', 'oathTOTPToken', 'vpnRouterScope']
ATTRIBUTES = ['uid', 'objectClass', 'pwdAccountLockedTime', 'givenName', 'sn', 'mail', 'oathTOTPToken', 'vpnRouterScope',
'displayName', 'description']
#: The vpnRouterScope of a service account.
SERVICE_SCOPE = 'service'
#: The object classes of a VPN user and of a Manager-only account.
@@ -147,6 +151,26 @@ def person(values, required):
return found
def labels(values):
"""The labels of a service account in values ({display_name, description}), checked, as {LDAP attribute: value}.
A key whose value is None is not given; '' means none. A display name has at most PERSON_NAME_MAX printable
characters and a description at most DESCRIPTION_MAX.
"""
found = {}
name = values.get('display_name')
if name is not None:
if not isinstance(name, str):
raise Failure('display_name must be a string', 2)
name = name.strip()
if len(name) > PERSON_NAME_MAX or any(not character.isprintable() for character in name):
raise Failure(f'the display name must have at most {PERSON_NAME_MAX} printable characters', 2)
found['displayName'] = name
if values.get('description') is not None:
found['description'] = check_description(values['description'])
return found
def dn_of(name):
return f'uid={name},{vpnrouter_ldap.USERS}'
@@ -195,17 +219,23 @@ def service(attributes):
def internal_accounts():
"""The names of the internal service accounts, which the router creates and manages for its own services: the one
metrics.service_account names while metrics are enabled."""
"""The internal service accounts, which the router creates and manages for its own services, by name, each with
the labels the router gives it ({display_name, description}): the one metrics.service_account names while metrics
are enabled."""
try:
metrics = vpnrouter.load_config()['metrics']
except SystemExit:
return set()
return {metrics['service_account']} if metrics['enabled'] else set()
return {}
if not metrics['enabled']:
return {}
return {metrics['service_account']: {
'display_name': 'Metrics',
'description': 'The account an OpenTelemetry Collector signs in with to read /metrics of the metrics service.',
}}
def summary(attributes):
"""A user as the answers show it; a person attribute the entry does not have is None.
"""A user as the answers show it; a person attribute or a label the entry does not have is None.
internal is whether it is an internal service account (internal_accounts).
"""
@@ -221,7 +251,7 @@ def summary(attributes):
'certificate': certificate(name)[0].is_file(),
'otp': 'oathTOTPToken' in attributes,
}
for key, attribute in PERSON:
for key, attribute in PERSON + LABELS:
values = attributes.get(attribute)
user[key] = values[0].decode() if values else None
return user
@@ -242,12 +272,14 @@ def list_users(connection):
def add(connection, name, password, values, service=False):
"""Add a user: a VPN user with its certificate when values gives all three person attributes, a Manager-only account
with at most the e-mail address when it gives neither name, and, with service, a service account, with at most the
e-mail address too."""
e-mail address and its labels (labels)."""
check_name(name)
details = person(values, required=False)
vpn_user = 'givenName' in details or 'sn' in details
if service and vpn_user:
raise Failure('a service account has no first or last name', 2)
if service:
details.update({attribute: value for attribute, value in labels(values).items() if value})
if vpn_user and len(details) != len(PERSON):
raise Failure('give first_name, last_name and email for a VPN user, or at most email for a Manager-only account', 2)
check_password(password)
@@ -277,9 +309,12 @@ def add(connection, name, password, values, service=False):
def update(connection, name, values):
"""Change the person attributes values gives; the cn follows the first and the last name.
A Manager-only account has only the e-mail address, which '' removes.
A Manager-only account has only the e-mail address, which '' removes; a service account also its labels
(update_service).
"""
current = entry(connection, name)
if service(current):
return update_service(connection, name, values)
if manager_only(current):
if values.get('first_name') is not None or values.get('last_name') is not None:
raise Failure(f'user {name} is a Manager-only account, which has no first or last name', 2)
@@ -303,6 +338,21 @@ def update(connection, name, values):
return summary(entry(connection, name))
def update_service(connection, name, values):
"""Change any of the e-mail address and the labels (labels) of a service account that values gives; '' removes one."""
if not service(entry(connection, name)):
raise Failure(f'{name} is not a service account', 3)
changes = [(ldap.MOD_REPLACE, attribute, [value.encode()] if value else []) for attribute, value in labels(values).items()]
email = values.get('email')
if email is not None:
address = '' if isinstance(email, str) and email.strip() == '' else person({'email': email}, required=False)['mail']
changes.append((ldap.MOD_REPLACE, 'mail', [address.encode()] if address else []))
if not changes:
raise Failure('give email, display_name or description', 2)
connection.modify_s(dn_of(name), changes)
return summary(entry(connection, name))
def passwd(connection, name, password):
check_password(password)
entry(connection, name)
+93 -9
View File
@@ -24,7 +24,9 @@ Pages:
/site-to-site
the site-to-site connections of the configuration: list, add, change and remove,
applied by vpn-router-setup (see site_to_site())
/auth sign-in with a user's password and TOTP code, refresh and sign-out (see auth())
/backup the backup settings, the last run and the local archives: show, change and run
now (see backup())
/auth sign-in with a user's password and TOTP code, refresh and sign-out (see auth())
/metrics/history
the metrics the metrics service keeps, for a view: hour, day, month or year (see
metrics_history())
@@ -32,7 +34,7 @@ Pages:
the databases of each PostgreSQL cluster on the router with their sizes, as JSON (see
postgresql_databases()); /config has the clusters' settings
/config, /metrics/history, /postgresql, /users, /service-accounts, /groups, /site-to-site and /account need an HS256 JWT in an Authorization: Bearer header signed
/config, /metrics/history, /postgresql, /users, /service-accounts, /groups, /site-to-site, /backup and /account need an HS256 JWT in an Authorization: Bearer header signed
with vpnrouter_auth.WEB_KEY (see vpnrouter_auth.verify). A token of /auth names a user in sub;
/account serves that user, and the other paths need a member of vpnrouter_users.ADMIN_GROUP. A
token of vpn-router-token has the admin claim and names no user; it may call every path but
@@ -67,8 +69,10 @@ the router has no features, and /config shows no configuration or interfaces.
"""
import collections
import configparser
import json
import os
import shutil
import signal
import sys
import threading
@@ -83,6 +87,7 @@ import ldap
import vpnrouter
import vpnrouter_auth
import vpnrouter_backup
import vpnrouter_metrics
import vpnrouter_net
import vpnrouter_serve
@@ -508,8 +513,8 @@ def service_accounts(method, path, body):
"""Answer a /service-accounts request with vpnrouter_users: the HTTP status and the JSON document.
GET /service-accounts the service accounts: {"service_accounts": [...]}
POST /service-accounts add one: {"name", "password", "email"?}
PUT /service-accounts/<name> change the e-mail address: {"email"}; "" removes it
POST /service-accounts add one: {"name", "password", "display_name"?, "description"?, "email"?}
PUT /service-accounts/<name> change any of {"display_name", "description", "email"}; "" removes one
PUT /service-accounts/<name>/password set the password: {"password": ...}
POST /service-accounts/<name>/disable lock it out
POST /service-accounts/<name>/enable let it in again
@@ -525,13 +530,13 @@ def service_accounts(method, path, body):
parts = path.strip('/').split('/')
route = (method, len(parts), parts[2] if len(parts) == 3 else None)
name = parts[1] if len(parts) > 1 else None
email = {'email': body.get('email')}
settings = {key: body.get(key) for key in ('email', 'display_name', 'description')}
actions = {
('GET', 1, None): lambda connection: {
'service_accounts': [user for user in vpnrouter_users.list_users(connection)['users'] if user['service']]},
('POST', 1, None): lambda connection: vpnrouter_users.add(connection, body.get('name'), body.get('password'), email,
('POST', 1, None): lambda connection: vpnrouter_users.add(connection, body.get('name'), body.get('password'), settings,
service=True),
('PUT', 2, None): lambda connection: vpnrouter_users.update(connection, name, email),
('PUT', 2, None): lambda connection: vpnrouter_users.update_service(connection, name, settings),
('PUT', 3, 'password'): lambda connection: vpnrouter_users.passwd(connection, name, body.get('password')),
('POST', 3, 'disable'): lambda connection: vpnrouter_users.disable(connection, name),
('POST', 3, 'enable'): lambda connection: vpnrouter_users.enable(connection, name),
@@ -575,7 +580,7 @@ def groups(state, method, path, body):
route = (method, len(parts), parts[2] if len(parts) > 2 else None)
members = body.get('members') if route == ('POST', 1, None) else [body.get('user')] if route == ('POST', 3, 'members') \
else [parts[3]] if route == ('DELETE', 4, 'members') else []
internal = sorted(vpnrouter_users.internal_accounts().intersection(
internal = sorted(set(vpnrouter_users.internal_accounts()).intersection(
member for member in (members if isinstance(members, list) else []) if isinstance(member, str)))
if internal:
return 403, {'error': f'{", ".join(internal)}: internal service accounts, which the router manages, are not group members'}
@@ -676,6 +681,82 @@ def site_to_site(method, path, body):
return (201 if method == 'POST' else 200), connection_of(name, cfg['site_to_site'][name], None)
def backup_settings(settings):
"""The backup section as /backup shows it: its settings but pbs_password, with pbs_password_set."""
return {**{option: value for option, value in settings.items() if option != 'pbs_password'},
'pbs_password_set': bool(settings['pbs_password'])}
def rclone_remotes():
"""The names of the remotes of vpnrouter_backup.RCLONE_CONFIG; [] when it cannot be read, as when it is encrypted."""
parser = configparser.ConfigParser(interpolation=None)
try:
parser.read(vpnrouter_backup.RCLONE_CONFIG, encoding='utf-8')
except (OSError, configparser.Error):
return []
return parser.sections()
def backup(method, path, body):
"""Answer a /backup request: the HTTP status and the JSON document.
GET /backup {"settings", "state", "archives", "next", "client", "rclone"}: the backup section without
pbs_password, with pbs_password_set; the last run (vpnrouter_backup.state), null before the
first; the local archives of this host, newest first, [{name, time, size}], [] with the pbs
target; the next run of the timer as systemd shows it, null while backups are disabled;
whether proxmox-backup-client is installed; and {"installed", "config", "remotes"} of rclone
PUT /backup change any of the settings of the backup section; an empty or absent pbs_password keeps the
current secret
POST /backup/run start a backup now, without waiting for it; 409 while one runs
A change is made as /site-to-site makes one, and applied by restarting vpn-router-setup without waiting for it.
"""
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
return 400, {'error': 'the body must be a JSON object'}
body = body or {}
if (method, path) == ('POST', '/backup/run'):
if vpnrouter_backup.running():
return 409, {'error': 'a backup is running'}
result = run('systemctl', 'start', '--no-block', vpnrouter_backup.UNIT)
if result is None or result.returncode != 0:
return 500, {'error': f'the backup cannot be started: {result.stderr.strip() if result else "no systemctl"}'}
return 202, {'started': True}
if (method, path) not in (('GET', '/backup'), ('PUT', '/backup')):
return 404, {'error': f'no route {method} {path}'}
try:
if method == 'GET':
settings = vpnrouter.load_config()['backup']
timer = run('systemctl', 'show', '-p', 'NextElapseUSecRealtime', '--value', vpnrouter_backup.TIMER)
next_run = timer.stdout.strip() if timer and timer.returncode == 0 else ''
return 200, {
'settings': backup_settings(settings),
'state': vpnrouter_backup.state() or None,
'archives': vpnrouter_backup.archives(settings['path']) if settings['target'] == 'local' else [],
'next': next_run if settings['enabled'] and next_run not in ('', 'n/a') else None,
'client': vpnrouter_backup.PBS_CLIENT.exists(),
'rclone': {'installed': shutil.which('rclone') is not None,
'config': vpnrouter_backup.RCLONE_CONFIG.is_file(), 'remotes': rclone_remotes()},
}
settings = {option: body[option] for option in vpnrouter.SCHEMA['backup'] if option in body}
if not settings.get('pbs_password'):
settings.pop('pbs_password', None)
with vpnrouter.config_lock():
data = vpnrouter.read_settings()
current = data.get('backup')
data['backup'] = {**(current if isinstance(current, dict) else {}), **settings}
cfg = vpnrouter.with_defaults(data)
vpnrouter.validate(cfg)
vpnrouter.save_config(data)
except vpnrouter.ConfigError as exc:
return 400, {'error': str(exc)}
except OSError as exc:
return 500, {'error': f'the configuration cannot be written: {exc}'}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
run('systemctl', 'restart', '--no-block', 'vpn-router-setup.service')
return 200, {'settings': backup_settings(cfg['backup'])}
def account(name, method, path, body):
"""Answer an /account request of the user named by the token with vpnrouter_users: the HTTP status and the JSON document.
@@ -864,7 +945,7 @@ class Handler(BaseHTTPRequestHandler):
# A body that is not read must not be taken for the next request.
if self.command not in ('GET', 'HEAD'):
self.close_connection = True
directory = next((name for name in ('users', 'service-accounts', 'groups', 'account', 'site-to-site')
directory = next((name for name in ('users', 'service-accounts', 'groups', 'account', 'site-to-site', 'backup')
if path == f'/{name}' or path.startswith(f'/{name}/')), None)
protected = path in ('/config', '/metrics/history', '/postgresql') or directory is not None
signing = path in ('/auth', '/auth/refresh', '/auth/logout')
@@ -905,6 +986,9 @@ class Handler(BaseHTTPRequestHandler):
elif directory == 'site-to-site':
code, document = site_to_site(self.command, path, payload)
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif directory == 'backup':
code, document = backup(self.command, path, payload)
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif directory == 'service-accounts':
code, document = service_accounts(self.command, path, payload)
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
+98
View File
@@ -0,0 +1,98 @@
#!/usr/bin/python3
"""Make, list and restore backups of the router (vpnrouter_backup).
run makes a backup as backup in /etc/vpn-router/vpn-router.json says, whether or not backups are
enabled, and prints its result; vpn-router-backup.service runs it on the schedule of its timer.
list prints the local archives of this host in backup.path, newest first, or with the pbs target
the snapshots of this host on the Proxmox Backup Server. restore puts back a local archive, or
with --pbs a snapshot, such as host/router/2026-10-25T03:00:00Z, after showing what it holds and
asking for yes, which --yes gives; what it replaces is kept in
/var/lib/vpn-router/backup/before-restore. A failure is a message on standard error; the exit
status is 2 for an invalid request and 1 for any other failure. -h or --help shows the options.
vpn-router-backup run
vpn-router-backup list
vpn-router-backup restore [--yes] ARCHIVE
vpn-router-backup restore [--yes] --pbs SNAPSHOT
"""
import argparse
import os
import sys
sys.path.insert(0, '/usr/lib/vpn-router')
import vpnrouter
import vpnrouter_backup
class Parser(argparse.ArgumentParser):
"""An argument parser whose errors end with status 2."""
def error(self, message):
print(f'{self.prog}: {message}; see {self.prog} --help', file=sys.stderr)
sys.exit(2)
def confirmed(manifest, yes):
"""Whether to restore the backup of the manifest: yes, or the answer yes to the question."""
print(f'Backup of {manifest.get("host")} made {manifest.get("time")} by vpn-router {manifest.get("version")}:')
for item in manifest.get('items', []):
print(f' {item}')
if yes:
return True
return input('Replace the router state with it? Type yes: ').strip() == 'yes'
def main(argv):
parser = Parser(prog='vpn-router-backup', description='Make, list and restore backups of the router.')
commands = parser.add_subparsers(dest='command', required=True, parser_class=Parser)
commands.add_parser('run', help='make a backup now')
commands.add_parser('list', help='list the backups of this host')
restorer = commands.add_parser('restore', help='put a backup back')
restorer.add_argument('source', metavar='ARCHIVE|SNAPSHOT', help='a local archive, or with --pbs a snapshot')
restorer.add_argument('--pbs', action='store_true', help='restore a snapshot of the Proxmox Backup Server')
restorer.add_argument('--yes', action='store_true', help='do not ask before replacing the router state')
options = parser.parse_args(argv)
os.umask(0o077)
try:
if options.command == 'run':
record = vpnrouter_backup.run()
if record['result'] == 'failed':
print(f'vpn-router-backup: {record["error"]}', file=sys.stderr)
return 1
print(f'Backup {record["archive"] or record["snapshot"]} made, {record["size"]} bytes.')
for name in record['removed']:
print(f'Removed {name}.')
if record['cloud']:
print(f'Cloud copy to {record["cloud"]["remote"]}: {record["cloud"]["error"] or "ok"}.')
return 0
cfg = vpnrouter.load_config()
if options.command == 'list':
if cfg['backup']['target'] == 'pbs':
print(vpnrouter_backup.snapshots(cfg), end='')
else:
for item in vpnrouter_backup.archives(cfg['backup']['path']):
print(f'{item["name"]} {item["size"]}')
return 0
if options.pbs and options.source.startswith('/'):
print('vpn-router-backup: a snapshot is type/id/time, such as host/router/2026-10-25T03:00:00Z',
file=sys.stderr)
return 2
saved = vpnrouter_backup.restore(options.source, options.pbs, lambda manifest: confirmed(manifest, options.yes))
except vpnrouter_backup.Failure as exc:
print(f'vpn-router-backup: {exc}', file=sys.stderr)
return 1
except OSError as exc:
print(f'vpn-router-backup: {exc}', file=sys.stderr)
return 1
if saved is None:
print('Nothing restored.')
return 1
print(f'Restored; what it replaced is in {saved}.')
return 0
if __name__ == '__main__':
sys.exit(main(sys.argv[1:]))
+8 -4
View File
@@ -11,8 +11,10 @@ user exists, 1 anything else. -h or --help shows the commands and their options.
vpn-router-user [--json] list
vpn-router-user [--json] add <name> --first-name F --last-name L --email E (password on stdin)
vpn-router-user [--json] add <name> [--email E] (a Manager-only account; password on stdin)
vpn-router-user [--json] add <name> --service [--email E] (a service account; password on stdin)
vpn-router-user [--json] add <name> --service [--display-name N] [--description D] [--email E]
(a service account; password on stdin)
vpn-router-user [--json] update <name> [--first-name F] [--last-name L] [--email E]
vpn-router-user [--json] update <name> [--display-name N] [--description D] [--email E] (a service account)
vpn-router-user [--json] passwd <name> (password on stdin)
vpn-router-user [--json] disable <name>
vpn-router-user [--json] enable <name>
@@ -57,7 +59,9 @@ HELP = {
}
#: The help of the person options.
PERSON_HELP = {'first_name': 'the first name', 'last_name': 'the last name', 'email': 'the e-mail address'}
PERSON_HELP = {'first_name': 'the first name', 'last_name': 'the last name', 'email': 'the e-mail address',
'display_name': "a service account's display name; '' removes it",
'description': "a service account's description; '' removes it"}
class Parser(argparse.ArgumentParser):
@@ -76,7 +80,7 @@ def parse(argv):
if command != 'list':
sub.add_argument('name', help='the user name')
if command in ('add', 'update'):
for key, _ in vpnrouter_users.PERSON:
for key, _ in vpnrouter_users.PERSON + vpnrouter_users.LABELS:
sub.add_argument(f'--{key.replace("_", "-")}', dest=key, metavar='TEXT', help=PERSON_HELP[key])
if command == 'add':
sub.add_argument('--service', action='store_true', help='add a service account, which cannot sign in to the web service')
@@ -95,7 +99,7 @@ def run(connection, options, secret):
command = options.command
if command == 'list':
return vpnrouter_users.list_users(connection)
values = {key: getattr(options, key, None) for key, _ in vpnrouter_users.PERSON}
values = {key: getattr(options, key, None) for key, _ in vpnrouter_users.PERSON + vpnrouter_users.LABELS}
if command == 'add':
return vpnrouter_users.add(connection, options.name, secret, values, service=options.service)
if command == 'update':
@@ -74,5 +74,24 @@
"keep_daily": "3 months",
"keep_monthly": "1 year",
"keep_yearly": "10 years"
},
"backup": {
"enabled": false,
"schedule": "daily",
"target": "local",
"path": "/var/backups/vpn-router",
"keep_hourly": 24,
"keep_daily": 7,
"keep_weekly": 4,
"keep_monthly": 12,
"keep_yearly": 0,
"metrics": false,
"rclone_remote": "",
"rclone_mode": "sync",
"pbs_repository": "",
"pbs_password": "",
"pbs_fingerprint": "",
"pbs_namespace": "",
"pbs_keyfile": ""
}
}