feature: the router makes scheduled backups to a local folder, copied to the cloud with rclone, or to a Proxmox Backup Server, and vpn-router-backup restores them.

2026-10-25 20:18:00 +01:00
1 parent 7d1f639508
commit fe45272f7c
@@ -143,6 +143,23 @@ must have the type shown below; an unknown section or setting is an error.
 | `metrics` | `keep_daily` | string | `3 months` | How long the daily summaries are kept |
 | `metrics` | `keep_monthly` | string | `1 year` | How long the monthly summaries are kept |
 | `metrics` | `keep_yearly` | string | `10 years` | How long the yearly summaries are kept |
+| `backup` | `enabled` | boolean | `false` | Scheduled backups (see Backups) |
+| `backup` | `schedule` | string | `daily` | `hourly`, `daily` or `weekly` |
+| `backup` | `target` | string | `local` | `local`, an archive in `backup.path`, or `pbs`, a Proxmox Backup Server |
+| `backup` | `path` | string | `/var/backups/vpn-router` | The folder of local archives, an absolute path |
+| `backup` | `keep_hourly` | integer | `24` | Hourly local archives kept, with the hourly schedule |
+| `backup` | `keep_daily` | integer | `7` | Daily local archives kept, with the hourly and the daily schedule |
+| `backup` | `keep_weekly` | integer | `4` | Weekly local archives kept |
+| `backup` | `keep_monthly` | integer | `12` | Monthly local archives kept |
+| `backup` | `keep_yearly` | integer | `0` | Yearly local archives kept |
+| `backup` | `metrics` | boolean | `false` | The backup holds the database of the metrics and the refresh tokens |
+| `backup` | `rclone_remote` | string | | `remote:path` of `/etc/vpn-router/rclone.conf` the local archives are copied to |
+| `backup` | `rclone_mode` | string | `sync` | `sync` mirrors the archives, `copy` deletes nothing in the cloud |
+| `backup` | `pbs_repository` | string | | `user@realm!token@host[:port]:datastore` |
+| `backup` | `pbs_password` | string | | The secret of the API token |
+| `backup` | `pbs_fingerprint` | string | | The SHA-256 fingerprint of the server's certificate, when the system's CAs do not verify it |
+| `backup` | `pbs_namespace` | string | | The namespace of the datastore, such as `routers/azure` |
+| `backup` | `pbs_keyfile` | string | | An encryption key of `proxmox-backup-client key create`, an absolute path |

 ### Site-to-site connections

@@ -826,6 +843,28 @@ is `{"error": ...}` with 400 for an invalid request or configuration, with the r
 connection that does not exist, 409 for a name that exists and 500 otherwise. Keys cross the
 internal network in plain HTTP, like the token.

+### The /backup API
+
+`/backup` shows and changes the backup settings and starts a backup, with the same token as
+`/users`.
+
+| Request | Body | Answer |
+| --- | --- | --- |
+| `GET /backup` | | `{"settings", "state", "archives", "next", "client", "rclone"}` |
+| `PUT /backup` | any of the settings of the `backup` section; an empty or absent `pbs_password` keeps the secret | `{"settings"}` |
+| `POST /backup/run` | | 202 and `{"started": true}`; 409 while a backup runs |
+
+The answer of `GET /backup` holds:
+
+- **`settings`:** the `backup` section without `pbs_password`, with `pbs_password_set`.
+- **`state`:** the last run, `null` before the first. Its `result` is `running`, `ok`, `warning` (the backup was made but the cloud copy failed), `failed` or `interrupted` (the run ended without recording its result). It also has `started`, `finished`, `target`, `archive` or `snapshot`, `size` in bytes, `removed` (the archives retention deleted), `error` and `cloud` (`{"remote", "mode", "result", "error"}`).
+- **`archives`:** the local archives of this host, newest first, as `{"name", "time", "size"}`.
+- **`next`:** the next run as systemd shows it, `null` while backups are off.
+- **`client`:** whether `proxmox-backup-client` is installed.
+- **`rclone`:** `{"installed", "config", "remotes"}`, the remotes being the sections of `rclone.conf`.
+
+A change is made, checked and applied as for `/site-to-site`.
+
 The service starts on its own, without waiting for `vpn-router-setup`, and keeps running. Every
 30 seconds it reads `web.listen`, `web.port` and `web.enabled` from `vpn-router.json` and works
 out the addresses of the interfaces `web.listen` names as `general.mode` directs; it listens on
@@ -968,11 +1007,107 @@ When `metrics.otlp_endpoint` is set, every sample is also pushed there as an OTL
 - **Authentication:** HTTP Basic with `metrics.otlp_username` and `metrics.otlp_password` when they are set.
 - **Failures:** a failed push is logged and not repeated; the next sample is pushed a minute later.

+## Backups
+
+While `backup.enabled` is `true`, `vpn-router-setup` runs `vpn-router-backup.timer` on `backup.schedule`. The timer starts `vpn-router-backup.service`, which runs `vpn-router-backup run`. The schedule is written to `/etc/systemd/system/vpn-router-backup.timer.d/schedule.conf`:
+
+- `hourly` runs at the start of each hour;
+- `daily` runs at midnight;
+- `weekly` runs at midnight between Sunday and Monday.
+
+A run missed while the router was off is made at the next boot.
+
+### What a backup holds
+
+A backup holds the state that `vpn-router-setup` cannot make again:
+
+| Item | Content |
+|---|---|
+| `/etc/vpn-router` | `vpn-router.json`, the secrets, `rclone.conf`, the PKI and the certificates of the users |
+| `/etc/wireguard` | The WireGuard keys and the peers |
+| `/etc/ufw`, `/etc/default/ufw` | The firewall rules |
+| `/etc/vpn-router-manager` | The Router Manager's settings, when it is installed |
+| `files/<path>` | Each `psk_file` outside `/etc/vpn-router` |
+| `ldap/vpn-router.ldif` | The user directory, written by `slapcat`: users, groups, service accounts, password hashes and TOTP secrets |
+| `postgresql/vpn-router.dump` | With `backup.metrics`, the database of the metrics and the refresh tokens, written by `pg_dump -Fc` |
+| `manifest.json` | The host, the time, the version of the package, the schedule and the items |
+
+The files and the directory are copied while the locks of the configuration and of the users are held, so no change of the API or the command-line tools falls between them; `pg_dump` reads a snapshot of its own. The copy is staged in `/var/lib/vpn-router/backup/staging`, which is removed after each run. The backup holds every secret of the router, so keep it as safe as the router itself.
+
+### Local archives
+
+With `backup.target` `local`, a backup is a tar archive compressed with zstd, `vpn-router-<host>-<UTC time>.tar.zst`, in `backup.path` (mode 0600, the folder 0700). The folder may be a mount, such as an NFS share or an Azure Files share, which exports the archives.
+
+After each backup, the archives of this host that the retention settings no longer keep are deleted. The rules are those of Proxmox Backup Server's prune:
+
+- The settings apply in order: `keep_hourly`, `keep_daily`, `keep_weekly` (ISO weeks, Monday to Sunday), `keep_monthly`, `keep_yearly`.
+- Each keeps the newest archive of each of its last N periods that have archives, in local time.
+- A period that has an archive a former setting kept is passed over.
+- A setting finer than the schedule does not apply: `keep_hourly` only with `hourly`, `keep_daily` with `hourly` and `daily`. At least one setting that applies must be above 0.
+
+### Cloud copy with rclone
+
+With `backup.rclone_remote` set, each local backup is followed by an rclone run that copies the archives of this host to the remote. rclone comes from the `rclone` package, which the package suggests:
+
+```sh
+sudo apt-get install rclone
+sudo rclone config --config /etc/vpn-router/rclone.conf
+sudo chmod 0600 /etc/vpn-router/rclone.conf
+sudo vpn-config-edit set backup.rclone_remote azure:backups/router
+sudo systemctl restart vpn-router-setup
+```
+
+The two modes:
+
+- `sync` (the default) makes the remote mirror the archives of this host, so an archive retention deletes is deleted remotely too.
+- `copy` uploads new archives and deletes nothing, so retention in the cloud is left to its own rules, such as a lifecycle policy.
+
+Other files in the remote path are left alone in both modes. A failed cloud copy leaves the local backup in place, and the run's result is `warning`.
+
+### Proxmox Backup Server
+
+With `backup.target` `pbs`, the staged copy is sent with `proxmox-backup-client` as the archive `router.pxar` of the snapshot `host/<host>/<time>`, in `backup.pbs_namespace` when it is set. The server keeps the snapshots by its own prune jobs; the router prunes nothing.
+
+The static client comes from Proxmox's client repository, which the package suggests but does not set up. Following Proxmox's installation guide, as `/etc/apt/sources.list.d/pbs-client.sources`:
+
+```text
+Types: deb
+URIs: http://download.proxmox.com/debian/pbs-client
+Suites: trixie
+Components: main
+Signed-By: /usr/share/keyrings/proxmox-archive-keyring.gpg
+```
+
+Get the key `https://enterprise.proxmox.com/debian/proxmox-archive-keyring-trixie.gpg` into `/usr/share/keyrings/proxmox-archive-keyring.gpg`, check it against the SHA-256 the [installation guide](https://pbs.proxmox.com/docs/installation.html) gives, and install `proxmox-backup-client-static`. Until it is installed, a backup to PBS fails with a message that names the package; nothing else depends on it.
+
+On the server, make an API token for the router and give the token itself the `DatastoreBackup` role on `/datastore/<store>`, or on the namespace path. `backup.pbs_repository` names the token, such as `backup@pbs!router@pbs.example.com:store`, and `backup.pbs_password` holds its secret. Set `backup.pbs_fingerprint` when the server's certificate is self-signed. With `backup.pbs_keyfile`, the snapshot is encrypted with a key of `proxmox-backup-client key create --kdf none <file>`. Keep a copy of that key away from the router, since a restore needs it.
+
+### Running, listing and restoring
+
+```sh
+sudo vpn-router-backup run          # a backup now, whether or not backups are enabled
+sudo vpn-router-backup list         # the local archives, or the snapshots on the server
+sudo vpn-router-backup restore /var/backups/vpn-router/vpn-router-router-20261025T030000Z.tar.zst
+sudo vpn-router-backup restore --pbs host/router/2026-10-25T03:00:00Z
+```
+
+Each run is recorded in `/var/lib/vpn-router/backup.json`, which `/backup` and the Router Manager show. Only one run or restore takes place at a time.
+
+`restore` is only on the command line. It shows what the backup holds and asks for `yes`, which `--yes` gives. It then:
+
+1. stops the web service, the metrics service and slapd;
+2. puts back the files;
+3. loads the directory with `slapadd` into an empty `/var/lib/ldap/vpn-router`;
+4. sets the database role's password to the restored `database.secret`, and restores the database with `pg_restore --clean` when the backup holds it;
+5. restarts `vpn-router-setup`, which regenerates everything else, followed by WireGuard, OpenVPN, strongSwan and the Router Manager, and reloads ufw.
+
+What it replaces is kept in `/var/lib/vpn-router/backup/before-restore/<UTC time>`. To restore onto a new VM, install the package first and let `vpn-router-setup` run once. The restored configuration names the interfaces of the old VM, so change them with `vpn-config-edit` if they differ.
+
 ## Router Manager

 The `vpn-router-manager` package, built in the repository `linux-cloud-router-manager`, is a web
-application that shows the state of the router and manages its users and site-to-site
-connections through the status web service. It can be installed on the router or on another Linux system. While it is installed on
+application that shows the state of the router and manages its users, site-to-site
+connections and backups through the status web service. It can be installed on the router or on another Linux system. While it is installed on
 the router, `vpn-router-setup` opens its port (`port` in `/etc/vpn-router-manager/manager.json`,
 7080 by default, HTTP; HTTPS is Caddy's, on `https.manager_port`) on the internal and the external
 interface; restart `vpn-router-setup` after changing it. The manager's `listen` chooses the
@@ -981,8 +1116,8 @@ interfaces it listens on.
 ## Removal

 `apt-get remove` stops `vpn-router-docker`, which removes the Docker rules. It terminates the
-tunnels and takes `wg-quick@wg0`, `openvpn-server@vpn-router`, `caddy` and `vpn-router-metrics`
-down, but leaves the
+tunnels and takes `wg-quick@wg0`, `openvpn-server@vpn-router`, `caddy`, `vpn-router-metrics` and
+`vpn-router-backup.timer` down, but leaves the
 configuration in place. `strongswan` is left enabled, since it is a shared service that
 may be in use by something else.

@@ -992,8 +1127,11 @@ before deleting it so a later reinstall starts clean. It also removes the web se
 signing key, the password file of the metrics service account, and the database of the refresh
 tokens and the metrics, its role and their password
 file when the package created them, and Caddy's configuration `/etc/caddy/vpn-router.Caddyfile`; the certificate in
-`/var/lib/caddy` belongs to the `caddy` package. Key material and users are never deleted: `/etc/vpn-router/pki`, `/etc/vpn-router/ldap`, the user directory in slapd, the
-WireGuard key pair and everything under `/etc/swanctl` are left alone.
+`/var/lib/caddy` belongs to the `caddy` package. It removes the backup timer's schedule and the
+record of the last backup. Key material, users and backups are never deleted: `/etc/vpn-router/pki`,
+`/etc/vpn-router/ldap`, `/etc/vpn-router/rclone.conf`, the user directory in slapd, the WireGuard
+key pair, everything under `/etc/swanctl`, the archives in `backup.path` and
+`/var/lib/vpn-router/backup/before-restore` are left alone.

 ## Building