Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
25c151841b | ||
|
|
68ebd541b2 | ||
|
|
f01be41191 | ||
|
|
d1970b77b7 | ||
|
|
bf651da5a6 | ||
|
|
4a5b15a559 | ||
|
|
a79510a02f | ||
|
|
38151bf5be | ||
|
|
83c1c50d8d | ||
|
|
650896131d | ||
|
|
dbc2046e24 | ||
|
|
45d73abe7b | ||
|
|
993a7a9fa9 | ||
|
|
581b53d447 | ||
|
|
8782edff0f | ||
|
|
fe17e83b49 | ||
|
|
17331eea37 | ||
|
|
af3127d1e8 | ||
|
|
b1e6d9cd0b |
No files matched your search
@@ -27,28 +27,40 @@ port is reachable from the external interface only when it is in `exposed_ports`
|
||||
in `DOCKER-USER` are not touched, because the package owns only its own chain. Restarting Docker
|
||||
restores the rules. The rules cover IPv4 only.
|
||||
|
||||
## ADR-2: A small status web service, bound to the internal address
|
||||
## ADR-2: A small status web service, on the interfaces web.listen chooses
|
||||
|
||||
Status: Accepted
|
||||
|
||||
Context: The router has no way to see its state without a shell. A check from the protected
|
||||
network, by a person or by a monitor, needs a page or an endpoint. The router is an
|
||||
internet-facing host, so what the service shows and where it listens matter.
|
||||
network, by a person or by a monitor, needs a page or an endpoint, and so does the platform that
|
||||
deploys the router, which reaches it on its public address. The router is an internet-facing
|
||||
host, so what the service shows and where it listens matter.
|
||||
|
||||
Decision: The `vpn-router-web` unit runs one Python script on the standard library's
|
||||
`http.server`, with the package's own modules and the Python packages of the Ubuntu archive that
|
||||
the package depends on, and no web framework. It listens on the address of the internal interface and on
|
||||
`web.port`, 7081 by default, and is on unless `web.enabled` is `false` (ADR-7). `/` is for
|
||||
people, in plain text for `curl` and `wget` and as a Tailwind CSS page for a browser, and
|
||||
`/status` is the same status as JSON. It shows the state of the services, the package version
|
||||
and the state of the site-to-site, road-warrior, OpenVPN and WireGuard features, and nothing from
|
||||
the configuration. `vpn-router-setup` opens the port on the internal interface in UFW and writes
|
||||
the features and the facts it resolves to `/run/vpn-router/web.json`.
|
||||
the package depends on, and no web framework. It listens on `web.port`, 7081 by default, on the
|
||||
address of the internal interface, of the external interface or of both, as `web.listen` sets
|
||||
(`internal`, `external` or `both`, the default), with a server for each address, and is on
|
||||
unless `web.enabled` is `false` (ADR-7). `/` is for people, in plain text for `curl` and `wget`
|
||||
and as a Tailwind CSS page for a browser, and `/status` is the same status as JSON. Both show as
|
||||
much as `web.status_content` says, since the router is reachable from the internet and the
|
||||
operator decides how much it tells strangers: `basic`, the default because it tells the least,
|
||||
the overall status and the Client IP, `standard` also the package versions and the number of
|
||||
active and other services, and `full` also each service and the state of the site-to-site, road-warrior, OpenVPN and
|
||||
WireGuard features. The Client IP is the address the request came from, through the trusted
|
||||
proxies, so a person can see the address the router sees. Nothing comes from the configuration.
|
||||
`/status` names its content, so clients such as manage-ui and the Router Manager show what it
|
||||
has. `vpn-router-setup` opens the
|
||||
port on both interfaces in UFW, so the choice of `web.listen` is made by the service alone, and
|
||||
writes the features and the facts it resolves to `/run/vpn-router/web.json`.
|
||||
|
||||
Consequences: Nothing outside the internal network can reach it, since it listens on the internal
|
||||
address only and the firewall opens the port on the internal interface only. `/` and `/status`
|
||||
have no authentication, which is acceptable because they are read only and show no
|
||||
configuration. It runs as root, because `swanctl` and `wg` need it, with `NoNewPrivileges`,
|
||||
Consequences: By default the service is reachable from the internet on the external address;
|
||||
`web.listen` set to `internal` keeps it to the internal network. It serves plain HTTP itself;
|
||||
HTTPS is Caddy's (ADR-21), so on the plain HTTP port the passwords of `/auth` and the tokens
|
||||
cross the internet unencrypted. A change of `web.listen`
|
||||
needs no run of setup, since the firewall does not depend on it; the package's upgrade runs
|
||||
setup, which writes the rules. `/` and `/status` have no authentication, which is acceptable because they are
|
||||
read only and show no configuration. It runs as root, because `swanctl` and `wg` need it, with `NoNewPrivileges`,
|
||||
`ProtectSystem=strict`, `ProtectHome` and `PrivateTmp`. The Tailwind style comes from a CDN, so a
|
||||
browser on a network without internet shows the unstyled page, and `curl` is unaffected.
|
||||
|
||||
@@ -60,7 +72,7 @@ Context: Clients such as the Router Manager, and people with `curl`, need the co
|
||||
the users and the groups of the router, which `/` and `/status` must not show. Both people and
|
||||
programs sign in with a user of the directory. A secret shared with the platform that deploys
|
||||
the router would let whoever holds it, and every copy of the deployment's state, make tokens.
|
||||
The service listens on plain HTTP on the internal network.
|
||||
The service listens on plain HTTP.
|
||||
|
||||
Decision: `/config`, `/users` and `/groups` need a JWT in an `Authorization: Bearer` header,
|
||||
HS256, verified with `hmac` and `hashlib`; no other algorithm is accepted. The key is
|
||||
@@ -143,8 +155,10 @@ Decision: `vpnrouter_users.py` holds the user operations, the checks and a sessi
|
||||
lock and the connection of the user manager account. The web service imports it and maps a
|
||||
`Failure` to the HTTP status; `vpn-router-user` imports it and adds the argument parsing and
|
||||
the output, text for people and JSON with `--json`. The web unit allows writing the paths the
|
||||
operations need: `/run/vpn-router` for the lock, `/etc/vpn-router/pki` for the certificates and
|
||||
the CA's state, and `/etc/swanctl/x509crl` for the published CRL.
|
||||
operations need: `/run/vpn-router` for the lock, its `RuntimeDirectory`, which systemd creates
|
||||
before the service starts, also before setup has run, and keeps when it stops, as setup's state
|
||||
files are there too; `/etc/vpn-router/pki` for the certificates and the CA's state, and
|
||||
`/etc/swanctl/x509crl` for the published CRL.
|
||||
|
||||
Consequences: The rules exist once. A request starts no process, and a password never leaves
|
||||
the process that received it. The web service imports `python3-ldap` and the package's
|
||||
@@ -156,22 +170,23 @@ and the holders of an admin token manage users.
|
||||
Status: Accepted
|
||||
|
||||
Context: The status page is the tool to see a problem of `vpn-router-setup`, so it must run
|
||||
whatever state setup is in, also before setup has run and while it fails. The internal address,
|
||||
`web.port` and `web.enabled` can change at any time.
|
||||
whatever state setup is in, also before setup has run and while it fails. The addresses,
|
||||
`web.listen`, `web.port` and `web.enabled` can change at any time.
|
||||
|
||||
Decision: The service depends on nothing to start and does not end on its own. A discovery
|
||||
thread works out, every 30 seconds, the internal address from `vpn-router.json` as
|
||||
`general.mode` directs, with the discovery code in `vpnrouter_net.py`, which `vpnrouter.py` uses
|
||||
too, and `web.port` and `web.enabled` from the same file. It runs the HTTP server in a thread of
|
||||
its own at that address and port, stops it and starts a new one when they change, and runs none
|
||||
while the service is disabled, the address is not known or it cannot be bound. Each request
|
||||
thread works out, every 30 seconds, the addresses of the interfaces `web.listen` names from
|
||||
`vpn-router.json` as `general.mode` directs, with the discovery code in `vpnrouter_net.py`,
|
||||
which `vpnrouter.py` uses too, and `web.port` and `web.enabled` from the same file. It runs an
|
||||
HTTP server in a thread of its own at each address and the port, stops one and starts a new one
|
||||
when they change, and runs none while the service is disabled or no address is known; an address
|
||||
that cannot be bound is tried again the next round. Each request
|
||||
reads the signing key and `vpn-router.json`, and the features and resolved facts from
|
||||
`web.json`, which setup writes when it applies the configuration, so the service shows what setup
|
||||
applied; while that file is missing the router shows no features. The unit is not
|
||||
ordered after `vpn-router-setup`, and setup does not restart it.
|
||||
|
||||
Consequences: The status page answers as soon as the internal address exists, whatever state
|
||||
setup is in, and follows a changed address, port or `web.enabled` within 30 seconds without a
|
||||
Consequences: The status page answers as soon as an address exists, whatever state setup is in,
|
||||
and follows a changed address, `web.listen`, port or `web.enabled` within 30 seconds without a
|
||||
restart. A new signing key applies to the next request. The interfaces are read every 30
|
||||
seconds, which costs a few `ip` commands.
|
||||
|
||||
@@ -184,10 +199,15 @@ that a secret is set tells a reader more than it needs.
|
||||
|
||||
Decision: The configuration that `vpn-router-setup` writes for the web service has no secret
|
||||
options at all. They are recognised by name: `psk`, `database`, whose connection string may hold
|
||||
a password, and options with `secret`, `password`, `token` or a `_key` ending in the name.
|
||||
a password, and options with `secret`, `password`, `token` or a `_key` ending in the name. The
|
||||
names are checked at every level, so the settings of each site-to-site connection lose their
|
||||
`psk` too; an object is kept whatever its name, so a connection named like a secret is still
|
||||
shown.
|
||||
|
||||
Consequences: No secret reaches the web service's state file or any answer of the API. A client
|
||||
cannot tell whether a secret is set. A new secret option needs a name that matches.
|
||||
Consequences: No secret reaches the web service's state file or any answer of the API. Through
|
||||
`/config` a client cannot tell whether a secret is set; `/site-to-site` says whether each
|
||||
connection has a key (ADR-25), as an admin managing connections needs to know. A new secret
|
||||
option needs a name that matches.
|
||||
|
||||
## ADR-9: The Router Manager is a separate package with a small Python server
|
||||
|
||||
@@ -207,9 +227,9 @@ router's `/auth`, keeps their tokens in a server-side session in PostgreSQL behi
|
||||
session's access token, refreshing it as needed; it holds no logic of the router's own and no
|
||||
key of it. It reads its settings from `/etc/vpn-router-manager/manager.json` at each request; on
|
||||
the router the settings not given come from `vpn-router.json`, and it listens on the internal
|
||||
address like the status web service, with the same discovery thread. `vpn-router-setup` opens
|
||||
the manager's port on the internal interface while the manager is installed, so the firewall
|
||||
stays in one place.
|
||||
address, the external address or both, as its own `listen` sets, like the status web service,
|
||||
with the same discovery thread. `vpn-router-setup` opens the manager's port on the internal and
|
||||
the external interface while the manager is installed, so the firewall stays in one place.
|
||||
|
||||
Consequences: The router needs no Node.js; the package needs only `python3` and packages of the
|
||||
Ubuntu archive. Tokens never reach the browser, and sessions survive a restart of the manager.
|
||||
@@ -318,8 +338,8 @@ within 15 minutes; a successful bind clears the count; `vpn-router-user unlock`
|
||||
`/auth` answers the same 401 for a wrong name, password or code and for a locked or disabled
|
||||
user. `/auth` and `/auth/refresh` allow 10 calls a minute per client address, counted in memory,
|
||||
and answer 429 with `Retry-After` beyond that. The client address is the peer, or the last
|
||||
`X-Forwarded-For` entry for a peer in `web.trusted_proxies`, by default the address the service
|
||||
listens on, where a manager on the router calls from.
|
||||
`X-Forwarded-For` entry for a peer in `web.trusted_proxies`, by default the address the call came
|
||||
in on, which is the address a manager on the router calls from.
|
||||
|
||||
Consequences: One lockout covers OpenVPN and `/auth`, kept by slapd alone. Someone who knows a
|
||||
user name can lock the user out for 15 minutes at a time. The rate limit restarts with the
|
||||
@@ -359,16 +379,22 @@ missing. PostgreSQL may also start after the services at boot.
|
||||
Decision: The web service makes its signing key and creates the refresh tokens' database itself,
|
||||
and the Router Manager its sessions database; neither `vpn-router-setup` nor a maintainer script
|
||||
creates them. Each service does it before it listens, retrying every 15 seconds and logging each
|
||||
failure, then listens. The first step is the PostgreSQL driver: psycopg finds libpq through the
|
||||
linker cache, which the package manager refreshes only at the end of an installation, so a
|
||||
service started during the installation cannot import it yet. The Manager imports psycopg only in
|
||||
its retry loop. On the router, `vpnrouter_auth` alone imports it, in `driver()`, which the web
|
||||
service's retry loop calls first, and turns a failed import and every database failure into
|
||||
failure, then listens. `vpnrouter_auth` turns every failure of the database into
|
||||
`DatabaseError`, the only database exception the web service and `vpn-router-user` handle.
|
||||
|
||||
psycopg (Ubuntu ships only its pure-Python implementation) finds libpq through the dynamic linker
|
||||
cache, which dpkg refreshes with its deferred `ldconfig` only at the end of an installation, so a
|
||||
service started during the installation cannot import it yet; a failed import cannot be repeated
|
||||
in the same process either, as psycopg caches the lookup. The services therefore import psycopg
|
||||
plainly, and their units wait instead: `ExecStartPre=wait-for-libpq`, a shell script of each
|
||||
package, waits until `ldconfig -p` lists `libpq.so.5`, within `TimeoutStartSec=5min`. The
|
||||
maintainer scripts start the units with `systemctl --no-block` (debhelper's start is turned off),
|
||||
because a start that waited would hold dpkg before its `ldconfig`.
|
||||
|
||||
Consequences: The order of installation and of the start of PostgreSQL and of the linker cache
|
||||
does not matter; a driver that cannot be loaded or a database that cannot be created is retried
|
||||
until the cause is gone, and the reason is in the service's log, without restarts. The web unit may write `/etc/vpn-router` for the key and the database password.
|
||||
does not matter; a database that cannot be created is retried until the cause is gone, and the
|
||||
reason is in the service's log. Outside an installation the wait ends at once. The web unit may
|
||||
write `/etc/vpn-router` for the key and the database password.
|
||||
|
||||
## ADR-17: Bootstrap diagnostics run only until the first setup succeeds
|
||||
|
||||
@@ -395,3 +421,261 @@ a `SystemExit` carrying its message, so setup records it.
|
||||
Consequences: The diagnostics need nothing the setup provides and cannot collide with the real
|
||||
services. After the first success they never run again, also not after an upgrade or a reboot.
|
||||
The page shows log lines to whoever reaches the internal network during the first setup.
|
||||
|
||||
## ADR-18: Manager-only accounts are a second kind of user; the package can create the first admin
|
||||
|
||||
Status: Accepted
|
||||
|
||||
Context: The Router Manager needs a first administrator before anyone has added users, as other
|
||||
packages create a default user from their debconf answers. Such an account, and others that only
|
||||
manage the router, have no person behind them and need no VPN access. A VPN user is an
|
||||
`inetOrgPerson`, which requires a last name and a common name, and its certificate needs the
|
||||
names. The Manager is configured before the router's directory exists during an installation.
|
||||
|
||||
Decision: A user is one of two kinds in `ou=users`, fixed when it is added: a VPN user, an
|
||||
`inetOrgPerson` with the three person attributes and a certificate, or a Manager-only account,
|
||||
an `account`, `simpleSecurityObject` and `vpnRouterAccount` (ADR-19) with the name, the password,
|
||||
an optional e-mail address for notifications, and no certificate. `add` with all three person
|
||||
attributes adds a VPN user and without the first and the last name a Manager-only account; one
|
||||
name without the rest is refused. The CLI and the API add both kinds; the Manager's dialog
|
||||
requires the names unless Manager access only is chosen, which offers the e-mail address only.
|
||||
Answers carry `manager_only`, and `update` changes only the e-mail address of a Manager-only
|
||||
account. The Manager's package asks `admin_username` and `admin_password`; with both
|
||||
it writes them to a root-only file, clears the answers, and `vpn-router-setup` creates the account
|
||||
after it provisions the directory, adds it to `admins` (creating the group when missing) and
|
||||
removes the file. Without a name nothing is created. `vpnrouter_users` knows only the directory,
|
||||
so setup can use it before PostgreSQL's driver can be loaded; `web` and `vpn-router-user` revoke
|
||||
refresh tokens after the operations of `vpnrouter_auth.REVOKING`.
|
||||
|
||||
Consequences: A Manager-only account cannot become a VPN user, as LDAP cannot change an entry's
|
||||
structural class and its password hash cannot be copied; it is removed and added again instead.
|
||||
It has no certificate, so OpenVPN and road-warrior access refuse it. The administrator's password
|
||||
is on disk only until setup has run.
|
||||
|
||||
## ADR-19: The package has an LDAP schema of its own under a UUID OID
|
||||
|
||||
Status: Accepted
|
||||
|
||||
Context: A Manager-only account (ADR-18) is an `account`, which allows no `mail`, yet notifications
|
||||
need an address for it; no standard auxiliary class allows `mail` except `extensibleObject`, which
|
||||
allows every attribute. Authorization scopes for users of both kinds may follow. An attribute or
|
||||
class of its own needs an OID, and the package has no registered arc.
|
||||
|
||||
Decision: The package defines its schema, `cn=vpn-router,cn=schema,cn=config`, which
|
||||
`vpnrouter_ldap.provision` adds when missing and keeps as `SCHEMA_ATTRIBUTE_TYPES` and
|
||||
`SCHEMA_OBJECT_CLASSES`. Its OID is `2.25.98477352595735834532404256531620013001`, made from a
|
||||
random UUID under the arc 2.25 (ITU-T X.667), which needs no registration; attribute types are
|
||||
under `.1` and object classes under `.2`. It has the attribute `vpnRouterScope` (multi-valued,
|
||||
case-insensitive strings) and the auxiliary class `vpnRouterAccount`,
|
||||
`MAY ( mail $ vpnRouterScope $ displayName $ description )`; `displayName` and `description`, the
|
||||
standard attributes, are the display name and the description of a service account, which
|
||||
`account` cannot hold. Manager-only accounts have the class from their creation, so an e-mail
|
||||
address can be added later without changing their classes; VPN users get it once scopes are used.
|
||||
`vpnrouter_ldap.provision` replaces the definitions in `cn=config` that differ, so an extended class
|
||||
reaches an existing directory at the next setup run.
|
||||
|
||||
Consequences: The schema check stays strict. New attributes extend `vpnRouterAccount`, and their
|
||||
OIDs continue the arcs. The scope `service` marks a service account (ADR-23). The OID must never
|
||||
be reused for anything else.
|
||||
|
||||
## ADR-20: Users manage their own account through /account, proving who they are again
|
||||
|
||||
Status: Accepted
|
||||
|
||||
Context: Users who are not members of `admins` can sign in but get 403 on every path, so they
|
||||
cannot change their password or enrol an authenticator without an administrator. A session
|
||||
token alone is a weak proof for such changes: whoever holds an open session could lock the user
|
||||
out. A wrong current password must not look like an ended session to the Manager, which signs a
|
||||
user out on 401.
|
||||
|
||||
Decision: `/account` serves the user named in the token's `sub`, admin or not: `GET` the user
|
||||
and whether they are a member of `admins`, `PUT /account/password`, and `POST` and
|
||||
`PUT /account/otp` to enrol or replace the TOTP enrolment. The routes do not take a user name.
|
||||
Each change sends the current password and, for an enrolled user, a current code, checked with a
|
||||
bind as the user like `/auth`, so it counts toward the lockout (ADR-14); a wrong one is 403. A
|
||||
new password is checked before the bind, so an invalid one does not use up a code. A user cannot
|
||||
remove their own enrolment; `/users` and `vpn-router-user` do. Tokens of `vpn-router-token` name
|
||||
no user and get 403 there.
|
||||
|
||||
Consequences: slapd's `otp` overlay accepts each code once, so a change right after signing in
|
||||
needs the next code. Changing the password revokes the user's refresh tokens, so the user's
|
||||
Manager session ends with its access token. An enrolled user who has lost the authenticator
|
||||
needs an administrator.
|
||||
|
||||
## ADR-21: Caddy from the Ubuntu archive serves HTTPS with a certificate from Let's Encrypt
|
||||
|
||||
Status: Accepted
|
||||
|
||||
Context: The status web service and the Router Manager are reachable from the internet (ADR-2),
|
||||
and `/auth` and the Manager's sign-in carry passwords. Both are small servers on the standard
|
||||
library's `http.server`, which would need certificates, their renewal and a TLS setup of their
|
||||
own. Browsers trust a public CA, not the router's.
|
||||
|
||||
Decision: Caddy, the `caddy` package of the Ubuntu archive, is the HTTPS proxy. While
|
||||
`https.enabled` is `true` (the default) and `wan.local_fqdn` is set, `vpn-router-setup` writes
|
||||
`/etc/caddy/vpn-router.Caddyfile`: a site for `wan.local_fqdn` on `https.web_port` (7444) for the
|
||||
web service and, while the Router Manager is installed, one on `https.manager_port` (7443) for
|
||||
the manager, each passing calls to the service on `127.0.0.1`. Caddy obtains and renews the
|
||||
certificate from Let's Encrypt with the HTTP challenge on port 80, which the firewall opens on
|
||||
the external interface; `auto_https disable_redirects` keeps port 80 to the challenge. The
|
||||
package's drop-in for `caddy.service` points it at that file and keeps it from starting before
|
||||
the file exists, so `/etc/caddy/Caddyfile`, a configuration file of the `caddy` package, is left
|
||||
unchanged. Both services always listen on `127.0.0.1` as well as on the interfaces their
|
||||
`listen` settings name, and trust `X-Forwarded-For` from it, which Caddy sets from the peer and
|
||||
never takes from the client.
|
||||
|
||||
Consequences: HTTPS needs `wan.local_fqdn` in public DNS pointing at the router and port 80
|
||||
reachable from the internet; until then Caddy retries and the HTTPS ports do not answer. The
|
||||
certificate and the ACME account are in `/var/lib/caddy`. Plain HTTP stays as the `listen`
|
||||
settings set it, so `internal` keeps passwords off the internet. Caddy's admin endpoint listens
|
||||
on `localhost:2019`, which `caddy reload` uses. Port 80 and the HTTPS ports are not available
|
||||
to Docker containers while HTTPS is on. The Manager's HTTPS port is a setting of the router, not
|
||||
of the manager.
|
||||
|
||||
## ADR-22: Metrics are kept in PostgreSQL and served in the Prometheus format, pushed with OTLP
|
||||
|
||||
Status: Accepted
|
||||
|
||||
Context: The router should keep its CPU, memory, disk, network, load and VPN client metrics for
|
||||
hour, day, month and year views, as Proxmox VE does, and expose them to OpenTelemetry. Ubuntu
|
||||
noble has no OpenTelemetry Python SDK and no OpenTelemetry Collector package. OTLP itself is
|
||||
push-only; a collector reads a scraped endpoint in the Prometheus format with its `prometheus`
|
||||
receiver. The router already runs PostgreSQL for the refresh tokens (ADR-15).
|
||||
|
||||
Decision: A service of its own, `vpn-router-metrics`, takes a sample every minute with `psutil`
|
||||
(`python3-psutil`) and the VPN readers it shares with the web service (`vpnrouter_status`). It
|
||||
keeps the samples in the router's database, in four detail tables (`metrics_system`,
|
||||
`metrics_network`, `metrics_vpn`, `metrics_site_to_site`, see ADR-25). Each detail table has hourly, daily, monthly and yearly summary
|
||||
tables, with the average and the maximum of each value per period, written when the clock passes a
|
||||
period from the level below with `INSERT ... ON CONFLICT DO NOTHING`, so a stopped service catches
|
||||
up. Garbage collection runs on start and once a day, not with every write, and deletes the rows
|
||||
older than the retention of their level, `metrics.keep_*`, PostgreSQL intervals. The service serves
|
||||
the last sample as `/metrics` in the Prometheus text format with `prometheus_client`
|
||||
(`python3-prometheus-client`) and pushes it as OTLP/HTTP JSON, built with the standard library, to
|
||||
`metrics.otlp_endpoint`. The names follow the OpenTelemetry semantic conventions. The web service
|
||||
answers the views from the tables at `/metrics/history`, protected like `/config`, for the Router
|
||||
Manager.
|
||||
|
||||
Consequences: The metrics live in the database of the refresh tokens and are dropped with it on
|
||||
purge. A view shows only what its level keeps, so a shorter retention empties the older part of a
|
||||
view. A month view needs the hourly rows and a year view the monthly rows. `/metrics` has only the
|
||||
last sample; history is not scraped.
|
||||
|
||||
## ADR-23: /metrics takes HTTP Basic authentication of an internal service account the router manages
|
||||
|
||||
Status: Accepted
|
||||
|
||||
Context: An OpenTelemetry Collector's `prometheus` receiver authenticates with HTTP Basic, not with
|
||||
the router's tokens, which last minutes (ADR-3). The scraper should not be able to sign in to the
|
||||
API or the Manager, and its password should be checked and locked out like any other. The account
|
||||
belongs to the package's metrics, not to the people using the router.
|
||||
|
||||
Decision: A service account is a Manager-only account (ADR-18) with `vpnRouterScope` `service`,
|
||||
which another system signs in with to use a router service. Administrators add service accounts
|
||||
with `vpn-router-user add --service` and the web service's `/service-accounts`.
|
||||
`/service-accounts` is a path of its own, apart from `/users`, which does not list or serve them:
|
||||
a service account has no names, certificate, profile or TOTP, and is not a person an
|
||||
administrator manages with the users. A service account has a display name and a description
|
||||
besides its name, so an administrator can tell what each is for. Internal service accounts are
|
||||
those the router creates and manages for its own services (`vpnrouter_users.internal_accounts`),
|
||||
with every property, display name and description included, so the system, not an
|
||||
administrator, owns them; at present that is the metrics service's, which setup names Metrics. `/service-accounts` marks
|
||||
them as `internal` and only lists them: every other request on one, and putting one in or taking
|
||||
one out of a group at `/groups`, gets 403, whatever the token. The router changes them through the
|
||||
directory's user-manager account, not the API, and root may still with `vpn-router-user`. For the
|
||||
metrics service, `vpn-router-setup` adds the account `metrics.service_account` names, `metrics` by
|
||||
default, while metrics are enabled, with a random password it writes to
|
||||
`/etc/vpn-router/metrics.secret` (root only), and sets a new random password, ending a lockout,
|
||||
when the file is missing or no longer holds the password. `/auth` refuses every service account with the message of a wrong password. `/metrics` accepts HTTP Basic only for that account, and
|
||||
only when it is a service account; the password is checked with a bind as the account, so the
|
||||
password policy's lockout applies (ADR-14). TLS is Caddy's, on `metrics.https_port` (ADR-21);
|
||||
plain HTTP is served on the interfaces `metrics.listen` names, `internal` by default.
|
||||
|
||||
Consequences: Every scrape binds to slapd once. The collector's operator copies the password from
|
||||
the router; deleting the file and running setup changes it. A user who already has the name is not
|
||||
taken over; setup fails until `metrics.service_account` names another. An account root disabled
|
||||
stays disabled. A scraper on the external interface over plain HTTP sends the password in the clear, so
|
||||
`metrics.listen` stays `internal` unless HTTPS is used.
|
||||
|
||||
## ADR-24: The Router Manager upgrades the router's packages in a transient unit outside its own
|
||||
|
||||
Status: Accepted
|
||||
|
||||
Context: An admin should upgrade `vpn-router` and `vpn-router-manager` from the Router Manager. The
|
||||
upgrade restarts the process that starts it: the manager's postinst restarts
|
||||
`vpn-router-manager.service`, and the router's restarts `vpn-router-web` and `vpn-router-setup`. A
|
||||
child of the manager would be stopped with the manager's cgroup in the middle of dpkg, leaving
|
||||
packages half configured, and the manager's unit cannot run apt at all, since `ProtectSystem=strict`
|
||||
makes `/usr` and `/var/lib/dpkg` read-only. The package lists are updated by a cron job, not by the
|
||||
manager.
|
||||
|
||||
Decision: The manager starts the helper `/usr/lib/vpn-router-manager/upgrade` with `systemd-run` as
|
||||
the transient unit `vpn-router-manager-upgrade`, of its own cgroup and without a sandbox, and does
|
||||
not wait for it. The helper:
|
||||
|
||||
- finishes an interrupted dpkg run with `dpkg --configure -a`;
|
||||
- runs `apt-get install --only-upgrade` of the two packages, non-interactive, keeping changed configuration files, with `needrestart` suspended, and waiting for the dpkg lock that the cron job may hold;
|
||||
- records the state and the result in `/var/lib/vpn-router-manager/upgrade.json`, replaced by a rename, and the output in `upgrade.log`.
|
||||
|
||||
A restarted manager reads them again. A state of running without an active unit is shown as interrupted. The manager reboots the router the same way, as a transient timer `vpn-router-manager-reboot` 5 seconds later, so the answer reaches the browser first. Only members of `admins`, as the router's `/account` answers, may start either.
|
||||
|
||||
Consequences: The fixed unit name allows one upgrade at a time. The page loses the manager and the
|
||||
router's web service for a while during an upgrade and reads again until they answer. Sessions
|
||||
survive in PostgreSQL. The manager can upgrade only the system it runs on, so a manager on another
|
||||
host upgrades that host's packages, not the router's. The operating system's other packages are
|
||||
left to unattended-upgrades.
|
||||
|
||||
## ADR-25: Site-to-site connections are a map of named connections, edited through the API and applied by setup
|
||||
|
||||
Status: Accepted
|
||||
|
||||
Context: The router had one IPsec site-to-site tunnel, the `remote` section, run as the swanctl
|
||||
connection `remote-site`, with the local identity mode in `wan` and the local subnets in `local`.
|
||||
A router often links more than one site, each with its own peer, subnets and key, and an admin
|
||||
should add them without a shell. The install answers and the Cloud Platform describe one
|
||||
connection, and the platform rewrites `vpn-router.json` when it rebuilds the VM.
|
||||
|
||||
Decision: `site_to_site` maps a connection name to its settings: `addrs`, `id`, `cidrs`,
|
||||
`local_cidrs`, `local_id_mode`, `psk` and `psk_file`. `remote` and `wan.local_id_mode` are gone,
|
||||
without a fallback for the old format. The name, a lowercase letter followed by at most 30
|
||||
lowercase letters, digits and hyphens, names the swanctl connection, its child and its secret
|
||||
`ike-psk-<name>`, so it is a valid swanctl section name and stays the same while the connection
|
||||
lives. The install answers and the Cloud Platform write the connection `remote-site`, the first
|
||||
one, so its name is the one the tunnel always had.
|
||||
|
||||
- **Local settings per connection:** each connection has its own `local_cidrs` and
|
||||
`local_id_mode`, as each peer expects its own traffic selectors and identity. `wan.local_fqdn`
|
||||
stays the router's one name, used by the `fqdn` mode, resolved once for the `public_ip` mode,
|
||||
and used by HTTPS, road-warrior and OpenVPN. `local.cidrs` stays the subnets the router
|
||||
routes, masquerades and offers to road-warrior and OpenVPN clients.
|
||||
- **One file:** `vpn-router-setup` writes every connection that is ready to
|
||||
`/etc/swanctl/conf.d/site-to-site.conf`, with the proposals and timers the one tunnel had, and
|
||||
removes the file when none is. The NAT exemptions pair each connection's `local_cidrs` with its
|
||||
`cidrs`.
|
||||
- **Editing:** the status web service serves `/site-to-site` to members of `admins` and admin
|
||||
tokens, like `/users`. It never returns a key, only whether one is set, and takes a new key only
|
||||
when one is given. `vpn-config-edit` edits the same map. Both read, check with the validation
|
||||
of `vpn-router-setup` and write `vpn-router.json` holding one lock, replacing the file in one
|
||||
step, so neither loses the other's change.
|
||||
- **Applying:** the web service restarts `vpn-router-setup` without waiting for it, the one
|
||||
place that renders, loads and opens the firewall, so a change through the API takes the same
|
||||
path as a change by hand.
|
||||
- **Metrics per connection:** the metrics service keeps, per connection, whether it is up, the
|
||||
bytes and packets per second through its child SAs and the seconds its IKE SA has been up, in
|
||||
a detail table of its own, `metrics_site_to_site`, keyed by the connection's name, with the
|
||||
summaries and retention of the other tables (ADR-22). One row per connection keeps the values
|
||||
of a tunnel together and lets the set of connections change without changing the schema;
|
||||
`metrics_vpn` counts only clients. swanctl's child SA counters start again at a rekey, so the
|
||||
service keeps rates computed between two samples, treating a lower counter as counted from
|
||||
zero, and `/metrics` and the OTLP push serve them as gauges rather than counters that would
|
||||
appear to reset.
|
||||
|
||||
Consequences: A configuration with `remote` is refused, so the package and a configuration in
|
||||
the new format go together. The status shows each connection. A rate across a rekey misses the
|
||||
traffic of the old child SA since the previous sample. A renamed connection starts a new series.
|
||||
A connection added through the API lives in
|
||||
`vpn-router.json` only; the Cloud Platform's rebuild of the VM writes the file again with its
|
||||
one connection, dropping the others. A change through the API is answered before setup has
|
||||
applied it, so a configuration that setup cannot apply shows in the setup state, not in the
|
||||
answer. A connection with a key file keeps it until a key is given through the API, which then
|
||||
replaces the file.
|
||||
@@ -9,6 +9,8 @@ A Debian package that configures a Linux host or virtual machine as a VPN router
|
||||
- optional WireGuard endpoint
|
||||
- routing and NAT for a protected subnet, TCP MSS clamping, and UFW firewall rules
|
||||
- firewall rules for the ports Docker publishes, when Docker is installed
|
||||
- metrics of the router kept in PostgreSQL, served for an OpenTelemetry Collector and pushed
|
||||
over OTLP
|
||||
|
||||
Nothing in the package is specific to a cloud provider. It is built on the external and
|
||||
internal NIC names, and a `mode` setting says how much of the rest you are supplying and
|
||||
@@ -101,16 +103,11 @@ must have the type shown below; an unknown section or setting is an error.
|
||||
| `general` | `mode` | string | `manual` | How much you supply: `manual`, `interfaces` or `auto` |
|
||||
| `interfaces` | `external` | string | | Name of the NIC facing the untrusted network |
|
||||
| `interfaces` | `internal` | string | | Name of the NIC facing the protected network |
|
||||
| `wan` | `local_fqdn` | string | | This router's name or IP address, used as its IKE identity in `fqdn` mode |
|
||||
| `wan` | `local_id_mode` | string | `fqdn` | IKE identity source: `fqdn`, `public_ip` or `internal_ip` |
|
||||
| `local` | `cidrs` | list | | Local subnets advertised into the tunnel |
|
||||
| `wan` | `local_fqdn` | string | | This router's name or IP address: its IKE identity in `fqdn` mode, the name resolved in `public_ip` mode, and the name of HTTPS, road-warrior and OpenVPN |
|
||||
| `local` | `cidrs` | list | | Local subnets routed via the internal side, masqueraded to the external one, and offered to road-warrior and OpenVPN clients |
|
||||
| `local` | `int_addr` | string | | This host's address on the internal network |
|
||||
| `local` | `int_gateway_ip` | string | | Next hop on the internal side |
|
||||
| `remote` | `addrs` | list | | Remote gateway addresses or FQDNs |
|
||||
| `remote` | `id` | string | | Remote IKE identity, an FQDN or an IPv4 or IPv6 address |
|
||||
| `remote` | `cidrs` | list | | Remote subnets reachable through the tunnel |
|
||||
| `remote` | `psk` | string | | Pre-shared key |
|
||||
| `remote` | `psk_file` | string | | Path to a file holding the key; wins over `psk` |
|
||||
| `site_to_site` | | object | | The site-to-site connections, each a name and an object of the settings below |
|
||||
| `p2s` | `enabled` | boolean | `false` | Road-warrior access |
|
||||
| `p2s` | `address_pool` | string | | CIDR assigned to road-warrior clients |
|
||||
| `p2s` | `ca_name` | string | `VPN Router CA` | Common name for a locally created CA |
|
||||
@@ -123,10 +120,52 @@ must have the type shown below; an unknown section or setting is an error.
|
||||
| `openvpn` | `protocol` | string | `udp` | `udp` or `tcp` |
|
||||
| `docker` | `mode` | string | `auto` | `auto` secures Docker when its service is installed, `on` requires it, `off` removes the rules |
|
||||
| `docker` | `exposed_ports` | list | | TCP ports and port ranges (`8000:8100`) reachable from the external interface; at most 15, a range counting as two |
|
||||
| `web` | `enabled` | boolean | `true` | The status web service on the internal address |
|
||||
| `web` | `enabled` | boolean | `true` | The status web service |
|
||||
| `web` | `listen` | string | `both` | The interfaces the status web service listens on: `internal`, `external` or `both` |
|
||||
| `web` | `port` | integer | `7081` | TCP port of the status web service |
|
||||
| `web` | `database` | string | | A PostgreSQL connection string for the refresh tokens; without it, the `vpn-router` database on this host that the web service creates |
|
||||
| `web` | `trusted_proxies` | list | the listening address | Addresses whose `X-Forwarded-For` header names the client for the `/auth` rate limit |
|
||||
| `web` | `trusted_proxies` | list | the address the call came in on | Addresses whose `X-Forwarded-For` header names the client for the `/auth` rate limit and the Client IP of the status |
|
||||
| `web` | `status_content` | string | `basic` | What `/` and `/status` show: `basic`, `standard` or `full` (see Status web service) |
|
||||
| `https` | `enabled` | boolean | `true` | Caddy serves the status web service and the Router Manager over HTTPS for `wan.local_fqdn` |
|
||||
| `https` | `email` | string | | The e-mail address of the Let's Encrypt account |
|
||||
| `https` | `web_port` | integer | `7444` | HTTPS port of the status web service |
|
||||
| `https` | `manager_port` | integer | `7443` | HTTPS port of the Router Manager |
|
||||
| `metrics` | `enabled` | boolean | `true` | The metrics service: collection, history and `/metrics` |
|
||||
| `metrics` | `listen` | string | `internal` | The interfaces `/metrics` is served on: `internal`, `external` or `both` |
|
||||
| `metrics` | `port` | integer | `9464` | TCP port of `/metrics` |
|
||||
| `metrics` | `https_port` | integer | `9465` | HTTPS port of `/metrics`, served by Caddy while `https.enabled` is true |
|
||||
| `metrics` | `service_account` | string | `metrics` | The service account `/metrics` takes, which `vpn-router-setup` adds and keeps |
|
||||
| `metrics` | `otlp_endpoint` | string | | An OTLP/HTTP URL the metrics are pushed to every minute, for example `https://collector.example.com:4318/v1/metrics` |
|
||||
| `metrics` | `otlp_username` | string | | The user name of HTTP Basic authentication at the OTLP endpoint |
|
||||
| `metrics` | `otlp_password` | string | | Its password |
|
||||
| `metrics` | `keep_details` | string | `24 hours` | How long the per-minute metrics are kept |
|
||||
| `metrics` | `keep_hourly` | string | `1 month` | How long the hourly summaries are kept |
|
||||
| `metrics` | `keep_daily` | string | `3 months` | How long the daily summaries are kept |
|
||||
| `metrics` | `keep_monthly` | string | `1 year` | How long the monthly summaries are kept |
|
||||
| `metrics` | `keep_yearly` | string | `10 years` | How long the yearly summaries are kept |
|
||||
|
||||
### Site-to-site connections
|
||||
|
||||
`site_to_site` maps a connection name to its settings. A name starts with a lowercase letter
|
||||
and has at most 31 lowercase letters, digits and hyphens; it names the strongSwan connection,
|
||||
its child and its secret (`ike-psk-<name>`). The install answers fill the connection
|
||||
`remote-site`; more are added in the file, with `vpn-config-edit`, or through `/site-to-site`
|
||||
of the status web service, which the Router Manager uses.
|
||||
|
||||
| Setting | Type | Default | Meaning |
|
||||
|---|---|---|---|
|
||||
| `addrs` | list | | Remote gateway addresses or FQDNs |
|
||||
| `id` | string | | Remote IKE identity, an FQDN or an IPv4 or IPv6 address |
|
||||
| `cidrs` | list | | Remote subnets reachable through the tunnel |
|
||||
| `local_cidrs` | list | | Local subnets advertised into the tunnel |
|
||||
| `local_id_mode` | string | `fqdn` | The router's IKE identity: `fqdn` is `wan.local_fqdn`, `public_ip` the address it resolves to, `internal_ip` the internal address |
|
||||
| `psk` | string | | Pre-shared key |
|
||||
| `psk_file` | string | | Path to a file holding the key; wins over `psk` |
|
||||
|
||||
A connection is set up once all of `addrs`, `id`, `cidrs`, `local_cidrs`, the key and its
|
||||
local identity are known; all such connections are written to
|
||||
`/etc/swanctl/conf.d/site-to-site.conf`. Traffic from a connection's `local_cidrs` to its
|
||||
`cidrs` is not masqueraded.
|
||||
|
||||
A minimal site-to-site configuration in `auto` mode:
|
||||
|
||||
@@ -135,16 +174,19 @@ A minimal site-to-site configuration in `auto` mode:
|
||||
"general": { "platform": "generic", "mode": "auto" },
|
||||
"wan": { "local_fqdn": "router.example.com" },
|
||||
"local": { "cidrs": ["10.0.0.0/24"] },
|
||||
"remote": {
|
||||
"addrs": ["peer.example.net"],
|
||||
"id": "peer.example.net",
|
||||
"cidrs": ["192.168.0.0/24"],
|
||||
"psk": "s3cr3t"
|
||||
"site_to_site": {
|
||||
"remote-site": {
|
||||
"addrs": ["peer.example.net"],
|
||||
"id": "peer.example.net",
|
||||
"cidrs": ["192.168.0.0/24"],
|
||||
"local_cidrs": ["10.0.0.0/24"],
|
||||
"psk": "s3cr3t"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The router's IKE identity (`wan.local_fqdn` in `fqdn` mode) and the remote one (`remote.id`) are
|
||||
The router's IKE identity (`wan.local_fqdn` in `fqdn` mode) and the remote one (`id`) are
|
||||
an FQDN or an IPv4 or IPv6 address. They are written to the strongSwan configuration as given;
|
||||
strongSwan reads an address as an IP identity and a name as an FQDN, so a leading `@` is not needed
|
||||
and is refused. Road-warrior and OpenVPN access need a name in `wan.local_fqdn`.
|
||||
@@ -234,6 +276,29 @@ vpn-router-setup` applies a changed configuration while Docker runs. With `docke
|
||||
`on`, the service fails when Docker is not installed or its daemon is not running; with
|
||||
`off` it removes the rules.
|
||||
|
||||
## Editing the configuration
|
||||
|
||||
`vpn-config-edit`, run as root, shows and changes `/etc/vpn-router/vpn-router.json`:
|
||||
|
||||
```sh
|
||||
vpn-config-edit show
|
||||
vpn-config-edit set web.status_content standard
|
||||
vpn-config-edit set web.port 7082
|
||||
vpn-config-edit set web.trusted_proxies 127.0.0.1,10.0.0.4
|
||||
vpn-config-edit set site_to_site.branch.addrs branch.example.net
|
||||
vpn-config-edit remove site_to_site.branch
|
||||
```
|
||||
|
||||
`show` prints the file as indented JSON and does not change it. `set` takes a setting as
|
||||
`section.option`, or as `site_to_site.NAME.option` for a site-to-site connection, which it
|
||||
creates with the defaults when it does not exist, and converts the value to the setting's type:
|
||||
a whole number, `true` or `false`, a list separated by commas (an empty value is an empty list),
|
||||
or a string; an unknown setting is refused. `remove site_to_site.NAME` removes a connection. It
|
||||
checks the whole configuration as `vpn-router-setup` does, and only then writes the file,
|
||||
indented, with mode 0600, and reminds that `vpn-router-setup` applies it. The status web service
|
||||
writes the file the same way, and a lock keeps the two from overwriting each other's change. A
|
||||
refused request exits with status 2, any other failure with 1.
|
||||
|
||||
## Applying changes
|
||||
|
||||
```sh
|
||||
@@ -324,7 +389,14 @@ is switched on:
|
||||
- its own database, `dc=vpn-router,dc=local` in `/var/lib/ldap/vpn-router`, next to the one the
|
||||
slapd package creates; root manages it through the local socket, with no administrator
|
||||
password;
|
||||
- `ou=users` holds one `inetOrgPerson` per user, named by `uid`;
|
||||
- `ou=users` holds one entry per user, named by `uid`: an `inetOrgPerson` for a VPN user, an
|
||||
`account` for a Manager-only account (see Managing users);
|
||||
- the package's own schema, `cn=vpn-router,cn=schema,cn=config`, under the OID
|
||||
`2.25.98477352595735834532404256531620013001` of the UUID arc: the auxiliary class
|
||||
`vpnRouterAccount` with `MAY ( mail $ vpnRouterScope $ displayName $ description )`, which
|
||||
Manager-only and service accounts have, and the attribute `vpnRouterScope`, authorization
|
||||
scopes, of which `service` marks a service account; `displayName` and `description` are the
|
||||
display name and the description of a service account;
|
||||
- `ou=groups` holds one `groupOfNames` per group, named by `cn`, with its members' DNs in
|
||||
`member` and an optional `description`;
|
||||
- `cn=user-manager,ou=services` is the account that changes users and groups; it may write
|
||||
@@ -346,9 +418,27 @@ The directory is kept on purge.
|
||||
|
||||
### Managing users
|
||||
|
||||
`vpn-router-user` manages the users. A user is the directory entry, with a first name, a last
|
||||
name and an e-mail address (`givenName`, `sn` and `mail`; `cn` is the first and the last name),
|
||||
and a client certificate with the user name as its common name, in `/etc/vpn-router/pki/users`.
|
||||
`vpn-router-user` manages the users. A user is a directory entry of one of three kinds, fixed when
|
||||
it is added:
|
||||
|
||||
- a VPN user, an `inetOrgPerson` with a first name, a last name and an e-mail address
|
||||
(`givenName`, `sn` and `mail`; `cn` is the first and the last name), and a client certificate
|
||||
with the user name as its common name, in `/etc/vpn-router/pki/users`;
|
||||
- a Manager-only account, an `account`, `simpleSecurityObject` and `vpnRouterAccount` with the
|
||||
name, the password and an optional e-mail address, for notifications: no names and no
|
||||
certificate, so no VPN access. It can sign in to `/auth` and, as a member of `admins`, manage
|
||||
the router. `add` without the first and the last name adds one; its names cannot be set later;
|
||||
- a service account, a Manager-only account whose `vpnRouterScope` is `service`, which another
|
||||
system signs in with to use a router service, such as an OpenTelemetry Collector reading
|
||||
`/metrics`; it cannot sign in to `/auth`. Besides its name, it has a display name and a
|
||||
description. `add --service` adds one, and the web service manages them at `/service-accounts`,
|
||||
not at `/users`. Internal service accounts are those the router creates and manages for its own
|
||||
services, display name and description included; at present that is the metrics service's, the
|
||||
account `metrics.service_account` names while metrics are enabled (see Metrics), which
|
||||
`vpn-router-setup` names Metrics. The API only lists
|
||||
them: `/service-accounts` refuses every other request on one, and `/groups` refuses to put one
|
||||
in or take one out of a group, with 403.
|
||||
|
||||
Passwords and TOTP secrets are read from the first line of standard input. The answer is text: a
|
||||
table of the users, the client profile, a TOTP secret or a short message, and a failure is a
|
||||
message on standard error.
|
||||
@@ -358,6 +448,8 @@ after a command its options.
|
||||
|
||||
```sh
|
||||
echo 's3cr3t-pass' | vpn-router-user add alice --first-name Alice --last-name Smith --email alice@example.com
|
||||
echo 'm4nager-pass' | vpn-router-user add ops
|
||||
echo 's3rvice-pass' | vpn-router-user add backup --service
|
||||
vpn-router-user update alice --email a.smith@example.com
|
||||
vpn-router-user list
|
||||
vpn-router-user profile alice > alice.ovpn
|
||||
@@ -374,9 +466,12 @@ vpn-router-user otp-remove alice
|
||||
|
||||
| Command | Effect |
|
||||
| --- | --- |
|
||||
| `list` | The users with their names and e-mail addresses, whether each is enabled or locked, has a certificate and is enrolled for TOTP |
|
||||
| `add <name> --first-name F --last-name L --email E` | Adds the user and issues the certificate; all three are required |
|
||||
| `list` | The users with their kind, names and e-mail addresses, whether each is enabled or locked, has a certificate and is enrolled for TOTP |
|
||||
| `add <name> --first-name F --last-name L --email E` | Adds a VPN user and issues the certificate |
|
||||
| `add <name> [--email E]` | Adds a Manager-only account, with the e-mail address when given; one name without the other two options is refused |
|
||||
| `add <name> --service [--display-name N] [--description D] [--email E]` | Adds a service account, with its display name, description and e-mail address when given; the names are refused |
|
||||
| `update <name> [--first-name F] [--last-name L] [--email E]` | Changes the names or the e-mail address |
|
||||
| `update <name> [--display-name N] [--description D] [--email E]` | Changes the display name, the description or the e-mail address of a service account; `''` removes one |
|
||||
| `passwd <name>` | Sets the password |
|
||||
| `disable <name>`, `enable <name>` | Locks the user out of OpenVPN and `/auth`, and lets the user in again |
|
||||
| `unlock <name>` | Ends a lockout after failed logins and clears the failure count; a disabled user stays disabled |
|
||||
@@ -390,9 +485,17 @@ A user name is 1 to 32 lowercase letters, digits and hyphens, starting with a le
|
||||
password has 8 to 128 characters; a first or last name has 1 to 64 printable characters; an
|
||||
e-mail address has one `@`, no spaces and a dot in its domain; a TOTP secret is 32 base32
|
||||
characters, as `otp-new` makes it, and a code 6 digits. Once enrolled, a user logs in to OpenVPN
|
||||
and `/auth` with the password followed by a current code. In the answers a user added
|
||||
without the names or the e-mail address has them as `null`. The exit status is 0 on success, 2 for an invalid request,
|
||||
3 when the user does not exist, 4 when the user exists and 1 for any other failure.
|
||||
and `/auth` with the password followed by a current code. In the answers `manager_only`, `service`
|
||||
and `internal` tell the kind, and a Manager-only account has its names as `null`; `update` changes only its e-mail
|
||||
address, and `--email ''` removes it. Changing a user's password, disabling, unlocking or removing the user also revokes its
|
||||
refresh tokens of `/auth`. The exit status is 0 on success, 2 for an invalid request, 3 when the
|
||||
user does not exist, 4 when the user exists and 1 for any other failure.
|
||||
|
||||
When the Router Manager is installed with an administrator (its debconf questions
|
||||
`vpn-router-manager/admin_username` and `admin_password`), its package leaves the answers in
|
||||
`/etc/vpn-router-manager/admin.json` (root only). `vpn-router-setup` then creates that user as a
|
||||
Manager-only account, adds it to the group `admins`, which it creates when missing, and removes
|
||||
the file.
|
||||
|
||||
The certificate is also accepted by road-warrior (IKEv2 EAP-TLS) access, which checks
|
||||
certificates only, so `disable` does not affect it; `remove` revokes the certificate for both.
|
||||
@@ -469,9 +572,22 @@ modules shipped with the package; the configuration file accepts any module inst
|
||||
|
||||
## Status web service
|
||||
|
||||
The `vpn-router-web` service shows the state of the router on its internal address and
|
||||
`web.port` (7081 by default). Its pages `/` and `/status` need no authentication and show no
|
||||
configuration: no key, identity, address or subnet.
|
||||
The `vpn-router-web` service shows the state of the router on the addresses of the interfaces
|
||||
`web.listen` names, the internal, the external or both (the default), and on `127.0.0.1`, where
|
||||
Caddy calls it (see HTTPS), with `web.port` (7081 by default). Its pages `/` and `/status` need no authentication and show no
|
||||
configuration: no key, identity, subnet, or address but the client's. They show as much as
|
||||
`web.status_content` says, read at each request:
|
||||
|
||||
| `status_content` | `/` and `/status` show |
|
||||
|---|---|
|
||||
| `basic` (default) | The overall status (`ok` or `degraded`), the time and the Client IP |
|
||||
| `standard` | Also the `vpn-router` version, the `vpn-router-manager` version or that it is not installed, and the number of active services and of services in any other state, without their names |
|
||||
| `full` | Also each service and its state, Docker, and the state of each site-to-site connection, road-warrior, OpenVPN and WireGuard |
|
||||
|
||||
The Client IP is the address the request came from: the last entry of `X-Forwarded-For` for a
|
||||
request from one of `web.trusted_proxies`, by default the address the request came in on, so
|
||||
behind Caddy or a Router Manager on the router it is the browser's. `/status` names the content in
|
||||
`content` and the Client IP in `client_ip`; `/` keeps the time and the link to `/status`.
|
||||
|
||||
The service first makes its signing key and the refresh tokens' database, retrying every 15
|
||||
seconds, for example while PostgreSQL is not running yet, and logs each failure; then it listens.
|
||||
@@ -501,15 +617,21 @@ curl http://10.0.0.4:17081/
|
||||
| `/auth` | Signing in with a user's password and TOTP code, refreshing and signing out |
|
||||
| `/config` | The configuration without its secret options, and what the router resolved and runs, as JSON; needs a token |
|
||||
| `/users` | The users of OpenVPN and road-warrior access; needs a token |
|
||||
| `/service-accounts` | The service accounts; needs a token |
|
||||
| `/groups` | The groups of users; needs a token |
|
||||
| `/site-to-site` | The site-to-site connections: list, add, change and remove; needs a token |
|
||||
| `/account` | The account of the signed-in user: its details, its password and its TOTP enrolment; needs a user's token |
|
||||
|
||||
The status holds the state of `vpn-router-setup` and `slapd`, of `strongswan` when site-to-site or
|
||||
The status holds the state of `vpn-router-setup`, `slapd` and each PostgreSQL cluster unit
|
||||
(`postgresql@<version>-<cluster>`, or `postgresql` as `no cluster` without one), of
|
||||
`vpn-router-manager` when that package is installed, of `strongswan` when site-to-site or
|
||||
road-warrior access is configured, of `wg-quick@wg0` when WireGuard is, and of
|
||||
`openvpn-server@vpn-router` when OpenVPN is; the number of connected
|
||||
OpenVPN clients, from the server's status file, which OpenVPN rewrites every minute; the package version; whether
|
||||
OpenVPN clients, from the server's status file, which OpenVPN rewrites every minute; the package version and,
|
||||
as `manager_version`, the installed version of `vpn-router-manager` (`null`, shown as not installed, without it); whether
|
||||
Docker is installed, and when it is, the state of `docker.service` and of `vpn-router-docker`
|
||||
(the Docker rules), which do not change `status`; whether the
|
||||
site-to-site tunnel is established; the number of connected road-warrior (IPsec P2S) clients; and
|
||||
(the Docker rules), which do not change `status`; as `site_to_site.connections`, whether each
|
||||
site-to-site connection that is set up is established (`null` when swanctl cannot be asked); the number of connected road-warrior (IPsec P2S) clients; and
|
||||
the WireGuard peers with the age of their last handshake and their traffic. `status` is `ok` when
|
||||
all these services are active, and `degraded` otherwise.
|
||||
|
||||
@@ -520,16 +642,18 @@ curl http://10.0.0.4:7081/status
|
||||
|
||||
### Tokens and signing in
|
||||
|
||||
`/config`, `/users` and `/groups` need a JWT in an `Authorization: Bearer` header, signed with
|
||||
`/config`, `/users`, `/service-accounts`, `/groups`, `/site-to-site` and `/account` need a JWT in an `Authorization: Bearer` header, signed with
|
||||
HS256 by the key the web service makes on its first run, `/etc/vpn-router/web.key` (root only).
|
||||
The key never leaves the router; while it does not exist these paths and `/auth` are not served
|
||||
(404). A missing, expired or wrongly signed token, or one of another algorithm, gets 401 with
|
||||
`WWW-Authenticate: Bearer`. There are two kinds of tokens:
|
||||
|
||||
- an access token of `/auth` names a user in `sub` and is valid for 5 minutes; the user must
|
||||
be a member of the group `admins`, otherwise the request gets 403;
|
||||
- an access token of `/auth` names a user in `sub` and is valid for 5 minutes; every user may
|
||||
call `/account`, and for the other paths the user must be a member of the group `admins`,
|
||||
otherwise the request gets 403;
|
||||
- a token of `vpn-router-token`, run as root on the router, has the claim `admin` and is valid
|
||||
for the lifetime given, without a maximum. It may call everything, and only such a token may
|
||||
for the lifetime given, without a maximum. It names no user, so `/account` refuses it with 403;
|
||||
it may call everything else, and only such a token may
|
||||
change the group `admins`: create it, rename a group to it, change or remove it, add or take
|
||||
out its members, or remove a user who is a member of it.
|
||||
|
||||
@@ -556,7 +680,7 @@ removed. Changing a user's password, disabling, unlocking or removing the user r
|
||||
user's refresh tokens. Five failed logins lock the account for 15 minutes (see the user
|
||||
directory). `/auth` and `/auth/refresh` take at most 10 calls a minute from one client address,
|
||||
and answer 429 with `Retry-After` beyond that; for a call from one of `web.trusted_proxies`, by
|
||||
default the address the service listens on, the client address is the last entry of
|
||||
default the address the call came in on, the client address is the last entry of
|
||||
`X-Forwarded-For`.
|
||||
|
||||
Refresh tokens are kept, as their SHA-256, in PostgreSQL: by default in the database and role
|
||||
@@ -576,11 +700,21 @@ curl -H "Authorization: Bearer $TOKEN" http://10.0.0.4:7081/config
|
||||
|
||||
| Key | Content |
|
||||
| --- | --- |
|
||||
| `configuration` | `vpn-router.json` with the defaults applied, without its secret options (`psk`, `database`, and any option named like a secret, password, token or key) |
|
||||
| `network` | The external and internal interface, their addresses, the internal gateway, the local and remote IKE identities and subnets, and the IPv4 routes as `ip -j -4 route` prints them |
|
||||
| `configuration` | `vpn-router.json` with the defaults applied, without its secret options at any level (`psk`, `database`, and any option named like a secret, password, token or key) |
|
||||
| `network` | The external and internal interface, their addresses, the internal gateway, `local.cidrs`, `site_to_site`: each connection with `name`, `local_identity`, `remote_identity`, `local_cidrs` and `remote_cidrs`, and the IPv4 routes as `ip -j -4 route` prints them |
|
||||
| `ipsec` | `connections` and `sas`: one object per connection and per IKE SA, read from `swanctl --list-conns --pretty` and `swanctl --list-sas --pretty`, with the connection's name under `name` and swanctl's keys and string values |
|
||||
| `wireguard` | The public key and port of `wg0`, and each peer's public key, endpoint, allowed IPs, handshake age and traffic; no private or preshared key |
|
||||
| `docker` | `docker.exposed_ports` and the rules of the `VPN-ROUTER-DOCKER` chain |
|
||||
| `postgresql` | `clusters`: each PostgreSQL cluster of `pg_lsclusters` with its version, name, port, status and data directory, and for one that is online its `settings`: `server_version`, `listen_addresses`, `port`, `max_connections`, `shared_buffers`, `effective_cache_size`, `work_mem`, `password_encryption`, `ssl` and `timezone`; `null` when `pg_lsclusters` fails |
|
||||
|
||||
### The /postgresql page
|
||||
|
||||
`/postgresql`, with the same token as `/config`, shows the databases of the PostgreSQL clusters on the
|
||||
router:
|
||||
|
||||
- **Content:** `clusters` lists each cluster with its version, name, port and status, its `databases` (name, owner, encoding and size in bytes, `pg_database_size`) and their total `size`.
|
||||
- **A cluster that is not online:** `databases` and `size` are `null`.
|
||||
- **Collection:** the web service reads it with `psql` as the `postgres` user at each request.
|
||||
|
||||
### The /users API
|
||||
|
||||
@@ -590,9 +724,9 @@ and answer bodies are JSON; a request body is at most 4096 bytes.
|
||||
|
||||
| Request | Body | Answer |
|
||||
| --- | --- | --- |
|
||||
| `GET /users` | | `{"users": [{"name", "enabled", "locked", "certificate", "otp", "first_name", "last_name", "email"}, ...]}` |
|
||||
| `POST /users` | `{"name", "password", "first_name", "last_name", "email"}`, all required | 201 and the user |
|
||||
| `PUT /users/<name>` | any of `{"first_name", "last_name", "email"}` | the user |
|
||||
| `GET /users` | | `{"users": [{"name", "manager_only", "service", "internal", "enabled", "locked", "certificate", "otp", "first_name", "last_name", "email"}, ...]}` |
|
||||
| `POST /users` | `{"name", "password", "first_name", "last_name", "email"}` for a VPN user; `{"name", "password", "email"?}` for a Manager-only account | 201 and the user |
|
||||
| `PUT /users/<name>` | any of `{"first_name", "last_name", "email"}`; for a Manager-only account only `{"email"}`, `""` removing it | the user |
|
||||
| `PUT /users/<name>/password` | `{"password": ...}` | the user |
|
||||
| `POST /users/<name>/disable` | | the user |
|
||||
| `POST /users/<name>/enable` | | the user |
|
||||
@@ -604,7 +738,8 @@ and answer bodies are JSON; a request body is at most 4096 bytes.
|
||||
| `DELETE /users/<name>/otp` | | the user |
|
||||
|
||||
A failure is `{"error": ...}` with 400 for an invalid request, 404 for a user that does not
|
||||
exist, 409 for a user that exists and 500 otherwise. `/users` works whether or not OpenVPN or
|
||||
exist or is a service account, 409 for a user that exists and 500 otherwise. `/users` does not
|
||||
list service accounts; `/service-accounts` serves them. `/users` works whether or not OpenVPN or
|
||||
road-warrior access is enabled; the profile needs OpenVPN. Passwords, private keys and TOTP
|
||||
secrets cross the internal network in plain HTTP, like the token.
|
||||
|
||||
@@ -614,6 +749,45 @@ curl -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
|
||||
http://10.0.0.4:7081/users
|
||||
```
|
||||
|
||||
### The /service-accounts API
|
||||
|
||||
`/service-accounts` manages the service accounts with the same token and the same module as
|
||||
`/users`.
|
||||
|
||||
| Request | Body | Answer |
|
||||
| --- | --- | --- |
|
||||
| `GET /service-accounts` | | `{"service_accounts": [{"name", "display_name", "description", "internal", "enabled", "locked", "email", ...}, ...]}`, each as `/users` shows a user |
|
||||
| `POST /service-accounts` | `{"name", "password", "display_name"?, "description"?, "email"?}` | 201 and the account |
|
||||
| `PUT /service-accounts/<name>` | any of `{"display_name", "description", "email"}`; `""` removes one | the account |
|
||||
| `PUT /service-accounts/<name>/password` | `{"password": ...}` | the account |
|
||||
| `POST /service-accounts/<name>/disable` | | the account |
|
||||
| `POST /service-accounts/<name>/enable` | | the account |
|
||||
| `POST /service-accounts/<name>/unlock` | | the account |
|
||||
| `DELETE /service-accounts/<name>` | | `{"name", "removed": true}` |
|
||||
|
||||
A display name has at most 64 printable characters and a description at most 256. The internal
|
||||
service accounts, `internal` true, are only listed: every other request on one gets 403, as the
|
||||
router manages them, their display name and description included. A failure is otherwise as for
|
||||
`/users`, with 404 also for a name that is not a service account.
|
||||
|
||||
### The /account API
|
||||
|
||||
`/account` serves the user the access token names, whether or not the user is a member of
|
||||
`admins`, with the same module as `/users`:
|
||||
|
||||
| Request | Body | Answer |
|
||||
| --- | --- | --- |
|
||||
| `GET /account` | | the user as `/users` shows it, and `"admin"`, whether the user is a member of `admins` |
|
||||
| `PUT /account/password` | `{"current_password", "current_code", "password"}` | the user |
|
||||
| `POST /account/otp` | | `{"name", "secret", "uri"}`, a new TOTP secret; nothing is stored |
|
||||
| `PUT /account/otp` | `{"current_password", "current_code", "secret", "code"}` | the user, once the code matches the secret |
|
||||
|
||||
`current_code` is needed only for a user enrolled for TOTP. A wrong current password or code gets
|
||||
403 and counts as a failed login; slapd accepts each code once, so a code used to sign in is not
|
||||
accepted again, and a change right after signing in waits for the next code. A user enrols or
|
||||
replaces the enrolment; removing it is left to `/users` and `vpn-router-user`. Changing the
|
||||
password revokes the user's refresh tokens. Other failures are those of `/users`.
|
||||
|
||||
### The /groups API
|
||||
|
||||
`/groups` manages groups with the same token and the same module as `/users`.
|
||||
@@ -632,36 +806,182 @@ member, 404 for a group or user that does not exist, 409 for a group that exists
|
||||
otherwise. `DELETE /users/<name>` is refused with 400 while the user is the last member of a
|
||||
group.
|
||||
|
||||
### The /site-to-site API
|
||||
|
||||
`/site-to-site` manages the site-to-site connections of `vpn-router.json` with the same token as
|
||||
`/users`.
|
||||
|
||||
| Request | Body | Answer |
|
||||
| --- | --- | --- |
|
||||
| `GET /site-to-site` | | `{"connections": [{"name", "addrs", "id", "cidrs", "local_cidrs", "local_id_mode", "psk_file", "psk_set", "established"}, ...]}` |
|
||||
| `POST /site-to-site` | `{"name", "addrs", "id", "cidrs", "local_cidrs", "local_id_mode", "psk"}`; `psk` is required | 201 and the connection |
|
||||
| `PUT /site-to-site/<name>` | any of the settings of `POST` but `name`; an empty or absent `psk` keeps the key, a `psk` given replaces `psk_file` | the connection |
|
||||
| `DELETE /site-to-site/<name>` | | `{"name", "removed": true}` |
|
||||
|
||||
A connection is shown without its key: `psk_set` says whether `psk` holds one, and `established`
|
||||
whether its tunnel is up, `null` when swanctl cannot be asked. Each change is made under the lock
|
||||
`vpn-config-edit` takes, checked as `vpn-router-setup` checks the configuration, written to
|
||||
`vpn-router.json`, and applied by restarting `vpn-router-setup` without waiting for it. A failure
|
||||
is `{"error": ...}` with 400 for an invalid request or configuration, with the reason, 404 for a
|
||||
connection that does not exist, 409 for a name that exists and 500 otherwise. Keys cross the
|
||||
internal network in plain HTTP, like the token.
|
||||
|
||||
The service starts on its own, without waiting for `vpn-router-setup`, and keeps running. Every
|
||||
30 seconds it works out the internal address from `vpn-router.json` as `general.mode` directs,
|
||||
and reads `web.port` and `web.enabled`; it listens there, moves when the address or the port
|
||||
changes, and stops listening while `web.enabled` is `false` or the address is not known. Each
|
||||
request reads the signing key and `vpn-router.json`, and the features and the facts for
|
||||
`/config` from `/run/vpn-router/web.json` (readable by root only), which `vpn-router-setup` writes;
|
||||
before that file exists the page shows no features. `vpn-router-setup` opens `web.port` on the
|
||||
internal interface in the firewall.
|
||||
30 seconds it reads `web.listen`, `web.port` and `web.enabled` from `vpn-router.json` and works
|
||||
out the addresses of the interfaces `web.listen` names as `general.mode` directs; it listens on
|
||||
each, moves when an address or the port changes, and stops listening while `web.enabled` is
|
||||
`false` or no address is known. A change of `web.listen` takes effect within 30 seconds, or when
|
||||
the service restarts. Each request reads the signing key and `vpn-router.json`, and the features
|
||||
and the facts for `/config` from `/run/vpn-router/web.json` (readable by root only), which
|
||||
`vpn-router-setup` writes; before that file exists the page shows no features.
|
||||
`vpn-router-setup` opens `web.port` on the internal and the external interface in the firewall.
|
||||
|
||||
## HTTPS
|
||||
|
||||
Caddy, from Ubuntu's `caddy` package, serves the status web service on `https.web_port` (7444 by
|
||||
default), while the Router Manager is installed the manager on `https.manager_port` (7443), and
|
||||
while metrics are enabled the metrics service on `metrics.https_port` (9465), all for `wan.local_fqdn`, while `https.enabled` is `true` and `wan.local_fqdn` is set. It obtains
|
||||
and renews the certificate from Let's Encrypt with the HTTP challenge, so `wan.local_fqdn` must
|
||||
resolve to the router's public address and port 80 must be reachable from the internet; the
|
||||
certificate and the ACME account are kept in `/var/lib/caddy`. Port 80 answers the challenge only;
|
||||
it does not redirect to HTTPS.
|
||||
|
||||
`vpn-router-setup` writes Caddy's configuration to `/etc/caddy/vpn-router.Caddyfile` (root, group
|
||||
`caddy`, the group Caddy runs as, mode 0640), which the
|
||||
package's drop-in `/usr/lib/systemd/system/caddy.service.d/vpn-router.conf` makes `caddy.service`
|
||||
read; `caddy.service` does not start before the file exists, and `/etc/caddy/Caddyfile` is not
|
||||
used. Setup starts Caddy, reloads it when the file changes, and stops it and removes the file when
|
||||
HTTPS is off. In the firewall it opens port 80 on the external interface and the HTTPS ports on the
|
||||
internal and the external interface. Caddy calls both services on `127.0.0.1`, where they listen
|
||||
whatever `web.listen` and the manager's `listen` say, and passes the client's address in
|
||||
`X-Forwarded-For`, which the manager trusts from `127.0.0.1` and the web service by default, as
|
||||
the address the call came in on; a `web.trusted_proxies` that is set must list `127.0.0.1`. The plain HTTP ports stay as
|
||||
`web.listen` and the manager's `listen` set them. `docker.exposed_ports` may not take port 80 or
|
||||
the HTTPS ports while `https.enabled` is `true`.
|
||||
|
||||
```bash
|
||||
curl https://router.example.com:7444/status
|
||||
```
|
||||
|
||||
## Metrics
|
||||
|
||||
While `metrics.enabled` is `true`, `vpn-router-setup` runs `vpn-router-metrics`, which every minute
|
||||
takes a sample of the router with `psutil`:
|
||||
|
||||
- the CPU utilization;
|
||||
- the memory used and in total;
|
||||
- the free and the total space of `/`;
|
||||
- the bytes received and sent per second on the external and the internal interface;
|
||||
- the 1, 5 and 15 minute load averages;
|
||||
- the VPN clients of each enabled VPN system: road-warrior (IPsec P2S) and OpenVPN clients, and
|
||||
WireGuard peers with a handshake in the last 3 minutes;
|
||||
- for each site-to-site connection that is set up: whether it is up (1 or 0), the bytes and the
|
||||
packets per second received and sent through its child SAs since the previous sample, and the
|
||||
seconds its IKE SA has been up. The child SAs' counters start again at a rekey; a counter lower
|
||||
than in the previous sample counts from zero.
|
||||
|
||||
### History
|
||||
|
||||
The samples are kept in the router's PostgreSQL database, the one of the refresh tokens
|
||||
(`web.database`), in four detail tables, one row per minute, and per interface, kind of client or
|
||||
site-to-site connection, and each is summarized hourly, daily, monthly and yearly. A summary row
|
||||
has the start of its period, in UTC, and the average and the maximum of each value; the average
|
||||
of a connection's `up` is the part of the period it was up:
|
||||
|
||||
| Level | Tables | Kept |
|
||||
|---|---|---|
|
||||
| Details | `metrics_system`, `metrics_network`, `metrics_vpn`, `metrics_site_to_site` | `metrics.keep_details` |
|
||||
| Hourly | `metrics_system_hourly`, `metrics_network_hourly`, `metrics_vpn_hourly`, `metrics_site_to_site_hourly` | `metrics.keep_hourly` |
|
||||
| Daily | `metrics_system_daily`, `metrics_network_daily`, `metrics_vpn_daily`, `metrics_site_to_site_daily` | `metrics.keep_daily` |
|
||||
| Monthly | `metrics_system_monthly`, `metrics_network_monthly`, `metrics_vpn_monthly`, `metrics_site_to_site_monthly` | `metrics.keep_monthly` |
|
||||
| Yearly | `metrics_system_yearly`, `metrics_network_yearly`, `metrics_vpn_yearly`, `metrics_site_to_site_yearly` | `metrics.keep_yearly` |
|
||||
|
||||
`metrics_site_to_site` has the columns `connection`, `up`, `rx`, `tx`, `packets_rx`, `packets_tx`
|
||||
and `uptime`.
|
||||
|
||||
How it works:
|
||||
|
||||
- **Summaries:** a period is summarized once it is complete, from the level below. A stopped service catches up on the periods it missed, as long as their rows are still kept.
|
||||
- **Garbage collection:** runs on start and once a day. It deletes the rows older than their retention, a count and a unit such as `24 hours`, `3 months` or `10 years`.
|
||||
- **History API:** `GET /metrics/history?range=hour|day|month|year` of the status web service, protected like `/config`, answers the views of the Router Manager's Metrics tab. Each point has the system values, `network` by interface, `clients` by kind and `site_to_site` by connection, each value as its `avg` and `max`:
|
||||
|
||||
| Range | Bucket | Source |
|
||||
|---|---|---|
|
||||
| `hour` | 1 minute | details |
|
||||
| `day` | 10 minutes | details |
|
||||
| `month` | 6 hours | hourly |
|
||||
| `year` | 1 month | monthly |
|
||||
|
||||
### Scraping
|
||||
|
||||
`GET /metrics` of the metrics service serves the last sample in the Prometheus text format, written by `prometheus_client`.
|
||||
|
||||
- **Metric names:** the OpenTelemetry semantic convention names in their Prometheus form:
|
||||
- `system_cpu_utilization`;
|
||||
- `system_memory_usage_bytes` and `system_memory_limit_bytes`;
|
||||
- `system_filesystem_usage_bytes` and `system_filesystem_utilization`;
|
||||
- `system_network_io_bytes_total`;
|
||||
- `system_cpu_load_average_1m`, `_5m` and `_15m`;
|
||||
- `vpn_router_clients`, by `kind`: `p2s`, `openvpn` and `wireguard`;
|
||||
- by `connection`, for each site-to-site connection: `vpn_router_site_to_site_up`, `vpn_router_site_to_site_uptime_seconds`, and, by `direction` too, `vpn_router_site_to_site_io_bytes_per_second` and `vpn_router_site_to_site_packets_per_second`. The rates are gauges, since the child SAs' counters start again at each rekey.
|
||||
- **Listening:** on the interfaces `metrics.listen` names, `internal` by default, at `metrics.port`, and on `127.0.0.1`. While HTTPS is on, Caddy serves it at `metrics.https_port` for `wan.local_fqdn`. `vpn-router-setup` opens both ports on the internal and the external interface.
|
||||
- **Authentication:** HTTP Basic as the service account `metrics.service_account` names, `metrics` by default. The password is checked with a bind to the user directory, so 5 failed attempts lock the account as for any user.
|
||||
- **The account:** `vpn-router-setup` owns it while metrics are enabled.
|
||||
- It adds the account with a random password, which it writes to `/etc/vpn-router/metrics.secret` (root only). Give that password to the collector.
|
||||
- When the file is missing or no longer holds the account's password, setup sets a new random password and ends any lockout. Delete the file and restart `vpn-router-setup` to change the password.
|
||||
- A disabled account is left as it is.
|
||||
- A user of that name who is not a service account is a setup failure.
|
||||
|
||||
An OpenTelemetry Collector scrapes it with its `prometheus` receiver:
|
||||
|
||||
```yaml
|
||||
receivers:
|
||||
prometheus:
|
||||
config:
|
||||
scrape_configs:
|
||||
- job_name: vpn-router
|
||||
scrape_interval: 60s
|
||||
scheme: https
|
||||
basic_auth:
|
||||
username: metrics
|
||||
password_file: /etc/otelcol/vpn-router-metrics.secret # a copy of /etc/vpn-router/metrics.secret
|
||||
static_configs:
|
||||
- targets: ['router.example.com:9465']
|
||||
```
|
||||
|
||||
### Pushing
|
||||
|
||||
When `metrics.otlp_endpoint` is set, every sample is also pushed there as an OTLP/HTTP JSON request.
|
||||
|
||||
- **Content:** the semantic convention names (`system.cpu.utilization`, `system.network.io` and so on), `vpn_router.clients`, the gauges `vpn_router.site_to_site.up`, `.uptime`, `.io` and `.packets` with the attribute `connection`, and the resource attributes `service.name` `vpn-router` and `host.name`.
|
||||
- **Authentication:** HTTP Basic with `metrics.otlp_username` and `metrics.otlp_password` when they are set.
|
||||
- **Failures:** a failed push is logged and not repeated; the next sample is pushed a minute later.
|
||||
|
||||
## Router Manager
|
||||
|
||||
The `vpn-router-manager` package, built in the repository `linux-cloud-router-manager`, is a web
|
||||
application that shows the state of the router and manages its users through the status web
|
||||
service. It can be installed on the router or on another Linux system. While it is installed on
|
||||
application that shows the state of the router and manages its users and site-to-site
|
||||
connections through the status web service. It can be installed on the router or on another Linux system. While it is installed on
|
||||
the router, `vpn-router-setup` opens its port (`port` in `/etc/vpn-router-manager/manager.json`,
|
||||
7080 by default, HTTP; 7443 is kept for it with TLS) on the internal interface; restart
|
||||
`vpn-router-setup` after changing it.
|
||||
7080 by default, HTTP; HTTPS is Caddy's, on `https.manager_port`) on the internal and the external
|
||||
interface; restart `vpn-router-setup` after changing it. The manager's `listen` chooses the
|
||||
interfaces it listens on.
|
||||
|
||||
## Removal
|
||||
|
||||
`apt-get remove` stops `vpn-router-docker`, which removes the Docker rules. It terminates the
|
||||
tunnels and takes `wg-quick@wg0` and `openvpn-server@vpn-router` down, but leaves the
|
||||
tunnels and takes `wg-quick@wg0`, `openvpn-server@vpn-router`, `caddy` and `vpn-router-metrics`
|
||||
down, but leaves the
|
||||
configuration in place. `strongswan` is left enabled, since it is a shared service that
|
||||
may be in use by something else.
|
||||
|
||||
`apt-get purge` additionally removes the generated files and the firewall rules, restoring
|
||||
`/etc/ufw/before.rules` to its original content, and hands `vpn-router.json` back to `ucf`
|
||||
before deleting it so a later reinstall starts clean. It also removes the web service's
|
||||
signing key, and the refresh tokens' database, its role and their password file when the package
|
||||
created them. Key material and users are never deleted: `/etc/vpn-router/pki`, `/etc/vpn-router/ldap`, the user directory in slapd, the
|
||||
signing key, the password file of the metrics service account, and the database of the refresh
|
||||
tokens and the metrics, its role and their password
|
||||
file when the package created them, and Caddy's configuration `/etc/caddy/vpn-router.Caddyfile`; the certificate in
|
||||
`/var/lib/caddy` belongs to the `caddy` package. Key material and users are never deleted: `/etc/vpn-router/pki`, `/etc/vpn-router/ldap`, the user directory in slapd, the
|
||||
WireGuard key pair and everything under `/etc/swanctl` are left alone.
|
||||
|
||||
## Building
|
||||
|
||||
@@ -1,3 +1,305 @@
|
||||
vpn-router (0.19.0-1) unstable; urgency=medium
|
||||
|
||||
feature: service accounts have a display name and a description, which the router sets for its internal ones.
|
||||
|
||||
* vpnRouterAccount of the package's schema also takes displayName and
|
||||
description; vpn-router-setup updates the schema of an existing
|
||||
directory.
|
||||
* /service-accounts shows display_name and description, takes them when
|
||||
adding a service account and changes them with PUT; "" removes one.
|
||||
/users shows them as null.
|
||||
* vpn-router-user add --service and update take --display-name and
|
||||
--description.
|
||||
* vpn-router-setup gives the metrics service's internal account the
|
||||
display name Metrics and its description, and sets them again when
|
||||
they differ.
|
||||
|
||||
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Mon, 26 Oct 2026 20:00:00 +0100
|
||||
|
||||
vpn-router (0.18.3-1) unstable; urgency=medium
|
||||
|
||||
fix: internal service accounts are the router's own, of which the metrics service's is the one at present, and the documentation says what a service account is.
|
||||
|
||||
* /service-accounts, /groups and the internal field of the answers
|
||||
take every internal service account, the accounts the router creates
|
||||
and manages for its own services, instead of only the metrics
|
||||
service's; the 403 answers say that the router manages them.
|
||||
* The README, the ADR and the users module describe a service account
|
||||
as the account another system, such as an OpenTelemetry Collector
|
||||
reading /metrics, signs in with, and the internal ones as managed by
|
||||
the router.
|
||||
|
||||
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Mon, 26 Oct 2026 19:00:00 +0100
|
||||
|
||||
vpn-router (0.18.2-1) unstable; urgency=medium
|
||||
|
||||
fix: service accounts are managed at /service-accounts, not /users, and the internal one of the metrics service is only listed.
|
||||
|
||||
* /service-accounts, for members of admins and admin tokens, lists the
|
||||
service accounts, adds one with a password and an optional e-mail
|
||||
address, changes the e-mail address, sets the password, disables,
|
||||
enables, unlocks and removes them.
|
||||
* /users no longer lists service accounts or takes "service" on POST,
|
||||
and answers 404 to a request naming one.
|
||||
* /service-accounts answers 403 to every request but the list on the
|
||||
account metrics.service_account names while metrics are enabled, and
|
||||
/groups to putting it in or taking it out of a group;
|
||||
vpn-router-setup and root's vpn-router-user still change it.
|
||||
|
||||
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Mon, 26 Oct 2026 18:00:00 +0100
|
||||
|
||||
vpn-router (0.18.1-1) unstable; urgency=medium
|
||||
|
||||
fix: vpn-router-setup owns only the creation of the metrics service's account, which is marked internal, and administrators add service accounts with vpn-router-user add --service and /users.
|
||||
|
||||
* POST /users takes "service": true and adds a service account, which
|
||||
cannot sign in to /auth; vpn-router-user add takes --service.
|
||||
* The users of /users and vpn-router-user have internal, true for the
|
||||
account metrics.service_account names while metrics are enabled;
|
||||
vpn-router-user list shows its kind as internal.
|
||||
* PUT /users/<name>/password refuses the internal service account,
|
||||
whose password vpn-router-setup keeps.
|
||||
|
||||
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Mon, 26 Oct 2026 17:00:00 +0100
|
||||
|
||||
vpn-router (0.18.0-2) unstable; urgency=medium
|
||||
|
||||
update: release.py tags a Debian revision other than 1 as x.y.z-n.
|
||||
|
||||
* release.py tags the newest entry x.y.z for the Debian revision 1 and
|
||||
x.y.z-n for a later revision, so a packaging change gets a release
|
||||
of its own.
|
||||
|
||||
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Mon, 26 Oct 2026 14:00:00 +0100
|
||||
|
||||
vpn-router (0.18.0-1) unstable; urgency=medium
|
||||
|
||||
feature: site_to_site holds any number of named site-to-site connections, which /site-to-site lists, adds, changes and removes.
|
||||
|
||||
* site_to_site maps a connection name to addrs, id, cidrs, local_cidrs,
|
||||
local_id_mode, psk and psk_file; it replaces remote and
|
||||
wan.local_id_mode, and a configuration with them is refused.
|
||||
* The install answers fill the connection remote-site; the local subnets
|
||||
fill both its local_cidrs and local.cidrs.
|
||||
* Every ready connection is written to
|
||||
/etc/swanctl/conf.d/site-to-site.conf as the swanctl connection and
|
||||
child of its name with the secret ike-psk-<name>; remote-site.conf is
|
||||
no longer written.
|
||||
* Traffic from each connection's local_cidrs to its cidrs is not
|
||||
masqueraded.
|
||||
* /site-to-site, for members of admins and admin tokens, lists the
|
||||
connections without their keys, adds, changes and removes them, and
|
||||
restarts vpn-router-setup to apply the change.
|
||||
* vpn-config-edit sets site_to_site.NAME.option, creating the
|
||||
connection, and removes site_to_site.NAME; it and the web service
|
||||
write vpn-router.json under one lock.
|
||||
* The status shows site_to_site.connections, whether each connection is
|
||||
established.
|
||||
* The metrics service keeps, per site-to-site connection, whether it is
|
||||
up, the bytes and packets per second received and sent, and the
|
||||
seconds its IKE SA has been up, in metrics_site_to_site and its
|
||||
summaries; /metrics/history has them in site_to_site of each point.
|
||||
* /metrics serves vpn_router_site_to_site_up,
|
||||
vpn_router_site_to_site_uptime_seconds,
|
||||
vpn_router_site_to_site_io_bytes_per_second and
|
||||
vpn_router_site_to_site_packets_per_second by connection, and the
|
||||
OTLP push the vpn_router.site_to_site gauges; vpn_router_clients and
|
||||
metrics_vpn no longer have the kind site_to_site.
|
||||
* /config shows each connection in network.site_to_site, and no psk at
|
||||
any level of the configuration.
|
||||
* Removing the package terminates every site-to-site connection; purging
|
||||
removes site-to-site.conf.
|
||||
|
||||
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Sun, 25 Oct 2026 23:00:00 +0100
|
||||
|
||||
vpn-router (0.17.0-1) unstable; urgency=medium
|
||||
|
||||
feature: /config shows the router's PostgreSQL clusters and their settings, and /postgresql their databases with their sizes.
|
||||
|
||||
* /config has a postgresql section: each cluster of pg_lsclusters with
|
||||
its version, name, port, status and data directory, and the main
|
||||
settings of each online cluster.
|
||||
* /postgresql, protected like /config, lists the databases of each
|
||||
online cluster with owner, encoding and size, and the total size.
|
||||
|
||||
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Sun, 25 Oct 2026 21:00:00 +0200
|
||||
|
||||
vpn-router (0.16.0-1) unstable; urgency=medium
|
||||
|
||||
feature: the router keeps its metrics in PostgreSQL, serves them for an OpenTelemetry Collector and pushes them over OTLP.
|
||||
|
||||
* vpn-router-metrics takes a sample every minute: CPU, memory, free
|
||||
space of /, bytes per second on the external and the internal
|
||||
interface, the load averages, and the VPN clients of each enabled
|
||||
system.
|
||||
* The samples are kept in metrics_system, metrics_network and
|
||||
metrics_vpn, and summarized hourly, daily, monthly and yearly in
|
||||
their _hourly, _daily, _monthly and _yearly tables; once a day the
|
||||
rows older than metrics.keep_details, keep_hourly, keep_daily,
|
||||
keep_monthly and keep_yearly are deleted.
|
||||
* /metrics serves the last sample in the Prometheus format on the
|
||||
interfaces metrics.listen names (internal by default) at
|
||||
metrics.port, and over HTTPS by Caddy at metrics.https_port, with
|
||||
HTTP Basic authentication of the service account
|
||||
metrics.service_account names (metrics by default).
|
||||
* metrics.otlp_endpoint pushes every sample as OTLP/HTTP JSON, with
|
||||
metrics.otlp_username and metrics.otlp_password.
|
||||
* /metrics/history of the web service answers the hour, day, month
|
||||
and year views for the Router Manager.
|
||||
* vpn-router-setup adds and keeps that service account, which cannot
|
||||
sign in to /auth, with a random password in
|
||||
/etc/vpn-router/metrics.secret, set anew when the file is missing.
|
||||
* Depends on python3-psutil and python3-prometheus-client.
|
||||
|
||||
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Sun, 25 Oct 2026 19:00:00 +0200
|
||||
|
||||
vpn-router (0.15.0-1) unstable; urgency=medium
|
||||
|
||||
feature: vpn-config-edit shows and changes the router's configuration.
|
||||
|
||||
* vpn-config-edit show prints /etc/vpn-router/vpn-router.json as
|
||||
indented JSON.
|
||||
* vpn-config-edit set SECTION.OPTION VALUE converts the value to the
|
||||
setting's type, checks the whole configuration as vpn-router-setup
|
||||
does, writes the file indented with mode 0600, and reminds that
|
||||
vpn-router-setup applies it.
|
||||
|
||||
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Sun, 25 Oct 2026 18:00:00 +0200
|
||||
|
||||
vpn-router (0.14.1-1) unstable; urgency=medium
|
||||
|
||||
fix: web.status_content is basic by default.
|
||||
|
||||
* Without web.status_content, or with a value that is not allowed, /
|
||||
and /status show only the overall status, the time and the Client
|
||||
IP; set it to standard or full to show more.
|
||||
|
||||
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Sun, 25 Oct 2026 17:00:00 +0200
|
||||
|
||||
vpn-router (0.14.0-1) unstable; urgency=medium
|
||||
|
||||
feature: the status page shows the Client IP, and web.status_content sets how much it shows.
|
||||
|
||||
* / and /status show the Client IP, the address the request came from:
|
||||
the last X-Forwarded-For entry for a request from a trusted proxy, so
|
||||
behind Caddy or the Router Manager the browser's; /status has it as
|
||||
client_ip.
|
||||
* web.status_content sets what / and /status show: basic the overall
|
||||
status, the time and the Client IP; standard also the package
|
||||
versions and the number of active and failed services; full, the
|
||||
default, everything as before. /status names it in content.
|
||||
|
||||
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Sun, 25 Oct 2026 16:00:00 +0200
|
||||
|
||||
vpn-router (0.13.0-1) unstable; urgency=medium
|
||||
|
||||
feature: Caddy serves the status page and the Router Manager over HTTPS with a certificate from Let's Encrypt.
|
||||
|
||||
* The package depends on caddy. While https.enabled is true, the
|
||||
default, and wan.local_fqdn is set, vpn-router-setup writes
|
||||
/etc/caddy/vpn-router.Caddyfile and runs Caddy, which serves the
|
||||
status page on https.web_port (7444) and the Router Manager, when it
|
||||
is installed, on https.manager_port (7443) for wan.local_fqdn, and
|
||||
obtains the certificate from Let's Encrypt with the HTTP challenge.
|
||||
* The package's drop-in for caddy.service makes Caddy read that file
|
||||
and keeps it from starting before the file exists. The file belongs
|
||||
to root and the group caddy, which Caddy runs as, with mode 0640.
|
||||
* The firewall opens port 80 on the external interface and the HTTPS
|
||||
ports on both interfaces while HTTPS is on.
|
||||
* The web service also listens on 127.0.0.1, where Caddy calls it.
|
||||
* docker.exposed_ports may not take port 80 or the HTTPS ports while
|
||||
HTTPS is on.
|
||||
* Removing the package stops and disables caddy; purging it removes
|
||||
/etc/caddy/vpn-router.Caddyfile.
|
||||
|
||||
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Sun, 25 Oct 2026 15:00:00 +0200
|
||||
|
||||
vpn-router (0.12.3-1) unstable; urgency=medium
|
||||
|
||||
fix: the web service listens on the internal address, the external address or both.
|
||||
|
||||
* web.listen in vpn-router.json sets the interfaces the web service
|
||||
listens on: internal, external or both, the default. It is read
|
||||
every 30 seconds, so a change takes effect without running setup,
|
||||
at the latest when vpn-router-web restarts.
|
||||
* The firewall opens the ports of the web service and of the Router
|
||||
Manager on the internal and the external interface.
|
||||
|
||||
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Sun, 25 Oct 2026 13:00:00 +0200
|
||||
|
||||
vpn-router (0.12.2-1) unstable; urgency=medium
|
||||
|
||||
fix: the web service manages users also when it started before setup.
|
||||
|
||||
* vpn-router-web creates /run/vpn-router when it starts, so the user
|
||||
operations take their lock there; started before vpn-router-setup had
|
||||
made the directory, they failed with "Read-only file system".
|
||||
|
||||
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Sun, 25 Oct 2026 12:00:00 +0200
|
||||
|
||||
vpn-router (0.12.1-1) unstable; urgency=medium
|
||||
|
||||
fix: the status page shows whether the Router Manager runs.
|
||||
|
||||
* When vpn-router-manager is installed, the services show the state of
|
||||
vpn-router-manager, and status is degraded while it is not active.
|
||||
|
||||
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Sun, 25 Oct 2026 11:00:00 +0200
|
||||
|
||||
vpn-router (0.12.0-1) unstable; urgency=medium
|
||||
|
||||
feature: every user manages their own account through /account.
|
||||
|
||||
* /account serves the signed-in user, admin or not: GET shows the user
|
||||
and whether they are a member of admins; PUT /account/password sets
|
||||
the password and POST and PUT /account/otp enrol or replace the TOTP
|
||||
enrolment, given the current password and, when enrolled, a current
|
||||
code. A wrong one is 403 and counts as a failed login. Tokens of
|
||||
vpn-router-token get 403 there.
|
||||
|
||||
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Sun, 25 Oct 2026 10:00:00 +0200
|
||||
|
||||
vpn-router (0.11.0-1) unstable; urgency=medium
|
||||
|
||||
feature: the status page shows the Router Manager's version.
|
||||
|
||||
* / and /status show the installed version of vpn-router-manager, as
|
||||
manager_version in JSON, or that it is not installed.
|
||||
* The services show the state of each PostgreSQL cluster, as
|
||||
postgresql@<version>-<cluster>, or postgresql: no cluster without one.
|
||||
|
||||
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Sun, 25 Oct 2026 09:00:00 +0200
|
||||
|
||||
vpn-router (0.10.0-1) unstable; urgency=medium
|
||||
|
||||
feature: Manager-only accounts, and the Router Manager's first administrator.
|
||||
|
||||
* vpn-router-user add without the person options, and POST /users without
|
||||
them, add a Manager-only account: a name and a password, no names, no
|
||||
e-mail address and no certificate, so no VPN access. Giving only some of
|
||||
the three is refused. The answers show the kind as manager_only, list
|
||||
shows a KIND column, and update refuses a Manager-only account.
|
||||
* vpn-router-setup creates the administrator the Router Manager's package
|
||||
leaves in /etc/vpn-router-manager/admin.json as a Manager-only account in
|
||||
the group admins, which it creates when missing, and removes the file.
|
||||
* The directory has the package's own schema, with the auxiliary class
|
||||
vpnRouterAccount and the attribute vpnRouterScope. A Manager-only account
|
||||
can have an e-mail address: add --email, update --email ('' removes it),
|
||||
and email in POST and PUT /users.
|
||||
|
||||
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Sun, 25 Oct 2026 08:00:00 +0200
|
||||
|
||||
vpn-router (0.9.2-1) unstable; urgency=medium
|
||||
|
||||
fix: the web service starts once libpq can be found.
|
||||
|
||||
* The web service unit waits, with wait-for-libpq, until libpq is in the
|
||||
linker cache before it starts the service, for at most 5 minutes; the
|
||||
service imports psycopg at start again.
|
||||
* The package starts the web service without waiting for it.
|
||||
|
||||
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Sun, 25 Oct 2026 06:00:00 +0200
|
||||
|
||||
vpn-router (0.9.1-1) unstable; urgency=medium
|
||||
|
||||
fix: the web service waits for the PostgreSQL driver instead of failing at start.
|
||||
|
||||
@@ -23,19 +23,23 @@ Depends: ${misc:Depends},
|
||||
python3-ldap,
|
||||
python3-pyotp,
|
||||
python3-psycopg,
|
||||
python3-psutil,
|
||||
python3-prometheus-client,
|
||||
postgresql,
|
||||
slapd,
|
||||
openvpn,
|
||||
openvpn-auth-ldap,
|
||||
iproute2,
|
||||
systemd-resolved
|
||||
systemd-resolved,
|
||||
caddy
|
||||
Description: Linux VPN router with IPSec, road-warrior, OpenVPN and WireGuard support
|
||||
Configures a Linux host or virtual machine as a VPN router: site-to-site
|
||||
IKEv2 IPSec (strongSwan swanctl), optional road-warrior access using EAP-TLS,
|
||||
optional OpenVPN with a certificate and a password per user, kept in a local
|
||||
LDAP directory, optional WireGuard, and the routing, NAT, MSS clamping and
|
||||
firewall rules a router needs. Nothing in the package is specific to a cloud
|
||||
provider.
|
||||
firewall rules a router needs. It keeps metrics of the router in PostgreSQL
|
||||
and serves them for an OpenTelemetry Collector. Nothing in the package is
|
||||
specific to a cloud provider.
|
||||
.
|
||||
Settings live in /etc/vpn-router/vpn-router.json, which is created on first
|
||||
install and owned by the administrator afterwards. Editing it and restarting
|
||||
|
||||
@@ -5,17 +5,23 @@ src/usr/lib/vpn-router/vpnrouter_net.py usr/lib/vpn-router/
|
||||
src/usr/lib/vpn-router/vpnrouter_serve.py usr/lib/vpn-router/
|
||||
src/usr/lib/vpn-router/vpnrouter_users.py usr/lib/vpn-router/
|
||||
src/usr/lib/vpn-router/vpnrouter_auth.py usr/lib/vpn-router/
|
||||
src/usr/lib/vpn-router/vpnrouter_metrics.py usr/lib/vpn-router/
|
||||
src/usr/lib/vpn-router/vpnrouter_status.py usr/lib/vpn-router/
|
||||
src/usr/lib/vpn-router/openvpn-same-user usr/lib/vpn-router/
|
||||
src/usr/sbin/vpn-router-user usr/sbin/
|
||||
src/usr/sbin/vpn-router-group usr/sbin/
|
||||
src/usr/sbin/vpn-router-token usr/sbin/
|
||||
src/usr/sbin/vpn-config-edit usr/sbin/
|
||||
src/usr/lib/vpn-router/simple-ca usr/lib/vpn-router/
|
||||
src/usr/lib/vpn-router/configure usr/lib/vpn-router/
|
||||
src/usr/lib/vpn-router/docker-rules usr/lib/vpn-router/
|
||||
src/usr/lib/vpn-router/setup usr/lib/vpn-router/
|
||||
src/usr/lib/vpn-router/web usr/lib/vpn-router/
|
||||
src/usr/lib/vpn-router/metrics usr/lib/vpn-router/
|
||||
src/usr/lib/vpn-router/bootstrap usr/lib/vpn-router/
|
||||
src/usr/lib/vpn-router/wait-for-libpq usr/lib/vpn-router/
|
||||
src/usr/lib/vpn-router/generate-config usr/lib/vpn-router/
|
||||
src/usr/lib/vpn-router/vpnrouter_platforms/*.py usr/lib/vpn-router/vpnrouter_platforms/
|
||||
src/usr/share/vpn-router/templates/* usr/share/vpn-router/templates/
|
||||
src/usr/lib/systemd/system/caddy.service.d/vpn-router.conf usr/lib/systemd/system/caddy.service.d/
|
||||
src/usr/share/doc/vpn-router/vpn-router.json.example usr/share/doc/vpn-router/
|
||||
@@ -70,6 +70,12 @@ case "$1" in
|
||||
# The firewall is not touched here. Enabling it, allowing SSH and
|
||||
# setting the forward policy all belong to vpn-router-setup, which does
|
||||
# them only once the configuration says this host is a router.
|
||||
|
||||
# The web service waits for libpq in the linker cache, which dpkg
|
||||
# refreshes after this script, so it is started without waiting for it.
|
||||
if [ -d /run/systemd/system ]; then
|
||||
systemctl --no-block restart vpn-router-web.service >/dev/null 2>&1 || true
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
|
||||
|
||||
@@ -11,10 +11,10 @@ strip_ufw_blocks() {
|
||||
|
||||
tmp="$(mktemp)"
|
||||
awk '
|
||||
/^# (IPSEC|WIREGUARD|OPENVPN|P2S DNS|WEB|BOOTSTRAP|MANAGER|ROUTER FORWARD|ROUTER NAT|ROUTER MSS) RULES START$/ {
|
||||
/^# (IPSEC|WIREGUARD|OPENVPN|P2S DNS|WEB|BOOTSTRAP|MANAGER|HTTPS|METRICS|ROUTER FORWARD|ROUTER NAT|ROUTER MSS) RULES START$/ {
|
||||
skip = 1; pending = 0; next
|
||||
}
|
||||
/^# (IPSEC|WIREGUARD|OPENVPN|P2S DNS|WEB|BOOTSTRAP|MANAGER|ROUTER FORWARD|ROUTER NAT|ROUTER MSS) RULES END$/ {
|
||||
/^# (IPSEC|WIREGUARD|OPENVPN|P2S DNS|WEB|BOOTSTRAP|MANAGER|HTTPS|METRICS|ROUTER FORWARD|ROUTER NAT|ROUTER MSS) RULES END$/ {
|
||||
skip = 0; next
|
||||
}
|
||||
skip { next }
|
||||
@@ -43,14 +43,16 @@ case "$1" in
|
||||
ucfr --purge vpn-router /etc/vpn-router/vpn-router.json
|
||||
fi
|
||||
|
||||
rm -f /etc/swanctl/conf.d/remote-site.conf \
|
||||
rm -f /etc/swanctl/conf.d/site-to-site.conf \
|
||||
/etc/swanctl/conf.d/road-warrior.conf \
|
||||
/etc/systemd/resolved.conf.d/p2s-forwarder.conf \
|
||||
/etc/wireguard/wg0.conf \
|
||||
/etc/openvpn/server/vpn-router.conf \
|
||||
/etc/openvpn/server/vpn-router-ldap.conf \
|
||||
/etc/caddy/vpn-router.Caddyfile \
|
||||
/etc/vpn-router/vpn-router.json \
|
||||
/etc/vpn-router/web.key \
|
||||
/etc/vpn-router/metrics.secret \
|
||||
/var/lib/vpn-router/setup.done
|
||||
rmdir --ignore-fail-on-non-empty /var/lib/vpn-router 2>/dev/null || true
|
||||
|
||||
|
||||
@@ -6,15 +6,21 @@ case "$1" in
|
||||
# Stop offering the tunnels, but leave strongSwan itself enabled: it is
|
||||
# a shared service and may be in use by something other than this
|
||||
# package. The connection files are removed on purge.
|
||||
# The site-to-site connections are the names of site_to_site in vpn-router.json.
|
||||
if [ -d /run/systemd/system ]; then
|
||||
swanctl --terminate --ike remote-site >/dev/null 2>&1 || true
|
||||
connections="$(python3 -c 'import json, sys; print(" ".join(json.load(open(sys.argv[1])).get("site_to_site", {})))' \
|
||||
/etc/vpn-router/vpn-router.json 2>/dev/null || true)"
|
||||
for connection in $connections; do
|
||||
swanctl --terminate --ike "$connection" >/dev/null 2>&1 || true
|
||||
done
|
||||
swanctl --terminate --ike road-warrior >/dev/null 2>&1 || true
|
||||
fi
|
||||
|
||||
# This package enabled wg-quick@wg0 and openvpn-server@vpn-router, so
|
||||
# it takes them down again. Use the debhelper wrappers rather than
|
||||
# systemctl, so the script behaves on a machine without systemd.
|
||||
for unit in wg-quick@wg0.service openvpn-server@vpn-router.service; do
|
||||
# vpn-router-setup enabled wg-quick@wg0, openvpn-server@vpn-router,
|
||||
# caddy and vpn-router-metrics, so the package takes them down again.
|
||||
# Use the debhelper wrappers rather than systemctl, so the script
|
||||
# behaves on a machine without systemd.
|
||||
for unit in wg-quick@wg0.service openvpn-server@vpn-router.service caddy.service vpn-router-metrics.service; do
|
||||
if command -v deb-systemd-invoke >/dev/null 2>&1; then
|
||||
deb-systemd-invoke stop "$unit" >/dev/null 2>&1 || true
|
||||
fi
|
||||
|
||||
@@ -5,5 +5,6 @@
|
||||
override_dh_installsystemd:
|
||||
dh_installsystemd --name=vpn-router-setup
|
||||
dh_installsystemd --name=vpn-router-docker
|
||||
dh_installsystemd --name=vpn-router-web
|
||||
dh_installsystemd --name=vpn-router-web --no-start
|
||||
dh_installsystemd --name=vpn-router-metrics --no-enable --no-start
|
||||
dh_installsystemd --name=vpn-router-bootstrap --no-enable --no-start
|
||||
@@ -66,7 +66,8 @@ Type: select
|
||||
Choices: fqdn, public_ip, internal_ip
|
||||
Default: fqdn
|
||||
Description: IKE local identity mode:
|
||||
How to derive the IKE identity advertised to the remote site:
|
||||
How to derive the IKE identity the first site-to-site connection
|
||||
advertises to the remote site:
|
||||
.
|
||||
fqdn - use the FQDN, which must match what the peer expects.
|
||||
.
|
||||
@@ -78,8 +79,9 @@ Template: vpn-router/local_cidrs
|
||||
Type: string
|
||||
Default:
|
||||
Description: Local subnet CIDR(s):
|
||||
Comma-separated list of local subnet CIDRs to advertise into the
|
||||
site-to-site tunnel (for example 10.0.0.0/24 or 10.0.0.0/24,10.0.1.0/24).
|
||||
Comma-separated list of local subnet CIDRs routed by this router and
|
||||
advertised into the first site-to-site connection (for example
|
||||
10.0.0.0/24 or 10.0.0.0/24,10.0.1.0/24).
|
||||
|
||||
Template: vpn-router/int_gateway_ip
|
||||
Type: string
|
||||
@@ -94,25 +96,26 @@ Type: string
|
||||
Default:
|
||||
Description: Remote site WAN IP address(es):
|
||||
Comma-separated list of remote gateway addresses or FQDNs for the
|
||||
site-to-site IPSec tunnel.
|
||||
first site-to-site connection.
|
||||
|
||||
Template: vpn-router/remote_id
|
||||
Type: string
|
||||
Default:
|
||||
Description: Remote site IKE identity:
|
||||
IKE identity of the remote peer, without a leading @.
|
||||
IKE identity of the remote peer of the first site-to-site connection,
|
||||
without a leading @.
|
||||
|
||||
Template: vpn-router/remote_cidrs
|
||||
Type: string
|
||||
Default:
|
||||
Description: Remote subnet CIDR(s):
|
||||
Comma-separated list of remote subnet CIDRs reachable through the
|
||||
site-to-site tunnel (for example 192.168.0.0/24).
|
||||
first site-to-site connection (for example 192.168.0.0/24).
|
||||
|
||||
Template: vpn-router/psk
|
||||
Type: password
|
||||
Description: Pre-shared key (PSK):
|
||||
Pre-shared key for the site-to-site IKEv2 tunnel. Must match the value
|
||||
Pre-shared key of the first site-to-site connection. Must match the value
|
||||
configured on the remote peer. Stored in the configuration file and cleared
|
||||
from the debconf database after installation.
|
||||
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
[Unit]
|
||||
Description=VPN Router metrics service
|
||||
Documentation=file:/usr/share/doc/vpn-router/README.md
|
||||
After=network-online.target postgresql.service vpn-router-web.service
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
ExecStartPre=/usr/lib/vpn-router/wait-for-libpq
|
||||
ExecStart=/usr/bin/python3 /usr/lib/vpn-router/metrics
|
||||
TimeoutStartSec=5min
|
||||
Restart=on-failure
|
||||
RestartSec=15
|
||||
NoNewPrivileges=yes
|
||||
ProtectSystem=strict
|
||||
RuntimeDirectory=vpn-router
|
||||
RuntimeDirectoryPreserve=yes
|
||||
ProtectHome=yes
|
||||
PrivateTmp=yes
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
@@ -6,12 +6,16 @@ Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
ExecStartPre=/usr/lib/vpn-router/wait-for-libpq
|
||||
ExecStart=/usr/bin/python3 /usr/lib/vpn-router/web
|
||||
TimeoutStartSec=5min
|
||||
Restart=on-failure
|
||||
RestartSec=15
|
||||
NoNewPrivileges=yes
|
||||
ProtectSystem=strict
|
||||
ReadWritePaths=-/run/vpn-router -/etc/vpn-router -/etc/swanctl/x509crl
|
||||
RuntimeDirectory=vpn-router
|
||||
RuntimeDirectoryPreserve=yes
|
||||
ReadWritePaths=-/etc/vpn-router -/etc/swanctl/x509crl
|
||||
ProtectHome=yes
|
||||
PrivateTmp=yes
|
||||
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
# Caddy is the HTTPS proxy of vpn-router: it reads the configuration vpn-router-setup writes, and
|
||||
# does not start until that exists.
|
||||
[Unit]
|
||||
ConditionPathExists=/etc/caddy/vpn-router.Caddyfile
|
||||
|
||||
[Service]
|
||||
ExecStart=
|
||||
ExecStart=/usr/bin/caddy run --environ --adapter caddyfile --config /etc/caddy/vpn-router.Caddyfile
|
||||
ExecReload=
|
||||
ExecReload=/usr/bin/caddy reload --adapter caddyfile --config /etc/caddy/vpn-router.Caddyfile --force
|
||||
@@ -18,7 +18,9 @@ on the next dpkg-reconfigure.
|
||||
|
||||
Answers arrive as VPN_ROUTER_* environment variables and are converted to the
|
||||
type of the setting they fill: comma-separated lists become arrays, "true" and
|
||||
"false" become booleans and digits become integers.
|
||||
"false" become booleans and digits become integers. The site-to-site answers
|
||||
fill the connection vpnrouter.FIRST_CONNECTION of site_to_site; the local
|
||||
subnets fill both its local_cidrs and local.cidrs.
|
||||
"""
|
||||
|
||||
import json
|
||||
@@ -29,27 +31,29 @@ sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
|
||||
|
||||
import vpnrouter
|
||||
|
||||
# environment variable -> (section, option)
|
||||
FIRST = ('site_to_site', vpnrouter.FIRST_CONNECTION)
|
||||
|
||||
# environment variable -> the settings it fills, each (section, option) or (site_to_site, name, option)
|
||||
ANSWERS = {
|
||||
'VPN_ROUTER_PLATFORM': ('general', 'platform'),
|
||||
'VPN_ROUTER_MODE': ('general', 'mode'),
|
||||
'VPN_ROUTER_INT_ADDR': ('local', 'int_addr'),
|
||||
'VPN_ROUTER_EXTERNAL_INTERFACE': ('interfaces', 'external'),
|
||||
'VPN_ROUTER_INTERNAL_INTERFACE': ('interfaces', 'internal'),
|
||||
'VPN_ROUTER_LOCAL_FQDN': ('wan', 'local_fqdn'),
|
||||
'VPN_ROUTER_LOCAL_ID_MODE': ('wan', 'local_id_mode'),
|
||||
'VPN_ROUTER_LOCAL_CIDRS': ('local', 'cidrs'),
|
||||
'VPN_ROUTER_INT_GATEWAY_IP': ('local', 'int_gateway_ip'),
|
||||
'VPN_ROUTER_REMOTE_ADDRS': ('remote', 'addrs'),
|
||||
'VPN_ROUTER_REMOTE_ID': ('remote', 'id'),
|
||||
'VPN_ROUTER_REMOTE_CIDRS': ('remote', 'cidrs'),
|
||||
'VPN_ROUTER_PSK': ('remote', 'psk'),
|
||||
'VPN_ROUTER_P2S_ENABLED': ('p2s', 'enabled'),
|
||||
'VPN_ROUTER_P2S_ADDRESS_POOL': ('p2s', 'address_pool'),
|
||||
'VPN_ROUTER_P2S_CA_NAME': ('p2s', 'ca_name'),
|
||||
'VPN_ROUTER_WG_ENABLED': ('wireguard', 'enabled'),
|
||||
'VPN_ROUTER_WG_ADDRESS': ('wireguard', 'address'),
|
||||
'VPN_ROUTER_WG_LISTEN_PORT': ('wireguard', 'listen_port'),
|
||||
'VPN_ROUTER_PLATFORM': (('general', 'platform'),),
|
||||
'VPN_ROUTER_MODE': (('general', 'mode'),),
|
||||
'VPN_ROUTER_INT_ADDR': (('local', 'int_addr'),),
|
||||
'VPN_ROUTER_EXTERNAL_INTERFACE': (('interfaces', 'external'),),
|
||||
'VPN_ROUTER_INTERNAL_INTERFACE': (('interfaces', 'internal'),),
|
||||
'VPN_ROUTER_LOCAL_FQDN': (('wan', 'local_fqdn'),),
|
||||
'VPN_ROUTER_LOCAL_ID_MODE': ((*FIRST, 'local_id_mode'),),
|
||||
'VPN_ROUTER_LOCAL_CIDRS': (('local', 'cidrs'), (*FIRST, 'local_cidrs')),
|
||||
'VPN_ROUTER_INT_GATEWAY_IP': (('local', 'int_gateway_ip'),),
|
||||
'VPN_ROUTER_REMOTE_ADDRS': ((*FIRST, 'addrs'),),
|
||||
'VPN_ROUTER_REMOTE_ID': ((*FIRST, 'id'),),
|
||||
'VPN_ROUTER_REMOTE_CIDRS': ((*FIRST, 'cidrs'),),
|
||||
'VPN_ROUTER_PSK': ((*FIRST, 'psk'),),
|
||||
'VPN_ROUTER_P2S_ENABLED': (('p2s', 'enabled'),),
|
||||
'VPN_ROUTER_P2S_ADDRESS_POOL': (('p2s', 'address_pool'),),
|
||||
'VPN_ROUTER_P2S_CA_NAME': (('p2s', 'ca_name'),),
|
||||
'VPN_ROUTER_WG_ENABLED': (('wireguard', 'enabled'),),
|
||||
'VPN_ROUTER_WG_ADDRESS': (('wireguard', 'address'),),
|
||||
'VPN_ROUTER_WG_LISTEN_PORT': (('wireguard', 'listen_port'),),
|
||||
}
|
||||
|
||||
|
||||
@@ -74,23 +78,28 @@ def main():
|
||||
vpnrouter.fail(f'cannot parse {vpnrouter.CONFIG_FILE}: the top level is not an object')
|
||||
|
||||
answered = False
|
||||
for variable, (section, option) in ANSWERS.items():
|
||||
value = os.environ.get(variable)
|
||||
if not value:
|
||||
for variable, paths in ANSWERS.items():
|
||||
answer = os.environ.get(variable)
|
||||
if not answer:
|
||||
continue
|
||||
# A value that does not convert is kept as a string for validate() to report.
|
||||
default = vpnrouter.SCHEMA[section][option]
|
||||
if isinstance(default, bool):
|
||||
value = value == 'true'
|
||||
elif isinstance(default, int):
|
||||
value = int(value) if value.isdigit() else value
|
||||
elif isinstance(default, list):
|
||||
value = [item.strip() for item in value.split(',') if item.strip()]
|
||||
if not isinstance(cfg.get(section, {}), dict):
|
||||
vpnrouter.fail(f'cannot parse {vpnrouter.CONFIG_FILE}: {section} is not an object')
|
||||
if cfg.setdefault(section, {}).get(option) != value:
|
||||
cfg[section][option] = value
|
||||
answered = True
|
||||
for path in paths:
|
||||
# A value that does not convert is kept as a string for validate() to report.
|
||||
default = vpnrouter.CONNECTION[path[2]] if path[0] == 'site_to_site' else vpnrouter.SCHEMA[path[0]][path[1]]
|
||||
value = answer
|
||||
if isinstance(default, bool):
|
||||
value = value == 'true'
|
||||
elif isinstance(default, int):
|
||||
value = int(value) if value.isdigit() else value
|
||||
elif isinstance(default, list):
|
||||
value = [item.strip() for item in value.split(',') if item.strip()]
|
||||
target = cfg
|
||||
for key in path[:-1]:
|
||||
if not isinstance(target.setdefault(key, {}), dict):
|
||||
vpnrouter.fail(f'cannot parse {vpnrouter.CONFIG_FILE}: {".".join(path[:-1])} is not an object')
|
||||
target = target[key]
|
||||
if target.get(path[-1]) != value:
|
||||
target[path[-1]] = value
|
||||
answered = True
|
||||
|
||||
# Without a change the candidate is the current file byte for byte, as ucf compares checksums.
|
||||
with open(output, 'w', encoding='utf-8') as handle:
|
||||
|
||||
+456
@@ -0,0 +1,456 @@
|
||||
#!/usr/bin/python3
|
||||
"""Collect the metrics of the router, keep them in PostgreSQL, serve them for scraping and push them with OTLP.
|
||||
|
||||
Run by vpn-router-metrics.service while metrics.enabled is true; vpn-router-setup enables and
|
||||
disables the unit.
|
||||
|
||||
Every SAMPLE_SECONDS, on the minute, a sample is taken with psutil: the CPU utilization since the
|
||||
previous sample, the memory used and in total, the free and the total space of /, the bytes
|
||||
received and sent per second on the external and the internal interface, and the load averages;
|
||||
and the VPN clients of each enabled VPN system (vpnrouter_status): road-warrior (p2s) and
|
||||
OpenVPN clients and WireGuard peers with a recent handshake; and of each site-to-site connection
|
||||
that is set up, whether it is up, the bytes and packets per second through its child SAs since
|
||||
the previous sample, and how long its IKE SA has been up. The interfaces, the enabled systems and
|
||||
the connections are those vpn-router-setup resolved, read from /run/vpn-router/web.json.
|
||||
|
||||
The sample is written to the detail tables of vpnrouter_metrics. When the clock has passed into
|
||||
a new hour, day, month or year, the complete periods are summarized, and once a day, after the
|
||||
daily summary, the rows older than their retention are deleted; both also run on start. A
|
||||
failure of the database is logged and tried again with the next sample.
|
||||
|
||||
GET /metrics answers the last sample in the Prometheus text format, written by
|
||||
prometheus_client, with OpenTelemetry semantic convention names in their Prometheus form, for an
|
||||
OpenTelemetry Collector's prometheus receiver. It needs HTTP Basic authentication as the service
|
||||
account metrics.service_account names, which vpn-router-setup keeps with the password in
|
||||
/etc/vpn-router/metrics.secret, checked with a bind to the router's directory, so failed attempts
|
||||
lock the account as for any user. The service listens on the interfaces metrics.listen
|
||||
names and on the loopback address, where Caddy serves it over HTTPS at metrics.https_port; the
|
||||
address of a call through Caddy is the last entry of its X-Forwarded-For header.
|
||||
|
||||
When metrics.otlp_endpoint is set, every sample is also pushed there as an OTLP/HTTP JSON
|
||||
ExportMetricsServiceRequest, with HTTP Basic authentication when metrics.otlp_username is set.
|
||||
"""
|
||||
|
||||
import base64
|
||||
import datetime
|
||||
import json
|
||||
import os
|
||||
import signal
|
||||
import socket
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from http.server import BaseHTTPRequestHandler
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
|
||||
|
||||
import ldap
|
||||
import psutil
|
||||
from prometheus_client import CONTENT_TYPE_LATEST, generate_latest
|
||||
from prometheus_client.core import CollectorRegistry, CounterMetricFamily, GaugeMetricFamily
|
||||
|
||||
import vpnrouter
|
||||
import vpnrouter_metrics
|
||||
import vpnrouter_net
|
||||
import vpnrouter_serve
|
||||
import vpnrouter_status
|
||||
import vpnrouter_users
|
||||
|
||||
STATE_FILE = '/run/vpn-router/web.json'
|
||||
CONFIG_FILE = '/etc/vpn-router/vpn-router.json'
|
||||
|
||||
SAMPLE_SECONDS = 60
|
||||
|
||||
#: Seconds between two discoveries of where the service should listen.
|
||||
DISCOVERY_SECONDS = 30
|
||||
|
||||
#: Seconds an OTLP push may take.
|
||||
OTLP_TIMEOUT = 10
|
||||
|
||||
#: The mount point whose space is measured.
|
||||
MOUNTPOINT = '/'
|
||||
|
||||
#: The last sample, as sample() makes it, or None before the first.
|
||||
_latest = {'sample': None}
|
||||
_latest_lock = threading.Lock()
|
||||
|
||||
|
||||
def log(message):
|
||||
print(f'vpn-router-metrics: {message}', file=sys.stderr, flush=True)
|
||||
|
||||
|
||||
def read_json(path):
|
||||
"""The object in a JSON file, or {} when the file is missing or holds no object."""
|
||||
try:
|
||||
with open(path, encoding='utf-8') as handle:
|
||||
data = json.load(handle)
|
||||
except (OSError, ValueError):
|
||||
return {}
|
||||
return data if isinstance(data, dict) else {}
|
||||
|
||||
|
||||
def section(configuration, name):
|
||||
"""A section of the configuration, or {} when it is missing or not an object."""
|
||||
value = configuration.get(name)
|
||||
return value if isinstance(value, dict) else {}
|
||||
|
||||
|
||||
def clients(state):
|
||||
"""The VPN clients of each enabled VPN system, None where it cannot be asked."""
|
||||
counts = {}
|
||||
if state.get('p2s'):
|
||||
counts['p2s'] = vpnrouter_status.established('road-warrior')
|
||||
if state.get('openvpn'):
|
||||
counts['openvpn'] = vpnrouter_status.openvpn_clients()
|
||||
if state.get('wireguard'):
|
||||
counts['wireguard'] = vpnrouter_status.wireguard_connected()
|
||||
return counts
|
||||
|
||||
|
||||
def sample(previous):
|
||||
"""A sample of the router, with the rates since the previous sample, None for the first."""
|
||||
state = read_json(STATE_FILE)
|
||||
network = section(state, 'network')
|
||||
names = [name for name in dict.fromkeys((network.get('external_interface'), network.get('internal_interface')))
|
||||
if isinstance(name, str) and name]
|
||||
counters = psutil.net_io_counters(pernic=True)
|
||||
memory = psutil.virtual_memory()
|
||||
disk = psutil.disk_usage(MOUNTPOINT)
|
||||
load1, load5, load15 = os.getloadavg()
|
||||
connections = state.get('site_to_site')
|
||||
tunnels = vpnrouter_status.site_to_site_tunnels(connections) if isinstance(connections, list) and connections else {}
|
||||
now = time.time()
|
||||
current = {
|
||||
'time': now,
|
||||
'monotonic': time.monotonic(),
|
||||
'system': {
|
||||
'cpu': psutil.cpu_percent(interval=None),
|
||||
'memory_used': memory.total - memory.available,
|
||||
'memory_total': memory.total,
|
||||
'disk_free': disk.free,
|
||||
'disk_total': disk.total,
|
||||
'load1': load1,
|
||||
'load5': load5,
|
||||
'load15': load15,
|
||||
},
|
||||
'counters': {name: (counters[name].bytes_recv, counters[name].bytes_sent) for name in names if name in counters},
|
||||
'clients': clients(state),
|
||||
'rates': {},
|
||||
'tunnels': tunnels or {},
|
||||
'site_to_site': {},
|
||||
}
|
||||
elapsed = current['monotonic'] - previous['monotonic'] if previous is not None else 0
|
||||
if elapsed > 0:
|
||||
for name, (received, sent) in current['counters'].items():
|
||||
if name in previous['counters']:
|
||||
before_received, before_sent = previous['counters'][name]
|
||||
current['rates'][name] = (max(received - before_received, 0) / elapsed, max(sent - before_sent, 0) / elapsed)
|
||||
# A connection's rates need its counters of the previous sample; a counter that went down was rekeyed and counts from zero.
|
||||
for name, tunnel in current['tunnels'].items():
|
||||
before = previous['tunnels'].get(name) if elapsed > 0 else None
|
||||
values = {'up': tunnel['up'], 'uptime': tunnel['uptime']}
|
||||
for value, counter in (('rx', 'bytes_in'), ('tx', 'bytes_out'), ('packets_rx', 'packets_in'), ('packets_tx', 'packets_out')):
|
||||
if before is None:
|
||||
values[value] = None
|
||||
else:
|
||||
count = tunnel[counter]
|
||||
values[value] = (count - before[counter] if count >= before[counter] else count) / elapsed
|
||||
current['site_to_site'][name] = values
|
||||
return current
|
||||
|
||||
|
||||
def periods(moment):
|
||||
"""The hour, the day, the month and the year of a UTC moment, keyed by the levels of vpnrouter_metrics.LEVELS."""
|
||||
return {'hourly': moment.strftime('%Y-%m-%dT%H'), 'daily': moment.strftime('%Y-%m-%d'),
|
||||
'monthly': moment.strftime('%Y-%m'), 'yearly': moment.strftime('%Y')}
|
||||
|
||||
|
||||
def store(cfg, current, done):
|
||||
"""Write the sample, summarize the periods the clock has passed and collect the garbage once a day.
|
||||
|
||||
done holds the periods summarized so far, and garbage the day of the last garbage collection;
|
||||
it is updated only for what succeeded.
|
||||
"""
|
||||
now = periods(datetime.datetime.now(datetime.timezone.utc))
|
||||
with vpnrouter_metrics.database(cfg) as connection:
|
||||
vpnrouter_metrics.write(connection, current['system'], current['rates'], current['clients'], current['site_to_site'])
|
||||
for level, _, _ in vpnrouter_metrics.LEVELS:
|
||||
if done.get(level) != now[level]:
|
||||
vpnrouter_metrics.summarize(connection, level)
|
||||
done[level] = now[level]
|
||||
if done.get('garbage') != now['daily']:
|
||||
vpnrouter_metrics.collect_garbage(connection, cfg)
|
||||
done['garbage'] = now['daily']
|
||||
|
||||
|
||||
def attributes(**values):
|
||||
"""OTLP attributes of string values."""
|
||||
return [{'key': key, 'value': {'stringValue': value}} for key, value in values.items()]
|
||||
|
||||
|
||||
def otlp_request(current, started):
|
||||
"""The OTLP/HTTP JSON ExportMetricsServiceRequest of a sample; started is when the counters were first read, in Unix seconds."""
|
||||
stamp = str(int(current['time'] * 1e9))
|
||||
start = str(int(started * 1e9))
|
||||
system = current['system']
|
||||
|
||||
def gauge(name, unit, points):
|
||||
return {'name': name, 'unit': unit, 'gauge': {'dataPoints': [
|
||||
{'timeUnixNano': stamp, 'asDouble': float(value), 'attributes': attributes(**labels)}
|
||||
for value, labels in points if value is not None]}}
|
||||
|
||||
metrics = [
|
||||
gauge('system.cpu.utilization', '1', [(system['cpu'] / 100, {})]),
|
||||
gauge('system.memory.usage', 'By', [(system['memory_used'], {'state': 'used'})]),
|
||||
gauge('system.memory.limit', 'By', [(system['memory_total'], {})]),
|
||||
gauge('system.filesystem.usage', 'By', [(system['disk_free'], {'mountpoint': MOUNTPOINT, 'state': 'free'}),
|
||||
(system['disk_total'] - system['disk_free'], {'mountpoint': MOUNTPOINT, 'state': 'used'})]),
|
||||
gauge('system.filesystem.utilization', '1',
|
||||
[(1 - system['disk_free'] / system['disk_total'] if system['disk_total'] else None, {'mountpoint': MOUNTPOINT})]),
|
||||
gauge('system.cpu.load_average.1m', '{thread}', [(system['load1'], {})]),
|
||||
gauge('system.cpu.load_average.5m', '{thread}', [(system['load5'], {})]),
|
||||
gauge('system.cpu.load_average.15m', '{thread}', [(system['load15'], {})]),
|
||||
gauge('vpn_router.clients', '{client}', [(count, {'kind': kind}) for kind, count in current['clients'].items()]),
|
||||
gauge('vpn_router.site_to_site.up', '1', [(values['up'], {'connection': name})
|
||||
for name, values in current['site_to_site'].items()]),
|
||||
gauge('vpn_router.site_to_site.uptime', 's', [(values['uptime'], {'connection': name})
|
||||
for name, values in current['site_to_site'].items()]),
|
||||
gauge('vpn_router.site_to_site.io', 'By/s', [(values[value], {'connection': name, 'direction': direction})
|
||||
for name, values in current['site_to_site'].items()
|
||||
for value, direction in (('rx', 'receive'), ('tx', 'transmit'))]),
|
||||
gauge('vpn_router.site_to_site.packets', '{packet}/s',
|
||||
[(values[value], {'connection': name, 'direction': direction})
|
||||
for name, values in current['site_to_site'].items()
|
||||
for value, direction in (('packets_rx', 'receive'), ('packets_tx', 'transmit'))]),
|
||||
{'name': 'system.network.io', 'unit': 'By', 'sum': {
|
||||
'aggregationTemporality': 2, 'isMonotonic': True, 'dataPoints': [
|
||||
{'startTimeUnixNano': start, 'timeUnixNano': stamp, 'asInt': str(value),
|
||||
'attributes': attributes(device=name, direction=direction)}
|
||||
for name, counters in current['counters'].items()
|
||||
for value, direction in zip(counters, ('receive', 'transmit'))]}},
|
||||
]
|
||||
return {'resourceMetrics': [{
|
||||
'resource': {'attributes': attributes(**{'service.name': 'vpn-router', 'host.name': socket.gethostname()})},
|
||||
'scopeMetrics': [{'scope': {'name': 'vpn-router-metrics'}, 'metrics': metrics}],
|
||||
}]}
|
||||
|
||||
|
||||
def push(metrics, current, started):
|
||||
"""Push the sample to metrics.otlp_endpoint; raises OSError when the collector does not take it."""
|
||||
request = urllib.request.Request(metrics['otlp_endpoint'], data=json.dumps(otlp_request(current, started)).encode(),
|
||||
headers={'Content-Type': 'application/json'}, method='POST')
|
||||
if metrics['otlp_username']:
|
||||
credentials = f"{metrics['otlp_username']}:{metrics['otlp_password']}".encode()
|
||||
request.add_header('Authorization', 'Basic ' + base64.b64encode(credentials).decode())
|
||||
with urllib.request.urlopen(request, timeout=OTLP_TIMEOUT) as response:
|
||||
response.read()
|
||||
|
||||
|
||||
def collector_loop():
|
||||
"""Take a sample on every minute, store it and push it; every new failure is logged once."""
|
||||
previous, done, reported = None, {}, {}
|
||||
started = time.time()
|
||||
psutil.cpu_percent(interval=None)
|
||||
|
||||
def report(kind, message):
|
||||
if message != reported.get(kind):
|
||||
if message:
|
||||
log(message)
|
||||
elif reported.get(kind):
|
||||
log(f'{kind} works again')
|
||||
reported[kind] = message
|
||||
|
||||
while True:
|
||||
time.sleep(SAMPLE_SECONDS - time.time() % SAMPLE_SECONDS)
|
||||
try:
|
||||
cfg = vpnrouter.load_config()
|
||||
except SystemExit as exc:
|
||||
report('configuration', str(exc.code))
|
||||
continue
|
||||
report('configuration', '')
|
||||
current = sample(previous)
|
||||
previous = current
|
||||
with _latest_lock:
|
||||
_latest['sample'] = current
|
||||
try:
|
||||
store(cfg, current, done)
|
||||
report('database', '')
|
||||
except vpnrouter_metrics.DatabaseError as exc:
|
||||
report('database', str(exc))
|
||||
metrics = cfg['metrics']
|
||||
if metrics['otlp_endpoint']:
|
||||
try:
|
||||
push(metrics, current, started)
|
||||
report('OTLP push', '')
|
||||
except (OSError, ValueError) as exc:
|
||||
report('OTLP push', f"pushing to {metrics['otlp_endpoint']} failed: {exc}")
|
||||
|
||||
|
||||
class Sample:
|
||||
"""A prometheus_client collector of a sample."""
|
||||
|
||||
def __init__(self, current):
|
||||
self.current = current
|
||||
|
||||
def collect(self):
|
||||
system = self.current['system']
|
||||
yield GaugeMetricFamily('system_cpu_utilization', 'CPU utilization since the previous sample, 0 to 1',
|
||||
value=system['cpu'] / 100)
|
||||
memory = GaugeMetricFamily('system_memory_usage_bytes', 'Memory in use', labels=['state'])
|
||||
memory.add_metric(['used'], system['memory_used'])
|
||||
yield memory
|
||||
yield GaugeMetricFamily('system_memory_limit_bytes', 'Total memory', value=system['memory_total'])
|
||||
usage = GaugeMetricFamily('system_filesystem_usage_bytes', 'Space of the file system', labels=['mountpoint', 'state'])
|
||||
usage.add_metric([MOUNTPOINT, 'free'], system['disk_free'])
|
||||
usage.add_metric([MOUNTPOINT, 'used'], system['disk_total'] - system['disk_free'])
|
||||
yield usage
|
||||
if system['disk_total']:
|
||||
utilization = GaugeMetricFamily('system_filesystem_utilization', 'Used part of the file system, 0 to 1',
|
||||
labels=['mountpoint'])
|
||||
utilization.add_metric([MOUNTPOINT], 1 - system['disk_free'] / system['disk_total'])
|
||||
yield utilization
|
||||
for period in ('1m', '5m', '15m'):
|
||||
yield GaugeMetricFamily(f'system_cpu_load_average_{period}', f'Load average over {period}',
|
||||
value=system[f'load{period[:-1]}'])
|
||||
network = CounterMetricFamily('system_network_io_bytes', 'Bytes received and sent', labels=['device', 'direction'])
|
||||
for name, (received, sent) in self.current['counters'].items():
|
||||
network.add_metric([name, 'receive'], received)
|
||||
network.add_metric([name, 'transmit'], sent)
|
||||
yield network
|
||||
vpn = GaugeMetricFamily('vpn_router_clients', 'Connected VPN clients', labels=['kind'])
|
||||
for kind, count in self.current['clients'].items():
|
||||
if count is not None:
|
||||
vpn.add_metric([kind], count)
|
||||
yield vpn
|
||||
tunnels = self.current['site_to_site']
|
||||
up = GaugeMetricFamily('vpn_router_site_to_site_up', 'Whether the site-to-site connection is established, 1 or 0',
|
||||
labels=['connection'])
|
||||
uptime = GaugeMetricFamily('vpn_router_site_to_site_uptime_seconds', 'Seconds the IKE SA of the site-to-site connection has been up',
|
||||
labels=['connection'])
|
||||
io = GaugeMetricFamily('vpn_router_site_to_site_io_bytes_per_second',
|
||||
'Bytes per second received and sent through the site-to-site connection', labels=['connection', 'direction'])
|
||||
packets = GaugeMetricFamily('vpn_router_site_to_site_packets_per_second',
|
||||
'Packets per second received and sent through the site-to-site connection',
|
||||
labels=['connection', 'direction'])
|
||||
for name, values in tunnels.items():
|
||||
up.add_metric([name], values['up'])
|
||||
if values['uptime'] is not None:
|
||||
uptime.add_metric([name], values['uptime'])
|
||||
for family, received, sent in ((io, 'rx', 'tx'), (packets, 'packets_rx', 'packets_tx')):
|
||||
for value, direction in ((received, 'receive'), (sent, 'transmit')):
|
||||
if values[value] is not None:
|
||||
family.add_metric([name, direction], values[value])
|
||||
yield from (up, uptime, io, packets)
|
||||
|
||||
|
||||
def authorized(header, account):
|
||||
"""Whether an Authorization header holds the password of the service account; LDAP errors propagate."""
|
||||
if not header.startswith('Basic '):
|
||||
return False
|
||||
try:
|
||||
name, _, password = base64.b64decode(header[6:].strip(), validate=True).decode().partition(':')
|
||||
except ValueError:
|
||||
return False
|
||||
if name != account:
|
||||
return False
|
||||
try:
|
||||
with vpnrouter_users.session() as connection:
|
||||
if not vpnrouter_users.service(vpnrouter_users.entry(connection, name)):
|
||||
return False
|
||||
vpnrouter_users.login(name, password, None)
|
||||
except vpnrouter_users.Failure:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
protocol_version = 'HTTP/1.1'
|
||||
|
||||
def client(self):
|
||||
"""The client address: the last X-Forwarded-For entry of a call through Caddy on the loopback address, else the peer."""
|
||||
peer = self.client_address[0]
|
||||
forwarded = self.headers.get('X-Forwarded-For', '')
|
||||
if peer == vpnrouter_net.LOOPBACK and forwarded.strip():
|
||||
return forwarded.split(',')[-1].strip()
|
||||
return peer
|
||||
|
||||
def answer(self, send_body):
|
||||
self.principal = None
|
||||
extra = []
|
||||
path = self.path.split('?', 1)[0]
|
||||
account = str(section(read_json(CONFIG_FILE), 'metrics').get('service_account',
|
||||
vpnrouter.SCHEMA['metrics']['service_account'])).strip()
|
||||
with _latest_lock:
|
||||
current = _latest['sample']
|
||||
if path != '/metrics':
|
||||
body, kind, code = b'Not found\n', 'text/plain; charset=utf-8', 404
|
||||
elif not account:
|
||||
body, kind, code = b'metrics.service_account is not set\n', 'text/plain; charset=utf-8', 503
|
||||
else:
|
||||
try:
|
||||
allowed = authorized(self.headers.get('Authorization', ''), account)
|
||||
except (ldap.LDAPError, OSError) as exc:
|
||||
allowed = None
|
||||
body, kind, code = f'the user directory failed: {exc}\n'.encode(), 'text/plain; charset=utf-8', 500
|
||||
if allowed is False:
|
||||
body, kind, code = b'Unauthorized\n', 'text/plain; charset=utf-8', 401
|
||||
extra.append(('WWW-Authenticate', 'Basic realm="vpn-router metrics", charset="UTF-8"'))
|
||||
elif allowed:
|
||||
self.principal = account
|
||||
if current is None:
|
||||
body, kind, code = b'no sample yet\n', 'text/plain; charset=utf-8', 503
|
||||
else:
|
||||
registry = CollectorRegistry()
|
||||
registry.register(Sample(current))
|
||||
body, kind, code = generate_latest(registry), CONTENT_TYPE_LATEST, 200
|
||||
self.send_response(code)
|
||||
self.send_header('Content-Type', kind)
|
||||
self.send_header('Content-Length', str(len(body)))
|
||||
self.send_header('Cache-Control', 'no-store')
|
||||
for name, value in extra:
|
||||
self.send_header(name, value)
|
||||
self.end_headers()
|
||||
if send_body:
|
||||
self.wfile.write(body)
|
||||
|
||||
def do_GET(self):
|
||||
self.answer(True)
|
||||
|
||||
def do_HEAD(self):
|
||||
self.answer(False)
|
||||
|
||||
def log_message(self, format, *args):
|
||||
principal = getattr(self, 'principal', None) or '-'
|
||||
sys.stderr.write('%s - %s [%s] %s\n' % (self.client(), principal, self.log_date_time_string(), format % args))
|
||||
|
||||
|
||||
def wanted():
|
||||
"""Where the service listens: the addresses of the interfaces metrics.listen names and LOOPBACK, at metrics.port."""
|
||||
configuration = read_json(CONFIG_FILE)
|
||||
metrics = section(configuration, 'metrics')
|
||||
defaults = vpnrouter.SCHEMA['metrics']
|
||||
if metrics.get('enabled', defaults['enabled']) is False:
|
||||
return [], 'metrics are disabled in the configuration'
|
||||
port = metrics.get('port', defaults['port'])
|
||||
if isinstance(port, bool) or not isinstance(port, int) or not 1 <= port <= 65535:
|
||||
port = defaults['port']
|
||||
addresses = vpnrouter_net.listen_addresses(configuration, metrics.get('listen', defaults['listen']))
|
||||
if not addresses:
|
||||
return [], 'the addresses to listen on are not known yet'
|
||||
return [(address, port) for address in [*addresses, vpnrouter_net.LOOPBACK]], ''
|
||||
|
||||
|
||||
def main():
|
||||
signal.signal(signal.SIGTERM, lambda *_: sys.exit(0))
|
||||
threading.Thread(target=collector_loop, name='collector', daemon=True).start()
|
||||
try:
|
||||
vpnrouter_serve.serve(wanted, Handler, log, DISCOVERY_SECONDS)
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
@@ -16,10 +16,12 @@ firewall and NAT rules are written against them, so choosing the wrong one
|
||||
would be a silent and security-relevant mistake.
|
||||
"""
|
||||
|
||||
import grp
|
||||
import json
|
||||
import os
|
||||
import pathlib
|
||||
import re
|
||||
import secrets
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
@@ -27,8 +29,11 @@ import time
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
|
||||
|
||||
import ldap
|
||||
|
||||
import vpnrouter
|
||||
import vpnrouter_ldap
|
||||
import vpnrouter_users
|
||||
from vpnrouter import fail
|
||||
|
||||
BEFORE_RULES = pathlib.Path('/etc/ufw/before.rules')
|
||||
@@ -49,8 +54,21 @@ BOOTSTRAP_PORTS = (17080, 17081)
|
||||
#: The Router Manager, vpn-router-manager, when it is installed, and its settings.
|
||||
MANAGER = pathlib.Path('/usr/lib/vpn-router-manager/manager')
|
||||
MANAGER_CONFIG = pathlib.Path('/etc/vpn-router-manager/manager.json')
|
||||
|
||||
#: The default admin of the Router Manager, {"username", "password"}, which its package writes from
|
||||
#: its debconf answers; setup creates the account and removes the file.
|
||||
MANAGER_ADMIN = pathlib.Path('/etc/vpn-router-manager/admin.json')
|
||||
MANAGER_PORT = 7080
|
||||
|
||||
#: The password of the metrics service account, which setup makes and keeps (root only).
|
||||
METRICS_SECRET = pathlib.Path('/etc/vpn-router/metrics.secret')
|
||||
|
||||
#: The configuration of Caddy, the HTTPS proxy, which the package's drop-in for caddy.service names;
|
||||
#: Caddy runs as the user and group CADDY_GROUP, which the caddy package creates.
|
||||
CADDYFILE = pathlib.Path('/etc/caddy/vpn-router.Caddyfile')
|
||||
CADDY_UNIT = 'caddy.service'
|
||||
CADDY_GROUP = 'caddy'
|
||||
|
||||
FILTER_ANCHOR = '# End required lines'
|
||||
|
||||
#: Configuration options left out wherever the configuration is shown.
|
||||
@@ -64,6 +82,8 @@ BLOCKS = (
|
||||
('# WEB RULES START', '# WEB RULES END'),
|
||||
('# BOOTSTRAP RULES START', '# BOOTSTRAP RULES END'),
|
||||
('# MANAGER RULES START', '# MANAGER RULES END'),
|
||||
('# HTTPS RULES START', '# HTTPS RULES END'),
|
||||
('# METRICS RULES START', '# METRICS RULES END'),
|
||||
('# ROUTER FORWARD RULES START', '# ROUTER FORWARD RULES END'),
|
||||
('# ROUTER NAT RULES START', '# ROUTER NAT RULES END'),
|
||||
('# ROUTER MSS RULES START', '# ROUTER MSS RULES END'),
|
||||
@@ -208,11 +228,13 @@ def filter_block(ctx):
|
||||
]
|
||||
lines += ['# P2S DNS RULES END']
|
||||
|
||||
if ctx['web_enabled'] and ctx['int_iface']:
|
||||
# The web service, the Router Manager and the metrics service choose their interfaces themselves, so their ports are open on both.
|
||||
interfaces = [name for name in (ctx['int_iface'], ctx['wan_iface']) if name]
|
||||
if ctx['web_enabled'] and interfaces:
|
||||
lines += [
|
||||
'',
|
||||
'# WEB RULES START',
|
||||
f'-A ufw-before-input -i {ctx["int_iface"]} -p tcp --dport {ctx["web_port"]} -j ACCEPT',
|
||||
*(f'-A ufw-before-input -i {name} -p tcp --dport {ctx["web_port"]} -j ACCEPT' for name in interfaces),
|
||||
'# WEB RULES END',
|
||||
]
|
||||
|
||||
@@ -224,15 +246,37 @@ def filter_block(ctx):
|
||||
'# BOOTSTRAP RULES END',
|
||||
]
|
||||
|
||||
if MANAGER.exists() and ctx['int_iface']:
|
||||
if MANAGER.exists() and interfaces:
|
||||
port = manager_port()
|
||||
lines += [
|
||||
'',
|
||||
'# MANAGER RULES START',
|
||||
f'-A ufw-before-input -i {ctx["int_iface"]} -p tcp --dport {port} -j ACCEPT',
|
||||
*(f'-A ufw-before-input -i {name} -p tcp --dport {port} -j ACCEPT' for name in interfaces),
|
||||
'# MANAGER RULES END',
|
||||
]
|
||||
|
||||
if vpnrouter.metrics_ready(ctx) and interfaces:
|
||||
lines += [
|
||||
'',
|
||||
'# METRICS RULES START',
|
||||
*(f'-A ufw-before-input -i {name} -p tcp --dport {ctx["metrics_port"]} -j ACCEPT' for name in interfaces),
|
||||
'# METRICS RULES END',
|
||||
]
|
||||
|
||||
# Let's Encrypt reaches the HTTP challenge on the external interface; the HTTPS ports are open on both.
|
||||
if vpnrouter.https_ready(ctx) and interfaces:
|
||||
ports = [port for wanted, port in ((ctx['web_enabled'], ctx['https_web_port']),
|
||||
(MANAGER.exists(), ctx['https_manager_port']),
|
||||
(vpnrouter.metrics_ready(ctx), ctx['metrics_https_port'])) if wanted]
|
||||
lines += [
|
||||
'',
|
||||
'# HTTPS RULES START',
|
||||
*([f'-A ufw-before-input -i {ctx["wan_iface"]} -p tcp --dport {vpnrouter.ACME_HTTP_PORT} -j ACCEPT']
|
||||
if ctx['wan_iface'] else []),
|
||||
*(f'-A ufw-before-input -i {name} -p tcp --dport {port} -j ACCEPT' for name in interfaces for port in ports),
|
||||
'# HTTPS RULES END',
|
||||
]
|
||||
|
||||
if vpnrouter.routing_ready(ctx):
|
||||
lines += ['', '# ROUTER FORWARD RULES START']
|
||||
for subnet in ctx['local_cidr_list']:
|
||||
@@ -250,7 +294,6 @@ def nat_and_mangle_block(ctx):
|
||||
|
||||
if vpnrouter.routing_ready(ctx):
|
||||
subnets = ctx['local_cidr_list']
|
||||
remote = ctx['remote_cidr_list']
|
||||
lines += [
|
||||
'',
|
||||
'# ROUTER NAT RULES START',
|
||||
@@ -258,11 +301,14 @@ def nat_and_mangle_block(ctx):
|
||||
':POSTROUTING ACCEPT [0:0]',
|
||||
'-F POSTROUTING',
|
||||
]
|
||||
# Every RETURN precedes every MASQUERADE, so tunnel-bound traffic
|
||||
# escapes NAT whichever local subnet it came from.
|
||||
for subnet in subnets:
|
||||
for cidr in remote:
|
||||
lines.append(f'-A POSTROUTING -s {subnet} -d {cidr} -j RETURN')
|
||||
# Every RETURN precedes every MASQUERADE, so traffic bound for a tunnel
|
||||
# escapes NAT; each ready connection pairs its local and remote subnets.
|
||||
for connection in ctx['connections']:
|
||||
if not connection['ready']:
|
||||
continue
|
||||
for subnet in connection['local_cidr_list']:
|
||||
for cidr in connection['remote_cidr_list']:
|
||||
lines.append(f'-A POSTROUTING -s {subnet} -d {cidr} -j RETURN')
|
||||
for subnet in subnets:
|
||||
lines.append(
|
||||
f'-A POSTROUTING -s {subnet} -o {ctx["wan_iface"]} -j MASQUERADE')
|
||||
@@ -384,6 +430,86 @@ def manage_directory():
|
||||
return ''
|
||||
|
||||
|
||||
def manage_admin():
|
||||
"""Create the Router Manager's default admin from MANAGER_ADMIN, then remove the file; the problem, or ''.
|
||||
|
||||
The admin is a Manager-only account and a member of the admins group, which is created with it
|
||||
when it does not exist.
|
||||
"""
|
||||
if not MANAGER_ADMIN.exists():
|
||||
return ''
|
||||
try:
|
||||
answers = json.loads(MANAGER_ADMIN.read_text(encoding='utf-8'))
|
||||
name, password = answers['username'], answers['password']
|
||||
with vpnrouter_users.session() as connection:
|
||||
vpnrouter_users.add(connection, name, password, {})
|
||||
try:
|
||||
vpnrouter_users.group_member_add(connection, vpnrouter_users.ADMIN_GROUP, name)
|
||||
except vpnrouter_users.Failure as failure:
|
||||
if failure.status != 3:
|
||||
raise
|
||||
vpnrouter_users.group_add(connection, vpnrouter_users.ADMIN_GROUP, 'Router administrators', [name])
|
||||
MANAGER_ADMIN.unlink()
|
||||
except (OSError, ValueError, KeyError, TypeError, ldap.LDAPError, vpnrouter_users.Failure) as exc:
|
||||
return f'the Router Manager admin cannot be created from {MANAGER_ADMIN}: {exc}'
|
||||
return ''
|
||||
|
||||
|
||||
def manage_metrics_account(ctx):
|
||||
"""Keep the service account metrics.service_account names, whose password is in METRICS_SECRET; the problem, or ''.
|
||||
|
||||
While metrics are enabled, setup owns the account, an internal service account: it adds it as a service account
|
||||
with a random password and the labels of vpnrouter_users.internal_accounts when it is missing, sets those labels
|
||||
when they differ, and gives it a new random password, ending any lockout, when METRICS_SECRET is missing or does
|
||||
not hold its password. The password of a disabled account is left as it is. An account of that name that is not a
|
||||
service account is left alone and is a problem.
|
||||
"""
|
||||
if not vpnrouter.metrics_ready(ctx):
|
||||
return ''
|
||||
name = ctx['metrics_service_account']
|
||||
wanted = vpnrouter_users.internal_accounts().get(name, {})
|
||||
try:
|
||||
vpnrouter_users.check_name(name)
|
||||
try:
|
||||
password = METRICS_SECRET.read_text(encoding='utf-8').strip()
|
||||
except OSError:
|
||||
password = ''
|
||||
with vpnrouter_users.session() as connection:
|
||||
try:
|
||||
current = vpnrouter_users.entry(connection, name)
|
||||
except vpnrouter_users.Failure as failure:
|
||||
if failure.status != 3:
|
||||
raise
|
||||
current = None
|
||||
if current is not None and not vpnrouter_users.service(current):
|
||||
return f'metrics.service_account: {name} is a user, not a service account'
|
||||
if current is not None and any(vpnrouter_users.summary(current)[key] != value for key, value in wanted.items()):
|
||||
vpnrouter_users.update_service(connection, name, wanted)
|
||||
# An administrator who disabled the account meant it; its password is left as it is.
|
||||
if current is not None and not vpnrouter_users.summary(current)['enabled']:
|
||||
return ''
|
||||
valid = False
|
||||
if current is not None and password:
|
||||
try:
|
||||
vpnrouter_users.login(name, password, None)
|
||||
valid = True
|
||||
except vpnrouter_users.Failure:
|
||||
pass
|
||||
if not valid:
|
||||
password = secrets.token_urlsafe(32)
|
||||
METRICS_SECRET.touch(mode=0o600)
|
||||
METRICS_SECRET.chmod(0o600)
|
||||
METRICS_SECRET.write_text(password + '\n', encoding='utf-8')
|
||||
if current is None:
|
||||
vpnrouter_users.add(connection, name, password, wanted, service=True)
|
||||
else:
|
||||
vpnrouter_users.passwd(connection, name, password)
|
||||
vpnrouter_users.unlock(connection, name)
|
||||
except (OSError, ldap.LDAPError, vpnrouter_users.Failure) as exc:
|
||||
return f'the metrics service account cannot be kept: {exc}'
|
||||
return ''
|
||||
|
||||
|
||||
def manage_openvpn(ctx, changed):
|
||||
"""Run the OpenVPN server when it is configured, and stop it when it is not."""
|
||||
unit = 'openvpn-server@vpn-router'
|
||||
@@ -399,6 +525,47 @@ def manage_openvpn(ctx, changed):
|
||||
run(['systemctl', 'restart', unit], check=False)
|
||||
|
||||
|
||||
def manage_caddy(ctx):
|
||||
"""Run Caddy, the HTTPS proxy of the web service, the Router Manager and the metrics service, while HTTPS is ready, and stop it when not.
|
||||
|
||||
CADDYFILE serves the web service on https.web_port, while the manager is installed the
|
||||
manager on https.manager_port, and while metrics are enabled the metrics service on
|
||||
metrics.https_port, all for wan.local_fqdn with a certificate from Let's Encrypt, and all
|
||||
calling the service on its loopback address.
|
||||
"""
|
||||
if not vpnrouter.https_ready(ctx):
|
||||
vpnrouter.remove(CADDYFILE)
|
||||
if unit_is_active(CADDY_UNIT):
|
||||
run(['systemctl', 'disable', '--now', CADDY_UNIT], check=False)
|
||||
return
|
||||
|
||||
try:
|
||||
group = grp.getgrnam(CADDY_GROUP).gr_gid
|
||||
except KeyError:
|
||||
fail(f'the group {CADDY_GROUP} does not exist; is the caddy package installed?')
|
||||
caddy_ctx = dict(ctx, manager_port=manager_port() if MANAGER.exists() else 0)
|
||||
changed = vpnrouter.render(vpnrouter._jinja_env(), caddy_ctx, 'Caddyfile.j2', CADDYFILE, 0o640)
|
||||
os.chown(CADDYFILE, 0, group)
|
||||
if not unit_is_active(CADDY_UNIT):
|
||||
run(['systemctl', 'enable', '--now', CADDY_UNIT], check=False)
|
||||
elif changed:
|
||||
run(['systemctl', 'reload', CADDY_UNIT], check=False)
|
||||
|
||||
|
||||
def manage_metrics(ctx):
|
||||
"""Run the metrics service while metrics are enabled, and stop it when not.
|
||||
|
||||
The service reads its settings itself, so it is restarted only to run the code of a new package version, which
|
||||
every run of setup after an upgrade does.
|
||||
"""
|
||||
unit = 'vpn-router-metrics.service'
|
||||
if not vpnrouter.metrics_ready(ctx):
|
||||
run(['systemctl', 'disable', '--now', unit], check=False)
|
||||
return
|
||||
run(['systemctl', 'enable', unit], check=False)
|
||||
run(['systemctl', 'restart', unit], check=False)
|
||||
|
||||
|
||||
def run_platform_setup(cfg, ctx):
|
||||
"""Let the platform module apply its own state, if it defines any."""
|
||||
module = vpnrouter.platform_module(cfg)
|
||||
@@ -430,31 +597,37 @@ def refresh_web(cfg, ctx):
|
||||
the configuration without its secret options and the facts resolved from
|
||||
the system, which it cannot work out with the standard library alone. The
|
||||
file is readable by root only. The service follows a changed address or
|
||||
port itself.
|
||||
port itself. Secret options are left out at every level, so also from each
|
||||
site-to-site connection.
|
||||
"""
|
||||
public = {
|
||||
section: {option: value for option, value in options.items() if not SECRET_OPTION.search(option)}
|
||||
for section, options in cfg.items()
|
||||
}
|
||||
def public(options):
|
||||
return {option: public(value) if isinstance(value, dict) else value
|
||||
for option, value in options.items()
|
||||
if isinstance(value, dict) or not SECRET_OPTION.search(option)}
|
||||
|
||||
WEB_STATE.parent.mkdir(parents=True, exist_ok=True)
|
||||
WEB_STATE.touch(mode=0o600)
|
||||
WEB_STATE.chmod(0o600)
|
||||
WEB_STATE.write_text(json.dumps({
|
||||
'site_to_site': bool(vpnrouter.s2s_ready(ctx)),
|
||||
'site_to_site': [connection['name'] for connection in ctx['connections'] if connection['ready']],
|
||||
'p2s': bool(vpnrouter.p2s_ready(ctx)),
|
||||
'wireguard': bool(vpnrouter.wg_ready(ctx)),
|
||||
'openvpn': bool(vpnrouter.openvpn_ready(ctx)),
|
||||
'configuration': public,
|
||||
'configuration': public(cfg),
|
||||
'network': {
|
||||
'external_interface': ctx['wan_iface'],
|
||||
'internal_interface': ctx['int_iface'],
|
||||
'external_addresses': ctx['local_addr_list'],
|
||||
'internal_address': ctx['int_addr'],
|
||||
'internal_gateway': ctx['int_gateway_ip'],
|
||||
'local_identity': ctx['local_id'],
|
||||
'remote_identity': ctx['remote_id'],
|
||||
'local_cidrs': ctx['local_cidr_list'],
|
||||
'remote_cidrs': ctx['remote_cidr_list'],
|
||||
'site_to_site': [{
|
||||
'name': connection['name'],
|
||||
'local_identity': connection['local_id'],
|
||||
'remote_identity': connection['remote_id'],
|
||||
'local_cidrs': connection['local_cidr_list'],
|
||||
'remote_cidrs': connection['remote_cidr_list'],
|
||||
} for connection in ctx['connections']],
|
||||
},
|
||||
}) + '\n', encoding='utf-8')
|
||||
|
||||
@@ -505,10 +678,10 @@ def main():
|
||||
if vpnrouter.routing_ready(ctx):
|
||||
apply_routes(ctx)
|
||||
|
||||
if ctx['local_id_mode'] == 'public_ip' and ctx['local_fqdn']:
|
||||
if ctx['local_fqdn'] and any(connection['local_id_mode'] == 'public_ip' for connection in ctx['connections']):
|
||||
progress('running', 'public address')
|
||||
address = resolve_public_ip(ctx['local_fqdn'])
|
||||
ctx, more = vpnrouter.configure(cfg, local_id_override=address)
|
||||
ctx, more = vpnrouter.configure(cfg, public_ip=address)
|
||||
changed |= more
|
||||
|
||||
progress('running', 'WireGuard keys')
|
||||
@@ -532,11 +705,22 @@ def main():
|
||||
progress('running', 'WireGuard')
|
||||
manage_wireguard(ctx, changed)
|
||||
|
||||
# The tunnels are up before the directory, whose problem ends setup only after every other step.
|
||||
progress('running', 'user directory')
|
||||
# The tunnels are up before the directory and the admin, whose problem ends setup only after every other step.
|
||||
failed_step = 'user directory'
|
||||
progress('running', failed_step)
|
||||
problem = manage_directory()
|
||||
if not problem:
|
||||
failed_step = 'Router Manager admin'
|
||||
progress('running', failed_step)
|
||||
problem = manage_admin()
|
||||
if not problem:
|
||||
failed_step = 'metrics service account'
|
||||
progress('running', failed_step)
|
||||
problem = manage_metrics_account(ctx)
|
||||
progress('running', 'OpenVPN')
|
||||
manage_openvpn(ctx, changed)
|
||||
progress('running', 'HTTPS proxy')
|
||||
manage_caddy(ctx)
|
||||
|
||||
if 'resolved' in changed:
|
||||
progress('running', 'resolver')
|
||||
@@ -548,8 +732,10 @@ def main():
|
||||
refresh_docker_rules()
|
||||
progress('running', 'web service state')
|
||||
refresh_web(cfg, ctx)
|
||||
progress('running', 'metrics service')
|
||||
manage_metrics(ctx)
|
||||
if problem:
|
||||
progress('failed', 'user directory', problem)
|
||||
progress('failed', failed_step, problem)
|
||||
print(f'vpn-router: error: {problem}', file=sys.stderr)
|
||||
return 1
|
||||
progress('done', 'done')
|
||||
|
||||
@@ -4,6 +4,8 @@ Everything derivable from the configuration file alone, with no running
|
||||
network required. Used by both 'configure' and 'setup'.
|
||||
"""
|
||||
|
||||
import contextlib
|
||||
import fcntl
|
||||
import ipaddress
|
||||
import json
|
||||
import os
|
||||
@@ -12,6 +14,8 @@ import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import urllib.parse
|
||||
|
||||
import jinja2
|
||||
|
||||
@@ -21,15 +25,16 @@ if str(LIB_DIR) not in sys.path:
|
||||
|
||||
import vpnrouter_ldap
|
||||
import vpnrouter_platforms
|
||||
from vpnrouter_net import ConfigError, detect_interfaces, interface_addresses, ip_output
|
||||
from vpnrouter_net import DEFAULT_LISTEN, LISTEN_CHOICES, LOOPBACK, ConfigError, detect_interfaces, interface_addresses, ip_output
|
||||
|
||||
CONFIG_FILE = pathlib.Path('/etc/vpn-router/vpn-router.json')
|
||||
CONFIG_LOCK = pathlib.Path('/run/vpn-router/config.lock')
|
||||
TEMPLATE_DIR = pathlib.Path('/usr/share/vpn-router/templates')
|
||||
PKI_DIR = pathlib.Path('/etc/vpn-router/pki')
|
||||
SIMPLE_CA = LIB_DIR / 'simple-ca'
|
||||
|
||||
SWANCTL_CONF_D = pathlib.Path('/etc/swanctl/conf.d')
|
||||
REMOTE_SITE = SWANCTL_CONF_D / 'remote-site.conf'
|
||||
SITE_TO_SITE = SWANCTL_CONF_D / 'site-to-site.conf'
|
||||
ROAD_WARRIOR = SWANCTL_CONF_D / 'road-warrior.conf'
|
||||
P2S_FORWARDER = pathlib.Path('/etc/systemd/resolved.conf.d/p2s-forwarder.conf')
|
||||
WG_CONF = pathlib.Path('/etc/wireguard/wg0.conf')
|
||||
@@ -47,16 +52,43 @@ TRUST_ANCHOR = pathlib.Path('/usr/local/share/ca-certificates/vpn-router-ca.cr
|
||||
SCHEMA = {
|
||||
'general': {'platform': 'none', 'mode': 'manual'},
|
||||
'interfaces': {'external': '', 'internal': ''},
|
||||
'wan': {'local_fqdn': '', 'local_id_mode': 'fqdn'},
|
||||
'wan': {'local_fqdn': ''},
|
||||
'local': {'cidrs': [], 'int_addr': '', 'int_gateway_ip': ''},
|
||||
'remote': {'addrs': [], 'id': '', 'cidrs': [], 'psk': '', 'psk_file': ''},
|
||||
'site_to_site': {},
|
||||
'p2s': {'enabled': False, 'address_pool': '', 'ca_name': 'VPN Router CA'},
|
||||
'wireguard': {'enabled': False, 'address': '', 'listen_port': 51820},
|
||||
'openvpn': {'enabled': False, 'address_pool': '', 'port': 1194, 'protocol': 'udp'},
|
||||
'docker': {'mode': 'auto', 'exposed_ports': []},
|
||||
'web': {'enabled': True, 'port': 7081, 'database': '', 'trusted_proxies': []},
|
||||
'web': {'enabled': True, 'listen': DEFAULT_LISTEN, 'port': 7081, 'database': '', 'trusted_proxies': [],
|
||||
'status_content': 'basic'},
|
||||
'https': {'enabled': True, 'email': '', 'web_port': 7444, 'manager_port': 7443},
|
||||
'metrics': {'enabled': True, 'listen': 'internal', 'port': 9464, 'https_port': 9465, 'service_account': 'metrics',
|
||||
'otlp_endpoint': '', 'otlp_username': '', 'otlp_password': '',
|
||||
'keep_details': '24 hours', 'keep_hourly': '1 month', 'keep_daily': '3 months',
|
||||
'keep_monthly': '1 year', 'keep_yearly': '10 years'},
|
||||
}
|
||||
|
||||
#: site_to_site maps a connection name to its settings: option -> default, the type of the default being the required type.
|
||||
CONNECTION = {'addrs': [], 'id': '', 'cidrs': [], 'local_cidrs': [], 'local_id_mode': 'fqdn', 'psk': '', 'psk_file': ''}
|
||||
|
||||
#: The name of a site_to_site connection, which also names its swanctl connection, child and secret.
|
||||
CONNECTION_NAME = re.compile(r'^[a-z][a-z0-9-]{0,30}$')
|
||||
|
||||
#: The site_to_site connection the install answers fill.
|
||||
FIRST_CONNECTION = 'remote-site'
|
||||
|
||||
#: The retention settings of the metrics, from the details to the yearly summaries.
|
||||
METRICS_KEEP = ('keep_details', 'keep_hourly', 'keep_daily', 'keep_monthly', 'keep_yearly')
|
||||
|
||||
#: A retention period as PostgreSQL reads an interval: a count and a unit.
|
||||
METRICS_INTERVAL = re.compile(r'^[1-9]\d{0,4} (hour|day|month|year)s?$')
|
||||
|
||||
#: The port of Let's Encrypt's HTTP challenge, which Caddy answers while HTTPS is on.
|
||||
ACME_HTTP_PORT = 80
|
||||
|
||||
#: What / and /status of the web service show, web.status_content, from the least to the most.
|
||||
STATUS_CONTENTS = ('basic', 'standard', 'full')
|
||||
|
||||
DOCKER_MODES = ('auto', 'on', 'off')
|
||||
|
||||
OPENVPN_PROTOCOLS = ('udp', 'tcp')
|
||||
@@ -74,7 +106,7 @@ DOCKER_PORT = re.compile(r'^\d{1,5}(:\d{1,5})?$')
|
||||
DOCKER_PORT_LIMIT = 15
|
||||
|
||||
TYPE_NAMES = {str: 'a string', list: 'a list of strings', bool: 'a boolean',
|
||||
int: 'an integer'}
|
||||
int: 'an integer', dict: 'an object'}
|
||||
|
||||
ID_MODES = ('fqdn', 'public_ip', 'internal_ip')
|
||||
|
||||
@@ -99,29 +131,76 @@ def fail(message):
|
||||
sys.exit(f'vpn-router: error: {message}')
|
||||
|
||||
|
||||
def load_config(path=None):
|
||||
"""Read the configuration, filling in defaults for anything absent.
|
||||
def read_settings(path=None):
|
||||
"""The settings in the configuration file, without the defaults; {} when the file does not exist.
|
||||
|
||||
The result maps section -> option -> value. Sections and options not in
|
||||
SCHEMA are kept so that validate() can report them.
|
||||
Raises ConfigError when the file is not JSON or not an object of sections.
|
||||
"""
|
||||
path = pathlib.Path(path) if path else CONFIG_FILE
|
||||
try:
|
||||
data = json.loads(path.read_text(encoding='utf-8'))
|
||||
except FileNotFoundError:
|
||||
return {}
|
||||
except ValueError as exc:
|
||||
raise ConfigError(f'cannot parse {path}: {exc}')
|
||||
if not isinstance(data, dict):
|
||||
raise ConfigError(f'cannot parse {path}: the top level is not an object')
|
||||
for section, options in data.items():
|
||||
if not isinstance(options, dict):
|
||||
raise ConfigError(f'cannot parse {path}: {section} is not an object')
|
||||
return data
|
||||
|
||||
|
||||
def with_defaults(data):
|
||||
"""The configuration of the settings of read_settings, with the defaults for anything absent.
|
||||
|
||||
The result maps section -> option -> value, and site_to_site maps each connection name to
|
||||
its settings filled from CONNECTION. Sections, options and connections that are not
|
||||
objects are kept so that validate() can report them.
|
||||
"""
|
||||
cfg = {section: dict(options) for section, options in SCHEMA.items()}
|
||||
for section, options in data.items():
|
||||
cfg.setdefault(section, {}).update(options)
|
||||
cfg['site_to_site'] = {name: {**CONNECTION, **options} if isinstance(options, dict) else options
|
||||
for name, options in cfg['site_to_site'].items()}
|
||||
return cfg
|
||||
|
||||
|
||||
def load_config(path=None):
|
||||
"""Read the configuration, filling in defaults for anything absent (see with_defaults).
|
||||
|
||||
The path is resolved when called, not when this function is defined, so
|
||||
CONFIG_FILE stays overridable.
|
||||
"""
|
||||
path = pathlib.Path(path) if path else CONFIG_FILE
|
||||
cfg = {section: dict(options) for section, options in SCHEMA.items()}
|
||||
if path.exists():
|
||||
try:
|
||||
data = json.loads(path.read_text(encoding='utf-8'))
|
||||
except ValueError as exc:
|
||||
fail(f'cannot parse {path}: {exc}')
|
||||
if not isinstance(data, dict):
|
||||
fail(f'cannot parse {path}: the top level is not an object')
|
||||
for section, options in data.items():
|
||||
if not isinstance(options, dict):
|
||||
fail(f'cannot parse {path}: {section} is not an object')
|
||||
cfg.setdefault(section, {}).update(options)
|
||||
return cfg
|
||||
try:
|
||||
return with_defaults(read_settings(path))
|
||||
except ConfigError as exc:
|
||||
fail(str(exc))
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def config_lock():
|
||||
"""Hold the lock of the configuration file from reading it to writing it with save_config."""
|
||||
CONFIG_LOCK.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(CONFIG_LOCK, 'a', encoding='utf-8') as handle:
|
||||
fcntl.flock(handle, fcntl.LOCK_EX)
|
||||
yield
|
||||
|
||||
|
||||
def save_config(data):
|
||||
"""Write the settings to the configuration file, indented, with mode 0600, replacing it in one step.
|
||||
|
||||
The caller holds config_lock.
|
||||
"""
|
||||
descriptor, name = tempfile.mkstemp(dir=CONFIG_FILE.parent, prefix='.vpn-router.json.')
|
||||
try:
|
||||
with os.fdopen(descriptor, 'w', encoding='utf-8') as handle:
|
||||
handle.write(json.dumps(data, indent=2) + '\n')
|
||||
os.chmod(name, 0o600)
|
||||
os.replace(name, CONFIG_FILE)
|
||||
except OSError:
|
||||
os.unlink(name)
|
||||
raise
|
||||
|
||||
|
||||
def _split_list(value):
|
||||
@@ -133,23 +212,37 @@ def _get(cfg, section, option):
|
||||
return value.strip() if isinstance(value, str) else value
|
||||
|
||||
|
||||
def _clean(values):
|
||||
return [item.strip() for item in values if item.strip()]
|
||||
|
||||
|
||||
def _get_list(cfg, section, option):
|
||||
return [item.strip() for item in cfg[section][option] if item.strip()]
|
||||
return _clean(cfg[section][option])
|
||||
|
||||
|
||||
def _check_options(options, defaults, prefix):
|
||||
for option, value in options.items():
|
||||
if option not in defaults:
|
||||
raise ConfigError(f'{prefix}.{option}: unknown setting')
|
||||
expected = type(defaults[option])
|
||||
if type(value) is not expected or (
|
||||
expected is list and
|
||||
not all(isinstance(item, str) for item in value)):
|
||||
raise ConfigError(
|
||||
f'{prefix}.{option} must be {TYPE_NAMES[expected]}')
|
||||
|
||||
|
||||
def _check_types(cfg):
|
||||
for section, options in cfg.items():
|
||||
if section not in SCHEMA:
|
||||
raise ConfigError(f'{section}: unknown section')
|
||||
for option, value in options.items():
|
||||
if option not in SCHEMA[section]:
|
||||
raise ConfigError(f'{section}.{option}: unknown setting')
|
||||
expected = type(SCHEMA[section][option])
|
||||
if type(value) is not expected or (
|
||||
expected is list and
|
||||
not all(isinstance(item, str) for item in value)):
|
||||
raise ConfigError(
|
||||
f'{section}.{option} must be {TYPE_NAMES[expected]}')
|
||||
if section != 'site_to_site':
|
||||
_check_options(options, SCHEMA[section], section)
|
||||
continue
|
||||
for name, connection in options.items():
|
||||
if not isinstance(connection, dict):
|
||||
raise ConfigError(f'site_to_site.{name} must be {TYPE_NAMES[dict]}')
|
||||
_check_options(connection, CONNECTION, f'site_to_site.{name}')
|
||||
|
||||
|
||||
def _check_networks(values, name):
|
||||
@@ -201,19 +294,28 @@ def validate(cfg):
|
||||
raise ConfigError(
|
||||
f'{section}.{option} is required in mode {mode or "manual"}')
|
||||
|
||||
for section, option in (('wan', 'local_fqdn'), ('remote', 'id')):
|
||||
if str(_get(cfg, section, option)).startswith('@'):
|
||||
identities = [('wan.local_fqdn', _get(cfg, 'wan', 'local_fqdn'))]
|
||||
for name, connection in cfg['site_to_site'].items():
|
||||
prefix = f'site_to_site.{name}'
|
||||
if not CONNECTION_NAME.match(name):
|
||||
raise ConfigError(
|
||||
f'{section}.{option}: remove the leading @; strongSwan reads an '
|
||||
f'{prefix}: a connection name starts with a lowercase letter and has at most 31 '
|
||||
'lowercase letters, digits and hyphens')
|
||||
identities.append((f'{prefix}.id', connection['id'].strip()))
|
||||
id_mode = connection['local_id_mode'].strip()
|
||||
if id_mode and id_mode not in ID_MODES:
|
||||
raise ConfigError(
|
||||
f'{prefix}.local_id_mode: {id_mode!r} is not one of {", ".join(ID_MODES)}')
|
||||
_check_networks(_clean(connection['cidrs']), f'{prefix}.cidrs')
|
||||
_check_networks(_clean(connection['local_cidrs']), f'{prefix}.local_cidrs')
|
||||
|
||||
for option, value in identities:
|
||||
if value.startswith('@'):
|
||||
raise ConfigError(
|
||||
f'{option}: remove the leading @; strongSwan reads an '
|
||||
'address as an IP identity and a name as an FQDN')
|
||||
|
||||
id_mode = _get(cfg, 'wan', 'local_id_mode')
|
||||
if id_mode and id_mode not in ID_MODES:
|
||||
raise ConfigError(
|
||||
f'wan.local_id_mode: {id_mode!r} is not one of {", ".join(ID_MODES)}')
|
||||
|
||||
_check_networks(_get_list(cfg, 'local', 'cidrs'), 'local.cidrs')
|
||||
_check_networks(_get_list(cfg, 'remote', 'cidrs'), 'remote.cidrs')
|
||||
pool = _get(cfg, 'p2s', 'address_pool')
|
||||
_check_networks([pool] if pool else [], 'p2s.address_pool')
|
||||
openvpn_pool = _get(cfg, 'openvpn', 'address_pool')
|
||||
@@ -259,9 +361,57 @@ def validate(cfg):
|
||||
if isinstance(web_port, bool) or not isinstance(web_port, int) or not 1 <= web_port <= 65535:
|
||||
raise ConfigError(f'web.port: {web_port!r} is not a valid port')
|
||||
|
||||
web_listen = _get(cfg, 'web', 'listen')
|
||||
if web_listen not in LISTEN_CHOICES:
|
||||
raise ConfigError(
|
||||
f'web.listen: {web_listen!r} is not one of {", ".join(LISTEN_CHOICES)}')
|
||||
|
||||
status_content = _get(cfg, 'web', 'status_content')
|
||||
if status_content not in STATUS_CONTENTS:
|
||||
raise ConfigError(
|
||||
f'web.status_content: {status_content!r} is not one of {", ".join(STATUS_CONTENTS)}')
|
||||
|
||||
https_ports = []
|
||||
for option in ('web_port', 'manager_port'):
|
||||
port = _get(cfg, 'https', option)
|
||||
if isinstance(port, bool) or not isinstance(port, int) or not 1 <= port <= 65535:
|
||||
raise ConfigError(f'https.{option}: {port!r} is not a valid port')
|
||||
if port == ACME_HTTP_PORT:
|
||||
raise ConfigError(f'https.{option}: {port} is the port of the HTTP challenge')
|
||||
https_ports.append(port)
|
||||
if https_ports[0] == https_ports[1]:
|
||||
raise ConfigError('https.web_port and https.manager_port must differ')
|
||||
|
||||
for item in _get_list(cfg, 'web', 'trusted_proxies'):
|
||||
_check_address(item, 'web.trusted_proxies')
|
||||
|
||||
taken = {web_port: 'web.port', ACME_HTTP_PORT: 'the HTTP challenge',
|
||||
https_ports[0]: 'https.web_port', https_ports[1]: 'https.manager_port'}
|
||||
for option in ('port', 'https_port'):
|
||||
port = _get(cfg, 'metrics', option)
|
||||
if isinstance(port, bool) or not isinstance(port, int) or not 1 <= port <= 65535:
|
||||
raise ConfigError(f'metrics.{option}: {port!r} is not a valid port')
|
||||
if port in taken:
|
||||
raise ConfigError(f'metrics.{option}: {port} is the port of {taken[port]}')
|
||||
taken[port] = f'metrics.{option}'
|
||||
|
||||
metrics_listen = _get(cfg, 'metrics', 'listen')
|
||||
if metrics_listen not in LISTEN_CHOICES:
|
||||
raise ConfigError(
|
||||
f'metrics.listen: {metrics_listen!r} is not one of {", ".join(LISTEN_CHOICES)}')
|
||||
|
||||
endpoint = _get(cfg, 'metrics', 'otlp_endpoint')
|
||||
if endpoint and urllib.parse.urlsplit(endpoint).scheme not in ('http', 'https'):
|
||||
raise ConfigError(f'metrics.otlp_endpoint: {endpoint!r} is not an http or https URL')
|
||||
if bool(_get(cfg, 'metrics', 'otlp_username')) != bool(_get(cfg, 'metrics', 'otlp_password')):
|
||||
raise ConfigError('metrics.otlp_username and metrics.otlp_password are given together')
|
||||
|
||||
for option in METRICS_KEEP:
|
||||
keep = _get(cfg, 'metrics', option)
|
||||
if not METRICS_INTERVAL.match(keep):
|
||||
raise ConfigError(
|
||||
f'metrics.{option}: {keep!r} is not a count and a unit, such as "24 hours" or "3 months"')
|
||||
|
||||
docker_mode = _get(cfg, 'docker', 'mode')
|
||||
if docker_mode not in DOCKER_MODES:
|
||||
raise ConfigError(
|
||||
@@ -275,21 +425,28 @@ def validate(cfg):
|
||||
raise ConfigError(
|
||||
f'docker.exposed_ports: {item!r} is not a port or a port range')
|
||||
weight += 2 if ':' in item else 1
|
||||
first, _, last = item.partition(':')
|
||||
if _get(cfg, 'https', 'enabled'):
|
||||
for port in (ACME_HTTP_PORT, *https_ports):
|
||||
if int(first) <= port <= int(last or first):
|
||||
raise ConfigError(
|
||||
f'docker.exposed_ports: {item!r} takes port {port}, which Caddy serves while '
|
||||
'https.enabled is true')
|
||||
if weight > DOCKER_PORT_LIMIT:
|
||||
raise ConfigError(
|
||||
f'docker.exposed_ports: at most {DOCKER_PORT_LIMIT} ports are allowed, '
|
||||
'a range counting as two')
|
||||
|
||||
|
||||
def resolve_psk(cfg):
|
||||
"""Return the pre-shared key, or '' when none is configured."""
|
||||
psk_file = _get(cfg, 'remote', 'psk_file')
|
||||
def resolve_psk(connection, name):
|
||||
"""Return the pre-shared key of the site_to_site connection: psk_file wins, else psk; '' when none is configured."""
|
||||
psk_file = connection['psk_file'].strip()
|
||||
if psk_file:
|
||||
path = pathlib.Path(psk_file)
|
||||
if not path.is_file():
|
||||
raise ConfigError(f'remote.psk_file: {psk_file} does not exist')
|
||||
raise ConfigError(f'site_to_site.{name}.psk_file: {psk_file} does not exist')
|
||||
return path.read_text(encoding='utf-8').strip()
|
||||
return cfg['remote']['psk']
|
||||
return connection['psk']
|
||||
|
||||
|
||||
def platform_module(cfg):
|
||||
@@ -373,29 +530,43 @@ def resolve_interfaces(cfg):
|
||||
return ext_iface, int_iface, local_addrs, int_addr, gateway
|
||||
|
||||
|
||||
def build_context(cfg, local_id_override=None):
|
||||
"""Assemble the template context from the configuration."""
|
||||
def build_context(cfg, public_ip=None):
|
||||
"""Assemble the template context from the configuration.
|
||||
|
||||
public_ip is the address wan.local_fqdn resolves to, the local identity of the
|
||||
site_to_site connections in mode public_ip; setup resolves it once the network is up and
|
||||
renders again, and without it such a connection is not ready.
|
||||
"""
|
||||
extras = vpnrouter_platforms.context(platform_module(cfg))
|
||||
|
||||
ext_iface, int_iface, local_addrs, int_addr, gateway = resolve_interfaces(cfg)
|
||||
|
||||
local_fqdn = _get(cfg, 'wan', 'local_fqdn')
|
||||
id_mode = _get(cfg, 'wan', 'local_id_mode') or 'fqdn'
|
||||
local_cidrs = _get_list(cfg, 'local', 'cidrs')
|
||||
remote_cidrs = _get_list(cfg, 'remote', 'cidrs')
|
||||
openvpn_pool = _get(cfg, 'openvpn', 'address_pool')
|
||||
openvpn_network = ipaddress.ip_network(openvpn_pool) if openvpn_pool else None
|
||||
identities = {'fqdn': local_fqdn, 'public_ip': public_ip or '', 'internal_ip': int_addr}
|
||||
|
||||
if local_id_override:
|
||||
local_id = local_id_override
|
||||
elif id_mode == 'fqdn':
|
||||
local_id = local_fqdn
|
||||
elif id_mode == 'internal_ip':
|
||||
local_id = int_addr
|
||||
else:
|
||||
# public_ip is resolved once the network is up, and the caller then
|
||||
# re-renders with local_id_override set.
|
||||
local_id = ''
|
||||
connections = []
|
||||
for name, connection in cfg['site_to_site'].items():
|
||||
id_mode = connection['local_id_mode'].strip() or 'fqdn'
|
||||
remote_cidrs = _clean(connection['cidrs'])
|
||||
connection_cidrs = _clean(connection['local_cidrs'])
|
||||
entry = {
|
||||
'name': name,
|
||||
'remote_addrs': ', '.join(_clean(connection['addrs'])),
|
||||
'remote_id': connection['id'].strip(),
|
||||
'remote_cidrs': ', '.join(remote_cidrs),
|
||||
'remote_cidr_list': remote_cidrs,
|
||||
'local_ts': _join(connection_cidrs, _split_list(extras['EXTRA_LOCAL_TS'])),
|
||||
'local_cidr_list': connection_cidrs,
|
||||
'local_id_mode': id_mode,
|
||||
'local_id': identities.get(id_mode, ''),
|
||||
'psk': resolve_psk(connection, name),
|
||||
}
|
||||
entry['ready'] = bool(local_addrs and connection_cidrs and all(
|
||||
entry[key] for key in ('remote_addrs', 'remote_id', 'remote_cidrs', 'local_id', 'psk')))
|
||||
connections.append(entry)
|
||||
|
||||
return {
|
||||
'platform': _get(cfg, 'general', 'platform') or PLATFORM_NONE,
|
||||
@@ -405,18 +576,12 @@ def build_context(cfg, local_id_override=None):
|
||||
'local_addrs': ', '.join(local_addrs),
|
||||
'local_addr_list': local_addrs,
|
||||
'local_fqdn': local_fqdn,
|
||||
'local_id_mode': id_mode,
|
||||
'local_id': local_id,
|
||||
'local_cidrs': ', '.join(local_cidrs),
|
||||
'local_cidr_list': local_cidrs,
|
||||
'local_ts': _join(local_cidrs, _split_list(extras['EXTRA_LOCAL_TS'])),
|
||||
'int_addr': int_addr,
|
||||
'int_gateway_ip': gateway,
|
||||
'remote_addrs': ', '.join(_get_list(cfg, 'remote', 'addrs')),
|
||||
'remote_id': _get(cfg, 'remote', 'id'),
|
||||
'remote_cidrs': ', '.join(remote_cidrs),
|
||||
'remote_cidr_list': remote_cidrs,
|
||||
'psk': resolve_psk(cfg),
|
||||
'connections': connections,
|
||||
'p2s_enabled': _get(cfg, 'p2s', 'enabled'),
|
||||
'p2s_address_pool': _get(cfg, 'p2s', 'address_pool'),
|
||||
'p2s_ca_name': _get(cfg, 'p2s', 'ca_name'),
|
||||
@@ -443,15 +608,21 @@ def build_context(cfg, local_id_override=None):
|
||||
'ldap_users': vpnrouter_ldap.USERS,
|
||||
'web_enabled': _get(cfg, 'web', 'enabled'),
|
||||
'web_port': _get(cfg, 'web', 'port'),
|
||||
'https_enabled': _get(cfg, 'https', 'enabled'),
|
||||
'https_email': _get(cfg, 'https', 'email'),
|
||||
'https_web_port': _get(cfg, 'https', 'web_port'),
|
||||
'https_manager_port': _get(cfg, 'https', 'manager_port'),
|
||||
'metrics_enabled': _get(cfg, 'metrics', 'enabled'),
|
||||
'metrics_port': _get(cfg, 'metrics', 'port'),
|
||||
'metrics_https_port': _get(cfg, 'metrics', 'https_port'),
|
||||
'metrics_service_account': _get(cfg, 'metrics', 'service_account'),
|
||||
'loopback': LOOPBACK,
|
||||
}
|
||||
|
||||
|
||||
def s2s_ready(ctx):
|
||||
return all([
|
||||
ctx['local_addrs'], ctx['local_fqdn'], ctx['local_cidrs'],
|
||||
ctx['remote_addrs'], ctx['remote_id'], ctx['remote_cidrs'],
|
||||
ctx['psk'], ctx['local_id'],
|
||||
])
|
||||
"""True when any site_to_site connection is ready."""
|
||||
return any(connection['ready'] for connection in ctx['connections'])
|
||||
|
||||
|
||||
def routing_ready(ctx):
|
||||
@@ -472,6 +643,16 @@ def openvpn_ready(ctx):
|
||||
return bool(ctx['openvpn_enabled'] and ctx['openvpn_pool'] and ctx['local_fqdn'])
|
||||
|
||||
|
||||
def https_ready(ctx):
|
||||
"""Caddy serves HTTPS once it is enabled and wan.local_fqdn names the certificate."""
|
||||
return bool(ctx['https_enabled'] and ctx['local_fqdn'])
|
||||
|
||||
|
||||
def metrics_ready(ctx):
|
||||
"""The metrics service collects, keeps and serves the metrics while metrics.enabled is true."""
|
||||
return bool(ctx['metrics_enabled'])
|
||||
|
||||
|
||||
def _jinja_env():
|
||||
return jinja2.Environment(
|
||||
loader=jinja2.FileSystemLoader(str(TEMPLATE_DIR)),
|
||||
@@ -606,7 +787,7 @@ def distribute_pki(ctx):
|
||||
return changed
|
||||
|
||||
|
||||
def configure(cfg=None, local_id_override=None):
|
||||
def configure(cfg=None, public_ip=None):
|
||||
"""Render configuration files and provision PKI.
|
||||
|
||||
Returns (ctx, changed) where changed is a set naming what needs reloading:
|
||||
@@ -622,7 +803,7 @@ def configure(cfg=None, local_id_override=None):
|
||||
|
||||
try:
|
||||
validate(cfg)
|
||||
ctx = build_context(cfg, local_id_override=local_id_override)
|
||||
ctx = build_context(cfg, public_ip=public_ip)
|
||||
except ConfigError as exc:
|
||||
fail(str(exc))
|
||||
|
||||
@@ -635,9 +816,9 @@ def configure(cfg=None, local_id_override=None):
|
||||
env = _jinja_env()
|
||||
|
||||
if s2s_ready(ctx):
|
||||
if render(env, ctx, 'remote-site.conf.j2', REMOTE_SITE, 0o600):
|
||||
if render(env, ctx, 'site-to-site.conf.j2', SITE_TO_SITE, 0o600):
|
||||
changed.add('swanctl')
|
||||
elif remove(REMOTE_SITE):
|
||||
elif remove(SITE_TO_SITE):
|
||||
changed.add('swanctl')
|
||||
|
||||
if p2s_ready(ctx):
|
||||
|
||||
@@ -5,9 +5,7 @@ router and that never leaves it. /auth issues an access token naming the user in
|
||||
a refresh token valid REFRESH_SECONDS; vpn-router-token issues tokens with the admin claim and
|
||||
any lifetime. A token without the admin claim must expire within MAX_LIFETIME seconds.
|
||||
|
||||
psycopg is imported by driver() when the database is used, not when the module is loaded, so the
|
||||
module loads while libpq cannot be found yet. A driver that cannot be imported and every
|
||||
failure of the database raise DatabaseError, the only database exception callers see.
|
||||
Every failure of the database raises DatabaseError, the only database exception callers see.
|
||||
|
||||
Refresh tokens are random strings kept in PostgreSQL as their SHA-256, in the database of
|
||||
web.database, by default the vpn-router database on this host that provision() creates on the
|
||||
@@ -24,12 +22,17 @@ import secrets
|
||||
import subprocess
|
||||
import time
|
||||
|
||||
import psycopg
|
||||
|
||||
WEB_KEY = pathlib.Path('/etc/vpn-router/web.key')
|
||||
DATABASE_SECRET = pathlib.Path('/etc/vpn-router/database.secret')
|
||||
|
||||
#: The role and the database provision() creates when web.database is not set.
|
||||
DATABASE = 'vpn-router'
|
||||
|
||||
#: The user operations of vpnrouter_users after which the user's refresh tokens are revoked.
|
||||
REVOKING = ('passwd', 'disable', 'unlock', 'remove')
|
||||
|
||||
#: Seconds an access token of /auth and a refresh token are valid.
|
||||
ACCESS_SECONDS = 300
|
||||
REFRESH_SECONDS = 8 * 3600
|
||||
@@ -44,7 +47,7 @@ SCHEMA = 'CREATE TABLE IF NOT EXISTS refresh_tokens (hash text PRIMARY KEY, name
|
||||
|
||||
|
||||
class DatabaseError(Exception):
|
||||
"""The PostgreSQL driver cannot be loaded, or the database of the refresh tokens failed."""
|
||||
"""The database of the refresh tokens failed."""
|
||||
|
||||
|
||||
def key():
|
||||
@@ -117,15 +120,6 @@ def admin_token(seconds):
|
||||
return {'token': sign({'admin': True, 'iat': now, 'exp': now + seconds}, secret), 'expires': now + seconds}
|
||||
|
||||
|
||||
def driver():
|
||||
"""The psycopg module; DatabaseError while libpq cannot be found."""
|
||||
try:
|
||||
import psycopg
|
||||
except ImportError as exc:
|
||||
raise DatabaseError(f'the PostgreSQL driver cannot be loaded: {exc}') from exc
|
||||
return psycopg
|
||||
|
||||
|
||||
def connection_string(cfg):
|
||||
"""web.database, or the database provision() creates; its password, of URL-safe characters, is written unquoted."""
|
||||
configured = cfg['web']['database'].strip()
|
||||
@@ -138,7 +132,6 @@ def connection_string(cfg):
|
||||
@contextlib.contextmanager
|
||||
def database(cfg):
|
||||
"""A connection to the database of the refresh tokens, with their table, closed at the end; DatabaseError on any failure."""
|
||||
psycopg = driver()
|
||||
try:
|
||||
with psycopg.connect(connection_string(cfg), autocommit=True) as connection:
|
||||
connection.execute(SCHEMA)
|
||||
|
||||
@@ -50,6 +50,22 @@ LOCKOUT_SECONDS = 900
|
||||
#: The overlays of the database, in the order they are configured.
|
||||
OVERLAYS = ('ppolicy', 'otp')
|
||||
|
||||
#: The package's own schema, under an OID of the UUID arc 2.25 (ITU-T X.667): attribute types
|
||||
#: under .1 and object classes under .2. vpnRouterAccount, auxiliary, adds properties of the
|
||||
#: package to a user of either kind: mail, for a Manager-only account, vpnRouterScope,
|
||||
#: authorization scopes, of which service marks a service account, and displayName and
|
||||
#: description, the labels of a service account.
|
||||
SCHEMA_OID = '2.25.98477352595735834532404256531620013001'
|
||||
SCHEMA_NAME = 'vpn-router'
|
||||
SCHEMA_ATTRIBUTE_TYPES = (
|
||||
f"( {SCHEMA_OID}.1.1 NAME 'vpnRouterScope' DESC 'VPN router: an authorization scope of the account' "
|
||||
"EQUALITY caseIgnoreMatch SUBSTR caseIgnoreSubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{256} )",
|
||||
)
|
||||
SCHEMA_OBJECT_CLASSES = (
|
||||
f"( {SCHEMA_OID}.2.1 NAME 'vpnRouterAccount' DESC 'VPN router: properties of a user or a Manager-only account' "
|
||||
"SUP top AUXILIARY MAY ( mail $ vpnRouterScope $ displayName $ description ) )",
|
||||
)
|
||||
|
||||
#: The TOTP parameters: digits, the OID of HMAC-SHA1, seconds per time step, and the time steps accepted before and after now.
|
||||
TOTP_DIGITS = 6
|
||||
TOTP_ALGORITHM = '1.2.840.113549.2.7'
|
||||
@@ -109,6 +125,23 @@ def _ensure_modules(connection):
|
||||
connection.modify_s(dn, [(ldap.MOD_ADD, 'olcModuleLoad', [name.encode() for name in missing])])
|
||||
|
||||
|
||||
def _ensure_schema(connection):
|
||||
"""Add the package's schema to cn=config when missing, else set its definitions that differ."""
|
||||
found = connection.search_s('cn=schema,cn=config', ldap.SCOPE_ONELEVEL, f'(cn=*{SCHEMA_NAME})',
|
||||
['olcAttributeTypes', 'olcObjectClasses'])
|
||||
attributes = {'olcAttributeTypes': [value.encode() for value in SCHEMA_ATTRIBUTE_TYPES],
|
||||
'olcObjectClasses': [value.encode() for value in SCHEMA_OBJECT_CLASSES]}
|
||||
if not found:
|
||||
connection.add_s(f'cn={SCHEMA_NAME},cn=schema,cn=config', ldap.modlist.addModlist(
|
||||
dict(attributes, objectClass=[b'olcSchemaConfig'], cn=[SCHEMA_NAME.encode()])))
|
||||
return
|
||||
dn, current = found[0]
|
||||
changes = [(ldap.MOD_REPLACE, name, values) for name, values in attributes.items()
|
||||
if unordered(current.get(name, [])) != [value.decode() for value in values]]
|
||||
if changes:
|
||||
connection.modify_s(dn, changes)
|
||||
|
||||
|
||||
def _ensure_password_hash(connection):
|
||||
dn = 'olcDatabase={-1}frontend,cn=config'
|
||||
current = connection.search_s(dn, ldap.SCOPE_BASE, attrlist=['olcPasswordHash'])[0][1].get('olcPasswordHash', [])
|
||||
@@ -188,10 +221,11 @@ def _ensure_manager(connection):
|
||||
|
||||
|
||||
def provision():
|
||||
"""Create or bring up to date the modules, database, ACLs, password policy, entries and service account."""
|
||||
"""Create or bring up to date the modules, the package's schema, database, ACLs, password policy, entries and service account."""
|
||||
connection = root()
|
||||
try:
|
||||
_ensure_modules(connection)
|
||||
_ensure_schema(connection)
|
||||
_ensure_password_hash(connection)
|
||||
database = _ensure_database(connection)
|
||||
_ensure_overlays(connection, database)
|
||||
|
||||
@@ -0,0 +1,185 @@
|
||||
"""The metrics of the router in PostgreSQL, for the metrics service and the web service.
|
||||
|
||||
The metrics service writes one detail row per minute to the four detail tables, of TABLES:
|
||||
metrics_system (CPU, memory, disk and load averages), metrics_network (bytes per second
|
||||
received and sent per interface), metrics_vpn (clients per kind) and metrics_site_to_site (per
|
||||
site-to-site connection: up, 1 or 0, bytes and packets per second received and sent, and the
|
||||
seconds its IKE SA has been up). Each detail table has four
|
||||
summary tables, of LEVELS: <table>_hourly from the details, <table>_daily from the hourly,
|
||||
<table>_monthly from the daily and <table>_yearly from the monthly rows. A summary row has the
|
||||
start of its period, in UTC, and the average and the maximum of each value over the period: of
|
||||
the details for the hourly rows, else the average of the averages and the maximum of the maxima
|
||||
below. A period is summarized once it is complete; summarize() catches up every complete period
|
||||
after the last summarized one, so a stopped service loses no summary of what was written.
|
||||
|
||||
collect_garbage() deletes the rows older than the retention of their level, metrics.keep_*;
|
||||
the service runs it on start and once a day. history() answers the views of the Metrics tab.
|
||||
|
||||
The tables are in the database of the refresh tokens (vpnrouter_auth.connection_string) and are
|
||||
created when missing. Every failure of the database raises DatabaseError.
|
||||
"""
|
||||
|
||||
import contextlib
|
||||
|
||||
import psycopg
|
||||
|
||||
import vpnrouter_auth
|
||||
|
||||
#: The detail tables: their key columns besides time, and their value columns.
|
||||
TABLES = {
|
||||
'metrics_system': ((), ('cpu', 'memory_used', 'memory_total', 'disk_free', 'disk_total', 'load1', 'load5', 'load15')),
|
||||
'metrics_network': (('interface',), ('rx', 'tx')),
|
||||
'metrics_vpn': (('kind',), ('clients',)),
|
||||
'metrics_site_to_site': (('connection',), ('up', 'rx', 'tx', 'packets_rx', 'packets_tx', 'uptime')),
|
||||
}
|
||||
|
||||
#: The summary levels, from the finest: the suffix of the tables, the period of date_trunc and the retention setting.
|
||||
LEVELS = (
|
||||
('hourly', 'hour', 'keep_hourly'),
|
||||
('daily', 'day', 'keep_daily'),
|
||||
('monthly', 'month', 'keep_monthly'),
|
||||
('yearly', 'year', 'keep_yearly'),
|
||||
)
|
||||
|
||||
#: The kinds of VPN clients of metrics_vpn.
|
||||
VPN_KINDS = ('p2s', 'openvpn', 'wireguard')
|
||||
|
||||
#: The views of history(): the summary level read (None: the details), the bucket and the span.
|
||||
RANGES = {
|
||||
'hour': (None, '1 minute', '1 hour'),
|
||||
'day': (None, '10 minutes', '1 day'),
|
||||
'month': ('hourly', '6 hours', '1 month'),
|
||||
'year': ('monthly', '1 month', '1 year'),
|
||||
}
|
||||
|
||||
|
||||
class DatabaseError(Exception):
|
||||
"""The metrics database failed."""
|
||||
|
||||
|
||||
def schema():
|
||||
"""The statements that create the detail and the summary tables."""
|
||||
statements = []
|
||||
for table, (keys, values) in TABLES.items():
|
||||
key_columns = ''.join(f', {key} text NOT NULL' for key in keys)
|
||||
primary = ', '.join(('time', *keys))
|
||||
detail = ''.join(f', {value} double precision' for value in values)
|
||||
statements.append(f'CREATE TABLE IF NOT EXISTS {table} (time timestamptz NOT NULL{key_columns}{detail}, '
|
||||
f'PRIMARY KEY ({primary}))')
|
||||
summary = ''.join(f', {value}_avg double precision, {value}_max double precision' for value in values)
|
||||
for suffix, _, _ in LEVELS:
|
||||
statements.append(f'CREATE TABLE IF NOT EXISTS {table}_{suffix} (time timestamptz NOT NULL{key_columns}{summary}, '
|
||||
f'PRIMARY KEY ({primary}))')
|
||||
return statements
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def database(cfg):
|
||||
"""A connection to the metrics database, with its tables, closed at the end; DatabaseError on any failure."""
|
||||
try:
|
||||
with psycopg.connect(vpnrouter_auth.connection_string(cfg), autocommit=True) as connection:
|
||||
for statement in schema():
|
||||
connection.execute(statement)
|
||||
yield connection
|
||||
except (psycopg.Error, OSError) as exc:
|
||||
raise DatabaseError(f'the metrics database failed: {exc}') from exc
|
||||
|
||||
|
||||
def write(connection, system, network, vpn, site_to_site):
|
||||
"""Write a sample as the detail rows of the current minute.
|
||||
|
||||
system maps the values of metrics_system to numbers; network maps an interface to its rx and
|
||||
tx; vpn maps a kind of VPN_KINDS to its clients; site_to_site maps a connection to the values
|
||||
of metrics_site_to_site. A value that is None is written as NULL.
|
||||
"""
|
||||
with connection.transaction():
|
||||
minute = connection.execute("SELECT date_trunc('minute', now(), 'UTC')").fetchone()[0]
|
||||
names = TABLES['metrics_system'][1]
|
||||
connection.execute(f"INSERT INTO metrics_system (time, {', '.join(names)}) "
|
||||
f"VALUES (%s{', %s' * len(names)}) ON CONFLICT DO NOTHING",
|
||||
(minute, *(system.get(name) for name in names)))
|
||||
for interface, (rx, tx) in network.items():
|
||||
connection.execute('INSERT INTO metrics_network (time, interface, rx, tx) VALUES (%s, %s, %s, %s) '
|
||||
'ON CONFLICT DO NOTHING', (minute, interface, rx, tx))
|
||||
for kind, clients in vpn.items():
|
||||
connection.execute('INSERT INTO metrics_vpn (time, kind, clients) VALUES (%s, %s, %s) ON CONFLICT DO NOTHING',
|
||||
(minute, kind, clients))
|
||||
names = TABLES['metrics_site_to_site'][1]
|
||||
for name, values in site_to_site.items():
|
||||
connection.execute(f"INSERT INTO metrics_site_to_site (time, connection, {', '.join(names)}) "
|
||||
f"VALUES (%s, %s{', %s' * len(names)}) ON CONFLICT DO NOTHING",
|
||||
(minute, name, *(values.get(value) for value in names)))
|
||||
|
||||
|
||||
def summarize(connection, level):
|
||||
"""Summarize the complete periods of a level of LEVELS that are not summarized yet, for every detail table."""
|
||||
index = [suffix for suffix, _, _ in LEVELS].index(level)
|
||||
_, period, _ = LEVELS[index]
|
||||
for table, (keys, values) in TABLES.items():
|
||||
target = f'{table}_{level}'
|
||||
source = table if index == 0 else f'{table}_{LEVELS[index - 1][0]}'
|
||||
if index == 0:
|
||||
aggregates = ', '.join(f'avg({value}), max({value})' for value in values)
|
||||
else:
|
||||
aggregates = ', '.join(f'avg({value}_avg), max({value}_max)' for value in values)
|
||||
columns = ', '.join(('time', *keys, *(f'{value}_{kind}' for value in values for kind in ('avg', 'max'))))
|
||||
groups = ''.join(f', {key}' for key in keys)
|
||||
connection.execute(
|
||||
f"INSERT INTO {target} ({columns}) "
|
||||
f"SELECT date_trunc('{period}', time, 'UTC') AS period{groups}, {aggregates} FROM {source} "
|
||||
f"WHERE time < date_trunc('{period}', now(), 'UTC') "
|
||||
f"AND time >= coalesce((SELECT max(time) FROM {target}) + interval '1 {period}', '-infinity') "
|
||||
f"GROUP BY period{groups} ON CONFLICT DO NOTHING")
|
||||
|
||||
|
||||
def collect_garbage(connection, cfg):
|
||||
"""Delete the rows older than the retention of their level: metrics.keep_details, keep_hourly and so on."""
|
||||
metrics = cfg['metrics']
|
||||
for table in TABLES:
|
||||
connection.execute(f'DELETE FROM {table} WHERE time < now() - %s::interval', (metrics['keep_details'],))
|
||||
for suffix, _, keep in LEVELS:
|
||||
connection.execute(f'DELETE FROM {table}_{suffix} WHERE time < now() - %s::interval', (metrics[keep],))
|
||||
|
||||
|
||||
def history(connection, view):
|
||||
"""The points of a view of RANGES, oldest first, each with the start of its bucket in Unix seconds.
|
||||
|
||||
A point has the average and the maximum of each value of metrics_system, network, which maps
|
||||
each interface to its rx and tx, clients, which maps each kind of VPN client to its clients,
|
||||
and site_to_site, which maps each site-to-site connection to the values of
|
||||
metrics_site_to_site; a value without data is None.
|
||||
"""
|
||||
level, bucket, span = RANGES[view]
|
||||
|
||||
def rows(table):
|
||||
keys, values = TABLES[table]
|
||||
source = table if level is None else f'{table}_{level}'
|
||||
if level is None:
|
||||
aggregates = ', '.join(f'avg({value}), max({value})' for value in values)
|
||||
else:
|
||||
aggregates = ', '.join(f'avg({value}_avg), max({value}_max)' for value in values)
|
||||
start = (f"date_bin('{bucket}', time, timestamptz '2000-01-01 00:00:00+00')" if view != 'year'
|
||||
else "date_trunc('month', time, 'UTC')")
|
||||
groups = ''.join(f', {key}' for key in keys)
|
||||
return connection.execute(
|
||||
f"SELECT extract(epoch FROM {start})::bigint AS bucket{groups}, {aggregates} FROM {source} "
|
||||
f"WHERE time >= now() - interval '{span}' GROUP BY bucket{groups} ORDER BY bucket").fetchall()
|
||||
|
||||
def pairs(row, values):
|
||||
return {value: {'avg': row[2 * index], 'max': row[2 * index + 1]} for index, value in enumerate(values)}
|
||||
|
||||
points = {}
|
||||
|
||||
def point(start):
|
||||
return points.setdefault(start, {'time': start, 'network': {}, 'clients': {}, 'site_to_site': {}})
|
||||
|
||||
for row in rows('metrics_system'):
|
||||
point(row[0]).update(pairs(row[1:], TABLES['metrics_system'][1]))
|
||||
for row in rows('metrics_network'):
|
||||
point(row[0])['network'][row[1]] = pairs(row[2:], TABLES['metrics_network'][1])
|
||||
for row in rows('metrics_vpn'):
|
||||
point(row[0])['clients'][row[1]] = pairs(row[2:], TABLES['metrics_vpn'][1])['clients']
|
||||
for row in rows('metrics_site_to_site'):
|
||||
point(row[0])['site_to_site'][row[1]] = pairs(row[2:], TABLES['metrics_site_to_site'][1])
|
||||
empty = {value: {'avg': None, 'max': None} for value in TABLES['metrics_system'][1]}
|
||||
return {'range': view, 'bucket': bucket, 'points': [{**empty, **points[start]} for start in sorted(points)]}
|
||||
@@ -7,6 +7,15 @@ this module too.
|
||||
import pathlib
|
||||
import subprocess
|
||||
|
||||
#: The values of web.listen and of the Router Manager's listen: the interfaces a service listens on.
|
||||
LISTEN_CHOICES = ('internal', 'external', 'both')
|
||||
|
||||
#: The interfaces a service listens on when it is not set.
|
||||
DEFAULT_LISTEN = 'both'
|
||||
|
||||
#: The address every service also listens on, where the HTTPS proxy calls it.
|
||||
LOOPBACK = '127.0.0.1'
|
||||
|
||||
|
||||
class ConfigError(Exception):
|
||||
"""A setting is present but malformed."""
|
||||
@@ -110,26 +119,58 @@ def internal_address(mode, internal, int_addr):
|
||||
return addresses[0] if addresses else ''
|
||||
|
||||
|
||||
def web_listen(configuration, default_port):
|
||||
"""Where the web service listens, from vpn-router.json read as configuration, as ((address, port), '').
|
||||
def external_address(mode, external):
|
||||
"""The router's address on the external interface, as general.mode directs, or '' when it is not known yet.
|
||||
|
||||
The address is the internal address as general.mode directs, the port web.port, else
|
||||
default_port. While web.enabled is false or the address is not known, the answer is (None,
|
||||
the reason).
|
||||
auto works out which interface that is; the other modes read the address of interfaces.external.
|
||||
"""
|
||||
def section(name):
|
||||
value = configuration.get(name)
|
||||
return value if isinstance(value, dict) else {}
|
||||
try:
|
||||
if mode == 'auto':
|
||||
external = detect_interfaces()[0]
|
||||
addresses = interface_addresses(external, 'interfaces.external')
|
||||
except ConfigError:
|
||||
return ''
|
||||
return addresses[0] if addresses else ''
|
||||
|
||||
web = section('web')
|
||||
|
||||
def _section(configuration, name):
|
||||
"""A section of a configuration, or {} when it is missing or not an object."""
|
||||
value = configuration.get(name)
|
||||
return value if isinstance(value, dict) else {}
|
||||
|
||||
|
||||
def listen_addresses(configuration, choice):
|
||||
"""The addresses a service listens on for choice of LISTEN_CHOICES, from vpn-router.json read as configuration.
|
||||
|
||||
The internal address comes first; an address not known yet is left out. A choice that is not
|
||||
one of LISTEN_CHOICES is DEFAULT_LISTEN. LOOPBACK is not among them; web_listen adds it.
|
||||
"""
|
||||
if choice not in LISTEN_CHOICES:
|
||||
choice = DEFAULT_LISTEN
|
||||
general, interfaces, local = (_section(configuration, name) for name in ('general', 'interfaces', 'local'))
|
||||
mode = str(general.get('mode') or 'manual')
|
||||
addresses = []
|
||||
if choice in ('internal', 'both'):
|
||||
addresses.append(internal_address(mode, str(interfaces.get('internal') or ''), str(local.get('int_addr') or '')))
|
||||
if choice in ('external', 'both'):
|
||||
addresses.append(external_address(mode, str(interfaces.get('external') or '')))
|
||||
return [address for address in addresses if address]
|
||||
|
||||
|
||||
def web_listen(configuration, default_port):
|
||||
"""Where the web service listens, from vpn-router.json read as configuration, as ([(address, port), ...], '').
|
||||
|
||||
The addresses are those of the interfaces web.listen names (listen_addresses), then
|
||||
LOOPBACK, the port web.port, else default_port. While web.enabled is false or no address of
|
||||
an interface is known, the answer is ([], the reason).
|
||||
"""
|
||||
web = _section(configuration, 'web')
|
||||
if web.get('enabled', True) is False:
|
||||
return None, 'the web service is disabled in the configuration'
|
||||
return [], 'the web service is disabled in the configuration'
|
||||
port = web.get('port', default_port)
|
||||
if isinstance(port, bool) or not isinstance(port, int) or not 1 <= port <= 65535:
|
||||
port = default_port
|
||||
general, interfaces, local = section('general'), section('interfaces'), section('local')
|
||||
address = internal_address(str(general.get('mode') or 'manual'), str(interfaces.get('internal') or ''),
|
||||
str(local.get('int_addr') or ''))
|
||||
if not address:
|
||||
return None, 'the internal address is not known yet'
|
||||
return (address, port), ''
|
||||
addresses = listen_addresses(configuration, web.get('listen', DEFAULT_LISTEN))
|
||||
if not addresses:
|
||||
return [], 'the addresses to listen on are not known yet'
|
||||
return [(address, port) for address in [*addresses, LOOPBACK]], ''
|
||||
@@ -10,31 +10,37 @@ from http.server import ThreadingHTTPServer
|
||||
|
||||
|
||||
def serve(wanted, handler, log, interval):
|
||||
"""Work out where to listen every interval seconds with wanted, and run an HTTP server of handler there.
|
||||
"""Work out where to listen every interval seconds with wanted, and run an HTTP server of handler at each place.
|
||||
|
||||
wanted answers ((address, port), '') or (None, the reason not to listen). When the address or
|
||||
the port changes, the server is stopped and a new one listens at the new place; while there is
|
||||
no place or it cannot be bound, no server runs and the next round tries again. Every new
|
||||
reason is logged with log. Does not return.
|
||||
wanted answers ([(address, port), ...], '') or ([], the reason not to listen). A server whose
|
||||
place is no longer wanted is stopped, and a new one listens at each new place; a place that
|
||||
cannot be bound is tried again the next round. Every new message is logged with log. Does
|
||||
not return.
|
||||
"""
|
||||
server, thread, listening, reported = None, None, None, None
|
||||
servers, reported = {}, None
|
||||
while True:
|
||||
target, reason = wanted()
|
||||
if target != listening and server is not None:
|
||||
targets, reason = wanted()
|
||||
for target in [target for target in servers if target not in targets]:
|
||||
server, thread = servers.pop(target)
|
||||
server.shutdown()
|
||||
server.server_close()
|
||||
thread.join()
|
||||
server, listening = None, None
|
||||
if target is not None and server is None:
|
||||
messages = [reason] if reason else []
|
||||
for target in targets:
|
||||
if target in servers:
|
||||
continue
|
||||
try:
|
||||
server = ThreadingHTTPServer(target, handler)
|
||||
except OSError as exc:
|
||||
reason = f'cannot listen on {target[0]}:{target[1]}: {exc}'
|
||||
else:
|
||||
thread = threading.Thread(target=server.serve_forever, name='server', daemon=True)
|
||||
thread.start()
|
||||
listening, reason = target, f'serving on http://{target[0]}:{target[1]}'
|
||||
if reason and reason != reported:
|
||||
log(reason)
|
||||
reported = reason
|
||||
messages.append(f'cannot listen on {target[0]}:{target[1]}: {exc}')
|
||||
continue
|
||||
thread = threading.Thread(target=server.serve_forever, name=f'server {target[0]}', daemon=True)
|
||||
thread.start()
|
||||
servers[target] = (server, thread)
|
||||
if servers:
|
||||
messages.append('serving on ' + ', '.join(f'http://{address}:{port}' for address, port in servers))
|
||||
message = '; '.join(messages)
|
||||
if message and message != reported:
|
||||
log(message)
|
||||
reported = message
|
||||
time.sleep(interval)
|
||||
@@ -0,0 +1,145 @@
|
||||
"""What the router's VPN systems report, for the web service and the metrics service.
|
||||
|
||||
Each reader asks the running system once and answers None when it cannot be asked: swanctl for
|
||||
the IKE SAs and the state and traffic of the site-to-site connections, wg for the WireGuard
|
||||
peers and the OpenVPN server's status file for its clients.
|
||||
"""
|
||||
|
||||
import subprocess
|
||||
import time
|
||||
|
||||
COMMAND_TIMEOUT = 5
|
||||
|
||||
OPENVPN_STATUS = '/run/openvpn-server/status-vpn-router.log'
|
||||
|
||||
#: Seconds since its last handshake within which a WireGuard peer counts as connected.
|
||||
WIREGUARD_CONNECTED_SECONDS = 180
|
||||
|
||||
|
||||
def run(*args):
|
||||
"""The finished process of a command, or None when it cannot be run."""
|
||||
try:
|
||||
return subprocess.run(args, capture_output=True, text=True, timeout=COMMAND_TIMEOUT)
|
||||
except (OSError, subprocess.TimeoutExpired):
|
||||
return None
|
||||
|
||||
|
||||
def swanctl(command):
|
||||
"""The entries of a swanctl list command, or None when swanctl cannot be asked.
|
||||
|
||||
The --pretty output has one item per line: "name {" opens a section, "key = [" a list
|
||||
whose items follow one per line, "}" and "]" close them, and "key = value" is a pair
|
||||
whose value may contain spaces. Every "... event {" section at the top holds one
|
||||
connection or SA; each becomes an entry with its name under "name". The closing
|
||||
"... reply {" section is left out. Values stay strings, as swanctl prints them.
|
||||
Output with more closing than opening lines gives None.
|
||||
"""
|
||||
result = run('swanctl', command, '--pretty')
|
||||
if result is None or result.returncode != 0:
|
||||
return None
|
||||
events = []
|
||||
stack = []
|
||||
for line in result.stdout.splitlines():
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
if line in ('}', ']'):
|
||||
if not stack:
|
||||
return None
|
||||
stack.pop()
|
||||
elif not stack:
|
||||
section = {}
|
||||
if line.endswith(' event {'):
|
||||
events.append(section)
|
||||
stack.append(section)
|
||||
elif isinstance(stack[-1], list):
|
||||
stack[-1].append(line)
|
||||
elif line.endswith(' {'):
|
||||
stack[-1][line[:-2]] = section = {}
|
||||
stack.append(section)
|
||||
elif line.endswith(' = ['):
|
||||
stack[-1][line[:-4]] = items = []
|
||||
stack.append(items)
|
||||
else:
|
||||
key, _, value = line.partition(' = ')
|
||||
stack[-1][key] = value
|
||||
return [dict(body, name=name) for event in events for name, body in event.items()]
|
||||
|
||||
|
||||
def established_each(connections):
|
||||
"""The number of established IKE SAs of each of the connections by name, None for each when swanctl cannot be asked."""
|
||||
sas = swanctl('--list-sas')
|
||||
return {name: None if sas is None else sum(1 for sa in sas if sa['name'] == name and sa.get('state') == 'ESTABLISHED')
|
||||
for name in connections}
|
||||
|
||||
|
||||
def _count(text):
|
||||
"""A counter swanctl prints as a string, as an integer; 0 when it is not a whole number."""
|
||||
return int(text) if isinstance(text, str) and text.isdigit() else 0
|
||||
|
||||
|
||||
def site_to_site_tunnels(connections):
|
||||
"""The state of each of the connections by name, or None when swanctl cannot be asked.
|
||||
|
||||
Each has up, 1 while it has an established IKE SA and 0 otherwise; uptime, the seconds since
|
||||
its oldest established IKE SA was established, None while it is down; and bytes_in,
|
||||
bytes_out, packets_in and packets_out, the sums over the child SAs of its established IKE
|
||||
SAs. The sums start again from zero when a child SA is rekeyed.
|
||||
"""
|
||||
sas = swanctl('--list-sas')
|
||||
if sas is None:
|
||||
return None
|
||||
tunnels = {}
|
||||
for name in connections:
|
||||
up = [sa for sa in sas if sa['name'] == name and sa.get('state') == 'ESTABLISHED']
|
||||
children = [child for sa in up for child in (sa.get('child-sas') or {}).values() if isinstance(child, dict)]
|
||||
tunnels[name] = {
|
||||
'up': 1 if up else 0,
|
||||
'uptime': max((_count(sa.get('established')) for sa in up), default=None),
|
||||
**{key.replace('-', '_'): sum(_count(child.get(key)) for child in children)
|
||||
for key in ('bytes-in', 'bytes-out', 'packets-in', 'packets-out')},
|
||||
}
|
||||
return tunnels
|
||||
|
||||
|
||||
def established(connection):
|
||||
"""The number of established IKE SAs of the connection, or None when swanctl cannot be asked."""
|
||||
return established_each([connection])[connection]
|
||||
|
||||
|
||||
def wireguard_peers():
|
||||
"""One entry per WireGuard peer, with its handshake age and traffic, or None when wg0 is down."""
|
||||
result = run('wg', 'show', 'wg0', 'dump')
|
||||
if result is None or result.returncode != 0:
|
||||
return None
|
||||
now = int(time.time())
|
||||
peers = []
|
||||
for line in result.stdout.splitlines()[1:]:
|
||||
fields = line.split('\t')
|
||||
if len(fields) < 7:
|
||||
continue
|
||||
handshake = int(fields[4])
|
||||
peers.append({
|
||||
'handshake_seconds_ago': now - handshake if handshake else None,
|
||||
'received_bytes': int(fields[5]),
|
||||
'sent_bytes': int(fields[6]),
|
||||
})
|
||||
return peers
|
||||
|
||||
|
||||
def wireguard_connected():
|
||||
"""The number of WireGuard peers with a handshake in the last WIREGUARD_CONNECTED_SECONDS, or None when wg0 is down."""
|
||||
peers = wireguard_peers()
|
||||
if peers is None:
|
||||
return None
|
||||
return sum(1 for peer in peers
|
||||
if peer['handshake_seconds_ago'] is not None and peer['handshake_seconds_ago'] < WIREGUARD_CONNECTED_SECONDS)
|
||||
|
||||
|
||||
def openvpn_clients():
|
||||
"""The number of connected OpenVPN clients, from the server's status file, or None when it cannot be read."""
|
||||
try:
|
||||
with open(OPENVPN_STATUS, encoding='utf-8') as handle:
|
||||
return sum(1 for line in handle if line.startswith('CLIENT_LIST,'))
|
||||
except OSError:
|
||||
return None
|
||||
@@ -1,9 +1,18 @@
|
||||
"""The users and groups of the router's directory, for vpn-router-user, vpn-router-group and the web service.
|
||||
|
||||
A user is an entry in the router's directory, with a first name, a last name and an e-mail
|
||||
address, whose password OpenVPN checks, and a client certificate of the router CA with the
|
||||
user name as its common name, which OpenVPN and IKEv2 accept. Users exist whether or not a
|
||||
P2S system is enabled.
|
||||
A user is an entry in ou=users of the router's directory, of one of two kinds, fixed when it is
|
||||
added. A VPN user is an inetOrgPerson with a first name, a last name and an e-mail address, whose
|
||||
password OpenVPN checks, and a client certificate of the router CA with the user name as its
|
||||
common name, which OpenVPN and IKEv2 accept. A Manager-only account is an account,
|
||||
simpleSecurityObject and vpnRouterAccount (the package's schema) with a name, a password and
|
||||
optionally an e-mail address, no names, no certificate and so no VPN access; it is added when
|
||||
neither the first nor the last name is given, and can sign in to the web service. A service
|
||||
account is a Manager-only account with the scope SERVICE_SCOPE in vpnRouterScope, which another
|
||||
system signs in with to use a router service, and which cannot sign in to the web service; it has
|
||||
a display name and a description (labels), in displayName and description of vpnRouterAccount. The
|
||||
internal service accounts are those the router creates and manages for its own services, labels
|
||||
included (internal_accounts); vpn-router-setup keeps the metrics service's.
|
||||
Users exist whether or not a P2S system is enabled.
|
||||
|
||||
A group is a groupOfNames in ou=groups with a name, which follows the rules of user names, an
|
||||
optional description and at least one member; a group cannot be empty, so the last member
|
||||
@@ -19,8 +28,8 @@ code of it. slapd's otp overlay then takes the password followed by a code at ev
|
||||
A user is disabled when pwdAccountLockedTime holds LOCKED, and locked when the password
|
||||
policy has set it after failed binds, for vpnrouter_ldap.LOCKOUT_SECONDS; unlock ends that.
|
||||
login() checks a user's password and code with a bind. Members of ADMIN_GROUP manage the
|
||||
router through the web service. Changing the password, disabling, unlocking and removing a
|
||||
user revoke its refresh tokens.
|
||||
router through the web service. The module knows only the directory; the callers revoke a
|
||||
user's refresh tokens after the operations of vpnrouter_auth.REVOKING.
|
||||
|
||||
Every operation runs in a session(), which holds the lock that serializes changes and the
|
||||
connection of the user manager account. A request that cannot be done raises Failure, whose
|
||||
@@ -44,7 +53,6 @@ import ldap.modlist
|
||||
import pyotp
|
||||
|
||||
import vpnrouter
|
||||
import vpnrouter_auth
|
||||
import vpnrouter_ldap
|
||||
|
||||
#: A user name is one lowercase DNS label, as it is also the common name of the certificate.
|
||||
@@ -58,8 +66,16 @@ EMAIL = re.compile(r'^[^@\s]+@[^@\s]+\.[^@\s]+$')
|
||||
EMAIL_MAX = 254
|
||||
#: The keys of the person attributes and their LDAP attributes.
|
||||
PERSON = (('first_name', 'givenName'), ('last_name', 'sn'), ('email', 'mail'))
|
||||
#: The keys of the labels of a service account and their LDAP attributes.
|
||||
LABELS = (('display_name', 'displayName'), ('description', 'description'))
|
||||
#: The attributes read for a user.
|
||||
ATTRIBUTES = ['uid', 'pwdAccountLockedTime', 'givenName', 'sn', 'mail', 'oathTOTPToken']
|
||||
ATTRIBUTES = ['uid', 'objectClass', 'pwdAccountLockedTime', 'givenName', 'sn', 'mail', 'oathTOTPToken', 'vpnRouterScope',
|
||||
'displayName', 'description']
|
||||
#: The vpnRouterScope of a service account.
|
||||
SERVICE_SCOPE = 'service'
|
||||
#: The object classes of a VPN user and of a Manager-only account.
|
||||
VPN_USER_CLASSES = [b'inetOrgPerson']
|
||||
MANAGER_ONLY_CLASSES = [b'account', b'simpleSecurityObject', b'vpnRouterAccount']
|
||||
#: The readable attributes of a TOTP token; oathSecret is never readable.
|
||||
TOKEN_ATTRIBUTES = ['oathTOTPToken', 'oathTOTPParams', 'oathSecretTime', 'oathTOTPLastTimeStep', 'oathTOTPTimeStepDrift',
|
||||
'oathLastLogin', 'oathLastFailure', 'oathFailureCount']
|
||||
@@ -135,6 +151,26 @@ def person(values, required):
|
||||
return found
|
||||
|
||||
|
||||
def labels(values):
|
||||
"""The labels of a service account in values ({display_name, description}), checked, as {LDAP attribute: value}.
|
||||
|
||||
A key whose value is None is not given; '' means none. A display name has at most PERSON_NAME_MAX printable
|
||||
characters and a description at most DESCRIPTION_MAX.
|
||||
"""
|
||||
found = {}
|
||||
name = values.get('display_name')
|
||||
if name is not None:
|
||||
if not isinstance(name, str):
|
||||
raise Failure('display_name must be a string', 2)
|
||||
name = name.strip()
|
||||
if len(name) > PERSON_NAME_MAX or any(not character.isprintable() for character in name):
|
||||
raise Failure(f'the display name must have at most {PERSON_NAME_MAX} printable characters', 2)
|
||||
found['displayName'] = name
|
||||
if values.get('description') is not None:
|
||||
found['description'] = check_description(values['description'])
|
||||
return found
|
||||
|
||||
|
||||
def dn_of(name):
|
||||
return f'uid={name},{vpnrouter_ldap.USERS}'
|
||||
|
||||
@@ -172,18 +208,50 @@ def locked_since(value):
|
||||
return 0
|
||||
|
||||
|
||||
def manager_only(attributes):
|
||||
"""Whether the entry is a Manager-only account."""
|
||||
return b'account' in (value.lower() for value in attributes.get('objectClass', []))
|
||||
|
||||
|
||||
def service(attributes):
|
||||
"""Whether the entry is a service account."""
|
||||
return SERVICE_SCOPE.encode() in (value.lower() for value in attributes.get('vpnRouterScope', []))
|
||||
|
||||
|
||||
def internal_accounts():
|
||||
"""The internal service accounts, which the router creates and manages for its own services, by name, each with
|
||||
the labels the router gives it ({display_name, description}): the one metrics.service_account names while metrics
|
||||
are enabled."""
|
||||
try:
|
||||
metrics = vpnrouter.load_config()['metrics']
|
||||
except SystemExit:
|
||||
return {}
|
||||
if not metrics['enabled']:
|
||||
return {}
|
||||
return {metrics['service_account']: {
|
||||
'display_name': 'Metrics',
|
||||
'description': 'The account an OpenTelemetry Collector signs in with to read /metrics of the metrics service.',
|
||||
}}
|
||||
|
||||
|
||||
def summary(attributes):
|
||||
"""A user as the answers show it; a person attribute the entry does not have is None."""
|
||||
"""A user as the answers show it; a person attribute or a label the entry does not have is None.
|
||||
|
||||
internal is whether it is an internal service account (internal_accounts).
|
||||
"""
|
||||
name = attributes['uid'][0].decode()
|
||||
lock = attributes.get('pwdAccountLockedTime', [None])[0]
|
||||
user = {
|
||||
'name': name,
|
||||
'manager_only': manager_only(attributes),
|
||||
'service': service(attributes),
|
||||
'internal': service(attributes) and name in internal_accounts(),
|
||||
'enabled': lock != LOCKED,
|
||||
'locked': lock not in (None, LOCKED) and locked_since(lock) > time.time() - vpnrouter_ldap.LOCKOUT_SECONDS,
|
||||
'certificate': certificate(name)[0].is_file(),
|
||||
'otp': 'oathTOTPToken' in attributes,
|
||||
}
|
||||
for key, attribute in PERSON:
|
||||
for key, attribute in PERSON + LABELS:
|
||||
values = attributes.get(attribute)
|
||||
user[key] = values[0].decode() if values else None
|
||||
return user
|
||||
@@ -196,38 +264,72 @@ def common_name(attributes, name):
|
||||
|
||||
|
||||
def list_users(connection):
|
||||
found = connection.search_s(vpnrouter_ldap.USERS, ldap.SCOPE_ONELEVEL, '(objectClass=inetOrgPerson)', ATTRIBUTES)
|
||||
found = connection.search_s(vpnrouter_ldap.USERS, ldap.SCOPE_ONELEVEL, '(|(objectClass=inetOrgPerson)(objectClass=account))',
|
||||
ATTRIBUTES)
|
||||
return {'users': sorted((summary(attributes) for _, attributes in found), key=lambda user: user['name'])}
|
||||
|
||||
|
||||
def add(connection, name, password, values):
|
||||
"""Add a user with the person attributes of values, all three required, and issue the certificate."""
|
||||
def add(connection, name, password, values, service=False):
|
||||
"""Add a user: a VPN user with its certificate when values gives all three person attributes, a Manager-only account
|
||||
with at most the e-mail address when it gives neither name, and, with service, a service account, with at most the
|
||||
e-mail address and its labels (labels)."""
|
||||
check_name(name)
|
||||
details = person(values, required=True)
|
||||
details = person(values, required=False)
|
||||
vpn_user = 'givenName' in details or 'sn' in details
|
||||
if service and vpn_user:
|
||||
raise Failure('a service account has no first or last name', 2)
|
||||
if service:
|
||||
details.update({attribute: value for attribute, value in labels(values).items() if value})
|
||||
if vpn_user and len(details) != len(PERSON):
|
||||
raise Failure('give first_name, last_name and email for a VPN user, or at most email for a Manager-only account', 2)
|
||||
check_password(password)
|
||||
if not (vpnrouter.PKI_DIR / 'ca_key.pem').is_file():
|
||||
if vpn_user and not (vpnrouter.PKI_DIR / 'ca_key.pem').is_file():
|
||||
raise Failure('the router CA does not exist; vpn-router-setup creates it once a platform is chosen')
|
||||
record = {'objectClass': [b'inetOrgPerson'], 'uid': [name.encode()],
|
||||
'cn': [common_name(details, name).encode()]}
|
||||
if vpn_user:
|
||||
record = {'objectClass': VPN_USER_CLASSES, 'uid': [name.encode()], 'cn': [common_name(details, name).encode()]}
|
||||
else:
|
||||
# simpleSecurityObject requires a userPassword; passwd_s sets the real one at once.
|
||||
record = {'objectClass': MANAGER_ONLY_CLASSES, 'uid': [name.encode()], 'userPassword': [b'{CRYPT}!']}
|
||||
if service:
|
||||
record['vpnRouterScope'] = [SERVICE_SCOPE.encode()]
|
||||
record.update({attribute: [value.encode()] for attribute, value in details.items()})
|
||||
try:
|
||||
connection.add_s(dn_of(name), ldap.modlist.addModlist(record))
|
||||
except ldap.ALREADY_EXISTS:
|
||||
raise Failure(f'user {name} exists', 4)
|
||||
connection.passwd_s(dn_of(name), None, password)
|
||||
vpnrouter.USER_CERT_DIR.mkdir(mode=0o700, parents=True, exist_ok=True)
|
||||
for path in certificate(name):
|
||||
path.unlink(missing_ok=True)
|
||||
simple_ca('make-cert', '--ca-dir', str(vpnrouter.PKI_DIR), '--cert-dir', str(vpnrouter.USER_CERT_DIR), name)
|
||||
if vpn_user:
|
||||
vpnrouter.USER_CERT_DIR.mkdir(mode=0o700, parents=True, exist_ok=True)
|
||||
for path in certificate(name):
|
||||
path.unlink(missing_ok=True)
|
||||
simple_ca('make-cert', '--ca-dir', str(vpnrouter.PKI_DIR), '--cert-dir', str(vpnrouter.USER_CERT_DIR), name)
|
||||
return summary(entry(connection, name))
|
||||
|
||||
|
||||
def update(connection, name, values):
|
||||
"""Change the person attributes values gives; the cn follows the first and the last name."""
|
||||
"""Change the person attributes values gives; the cn follows the first and the last name.
|
||||
|
||||
A Manager-only account has only the e-mail address, which '' removes; a service account also its labels
|
||||
(update_service).
|
||||
"""
|
||||
current = entry(connection, name)
|
||||
if service(current):
|
||||
return update_service(connection, name, values)
|
||||
if manager_only(current):
|
||||
if values.get('first_name') is not None or values.get('last_name') is not None:
|
||||
raise Failure(f'user {name} is a Manager-only account, which has no first or last name', 2)
|
||||
email = values.get('email')
|
||||
if email is None:
|
||||
raise Failure('give email', 2)
|
||||
if isinstance(email, str) and email.strip() == '':
|
||||
if 'mail' in current:
|
||||
connection.modify_s(dn_of(name), [(ldap.MOD_DELETE, 'mail', None)])
|
||||
else:
|
||||
connection.modify_s(dn_of(name), [(ldap.MOD_REPLACE, 'mail', [person({'email': email}, required=False)['mail'].encode()])])
|
||||
return summary(entry(connection, name))
|
||||
details = person(values, required=False)
|
||||
if not details:
|
||||
raise Failure('give at least one of first_name, last_name and email', 2)
|
||||
current = entry(connection, name)
|
||||
names = {attribute: current[attribute][0].decode() for attribute in ('givenName', 'sn') if current.get(attribute)}
|
||||
names.update({attribute: details[attribute] for attribute in ('givenName', 'sn') if attribute in details})
|
||||
changes = [(ldap.MOD_REPLACE, attribute, [value.encode()]) for attribute, value in details.items()]
|
||||
@@ -236,18 +338,31 @@ def update(connection, name, values):
|
||||
return summary(entry(connection, name))
|
||||
|
||||
|
||||
def update_service(connection, name, values):
|
||||
"""Change any of the e-mail address and the labels (labels) of a service account that values gives; '' removes one."""
|
||||
if not service(entry(connection, name)):
|
||||
raise Failure(f'{name} is not a service account', 3)
|
||||
changes = [(ldap.MOD_REPLACE, attribute, [value.encode()] if value else []) for attribute, value in labels(values).items()]
|
||||
email = values.get('email')
|
||||
if email is not None:
|
||||
address = '' if isinstance(email, str) and email.strip() == '' else person({'email': email}, required=False)['mail']
|
||||
changes.append((ldap.MOD_REPLACE, 'mail', [address.encode()] if address else []))
|
||||
if not changes:
|
||||
raise Failure('give email, display_name or description', 2)
|
||||
connection.modify_s(dn_of(name), changes)
|
||||
return summary(entry(connection, name))
|
||||
|
||||
|
||||
def passwd(connection, name, password):
|
||||
check_password(password)
|
||||
entry(connection, name)
|
||||
connection.passwd_s(dn_of(name), None, password)
|
||||
vpnrouter_auth.revoke(vpnrouter.load_config(), name)
|
||||
return summary(entry(connection, name))
|
||||
|
||||
|
||||
def disable(connection, name):
|
||||
if entry(connection, name).get('pwdAccountLockedTime') != [LOCKED]:
|
||||
connection.modify_s(dn_of(name), [(ldap.MOD_REPLACE, 'pwdAccountLockedTime', [LOCKED])])
|
||||
vpnrouter_auth.revoke(vpnrouter.load_config(), name)
|
||||
return summary(entry(connection, name))
|
||||
|
||||
|
||||
@@ -266,7 +381,6 @@ def unlock(connection, name):
|
||||
changes.append((ldap.MOD_DELETE, 'pwdAccountLockedTime', None))
|
||||
if changes:
|
||||
connection.modify_s(dn_of(name), changes)
|
||||
vpnrouter_auth.revoke(vpnrouter.load_config(), name)
|
||||
return summary(entry(connection, name))
|
||||
|
||||
|
||||
@@ -361,7 +475,6 @@ def remove(connection, name):
|
||||
cert.unlink(missing_ok=True)
|
||||
key.unlink(missing_ok=True)
|
||||
connection.delete_s(dn_of(name))
|
||||
vpnrouter_auth.revoke(vpnrouter.load_config(), name)
|
||||
return {'name': name, 'removed': True}
|
||||
|
||||
|
||||
|
||||
+12
@@ -0,0 +1,12 @@
|
||||
#!/bin/sh
|
||||
# Wait until libpq is in the dynamic linker cache, where psycopg looks it up.
|
||||
# ExecStartPre= of vpn-router-web.service; TimeoutStartSec= of the unit bounds the wait.
|
||||
|
||||
reported=
|
||||
until ldconfig -p | grep -q '^[[:space:]]*libpq\.so\.5 '; do
|
||||
if [ -z "$reported" ]; then
|
||||
echo 'waiting for libpq.so.5 in the linker cache'
|
||||
reported=yes
|
||||
fi
|
||||
sleep 2
|
||||
done
|
||||
@@ -1,5 +1,5 @@
|
||||
#!/usr/bin/python3
|
||||
"""Serve the status of the router on its internal address.
|
||||
"""Serve the status of the router on its internal address, its external address or both, as web.listen sets.
|
||||
|
||||
Run by vpn-router-web.service. Users are managed with vpnrouter_users, the module that
|
||||
vpn-router-user uses too.
|
||||
@@ -12,35 +12,55 @@ Pages:
|
||||
runs, as JSON
|
||||
/users the users in the router's directory, whether or not a P2S system is enabled: list,
|
||||
add, change, set the password, disable, enable, remove, the OpenVPN client
|
||||
profile, and TOTP enrolment (see users())
|
||||
profile, and TOTP enrolment (see users()); service accounts are not among them
|
||||
/service-accounts
|
||||
the service accounts: list, add, change the e-mail address, set the password,
|
||||
disable, enable, unlock and remove; the internal ones are only listed (see
|
||||
service_accounts())
|
||||
/groups the groups of the router's directory: list, add, rename or describe, remove, and
|
||||
add and take out members (see groups())
|
||||
/account the account of the signed-in user: its details, its password and its TOTP
|
||||
enrolment (see account())
|
||||
/site-to-site
|
||||
the site-to-site connections of the configuration: list, add, change and remove,
|
||||
applied by vpn-router-setup (see site_to_site())
|
||||
/auth sign-in with a user's password and TOTP code, refresh and sign-out (see auth())
|
||||
/metrics/history
|
||||
the metrics the metrics service keeps, for a view: hour, day, month or year (see
|
||||
metrics_history())
|
||||
/postgresql
|
||||
the databases of each PostgreSQL cluster on the router with their sizes, as JSON (see
|
||||
postgresql_databases()); /config has the clusters' settings
|
||||
|
||||
/config, /users and /groups need an HS256 JWT in an Authorization: Bearer header signed with
|
||||
vpnrouter_auth.WEB_KEY (see vpnrouter_auth.verify). A token of /auth names a user in sub, who
|
||||
must be a member of vpnrouter_users.ADMIN_GROUP; a token of vpn-router-token has the admin
|
||||
claim, and only such a token may change ADMIN_GROUP: create, rename to it, change, remove it,
|
||||
/config, /metrics/history, /postgresql, /users, /service-accounts, /groups, /site-to-site and /account need an HS256 JWT in an Authorization: Bearer header signed
|
||||
with vpnrouter_auth.WEB_KEY (see vpnrouter_auth.verify). A token of /auth names a user in sub;
|
||||
/account serves that user, and the other paths need a member of vpnrouter_users.ADMIN_GROUP. A
|
||||
token of vpn-router-token has the admin claim and names no user; it may call every path but
|
||||
/account, and only such a token may change ADMIN_GROUP: create, rename to it, change, remove it,
|
||||
add or take out its members, and remove one of its members. These paths and /auth are not
|
||||
served (404) while the key does not exist. /auth is limited to AUTH_LIMIT calls per client
|
||||
address in AUTH_WINDOW seconds; the address of a call from one of web.trusted_proxies, by
|
||||
default the address the service listens on, is the last entry of its X-Forwarded-For header.
|
||||
default the address the call came in on, is the last entry of its X-Forwarded-For header.
|
||||
|
||||
/ and /status show only the status: whether the services run, the package version, whether
|
||||
Docker is installed, and the state of site-to-site, road-warrior (IPsec P2S), OpenVPN and
|
||||
WireGuard. No configuration data, key or address is shown.
|
||||
/ and /status show only the status, as much as web.status_content says (see shown): basic, the
|
||||
default, the overall status, the time and the client's address (its address as for the rate limit
|
||||
below); standard also the package version, the installed version of vpn-router-manager
|
||||
(manager_version, null when it is not installed) and the number of active and other services; full
|
||||
whether each service runs, whether Docker is installed, and the state of each site-to-site connection, road-warrior
|
||||
(IPsec P2S), OpenVPN and WireGuard. No configuration data or key is shown, and no address but the
|
||||
client's.
|
||||
|
||||
The service first initializes, without listening: it loads the PostgreSQL driver
|
||||
(vpnrouter_auth.driver; psycopg finds libpq through the linker cache, which the package manager
|
||||
refreshes only at the end of an installation), then it makes the signing key and the refresh
|
||||
The service first initializes, without listening: it makes the signing key and the refresh
|
||||
tokens' database (vpnrouter_auth), retrying every INIT_RETRY_SECONDS until both are ready and
|
||||
logging each failure. Then it listens.
|
||||
logging each failure. Then it listens. Its unit starts it only once wait-for-libpq has found libpq
|
||||
in the linker cache, where psycopg looks it up.
|
||||
|
||||
The service needs nothing else to start and does not end on its own. Every DISCOVERY_SECONDS it
|
||||
works out the internal address from /etc/vpn-router/vpn-router.json as general.mode directs,
|
||||
with web.port and web.enabled (vpnrouter_net.web_listen), and runs the server there
|
||||
(vpnrouter_serve.serve): it starts it, moves it when the address or the port changes, and stops
|
||||
it while the service is disabled or the address is not known. Every request reads what it needs when it is made: the signing
|
||||
works out from /etc/vpn-router/vpn-router.json the addresses of the interfaces web.listen names,
|
||||
internal, external or both, as general.mode directs, with web.port and web.enabled
|
||||
(vpnrouter_net.web_listen), and runs a server on each (vpnrouter_serve.serve): it starts them,
|
||||
moves them when an address or the port changes, and stops them while the service is disabled or
|
||||
no address is known. Every request reads what it needs when it is made: the signing
|
||||
key, the configuration, and the features of the router and the facts shown by /config from
|
||||
/run/vpn-router/web.json, which vpn-router-setup writes. Before setup has written that file
|
||||
the router has no features, and /config shows no configuration or interfaces.
|
||||
@@ -50,10 +70,10 @@ import collections
|
||||
import json
|
||||
import os
|
||||
import signal
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
import urllib.parse
|
||||
from html import escape
|
||||
from http.server import BaseHTTPRequestHandler
|
||||
|
||||
@@ -63,9 +83,11 @@ import ldap
|
||||
|
||||
import vpnrouter
|
||||
import vpnrouter_auth
|
||||
import vpnrouter_metrics
|
||||
import vpnrouter_net
|
||||
import vpnrouter_serve
|
||||
import vpnrouter_users
|
||||
from vpnrouter_status import established, established_each, openvpn_clients, run, swanctl, wireguard_peers
|
||||
|
||||
STATE_FILE = '/run/vpn-router/web.json'
|
||||
CONFIG_FILE = '/etc/vpn-router/vpn-router.json'
|
||||
@@ -78,7 +100,6 @@ DEFAULT_PORT = 7081
|
||||
|
||||
#: Seconds between two discoveries of where the service should listen.
|
||||
DISCOVERY_SECONDS = 30
|
||||
COMMAND_TIMEOUT = 5
|
||||
CACHE_SECONDS = 2
|
||||
|
||||
#: The calls of /auth allowed per client address in AUTH_WINDOW seconds.
|
||||
@@ -90,11 +111,16 @@ TAILWIND = 'https://cdn.jsdelivr.net/npm/@tailwindcss/browser@4'
|
||||
#: Seconds after which the status page in a browser reloads itself.
|
||||
REFRESH_SECONDS = 60
|
||||
|
||||
OPENVPN_STATUS = '/run/openvpn-server/status-vpn-router.log'
|
||||
#: The PostgreSQL settings /config shows for each cluster.
|
||||
POSTGRESQL_SETTINGS = ('server_version', 'listen_addresses', 'port', 'max_connections', 'shared_buffers',
|
||||
'effective_cache_size', 'work_mem', 'password_encryption', 'ssl', 'timezone')
|
||||
|
||||
#: The largest request body accepted, in bytes.
|
||||
BODY_LIMIT = 4096
|
||||
|
||||
#: The settings of a site_to_site connection that /site-to-site adds and changes.
|
||||
CONNECTION_SETTINGS = ('addrs', 'id', 'cidrs', 'local_cidrs', 'local_id_mode', 'psk')
|
||||
|
||||
#: The HTTP status for each status of a vpnrouter_users.Failure; any other is 500.
|
||||
USER_STATUS = {2: 400, 3: 404, 4: 409, 5: 401}
|
||||
|
||||
@@ -131,22 +157,15 @@ def current():
|
||||
return dict(read_json(STATE_FILE), key=vpnrouter_auth.key())
|
||||
|
||||
|
||||
def run(*args):
|
||||
"""The finished process of a command, or None when it cannot be run."""
|
||||
try:
|
||||
return subprocess.run(args, capture_output=True, text=True, timeout=COMMAND_TIMEOUT)
|
||||
except (OSError, subprocess.TimeoutExpired):
|
||||
return None
|
||||
|
||||
|
||||
def unit_state(unit):
|
||||
result = run('systemctl', 'is-active', unit)
|
||||
return result.stdout.strip() if result and result.stdout.strip() else 'unknown'
|
||||
|
||||
|
||||
def package_version():
|
||||
result = run('dpkg-query', '-W', '-f=${Version}', 'vpn-router')
|
||||
return result.stdout.strip() if result and result.returncode == 0 else None
|
||||
def package_version(package='vpn-router'):
|
||||
"""The installed version of the package, or None when it is not installed."""
|
||||
result = run('dpkg-query', '-W', '-f=${Version}', package)
|
||||
return (result.stdout.strip() or None) if result and result.returncode == 0 else None
|
||||
|
||||
|
||||
def docker_status():
|
||||
@@ -161,88 +180,23 @@ def docker_status():
|
||||
}
|
||||
|
||||
|
||||
def swanctl(command):
|
||||
"""The entries of a swanctl list command, or None when swanctl cannot be asked.
|
||||
def postgresql_clusters():
|
||||
"""The state of each PostgreSQL cluster unit (postgresql@<version>-<name>), or {'postgresql': 'no cluster'} without one.
|
||||
|
||||
The --pretty output has one item per line: "name {" opens a section, "key = [" a list
|
||||
whose items follow one per line, "}" and "]" close them, and "key = value" is a pair
|
||||
whose value may contain spaces. Every "... event {" section at the top holds one
|
||||
connection or SA; each becomes an entry with its name under "name". The closing
|
||||
"... reply {" section is left out. Values stay strings, as swanctl prints them.
|
||||
Output with more closing than opening lines gives None.
|
||||
postgresql.service, which only runs /bin/true, is not reported.
|
||||
"""
|
||||
result = run('swanctl', command, '--pretty')
|
||||
if result is None or result.returncode != 0:
|
||||
return None
|
||||
events = []
|
||||
stack = []
|
||||
for line in result.stdout.splitlines():
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
if line in ('}', ']'):
|
||||
if not stack:
|
||||
return None
|
||||
stack.pop()
|
||||
elif not stack:
|
||||
section = {}
|
||||
if line.endswith(' event {'):
|
||||
events.append(section)
|
||||
stack.append(section)
|
||||
elif isinstance(stack[-1], list):
|
||||
stack[-1].append(line)
|
||||
elif line.endswith(' {'):
|
||||
stack[-1][line[:-2]] = section = {}
|
||||
stack.append(section)
|
||||
elif line.endswith(' = ['):
|
||||
stack[-1][line[:-4]] = items = []
|
||||
stack.append(items)
|
||||
else:
|
||||
key, _, value = line.partition(' = ')
|
||||
stack[-1][key] = value
|
||||
return [dict(body, name=name) for event in events for name, body in event.items()]
|
||||
|
||||
|
||||
def established(connection):
|
||||
"""The number of established IKE SAs of the connection, or None when swanctl cannot be asked."""
|
||||
sas = swanctl('--list-sas')
|
||||
if sas is None:
|
||||
return None
|
||||
return sum(1 for sa in sas if sa['name'] == connection and sa.get('state') == 'ESTABLISHED')
|
||||
|
||||
|
||||
def wireguard_peers():
|
||||
"""One entry per WireGuard peer, with its handshake age and traffic, or None when wg0 is down."""
|
||||
result = run('wg', 'show', 'wg0', 'dump')
|
||||
if result is None or result.returncode != 0:
|
||||
return None
|
||||
now = int(time.time())
|
||||
peers = []
|
||||
for line in result.stdout.splitlines()[1:]:
|
||||
fields = line.split('\t')
|
||||
if len(fields) < 7:
|
||||
continue
|
||||
handshake = int(fields[4])
|
||||
peers.append({
|
||||
'handshake_seconds_ago': now - handshake if handshake else None,
|
||||
'received_bytes': int(fields[5]),
|
||||
'sent_bytes': int(fields[6]),
|
||||
})
|
||||
return peers
|
||||
|
||||
|
||||
def openvpn_clients():
|
||||
"""The number of connected OpenVPN clients, from the server's status file, or None when it cannot be read."""
|
||||
try:
|
||||
with open(OPENVPN_STATUS, encoding='utf-8') as handle:
|
||||
return sum(1 for line in handle if line.startswith('CLIENT_LIST,'))
|
||||
except OSError:
|
||||
return None
|
||||
lines = lines_of('systemctl', 'list-units', 'postgresql@*.service', '--all', '--plain', '--no-legend') or []
|
||||
names = [line.split()[0].removesuffix('.service') for line in lines if line.strip()]
|
||||
return {name: unit_state(f'{name}.service') for name in names} or {'postgresql': 'no cluster'}
|
||||
|
||||
|
||||
def collect(state):
|
||||
"""The status document. state says which features the router has."""
|
||||
units = {'vpn-router-setup': unit_state('vpn-router-setup.service'), 'slapd': unit_state('slapd.service')}
|
||||
units = {'vpn-router-setup': unit_state('vpn-router-setup.service'), 'slapd': unit_state('slapd.service'),
|
||||
**postgresql_clusters()}
|
||||
manager_version = package_version('vpn-router-manager')
|
||||
if manager_version:
|
||||
units['vpn-router-manager'] = unit_state('vpn-router-manager.service')
|
||||
if state.get('site_to_site') or state.get('p2s'):
|
||||
units['strongswan'] = unit_state('strongswan.service')
|
||||
if state.get('wireguard'):
|
||||
@@ -253,13 +207,15 @@ def collect(state):
|
||||
status = {
|
||||
'status': 'ok' if all(value == 'active' for value in units.values()) else 'degraded',
|
||||
'version': package_version(),
|
||||
'manager_version': manager_version,
|
||||
'time': time.strftime('%Y-%m-%dT%H:%M:%S%z'),
|
||||
'units': units,
|
||||
'docker': docker_status(),
|
||||
}
|
||||
if state.get('site_to_site'):
|
||||
count = established('remote-site')
|
||||
status['site_to_site'] = {'established': None if count is None else count > 0}
|
||||
names = state.get('site_to_site')
|
||||
if isinstance(names, list) and names:
|
||||
status['site_to_site'] = {'connections': {name: None if count is None else count > 0
|
||||
for name, count in established_each(names).items()}}
|
||||
if state.get('p2s'):
|
||||
status['p2s'] = {'clients': established('road-warrior')}
|
||||
if state.get('wireguard'):
|
||||
@@ -269,6 +225,26 @@ def collect(state):
|
||||
return status
|
||||
|
||||
|
||||
def shown(status, content, client):
|
||||
"""The status document as web.status_content shows it to the client at its address.
|
||||
|
||||
Every content has status, time and client_ip, the client's address, and content, which one it is: basic nothing
|
||||
more; standard adds version, manager_version and services, the number of active units and of units in every
|
||||
other state; full adds the whole status document of collect. Content that is not one of
|
||||
vpnrouter.STATUS_CONTENTS, or not set, is basic.
|
||||
"""
|
||||
if content not in vpnrouter.STATUS_CONTENTS:
|
||||
content = 'basic'
|
||||
if content == 'full':
|
||||
return {**status, 'content': content, 'client_ip': client}
|
||||
document = {'status': status['status'], 'content': content, 'time': status['time'], 'client_ip': client}
|
||||
if content == 'standard':
|
||||
active = sum(1 for value in status['units'].values() if value == 'active')
|
||||
document.update(version=status['version'], manager_version=status['manager_version'],
|
||||
services={'active': active, 'failed': len(status['units']) - active})
|
||||
return document
|
||||
|
||||
|
||||
def lines_of(*args):
|
||||
"""The output lines of a command, or None when it fails."""
|
||||
result = run(*args)
|
||||
@@ -307,6 +283,65 @@ def wireguard_details():
|
||||
}
|
||||
|
||||
|
||||
def postgresql_query(cluster, sql):
|
||||
"""The JSON value an SQL statement answers on a cluster (version/name), run by psql as postgres, or None when it fails."""
|
||||
result = run('runuser', '-u', 'postgres', '--', 'psql', '--cluster', cluster, '-X', '-q', '-t', '-A',
|
||||
'-v', 'ON_ERROR_STOP=1', '-d', 'postgres', '-c', sql)
|
||||
if result is None or result.returncode != 0:
|
||||
return None
|
||||
try:
|
||||
return json.loads(result.stdout)
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
def postgresql_clusters_of():
|
||||
"""The PostgreSQL clusters of pg_lsclusters, each with version, name, port, status and data_directory, or None when it fails."""
|
||||
lines = lines_of('pg_lsclusters', '--no-header')
|
||||
if lines is None:
|
||||
return None
|
||||
clusters = []
|
||||
for line in lines:
|
||||
fields = line.split()
|
||||
if len(fields) >= 6:
|
||||
clusters.append({'version': fields[0], 'name': fields[1], 'port': int(fields[2]) if fields[2].isdigit() else None,
|
||||
'status': fields[3], 'data_directory': fields[5]})
|
||||
return clusters
|
||||
|
||||
|
||||
def postgresql_configuration():
|
||||
"""The PostgreSQL clusters on the router with the settings of POSTGRESQL_SETTINGS of each one that is online (None
|
||||
for one that is not), or None when pg_lsclusters fails."""
|
||||
clusters = postgresql_clusters_of()
|
||||
if clusters is None:
|
||||
return None
|
||||
names = ', '.join(f"'{name}'" for name in POSTGRESQL_SETTINGS)
|
||||
for cluster in clusters:
|
||||
cluster['settings'] = postgresql_query(
|
||||
f"{cluster['version']}/{cluster['name']}",
|
||||
f'SELECT json_object_agg(name, current_setting(name) ORDER BY name) FROM pg_settings WHERE name IN ({names})',
|
||||
) if cluster['status'].startswith('online') else None
|
||||
return {'clusters': clusters}
|
||||
|
||||
|
||||
def postgresql_databases():
|
||||
"""What /postgresql shows: the PostgreSQL clusters on the router, each with its databases (name, owner, encoding and
|
||||
size in bytes) and their total size, None for a cluster that is not online; clusters is None when pg_lsclusters fails."""
|
||||
clusters = postgresql_clusters_of()
|
||||
if clusters is None:
|
||||
return {'clusters': None}
|
||||
for cluster in clusters:
|
||||
databases = postgresql_query(
|
||||
f"{cluster['version']}/{cluster['name']}",
|
||||
"SELECT coalesce(json_agg(json_build_object('name', datname, 'owner', pg_get_userbyid(datdba), "
|
||||
"'encoding', pg_encoding_to_char(encoding), 'size', pg_database_size(oid)) ORDER BY datname), '[]') "
|
||||
'FROM pg_database',
|
||||
) if cluster['status'].startswith('online') else None
|
||||
cluster['databases'] = databases
|
||||
cluster['size'] = sum(database['size'] for database in databases) if isinstance(databases, list) else None
|
||||
return {'clusters': clusters}
|
||||
|
||||
|
||||
def details(state):
|
||||
"""What /config shows: the configuration without its secrets, and what the router resolved and runs."""
|
||||
configuration = state.get('configuration', {})
|
||||
@@ -322,19 +357,27 @@ def details(state):
|
||||
'exposed_ports': configuration.get('docker', {}).get('exposed_ports', []),
|
||||
'rules': lines_of('iptables', '-S', 'VPN-ROUTER-DOCKER'),
|
||||
},
|
||||
'postgresql': postgresql_configuration(),
|
||||
}
|
||||
|
||||
|
||||
def text_of(status):
|
||||
"""The status for people, as plain text."""
|
||||
lines = ['VPN Router', f'Status: {status["status"]}', f'Version: {status["version"] or "unknown"}', '']
|
||||
lines += [f'{name}: {value}' for name, value in status['units'].items()]
|
||||
docker = status['docker']
|
||||
lines.append('Docker: not installed' if not docker['installed']
|
||||
else f'Docker: installed, service {docker["service"]}, rules {docker["rules"]}')
|
||||
if 'site_to_site' in status:
|
||||
value = status['site_to_site']['established']
|
||||
lines.append(f'Site to site: {"unknown" if value is None else "established" if value else "not established"}')
|
||||
"""The status document of shown for people, as plain text: a header, a blank line, then name: value lines."""
|
||||
lines = ['VPN Router', f'Status: {status["status"]}']
|
||||
if 'version' in status:
|
||||
lines.append(f'Version: {status["version"] or "unknown"}')
|
||||
lines += ['', f'Client IP: {status["client_ip"]}']
|
||||
if 'manager_version' in status:
|
||||
lines.append(f'Router Manager: {status["manager_version"] or "not installed"}')
|
||||
if 'services' in status:
|
||||
lines.append(f'Services: {status["services"]["active"]} active, {status["services"]["failed"]} failed')
|
||||
lines += [f'{name}: {value}' for name, value in status.get('units', {}).items()]
|
||||
if 'docker' in status:
|
||||
docker = status['docker']
|
||||
lines.append('Docker: not installed' if not docker['installed']
|
||||
else f'Docker: installed, service {docker["service"]}, rules {docker["rules"]}')
|
||||
for name, value in status.get('site_to_site', {}).get('connections', {}).items():
|
||||
lines.append(f'Site to site {name}: {"unknown" if value is None else "established" if value else "not established"}')
|
||||
if 'p2s' in status:
|
||||
clients = status['p2s']['clients']
|
||||
lines.append(f'Road-warrior (IPsec P2S): {"unknown" if clients is None else f"{clients} connected"}')
|
||||
@@ -355,11 +398,12 @@ def text_of(status):
|
||||
|
||||
|
||||
def html_of(status):
|
||||
"""The status for a browser, as a page styled with Tailwind CSS."""
|
||||
"""The status document of shown for a browser, as a page styled with Tailwind CSS."""
|
||||
colour = 'bg-emerald-600 dark:bg-emerald-700' if status['status'] == 'ok' else 'bg-amber-600 dark:bg-amber-700'
|
||||
# Each "name: value" line of the text is a row; indented lines, such as WireGuard peers, are indented here too.
|
||||
version = f', version {status["version"] or "unknown"}' if 'version' in status else ''
|
||||
# Each "name: value" line after the header of the text is a row; indented lines, such as WireGuard peers, are indented here too.
|
||||
rows = []
|
||||
for line in text_of(status).splitlines()[4:]:
|
||||
for line in text_of(status).split('\n\n', 1)[1].splitlines():
|
||||
if not line:
|
||||
continue
|
||||
name, _, value = line.partition(': ')
|
||||
@@ -383,7 +427,7 @@ def html_of(status):
|
||||
<div class="overflow-hidden rounded-2xl border border-slate-200 bg-white shadow-lg dark:border-slate-700 dark:bg-slate-800">
|
||||
<div class="{colour} px-6 py-4 text-white">
|
||||
<h1 class="text-xl font-semibold">VPN Router</h1>
|
||||
<p class="mt-1 text-sm">Status: {escape(status['status'])}, version {escape(status['version'] or 'unknown')}</p>
|
||||
<p class="mt-1 text-sm">Status: {escape(status['status'] + version)}</p>
|
||||
</div>
|
||||
<div class="px-6 py-4">
|
||||
<table class="w-full text-sm"><tbody>{body}</tbody></table>
|
||||
@@ -400,7 +444,8 @@ def users(state, method, path, body):
|
||||
"""Answer a /users request with vpnrouter_users: the HTTP status and the JSON document.
|
||||
|
||||
GET /users the users
|
||||
POST /users add a user: {"name", "password", "first_name", "last_name", "email"}
|
||||
POST /users add a user: {"name", "password", "first_name", "last_name", "email"};
|
||||
without the names a Manager-only account
|
||||
PUT /users/<name> change any of {"first_name", "last_name", "email"}
|
||||
PUT /users/<name>/password set the password: {"password": ...}
|
||||
POST /users/<name>/disable lock the user out of OpenVPN
|
||||
@@ -411,6 +456,8 @@ def users(state, method, path, body):
|
||||
POST /users/<name>/otp a new TOTP secret and its otpauth URI; nothing is stored
|
||||
PUT /users/<name>/otp store a TOTP secret: {"secret", "code"}, code a code of the secret
|
||||
DELETE /users/<name>/otp remove the TOTP token
|
||||
|
||||
Service accounts are served by /service-accounts: they are not listed, and a request naming one is 404.
|
||||
"""
|
||||
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
|
||||
return 400, {'error': 'the body must be a JSON object'}
|
||||
@@ -419,7 +466,8 @@ def users(state, method, path, body):
|
||||
route = (method, len(parts), parts[2] if len(parts) == 3 else None)
|
||||
name = parts[1] if len(parts) > 1 else None
|
||||
actions = {
|
||||
('GET', 1, None): lambda connection: vpnrouter_users.list_users(connection),
|
||||
('GET', 1, None): lambda connection: {
|
||||
'users': [user for user in vpnrouter_users.list_users(connection)['users'] if not user['service']]},
|
||||
('POST', 1, None): lambda connection: vpnrouter_users.add(connection, body.get('name'), body.get('password'), body),
|
||||
('PUT', 2, None): lambda connection: vpnrouter_users.update(connection, name, body),
|
||||
('PUT', 3, 'password'): lambda connection: vpnrouter_users.passwd(connection, name, body.get('password')),
|
||||
@@ -432,11 +480,18 @@ def users(state, method, path, body):
|
||||
('PUT', 3, 'otp'): lambda connection: vpnrouter_users.otp_enrol(connection, name, body.get('secret'), body.get('code')),
|
||||
('DELETE', 3, 'otp'): lambda connection: vpnrouter_users.otp_remove(connection, name),
|
||||
}
|
||||
# The vpnrouter_users operation of each route that may revoke refresh tokens.
|
||||
operations = {('PUT', 3, 'password'): 'passwd', ('POST', 3, 'disable'): 'disable', ('POST', 3, 'unlock'): 'unlock',
|
||||
('DELETE', 2, None): 'remove'}
|
||||
if route not in actions:
|
||||
return 404, {'error': f'no route {method} {path}'}
|
||||
try:
|
||||
with vpnrouter_users.session() as connection:
|
||||
if name is not None and vpnrouter_users.service(vpnrouter_users.entry(connection, name)):
|
||||
raise vpnrouter_users.Failure(f'{name} is a service account; see /service-accounts', 3)
|
||||
document = actions[route](connection)
|
||||
if operations.get(route) in vpnrouter_auth.REVOKING:
|
||||
vpnrouter_auth.revoke(vpnrouter.load_config(), name)
|
||||
except vpnrouter_users.Failure as failure:
|
||||
return USER_STATUS.get(failure.status, 500), {'error': str(failure)}
|
||||
except (ldap.LDAPError, OSError) as exc:
|
||||
@@ -449,6 +504,56 @@ def users(state, method, path, body):
|
||||
return (201 if route == ('POST', 1, None) else 200), document
|
||||
|
||||
|
||||
def service_accounts(method, path, body):
|
||||
"""Answer a /service-accounts request with vpnrouter_users: the HTTP status and the JSON document.
|
||||
|
||||
GET /service-accounts the service accounts: {"service_accounts": [...]}
|
||||
POST /service-accounts add one: {"name", "password", "display_name"?, "description"?, "email"?}
|
||||
PUT /service-accounts/<name> change any of {"display_name", "description", "email"}; "" removes one
|
||||
PUT /service-accounts/<name>/password set the password: {"password": ...}
|
||||
POST /service-accounts/<name>/disable lock it out
|
||||
POST /service-accounts/<name>/enable let it in again
|
||||
POST /service-accounts/<name>/unlock end a lockout after failed logins
|
||||
DELETE /service-accounts/<name> remove it
|
||||
|
||||
A name that is not a service account is 404. The internal service accounts (vpnrouter_users.internal_accounts),
|
||||
which the router manages, are only listed: every other request on one is refused with 403.
|
||||
"""
|
||||
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
|
||||
return 400, {'error': 'the body must be a JSON object'}
|
||||
body = body or {}
|
||||
parts = path.strip('/').split('/')
|
||||
route = (method, len(parts), parts[2] if len(parts) == 3 else None)
|
||||
name = parts[1] if len(parts) > 1 else None
|
||||
settings = {key: body.get(key) for key in ('email', 'display_name', 'description')}
|
||||
actions = {
|
||||
('GET', 1, None): lambda connection: {
|
||||
'service_accounts': [user for user in vpnrouter_users.list_users(connection)['users'] if user['service']]},
|
||||
('POST', 1, None): lambda connection: vpnrouter_users.add(connection, body.get('name'), body.get('password'), settings,
|
||||
service=True),
|
||||
('PUT', 2, None): lambda connection: vpnrouter_users.update_service(connection, name, settings),
|
||||
('PUT', 3, 'password'): lambda connection: vpnrouter_users.passwd(connection, name, body.get('password')),
|
||||
('POST', 3, 'disable'): lambda connection: vpnrouter_users.disable(connection, name),
|
||||
('POST', 3, 'enable'): lambda connection: vpnrouter_users.enable(connection, name),
|
||||
('POST', 3, 'unlock'): lambda connection: vpnrouter_users.unlock(connection, name),
|
||||
('DELETE', 2, None): lambda connection: vpnrouter_users.remove(connection, name),
|
||||
}
|
||||
if route not in actions:
|
||||
return 404, {'error': f'no route {method} {path}'}
|
||||
if name is not None and name in vpnrouter_users.internal_accounts():
|
||||
return 403, {'error': f'{name} is an internal service account, which the router manages'}
|
||||
try:
|
||||
with vpnrouter_users.session() as connection:
|
||||
if name is not None and not vpnrouter_users.service(vpnrouter_users.entry(connection, name)):
|
||||
raise vpnrouter_users.Failure(f'{name} is not a service account', 3)
|
||||
document = actions[route](connection)
|
||||
except vpnrouter_users.Failure as failure:
|
||||
return USER_STATUS.get(failure.status, 500), {'error': str(failure)}
|
||||
except (ldap.LDAPError, OSError) as exc:
|
||||
return 500, {'error': f'the user directory failed: {exc}'}
|
||||
return (201 if route == ('POST', 1, None) else 200), document
|
||||
|
||||
|
||||
def groups(state, method, path, body):
|
||||
"""Answer a /groups request with vpnrouter_users: the HTTP status and the JSON document.
|
||||
|
||||
@@ -458,6 +563,9 @@ def groups(state, method, path, body):
|
||||
DELETE /groups/<name> remove the group; its members are kept
|
||||
POST /groups/<name>/members add a member: {"user": ...}
|
||||
DELETE /groups/<name>/members/<user> take a member out; the last member cannot leave
|
||||
|
||||
An internal service account (vpnrouter_users.internal_accounts) is not put in or taken out of a group: such a
|
||||
request is refused with 403.
|
||||
"""
|
||||
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
|
||||
return 400, {'error': 'the body must be a JSON object'}
|
||||
@@ -465,6 +573,12 @@ def groups(state, method, path, body):
|
||||
parts = path.strip('/').split('/')
|
||||
name = parts[1] if len(parts) > 1 else None
|
||||
route = (method, len(parts), parts[2] if len(parts) > 2 else None)
|
||||
members = body.get('members') if route == ('POST', 1, None) else [body.get('user')] if route == ('POST', 3, 'members') \
|
||||
else [parts[3]] if route == ('DELETE', 4, 'members') else []
|
||||
internal = sorted(set(vpnrouter_users.internal_accounts()).intersection(
|
||||
member for member in (members if isinstance(members, list) else []) if isinstance(member, str)))
|
||||
if internal:
|
||||
return 403, {'error': f'{", ".join(internal)}: internal service accounts, which the router manages, are not group members'}
|
||||
actions = {
|
||||
('GET', 1, None): lambda connection: vpnrouter_users.list_groups(connection),
|
||||
('POST', 1, None): lambda connection: vpnrouter_users.group_add(
|
||||
@@ -487,8 +601,148 @@ def groups(state, method, path, body):
|
||||
return (201 if route == ('POST', 1, None) else 200), document
|
||||
|
||||
|
||||
def connection_of(name, connection, count):
|
||||
"""A site_to_site connection as /site-to-site shows it: its settings but psk, psk_set and established.
|
||||
|
||||
count is the number of its established IKE SAs, None when it is not known.
|
||||
"""
|
||||
return {'name': name, **{option: value for option, value in connection.items() if option != 'psk'},
|
||||
'psk_set': bool(connection['psk']), 'established': None if count is None else count > 0}
|
||||
|
||||
|
||||
def site_to_site(method, path, body):
|
||||
"""Answer a /site-to-site request: the HTTP status and the JSON document.
|
||||
|
||||
GET /site-to-site the connections, {"connections": [...]}: each with its settings but psk, with
|
||||
psk_set, whether psk holds a key, and established, whether its tunnel is up,
|
||||
null when swanctl cannot be asked
|
||||
POST /site-to-site add a connection: {"name", "addrs", "id", "cidrs", "local_cidrs", "local_id_mode",
|
||||
"psk"}; psk is required, and an existing name is 409
|
||||
PUT /site-to-site/<name> change any of the settings of POST but name; an empty or absent psk keeps the
|
||||
current key, and a psk given replaces psk_file
|
||||
DELETE /site-to-site/<name> remove the connection
|
||||
|
||||
A change is made holding vpnrouter.config_lock, checked as vpn-router-setup checks the configuration (400 with
|
||||
the reason), written with vpnrouter.save_config, and applied by restarting vpn-router-setup without waiting for it.
|
||||
"""
|
||||
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
|
||||
return 400, {'error': 'the body must be a JSON object'}
|
||||
body = body or {}
|
||||
parts = path.strip('/').split('/')
|
||||
name = parts[1] if len(parts) == 2 else None
|
||||
if (method, len(parts)) not in (('GET', 1), ('POST', 1), ('PUT', 2), ('DELETE', 2)):
|
||||
return 404, {'error': f'no route {method} {path}'}
|
||||
try:
|
||||
if method == 'GET':
|
||||
connections = vpnrouter.load_config()['site_to_site']
|
||||
counts = established_each(connections)
|
||||
return 200, {'connections': [connection_of(key, value, counts[key]) for key, value in connections.items()]}
|
||||
settings = {option: body[option] for option in CONNECTION_SETTINGS if option in body}
|
||||
with vpnrouter.config_lock():
|
||||
data = vpnrouter.read_settings()
|
||||
connections = data.setdefault('site_to_site', {})
|
||||
if method == 'POST':
|
||||
name = body.get('name')
|
||||
if not isinstance(name, str) or not name:
|
||||
return 400, {'error': 'name is required'}
|
||||
if name in connections:
|
||||
return 409, {'error': f'connection {name} exists'}
|
||||
if not isinstance(settings.get('psk'), str) or not settings['psk']:
|
||||
return 400, {'error': 'psk is required'}
|
||||
connections[name] = settings
|
||||
elif name not in connections:
|
||||
return 404, {'error': f'no connection {name}'}
|
||||
elif method == 'PUT':
|
||||
connection = dict(connections[name]) if isinstance(connections[name], dict) else {}
|
||||
if settings.get('psk'):
|
||||
connection.pop('psk_file', None)
|
||||
else:
|
||||
settings.pop('psk', None)
|
||||
connections[name] = {**connection, **settings}
|
||||
else:
|
||||
del connections[name]
|
||||
cfg = vpnrouter.with_defaults(data)
|
||||
vpnrouter.validate(cfg)
|
||||
vpnrouter.save_config(data)
|
||||
except vpnrouter.ConfigError as exc:
|
||||
return 400, {'error': str(exc)}
|
||||
except OSError as exc:
|
||||
return 500, {'error': f'the configuration cannot be written: {exc}'}
|
||||
except SystemExit as exc:
|
||||
return 500, {'error': str(exc.code)}
|
||||
run('systemctl', 'restart', '--no-block', 'vpn-router-setup.service')
|
||||
if method == 'DELETE':
|
||||
return 200, {'name': name, 'removed': True}
|
||||
return (201 if method == 'POST' else 200), connection_of(name, cfg['site_to_site'][name], None)
|
||||
|
||||
|
||||
def account(name, method, path, body):
|
||||
"""Answer an /account request of the user named by the token with vpnrouter_users: the HTTP status and the JSON document.
|
||||
|
||||
GET /account the user, with admin, whether the user is a member of ADMIN_GROUP
|
||||
PUT /account/password set the password: {"current_password", "current_code"?, "password"}
|
||||
POST /account/otp a new TOTP secret and its otpauth URI; nothing is stored
|
||||
PUT /account/otp store a TOTP secret: {"current_password", "current_code"?, "secret", "code"},
|
||||
code a code of the secret
|
||||
|
||||
A change needs the current password and, for an enrolled user, a current code; a wrong one is 403 and counts as
|
||||
a failed login. A password change revokes the user's refresh tokens.
|
||||
"""
|
||||
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
|
||||
return 400, {'error': 'the body must be a JSON object'}
|
||||
body = body or {}
|
||||
route = (method, path.rstrip('/'))
|
||||
if route not in (('GET', '/account'), ('PUT', '/account/password'), ('POST', '/account/otp'), ('PUT', '/account/otp')):
|
||||
return 404, {'error': f'no route {method} {path}'}
|
||||
try:
|
||||
if route == ('PUT', '/account/password'):
|
||||
vpnrouter_users.check_password(body.get('password'))
|
||||
if method == 'PUT':
|
||||
vpnrouter_users.login(name, body.get('current_password'), body.get('current_code'))
|
||||
with vpnrouter_users.session() as connection:
|
||||
if route == ('GET', '/account'):
|
||||
document = {**vpnrouter_users.summary(vpnrouter_users.entry(connection, name)),
|
||||
'admin': vpnrouter_users.is_admin(connection, name)}
|
||||
elif route == ('PUT', '/account/password'):
|
||||
document = vpnrouter_users.passwd(connection, name, body['password'])
|
||||
elif route == ('POST', '/account/otp'):
|
||||
document = vpnrouter_users.otp_new(connection, name)
|
||||
else:
|
||||
document = vpnrouter_users.otp_enrol(connection, name, body.get('secret'), body.get('code'))
|
||||
if route == ('PUT', '/account/password'):
|
||||
vpnrouter_auth.revoke(vpnrouter.load_config(), name)
|
||||
except vpnrouter_users.Failure as failure:
|
||||
if failure.status == 5:
|
||||
return 403, {'error': 'the current password or code is wrong'}
|
||||
return USER_STATUS.get(failure.status, 500), {'error': str(failure)}
|
||||
except (ldap.LDAPError, OSError) as exc:
|
||||
return 500, {'error': f'the user directory failed: {exc}'}
|
||||
except vpnrouter_auth.DatabaseError as exc:
|
||||
return 500, {'error': str(exc)}
|
||||
except SystemExit as exc:
|
||||
return 500, {'error': str(exc.code)}
|
||||
return 200, document
|
||||
|
||||
|
||||
def metrics_history(view):
|
||||
"""Answer GET /metrics/history?range=<view> with vpnrouter_metrics.history: the HTTP status and the JSON document.
|
||||
|
||||
view is hour, the default, day, month or year (vpnrouter_metrics.RANGES).
|
||||
"""
|
||||
if view not in vpnrouter_metrics.RANGES:
|
||||
return 400, {'error': f'range must be one of {", ".join(vpnrouter_metrics.RANGES)}'}
|
||||
try:
|
||||
with vpnrouter_metrics.database(vpnrouter.load_config()) as connection:
|
||||
return 200, vpnrouter_metrics.history(connection, view)
|
||||
except vpnrouter_metrics.DatabaseError as exc:
|
||||
return 500, {'error': str(exc)}
|
||||
except SystemExit as exc:
|
||||
return 500, {'error': str(exc.code)}
|
||||
|
||||
|
||||
def changes_admins(connection, method, parts, body):
|
||||
"""Whether a /users or /groups request creates, renames to, changes or removes ADMIN_GROUP, or removes a member of it."""
|
||||
"""Whether a /users, /service-accounts or /groups request creates, renames to, changes or removes ADMIN_GROUP, or
|
||||
removes a member of it."""
|
||||
admins = vpnrouter_users.ADMIN_GROUP
|
||||
if parts[0] == 'groups':
|
||||
if method == 'POST' and len(parts) == 1:
|
||||
@@ -496,16 +750,18 @@ def changes_admins(connection, method, parts, body):
|
||||
if method != 'GET' and len(parts) > 1 and parts[1] == admins:
|
||||
return True
|
||||
return method == 'PUT' and len(parts) == 2 and body.get('name') == admins
|
||||
if parts[0] == 'users' and method == 'DELETE' and len(parts) == 2:
|
||||
if parts[0] in ('users', 'service-accounts') and method == 'DELETE' and len(parts) == 2:
|
||||
return admins in vpnrouter_users.groups_of(connection, parts[1])
|
||||
return False
|
||||
|
||||
|
||||
def refusal(claims, method, path, body):
|
||||
"""Why the holder of the claims may not make the request to a protected path, or '' when it may."""
|
||||
name = claims.get('sub')
|
||||
if path == '/account' or path.startswith('/account/'):
|
||||
return '' if isinstance(name, str) else 'the token names no user'
|
||||
if claims.get('admin') is True:
|
||||
return ''
|
||||
name = claims.get('sub')
|
||||
if not isinstance(name, str):
|
||||
return 'the token names no user'
|
||||
admins = vpnrouter_users.ADMIN_GROUP
|
||||
@@ -530,6 +786,9 @@ def auth(path, body):
|
||||
cfg = vpnrouter.load_config()
|
||||
if path == '/auth':
|
||||
vpnrouter_users.login(body.get('name'), body.get('password'), body.get('code'))
|
||||
with vpnrouter_users.session() as connection:
|
||||
if vpnrouter_users.service(vpnrouter_users.entry(connection, body['name'])):
|
||||
raise vpnrouter_users.Failure('the name, the password or the code is wrong', 5)
|
||||
return 200, vpnrouter_auth.issue(cfg, body['name'])
|
||||
name = vpnrouter_auth.take(cfg, body.get('refresh_token'))
|
||||
if path == '/auth/logout':
|
||||
@@ -605,8 +864,9 @@ class Handler(BaseHTTPRequestHandler):
|
||||
# A body that is not read must not be taken for the next request.
|
||||
if self.command not in ('GET', 'HEAD'):
|
||||
self.close_connection = True
|
||||
directory = next((name for name in ('users', 'groups') if path == f'/{name}' or path.startswith(f'/{name}/')), None)
|
||||
protected = path == '/config' or directory is not None
|
||||
directory = next((name for name in ('users', 'service-accounts', 'groups', 'account', 'site-to-site')
|
||||
if path == f'/{name}' or path.startswith(f'/{name}/')), None)
|
||||
protected = path in ('/config', '/metrics/history', '/postgresql') or directory is not None
|
||||
signing = path in ('/auth', '/auth/refresh', '/auth/logout')
|
||||
state = current() if protected or signing else {}
|
||||
claims = self.claims(state['key']) if protected and state['key'] else None
|
||||
@@ -633,19 +893,35 @@ class Handler(BaseHTTPRequestHandler):
|
||||
body = json.dumps({'error': refused[1]}, indent=2).encode() + b'\n'
|
||||
elif path == '/config' and self.command in ('GET', 'HEAD'):
|
||||
body, kind, code = json.dumps(details(state), indent=2).encode() + b'\n', 'application/json', 200
|
||||
elif protected and path != '/config':
|
||||
elif path == '/postgresql' and self.command in ('GET', 'HEAD'):
|
||||
body, kind, code = json.dumps(postgresql_databases(), indent=2).encode() + b'\n', 'application/json', 200
|
||||
elif path == '/metrics/history' and self.command in ('GET', 'HEAD'):
|
||||
query = urllib.parse.parse_qs(urllib.parse.urlsplit(self.path).query)
|
||||
code, document = metrics_history(query.get('range', ['hour'])[0])
|
||||
body, kind = json.dumps(document).encode() + b'\n', 'application/json'
|
||||
elif directory == 'account':
|
||||
code, document = account(claims['sub'], self.command, path, payload)
|
||||
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
|
||||
elif directory == 'site-to-site':
|
||||
code, document = site_to_site(self.command, path, payload)
|
||||
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
|
||||
elif directory == 'service-accounts':
|
||||
code, document = service_accounts(self.command, path, payload)
|
||||
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
|
||||
elif directory is not None:
|
||||
code, document = (users if directory == 'users' else groups)(state, self.command, path, payload)
|
||||
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
|
||||
elif self.command not in ('GET', 'HEAD'):
|
||||
body, kind, code = b'Method not allowed\n', 'text/plain; charset=utf-8', 405
|
||||
elif path == '/status':
|
||||
body, kind, code = json.dumps(self.status(), indent=2).encode() + b'\n', 'application/json', 200
|
||||
elif path == '/':
|
||||
elif path in ('/status', '/'):
|
||||
document = shown(self.status(), section(read_json(CONFIG_FILE), 'web').get('status_content'), self.client())
|
||||
agent = self.headers.get('User-Agent', '').lower()
|
||||
if 'curl' in agent or 'wget' in agent:
|
||||
body, kind = text_of(self.status()).encode(), 'text/plain; charset=utf-8'
|
||||
if path == '/status':
|
||||
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
|
||||
elif 'curl' in agent or 'wget' in agent:
|
||||
body, kind = text_of(document).encode(), 'text/plain; charset=utf-8'
|
||||
else:
|
||||
body, kind = html_of(self.status()).encode(), 'text/html; charset=utf-8'
|
||||
body, kind = html_of(document).encode(), 'text/html; charset=utf-8'
|
||||
code = 200
|
||||
else:
|
||||
body, kind, code = b'Not found\n', 'text/plain; charset=utf-8', 404
|
||||
@@ -697,13 +973,11 @@ def log(message):
|
||||
|
||||
|
||||
def initialize():
|
||||
"""Load the PostgreSQL driver, make the signing key and the refresh tokens' database, retrying every INIT_RETRY_SECONDS until all are ready."""
|
||||
"""Make the signing key and the refresh tokens' database, retrying every INIT_RETRY_SECONDS until both are ready."""
|
||||
reported = None
|
||||
while True:
|
||||
phase = 'loading the PostgreSQL driver'
|
||||
phase = 'reading the configuration'
|
||||
try:
|
||||
vpnrouter_auth.driver()
|
||||
phase = 'reading the configuration'
|
||||
cfg = vpnrouter.load_config()
|
||||
phase = 'making the signing key'
|
||||
vpnrouter_auth.ensure_key()
|
||||
|
||||
Executable
+132
@@ -0,0 +1,132 @@
|
||||
#!/usr/bin/python3
|
||||
"""Show and change the router's configuration, /etc/vpn-router/vpn-router.json.
|
||||
|
||||
show prints the file as indented JSON; the file is not changed. set takes a setting as
|
||||
section.option, such as web.port, or as site_to_site.NAME.option for a site-to-site connection,
|
||||
which it creates with the defaults (vpnrouter.CONNECTION) when it does not exist, and a value,
|
||||
which it converts to the type the package's schema (vpnrouter.SCHEMA) gives the setting: a whole
|
||||
number, true or false, a list of strings separated by commas (an empty value is an empty list),
|
||||
or a string. remove takes site_to_site.NAME and removes that connection. The configuration with
|
||||
the change is checked as vpn-router-setup checks it, and only then written with
|
||||
vpnrouter.save_config, holding vpnrouter.config_lock; vpn-router-setup applies it at its next
|
||||
run, which set and remove remind of. A failure is a message on standard error; the exit status
|
||||
is 2 for an invalid request and 1 for any other failure. -h or --help shows the options.
|
||||
|
||||
vpn-config-edit show
|
||||
vpn-config-edit set SECTION.OPTION VALUE
|
||||
vpn-config-edit set site_to_site.NAME.OPTION VALUE
|
||||
vpn-config-edit remove site_to_site.NAME
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import copy
|
||||
import json
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, '/usr/lib/vpn-router')
|
||||
|
||||
import vpnrouter
|
||||
|
||||
|
||||
class Failure(Exception):
|
||||
"""A request that cannot be done; status is 2 invalid, 1 other."""
|
||||
|
||||
def __init__(self, message, status=1):
|
||||
super().__init__(message)
|
||||
self.status = status
|
||||
|
||||
|
||||
class Parser(argparse.ArgumentParser):
|
||||
"""An argument parser whose errors are Failures with status 2."""
|
||||
|
||||
def error(self, message):
|
||||
raise Failure(f'{message}; see {self.prog} --help', 2)
|
||||
|
||||
|
||||
def read():
|
||||
"""The settings in the configuration file, without the defaults; {} when the file does not exist."""
|
||||
try:
|
||||
return vpnrouter.read_settings()
|
||||
except vpnrouter.ConfigError as exc:
|
||||
raise Failure(str(exc))
|
||||
|
||||
|
||||
def converted(key, text):
|
||||
"""The path of the setting key, section.option or site_to_site.NAME.option, and text as the type the schema gives it."""
|
||||
path = key.split('.')
|
||||
if path[0] == 'site_to_site' and len(path) == 3 and path[2] in vpnrouter.CONNECTION:
|
||||
kind = type(vpnrouter.CONNECTION[path[2]])
|
||||
elif path[0] != 'site_to_site' and len(path) == 2 and path[0] in vpnrouter.SCHEMA and path[1] in vpnrouter.SCHEMA[path[0]]:
|
||||
kind = type(vpnrouter.SCHEMA[path[0]][path[1]])
|
||||
else:
|
||||
raise Failure(f'{key}: unknown setting; settings are section.option, such as web.port, '
|
||||
'or site_to_site.NAME.option', 2)
|
||||
if kind is bool:
|
||||
if text not in ('true', 'false'):
|
||||
raise Failure(f'{key} takes true or false', 2)
|
||||
return path, text == 'true'
|
||||
if kind is int:
|
||||
try:
|
||||
return path, int(text)
|
||||
except ValueError:
|
||||
raise Failure(f'{key} takes a whole number', 2)
|
||||
if kind is list:
|
||||
return path, [item.strip() for item in text.split(',') if item.strip()]
|
||||
return path, text
|
||||
|
||||
|
||||
def main(argv):
|
||||
try:
|
||||
parser = Parser(prog='vpn-config-edit', description='Show and change the router configuration.')
|
||||
commands = parser.add_subparsers(dest='command', required=True, parser_class=Parser)
|
||||
commands.add_parser('show', help='print the configuration as indented JSON')
|
||||
setter = commands.add_parser('set', help='set a setting and write the configuration')
|
||||
setter.add_argument('key', metavar='SECTION.OPTION',
|
||||
help='the setting, such as web.port or site_to_site.NAME.option')
|
||||
setter.add_argument('value', metavar='VALUE', help='its value; a list separated by commas')
|
||||
remover = commands.add_parser('remove', help='remove a site-to-site connection and write the configuration')
|
||||
remover.add_argument('key', metavar='site_to_site.NAME', help='the connection')
|
||||
options = parser.parse_args(argv)
|
||||
|
||||
if options.command == 'show':
|
||||
print(json.dumps(read(), indent=2))
|
||||
return 0
|
||||
|
||||
with vpnrouter.config_lock():
|
||||
data = read()
|
||||
if options.command == 'set':
|
||||
path, value = converted(options.key, options.value)
|
||||
if path[0] == 'site_to_site':
|
||||
connections = data.setdefault('site_to_site', {})
|
||||
connection = connections.setdefault(path[1], copy.deepcopy(vpnrouter.CONNECTION))
|
||||
if not isinstance(connection, dict):
|
||||
raise Failure(f'site_to_site.{path[1]} is not an object')
|
||||
connection[path[2]] = value
|
||||
else:
|
||||
data.setdefault(path[0], {})[path[1]] = value
|
||||
else:
|
||||
section, _, name = options.key.partition('.')
|
||||
if section != 'site_to_site' or not name or name not in data.get('site_to_site', {}):
|
||||
raise Failure(f'{options.key}: no such site-to-site connection', 2)
|
||||
del data['site_to_site'][name]
|
||||
try:
|
||||
vpnrouter.validate(vpnrouter.with_defaults(data))
|
||||
except vpnrouter.ConfigError as exc:
|
||||
raise Failure(str(exc), 2)
|
||||
vpnrouter.save_config(data)
|
||||
except Failure as failure:
|
||||
print(f'vpn-config-edit: {failure}', file=sys.stderr)
|
||||
return failure.status
|
||||
except OSError as exc:
|
||||
print(f'vpn-config-edit: {exc}', file=sys.stderr)
|
||||
return 1
|
||||
if options.command == 'set':
|
||||
print(f'{options.key} set to {json.dumps(value)} in {vpnrouter.CONFIG_FILE}.')
|
||||
else:
|
||||
print(f'{options.key} removed from {vpnrouter.CONFIG_FILE}.')
|
||||
print('Apply it with: systemctl restart vpn-router-setup')
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main(sys.argv[1:]))
|
||||
@@ -10,7 +10,11 @@ user exists, 1 anything else. -h or --help shows the commands and their options.
|
||||
|
||||
vpn-router-user [--json] list
|
||||
vpn-router-user [--json] add <name> --first-name F --last-name L --email E (password on stdin)
|
||||
vpn-router-user [--json] add <name> [--email E] (a Manager-only account; password on stdin)
|
||||
vpn-router-user [--json] add <name> --service [--display-name N] [--description D] [--email E]
|
||||
(a service account; password on stdin)
|
||||
vpn-router-user [--json] update <name> [--first-name F] [--last-name L] [--email E]
|
||||
vpn-router-user [--json] update <name> [--display-name N] [--description D] [--email E] (a service account)
|
||||
vpn-router-user [--json] passwd <name> (password on stdin)
|
||||
vpn-router-user [--json] disable <name>
|
||||
vpn-router-user [--json] enable <name>
|
||||
@@ -30,6 +34,7 @@ sys.path.insert(0, '/usr/lib/vpn-router')
|
||||
|
||||
import ldap
|
||||
|
||||
import vpnrouter
|
||||
import vpnrouter_auth
|
||||
import vpnrouter_users
|
||||
from vpnrouter_users import Failure
|
||||
@@ -37,8 +42,11 @@ from vpnrouter_users import Failure
|
||||
#: The help line of each command.
|
||||
HELP = {
|
||||
'list': 'list the users',
|
||||
'add': 'add a user and issue the certificate; the password is read from standard input',
|
||||
'update': 'change the first name, the last name or the e-mail address',
|
||||
'add': 'add a VPN user with the three person options and issue the certificate, or without the names a '
|
||||
'Manager-only account, or with --service a service account, each with an optional e-mail address; the '
|
||||
'password is read from standard input',
|
||||
'update': "change the first name, the last name or the e-mail address; a Manager-only account has only the "
|
||||
"e-mail address, which '' removes",
|
||||
'passwd': 'set the password, read from standard input',
|
||||
'disable': 'lock the user out of OpenVPN',
|
||||
'enable': 'let the user in again',
|
||||
@@ -51,7 +59,9 @@ HELP = {
|
||||
}
|
||||
|
||||
#: The help of the person options.
|
||||
PERSON_HELP = {'first_name': 'the first name', 'last_name': 'the last name', 'email': 'the e-mail address'}
|
||||
PERSON_HELP = {'first_name': 'the first name', 'last_name': 'the last name', 'email': 'the e-mail address',
|
||||
'display_name': "a service account's display name; '' removes it",
|
||||
'description': "a service account's description; '' removes it"}
|
||||
|
||||
|
||||
class Parser(argparse.ArgumentParser):
|
||||
@@ -70,9 +80,10 @@ def parse(argv):
|
||||
if command != 'list':
|
||||
sub.add_argument('name', help='the user name')
|
||||
if command in ('add', 'update'):
|
||||
for key, _ in vpnrouter_users.PERSON:
|
||||
sub.add_argument(f'--{key.replace("_", "-")}', dest=key, metavar='TEXT', required=command == 'add',
|
||||
help=PERSON_HELP[key])
|
||||
for key, _ in vpnrouter_users.PERSON + vpnrouter_users.LABELS:
|
||||
sub.add_argument(f'--{key.replace("_", "-")}', dest=key, metavar='TEXT', help=PERSON_HELP[key])
|
||||
if command == 'add':
|
||||
sub.add_argument('--service', action='store_true', help='add a service account, which cannot sign in to the web service')
|
||||
if command == 'otp-enrol':
|
||||
sub.add_argument('code', help='a code of the secret from the authenticator')
|
||||
return parser.parse_args(argv)
|
||||
@@ -88,9 +99,9 @@ def run(connection, options, secret):
|
||||
command = options.command
|
||||
if command == 'list':
|
||||
return vpnrouter_users.list_users(connection)
|
||||
values = {key: getattr(options, key, None) for key, _ in vpnrouter_users.PERSON}
|
||||
values = {key: getattr(options, key, None) for key, _ in vpnrouter_users.PERSON + vpnrouter_users.LABELS}
|
||||
if command == 'add':
|
||||
return vpnrouter_users.add(connection, options.name, secret, values)
|
||||
return vpnrouter_users.add(connection, options.name, secret, values, service=options.service)
|
||||
if command == 'update':
|
||||
return vpnrouter_users.update(connection, options.name, values)
|
||||
if command == 'passwd':
|
||||
@@ -111,8 +122,9 @@ def text_of(answer):
|
||||
users = answer['users'] if 'users' in answer else [answer]
|
||||
if not users:
|
||||
return 'no users'
|
||||
head = ('USER', 'STATE', 'CERTIFICATE', 'OTP', 'FIRST NAME', 'LAST NAME', 'E-MAIL')
|
||||
rows = [(user['name'], 'disabled' if not user['enabled'] else 'locked' if user['locked'] else 'enabled',
|
||||
head = ('USER', 'KIND', 'STATE', 'CERTIFICATE', 'OTP', 'FIRST NAME', 'LAST NAME', 'E-MAIL')
|
||||
rows = [(user['name'], 'internal' if user['internal'] else 'service' if user['service'] else 'manager' if user['manager_only'] else 'vpn',
|
||||
'disabled' if not user['enabled'] else 'locked' if user['locked'] else 'enabled',
|
||||
'issued' if user['certificate'] else 'missing',
|
||||
'enrolled' if user['otp'] else '-', user['first_name'] or '-', user['last_name'] or '-', user['email'] or '-')
|
||||
for user in users]
|
||||
@@ -131,6 +143,8 @@ def main(argv):
|
||||
secret = password() if options.command in ('add', 'passwd', 'otp-enrol') else None
|
||||
with vpnrouter_users.session() as connection:
|
||||
answer = run(connection, options, secret)
|
||||
if options.command in vpnrouter_auth.REVOKING:
|
||||
vpnrouter_auth.revoke(vpnrouter.load_config(), options.name)
|
||||
except Failure as failure:
|
||||
message, status = str(failure), failure.status
|
||||
except (ldap.LDAPError, OSError) as exc:
|
||||
|
||||
@@ -8,20 +8,23 @@
|
||||
"internal": "eth1"
|
||||
},
|
||||
"wan": {
|
||||
"local_fqdn": "router.example.com",
|
||||
"local_id_mode": "fqdn"
|
||||
"local_fqdn": "router.example.com"
|
||||
},
|
||||
"local": {
|
||||
"cidrs": ["10.0.0.0/24"],
|
||||
"int_addr": "10.1.1.4",
|
||||
"int_gateway_ip": "10.1.1.1"
|
||||
},
|
||||
"remote": {
|
||||
"addrs": ["peer.example.net"],
|
||||
"id": "peer.example.net",
|
||||
"cidrs": ["192.168.0.0/24"],
|
||||
"psk": "change-me",
|
||||
"psk_file": ""
|
||||
"site_to_site": {
|
||||
"remote-site": {
|
||||
"addrs": ["peer.example.net"],
|
||||
"id": "peer.example.net",
|
||||
"cidrs": ["192.168.0.0/24"],
|
||||
"local_cidrs": ["10.0.0.0/24"],
|
||||
"local_id_mode": "fqdn",
|
||||
"psk": "change-me",
|
||||
"psk_file": ""
|
||||
}
|
||||
},
|
||||
"p2s": {
|
||||
"enabled": false,
|
||||
@@ -45,8 +48,31 @@
|
||||
},
|
||||
"web": {
|
||||
"enabled": true,
|
||||
"listen": "both",
|
||||
"port": 7081,
|
||||
"database": "",
|
||||
"trusted_proxies": []
|
||||
"trusted_proxies": [],
|
||||
"status_content": "basic"
|
||||
},
|
||||
"https": {
|
||||
"enabled": true,
|
||||
"email": "",
|
||||
"web_port": 7444,
|
||||
"manager_port": 7443
|
||||
},
|
||||
"metrics": {
|
||||
"enabled": true,
|
||||
"listen": "internal",
|
||||
"port": 9464,
|
||||
"https_port": 9465,
|
||||
"service_account": "metrics",
|
||||
"otlp_endpoint": "",
|
||||
"otlp_username": "",
|
||||
"otlp_password": "",
|
||||
"keep_details": "24 hours",
|
||||
"keep_hourly": "1 month",
|
||||
"keep_daily": "3 months",
|
||||
"keep_monthly": "1 year",
|
||||
"keep_yearly": "10 years"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
# Written by vpn-router-setup from /etc/vpn-router/vpn-router.json; edits are overwritten.
|
||||
{
|
||||
auto_https disable_redirects
|
||||
{%- if https_email %}
|
||||
email {{ https_email }}
|
||||
{%- endif %}
|
||||
}
|
||||
{% if web_enabled %}
|
||||
{{ local_fqdn }}:{{ https_web_port }} {
|
||||
reverse_proxy {{ loopback }}:{{ web_port }}
|
||||
}
|
||||
{% endif %}
|
||||
{%- if manager_port %}
|
||||
{{ local_fqdn }}:{{ https_manager_port }} {
|
||||
reverse_proxy {{ loopback }}:{{ manager_port }}
|
||||
}
|
||||
{% endif %}
|
||||
{%- if metrics_enabled %}
|
||||
{{ local_fqdn }}:{{ metrics_https_port }} {
|
||||
reverse_proxy {{ loopback }}:{{ metrics_port }}
|
||||
}
|
||||
{% endif %}
|
||||
@@ -1,46 +0,0 @@
|
||||
connections {
|
||||
remote-site {
|
||||
version = 2
|
||||
proposals = aes256-sha256-modp2048,aes256-sha1-modp1024
|
||||
reauth_time = 28800
|
||||
unique = replace
|
||||
|
||||
local_addrs = {{ local_addrs }}
|
||||
remote_addrs = {{ remote_addrs }}
|
||||
|
||||
local {
|
||||
id = {{ local_id }}
|
||||
auth = psk
|
||||
}
|
||||
|
||||
remote {
|
||||
id = {{ remote_id }}
|
||||
auth = psk
|
||||
}
|
||||
|
||||
children {
|
||||
site2site {
|
||||
mode = tunnel
|
||||
local_ts = {{ local_ts }}
|
||||
remote_ts = {{ remote_cidrs }}
|
||||
esp_proposals = aes256-sha256,aes256-sha1
|
||||
life_time = 3600
|
||||
dpd_action = restart
|
||||
start_action = start
|
||||
close_action = none
|
||||
}
|
||||
}
|
||||
|
||||
rekey_time = 10800
|
||||
dpd_delay = 10
|
||||
dpd_timeout = 120
|
||||
}
|
||||
}
|
||||
|
||||
secrets {
|
||||
ike-psk {
|
||||
id-1 = {{ local_id }}
|
||||
id-2 = {{ remote_id }}
|
||||
secret = "{{ psk }}"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,50 @@
|
||||
connections {
|
||||
{%- for connection in connections if connection.ready %}
|
||||
{{ connection.name }} {
|
||||
version = 2
|
||||
proposals = aes256-sha256-modp2048,aes256-sha1-modp1024
|
||||
reauth_time = 28800
|
||||
unique = replace
|
||||
|
||||
local_addrs = {{ local_addrs }}
|
||||
remote_addrs = {{ connection.remote_addrs }}
|
||||
|
||||
local {
|
||||
id = {{ connection.local_id }}
|
||||
auth = psk
|
||||
}
|
||||
|
||||
remote {
|
||||
id = {{ connection.remote_id }}
|
||||
auth = psk
|
||||
}
|
||||
|
||||
children {
|
||||
{{ connection.name }} {
|
||||
mode = tunnel
|
||||
local_ts = {{ connection.local_ts }}
|
||||
remote_ts = {{ connection.remote_cidrs }}
|
||||
esp_proposals = aes256-sha256,aes256-sha1
|
||||
life_time = 3600
|
||||
dpd_action = restart
|
||||
start_action = start
|
||||
close_action = none
|
||||
}
|
||||
}
|
||||
|
||||
rekey_time = 10800
|
||||
dpd_delay = 10
|
||||
dpd_timeout = 120
|
||||
}
|
||||
{%- endfor %}
|
||||
}
|
||||
|
||||
secrets {
|
||||
{%- for connection in connections if connection.ready %}
|
||||
ike-psk-{{ connection.name }} {
|
||||
id-1 = {{ connection.local_id }}
|
||||
id-2 = {{ connection.remote_id }}
|
||||
secret = "{{ connection.psk }}"
|
||||
}
|
||||
{%- endfor %}
|
||||
}
|
||||
@@ -15,13 +15,17 @@ locals {
|
||||
# Contents of /etc/vpn-router/vpn-router.json.
|
||||
config = {
|
||||
general = { platform = "azure", mode = "auto" }
|
||||
wan = { local_fqdn = var.local_fqdn, local_id_mode = var.local_id_mode }
|
||||
local = { cidrs = [for c in split(",", var.local_cidrs) : trimspace(c) if trimspace(c) != ""] }
|
||||
remote = {
|
||||
addrs = [for a in split(",", var.remote_addrs) : trimspace(a) if trimspace(a) != ""]
|
||||
id = var.remote_id
|
||||
cidrs = local.remote_cidr_list
|
||||
psk = var.psk
|
||||
wan = { local_fqdn = var.local_fqdn }
|
||||
local = { cidrs = local.local_cidr_list }
|
||||
site_to_site = {
|
||||
"remote-site" = {
|
||||
addrs = [for a in split(",", var.remote_addrs) : trimspace(a) if trimspace(a) != ""]
|
||||
id = var.remote_id
|
||||
cidrs = local.remote_cidr_list
|
||||
local_cidrs = local.local_cidr_list
|
||||
local_id_mode = var.local_id_mode
|
||||
psk = var.psk
|
||||
}
|
||||
}
|
||||
p2s = { enabled = var.p2s_enabled, address_pool = var.p2s_address_pool, ca_name = "VPN Router CA" }
|
||||
wireguard = { enabled = var.wireguard_enabled, address = var.wireguard_address, listen_port = var.wireguard_listen_port }
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
locals {
|
||||
remote_cidr_list = [for c in split(",", var.remote_cidrs) : trimspace(c) if trimspace(c) != ""]
|
||||
local_cidr_list = [for c in split(",", var.local_cidrs) : trimspace(c) if trimspace(c) != ""]
|
||||
}
|
||||
|
||||
resource "azurerm_route_table" "router" {
|
||||
|
||||
+11
-7
@@ -15,13 +15,17 @@ locals {
|
||||
# Contents of /etc/vpn-router/vpn-router.json.
|
||||
config = {
|
||||
general = { platform = "gcp", mode = "auto" }
|
||||
wan = { local_fqdn = var.local_fqdn, local_id_mode = var.local_id_mode }
|
||||
local = { cidrs = [for c in split(",", var.local_cidrs) : trimspace(c) if trimspace(c) != ""] }
|
||||
remote = {
|
||||
addrs = [for a in split(",", var.remote_addrs) : trimspace(a) if trimspace(a) != ""]
|
||||
id = var.remote_id
|
||||
cidrs = local.remote_cidr_list
|
||||
psk = var.psk
|
||||
wan = { local_fqdn = var.local_fqdn }
|
||||
local = { cidrs = local.local_cidr_list }
|
||||
site_to_site = {
|
||||
"remote-site" = {
|
||||
addrs = [for a in split(",", var.remote_addrs) : trimspace(a) if trimspace(a) != ""]
|
||||
id = var.remote_id
|
||||
cidrs = local.remote_cidr_list
|
||||
local_cidrs = local.local_cidr_list
|
||||
local_id_mode = var.local_id_mode
|
||||
psk = var.psk
|
||||
}
|
||||
}
|
||||
p2s = { enabled = var.p2s_enabled, address_pool = var.p2s_address_pool, ca_name = "VPN Router CA" }
|
||||
wireguard = { enabled = var.wireguard_enabled, address = var.wireguard_address, listen_port = var.wireguard_listen_port }
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
locals {
|
||||
remote_cidr_list = [for c in split(",", var.remote_cidrs) : trimspace(c) if trimspace(c) != ""]
|
||||
local_cidr_list = [for c in split(",", var.local_cidrs) : trimspace(c) if trimspace(c) != ""]
|
||||
}
|
||||
|
||||
resource "google_compute_route" "to_remote" {
|
||||
|
||||
+10
-7
@@ -47,19 +47,22 @@ cat > /etc/vpn-router/vpn-router.json <<EOF
|
||||
"internal": "${INTERNAL_INTERFACE}"
|
||||
},
|
||||
"wan": {
|
||||
"local_fqdn": "${LOCAL_FQDN}",
|
||||
"local_id_mode": "${LOCAL_ID_MODE}"
|
||||
"local_fqdn": "${LOCAL_FQDN}"
|
||||
},
|
||||
"local": {
|
||||
"cidrs": ${LOCAL_CIDRS},
|
||||
"int_addr": "${INT_ADDR}",
|
||||
"int_gateway_ip": "${INT_GATEWAY_IP}"
|
||||
},
|
||||
"remote": {
|
||||
"addrs": ${REMOTE_ADDRS},
|
||||
"id": "${REMOTE_ID}",
|
||||
"cidrs": ${REMOTE_CIDRS},
|
||||
"psk_file": "/etc/vpn-router/psk"
|
||||
"site_to_site": {
|
||||
"remote-site": {
|
||||
"addrs": ${REMOTE_ADDRS},
|
||||
"id": "${REMOTE_ID}",
|
||||
"cidrs": ${REMOTE_CIDRS},
|
||||
"local_cidrs": ${LOCAL_CIDRS},
|
||||
"local_id_mode": "${LOCAL_ID_MODE}",
|
||||
"psk_file": "/etc/vpn-router/psk"
|
||||
}
|
||||
},
|
||||
"p2s": {
|
||||
"enabled": ${P2S_ENABLED},
|
||||
|
||||
+6
-5
@@ -1,9 +1,9 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Commit everything, tag the version of the newest changelog entry and push main with the tags.
|
||||
|
||||
The tag is the upstream version of the first entry of debian-package/debian/changelog,
|
||||
without the Debian revision: vpn-router (0.5.1-1) is tagged 0.5.1, which starts the
|
||||
release workflow.
|
||||
The tag is the version of the first entry of debian-package/debian/changelog, without the
|
||||
Debian revision 1: vpn-router (0.5.1-1) is tagged 0.5.1 and (0.5.1-2) 0.5.1-2. The tag
|
||||
starts the release workflow.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
@@ -48,13 +48,14 @@ def summary():
|
||||
|
||||
|
||||
def version():
|
||||
"""The upstream version of the first changelog entry."""
|
||||
"""The tag of the first changelog entry: its version, without the Debian revision when that is 1."""
|
||||
first = CHANGELOG.read_text(encoding='utf-8').splitlines()[0]
|
||||
start, end = first.find('('), first.find(')')
|
||||
if start == -1 or end < start:
|
||||
sys.exit(f'{CHANGELOG}: the first line has no version: {first}')
|
||||
full = first[start + 1:end]
|
||||
return full.rsplit('-', 1)[0] if '-' in full else full
|
||||
upstream, _, revision = full.rpartition('-')
|
||||
return upstream if upstream and revision == '1' else full
|
||||
|
||||
|
||||
def main():
|
||||
|
||||
Reference in new issue
Block a user