Compare commits

...
21 Commits
Author SHA1 Message Date
slawek ff0c93ed73 fix: the webui does not allow to disable the last admin now.
Release / release (push) Failing after 4s
2026-11-22 19:48:02 +01:00
slawek 5fff009320 fix: update remote LXD scripts to use OVN backed NAT-ed networks. 2026-11-22 19:27:53 +01:00
slawek da889b4865 fix: connectivity panel rearrangement. LXD VM for repeatable tests. 2026-11-22 15:59:40 +01:00
slawek 85a1ffa21f feat: rebranded from vpn-router to linux-manager. 2026-11-22 09:51:07 +01:00
slawek 8efe21bfa6 feature: the package installs the Router Manager, which is no longer a package of its own.
Release / release (push) Failing after 4s
2026-11-14 12:52:44 +01:00
slawek 6d90441bee fix: the web service starts again after its first start with 0.39.0, and the name server's domains can be changed from any process.
Release / release (push) Successful in 14s
2026-11-06 20:17:26 +01:00
slawek ee1161f534 update: vpn-router is published in the koszewscy package registry.
Release / release (push) Successful in 15s
2026-11-06 20:09:06 +01:00
slawek 7f85a1a2e5 feature: the name server's domains have wan.local_fqdn and dns.secondary as their name servers, and a site with dns router gets records of the router's public, internal or external address.
Release / release (push) Successful in 35s
2026-11-06 19:06:48 +01:00
slawek c2c33cfca0 fix: /https/settings shows the https section and switches https.acme_staging, for the Router Manager.
Release / release (push) Successful in 56s
2026-10-30 02:46:07 +01:00
slawek f36e2a3b04 feature: https.acme_staging takes Caddy's certificates from Let's Encrypt's staging environment.
Release / release (push) Successful in 14s
2026-10-30 02:35:21 +01:00
slawek 1e0e375d60 fix: an answer whose client went away ends quietly, and a shell whose client left before it opened is stopped.
Release / release (push) Successful in 15s
2026-10-30 02:03:58 +01:00
slawek dd489e65aa fix: the web service, the metrics service and the bootstrap server log a failure in one line instead of a traceback, and nothing when a client goes away.
Release / release (push) Successful in 17s
2026-10-30 02:01:27 +01:00
slawek 3de030b06b fix: /config answers the OpenID Provider's state and issuer.
Release / release (push) Successful in 15s
2026-10-30 01:54:50 +01:00
slawek 3c4c5a8c32 feature: the OpenID Provider has the client credentials and device authorization grants, public clients, and service accounts as resources with roles and scopes in JWT access tokens.
Release / release (push) Successful in 17s
2026-10-30 01:01:45 +01:00
slawek 3ed79c7563 feature: a shell opens with one of the user's passkeys in place of the password and the TOTP code.
Release / release (push) Successful in 17s
2026-10-29 23:13:11 +01:00
slawek 3e78d520ae update: the package is published for Ubuntu 26.04 (resolute) as well as 24.04 (noble).
Release / release (push) Successful in 33s
2026-10-29 21:42:15 +01:00
slawek d99e4ea425 fix: the web service answers /, /status and /status/full again while IKEv2 runs.
Release / release (push) Successful in 22s
2026-10-29 19:04:11 +01:00
slawek 46d3550eab update: the router web service is split into the package vpnrouter_web, with one module per family of paths.
Release / release (push) Successful in 14s
2026-10-29 18:52:51 +01:00
slawek 3b14b7723e feature: vre opens files from the Router Manager's shell in the editor of its Shell page.
Release / release (push) Successful in 17s
2026-10-29 18:10:58 +01:00
slawek 5303779559 feature: the web service shows and changes the settings of the metrics service at /metrics/settings, and OpenVPN starts on a new router, as setup installs CRLs that revoke nothing.
Release / release (push) Failing after 14m34s
2026-10-28 22:31:27 +01:00
slawek 2dc4db440f fix: OpenVPN starts on a new router, as setup installs CRLs that revoke nothing.
Release / release (push) Failing after 15m31s
2026-10-28 22:00:37 +01:00
272 changed files with 27508 additions and 5610 deletions

No files matched your search

+11
View File
@@ -0,0 +1,11 @@
.git
.claude
.vscode
out
tmp
examples
web/node_modules
web/dist
.venv
dev.json
**/__pycache__
+4
View File
@@ -12,3 +12,7 @@ tmp
*.tfstate
*.tfstate.*
*.auto.tfvars
node_modules
web/dist
.venv
dev.json
+693 -327
View File
File diff suppressed because it is too large. Load diff
+16 -2
View File
@@ -1,4 +1,13 @@
ARG UBUNTU_VERSION=24.04
ARG NODE_VERSION=24
ARG UBUNTU_VERSION=26.04
FROM node:${NODE_VERSION}-bookworm-slim AS web
WORKDIR /web
COPY web/package.json web/package-lock.json ./
RUN npm ci
COPY web/ ./
RUN npm run build
FROM ubuntu:${UBUNTU_VERSION}
ARG HOST_UID
ARG HOST_GID
@@ -12,7 +21,12 @@ RUN apt-get update \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /build
COPY . src/
COPY debian-package/ src/
COPY docs/ src/docs/
COPY --from=web /web/dist src/web/dist/
# DEB_VERSION, when set, replaces the version of the newest changelog entry in the image only.
ARG DEB_VERSION
RUN if [ -n "$DEB_VERSION" ]; then sed -i "1s/([^)]*)/($DEB_VERSION)/" src/debian/changelog; fi
RUN cd src && dpkg-buildpackage -us -uc -b
RUN lintian --no-tag-display-limit /build/*.changes || true
+298
View File
@@ -0,0 +1,298 @@
Private Enterprise Number 67380 S. Koszewski
Category: Registry November 2026
OID Registry for the Linux Manager Arc 1.3.6.1.4.1.67380.10.1
Abstract
This document is the registry of Object Identifiers (OIDs) under the
arc 1.3.6.1.4.1.67380.10.1, which the registry of the Private
Enterprise Number 67380 delegates to Linux Manager. It defines the
tree of the arc, the rules for allocating OIDs in it, and lists every
allocation.
Table of Contents
1. Introduction
2. Root
3. Rules
4. Tree
5. Statuses
6. Assignments
7. Requesting an OID
8. References
Author's Address
1. Introduction
The tree of the arc follows the layout of the root registry of the
Private Enterprise Number 67380 [PEN-REGISTRY]: it is split by
function at its top level, and another arc holds experimental
definitions.
In this document, an OID written with a leading dot, such as ".2.1",
is relative to the OID root given in Section 2.
2. Root
+===========================+==========================+
| Item | Value |
+===========================+==========================+
| OID root | 1.3.6.1.4.1.67380.10.1 |
+---------------------------+--------------------------+
| Parent registry entry | .10.1 linux-manager |
+---------------------------+--------------------------+
Table 1: Root
3. Rules
* A number is used once among its siblings. Arcs and leaves are
never reused or renumbered.
* An allocated OID is never deleted. A retired OID is marked
deprecated and stays listed.
* A frozen arc takes no new children. Its existing children keep
their status.
* Leaves under an arc are assigned in order, starting from 1.
* Function arc numbers have the same meaning as in the root
registry, both under the root of this arc and under its
experimental arc ".99".
* Every leaf is listed in this document before it is used in
released software.
* An experimental OID under ".99" is never used in released
software or in production. A definition that leaves the
experimental arc gets a new OID; the experimental OID is not
moved.
* LDAP descriptors (short names) of schema elements under ".2"
carry the prefix "linuxManager".
* Documentation and test examples use "1.3.6.1.4.1.32473" [RFC5612]
or "2.999", never this arc.
4. Tree
1.3.6.1.4.1.67380.10.1 Linux Manager
.0 reserved, never allocated
.1 SNMP (reserved)
.2 LDAP / directory
.2.1 attribute types
.2.2 object classes
.3 PKI / X.509
.3.1 certificate policies
.3.2 extended key usages (KeyPurposeId)
.99 Experimental: mirrors the function arcs
Figure 1: Tree
5. Statuses
The statuses are those of the root registry [PEN-REGISTRY]:
reserved, active, frozen and deprecated.
6. Assignments
Each entry gives the OID and its name, followed by its fields. The
owner of every entry is "Slawomir Koszewski Org.".
6.1. Arcs
.0 reserved
Description: Never allocated
Status: reserved
Allocated: 2026-11-21
Reference: none
.1 snmp
Description: SNMP MIB modules, held for later use
Status: reserved
Allocated: 2026-11-21
Reference: [RFC2578]
.2 ldap
Description: The LDAP schema of the package
Status: active
Allocated: 2026-11-21
Reference: [RFC4512]
.2.1 ldap-attribute-types
Description: LDAP attribute types
Status: active
Allocated: 2026-11-21
Reference: [RFC4512], Section 4.1.2
.2.2 ldap-object-classes
Description: LDAP object classes
Status: active
Allocated: 2026-11-21
Reference: [RFC4512], Section 4.1.1
.3 pki
Description: PKI and X.509
Status: active
Allocated: 2026-11-21
Reference: [RFC5280]
.3.1 pki-certificate-policies
Description: Certificate policies
Status: active
Allocated: 2026-11-21
Reference: [RFC5280], Section 4.2.1.4
.3.2 pki-extended-key-usages
Description: Extended key usages (KeyPurposeId)
Status: active
Allocated: 2026-11-21
Reference: [RFC5280], Section 4.2.1.12
.99 experimental
Description: Experimental definitions, never in
production
Status: active
Allocated: 2026-11-21
Reference: [RFC4520], Section 3.1
6.2. LDAP Attribute Types
Every attribute type below has the status active, was allocated on
2026-11-21, and is defined in "linuxmanager_ldap.py" of this
repository.
.2.1.1 linuxManagerScope
An authorization scope of the account
.2.1.2 linuxManagerPrivateKey
A private key, PEM
.2.1.3 linuxManagerCertificateRequest
The CSR of a CA, PEM
.2.1.4 linuxManagerCRLNumber
The number of the last CRL of a CA
.2.1.5 linuxManagerCRLSource
Signed or uploaded, where the CRL of a CA came from
.2.1.6 linuxManagerRetired
When a CA was retired
.2.1.7 linuxManagerKind
Ca, router, user or external, the kind of a certificate
.2.1.8 linuxManagerTemplate
The template of a certificate
.2.1.9 linuxManagerSubjectAltName
A subject alternative name of a certificate
.2.1.10 linuxManagerNotAfter
When a certificate expires
.2.1.11 linuxManagerRevoked
When a certificate was revoked
.2.1.12 linuxManagerRevocationReason
Why a certificate was revoked
.2.1.13 linuxManagerConsoleLogin
Whether the user may use a console
.2.1.14 linuxManagerConsoleAccount
The local account the console of the user runs as
.2.1.15 linuxManagerWireGuardPeer
A WireGuard peer of the user, its public key and IPs
.2.1.16 linuxManagerPasskey
A passkey of the user, its credential as JSON
.2.1.17 linuxManagerRedirectUri
A redirect URI of an OpenID Connect client
.2.1.18 linuxManagerPostLogoutRedirectUri
A post-logout redirect URI of an OpenID Connect client
.2.1.19 linuxManagerAllowedGroup
A group whose members may sign in to an OpenID Connect
client
.2.1.20 linuxManagerGrantType
A grant an OpenID Connect client may use besides the
authorization code
.2.1.21 linuxManagerPublicClient
Whether an OpenID Connect client has no secret
.2.1.22 linuxManagerClientScope
A scope of a resource an OpenID Connect client may ask for
.2.1.23 linuxManagerResourceScope
A scope of a resource of the OpenID Provider
6.3. LDAP Object Classes
Every object class below has the status active, was allocated on
2026-11-21, and is defined in "linuxmanager_ldap.py" of this
repository.
.2.2.1 linuxManagerAccount
Auxiliary: properties of a user or a Manager-only account
.2.2.2 linuxManagerAuthority
Structural: a CA of the PKI
.2.2.3 linuxManagerCertificate
Structural: a certificate a CA of the PKI issued
.2.2.4 linuxManagerRole
Structural: a role of a resource of the OpenID Provider
7. Requesting an OID
An allocation is a change to this document. The change adds an
entry to Section 6 with every field filled in, and for a new arc
also updates Figure 1. A new function arc is first allocated in the
root registry [PEN-REGISTRY].
8. References
[PEN-REGISTRY]
Koszewski, S., "OID Registry for Private Enterprise
Number 67380", November 2026.
[RFC2578] McCloghrie, K., Ed., Perkins, D., Ed., and J.
Schoenwaelder, Ed., "Structure of Management Information
Version 2 (SMIv2)", STD 58, RFC 2578,
DOI 10.17487/RFC2578, April 1999,
<https://www.rfc-editor.org/info/rfc2578>.
[RFC4512] Zeilenga, K., Ed., "Lightweight Directory Access Protocol
(LDAP): Directory Information Models", RFC 4512,
DOI 10.17487/RFC4512, June 2006,
<https://www.rfc-editor.org/info/rfc4512>.
[RFC4520] Zeilenga, K., "Internet Assigned Numbers Authority (IANA)
Considerations for the Lightweight Directory Access
Protocol (LDAP)", BCP 64, RFC 4520, DOI 10.17487/RFC4520,
June 2006, <https://www.rfc-editor.org/info/rfc4520>.
[RFC5280] Cooper, D., Santesson, S., Farrell, S., Boeyen, S.,
Housley, R., and W. Polk, "Internet X.509 Public Key
Infrastructure Certificate and Certificate Revocation List
(CRL) Profile", RFC 5280, DOI 10.17487/RFC5280, May 2008,
<https://www.rfc-editor.org/info/rfc5280>.
[RFC5612] Eronen, P. and D. Harrington, "Enterprise Number for
Documentation Use", RFC 5612, DOI 10.17487/RFC5612,
August 2009, <https://www.rfc-editor.org/info/rfc5612>.
Author's Address
Slawomir Koszewski
+1095 -474
View File
File diff suppressed because it is too large. Load diff
Executable
+20
View File
@@ -0,0 +1,20 @@
#!/bin/sh
set -e
# CONTAINER_RUNTIME from the environment, for example docker with DOCKER_CONTEXT naming a remote engine;
# otherwise Apple container CLI on macOS and Docker CE on Linux.
if [ -z "${CONTAINER_RUNTIME:-}" ]; then
case "$(uname -s)" in
Darwin) CONTAINER_RUNTIME=container ;;
Linux) CONTAINER_RUNTIME=docker ;;
*) printf 'ERROR: unsupported platform %s\n' "$(uname -s)" >&2; exit 1 ;;
esac
fi
mkdir -p out
$CONTAINER_RUNTIME build --build-arg HOST_UID=$(id -u) --build-arg HOST_GID=$(id -g) -t linux-manager-builder "$@" .
# Artifacts are streamed out as a tar archive, so no bind mount is needed.
$CONTAINER_RUNTIME run --rm linux-manager-builder tar -cf - -C /out . | tar -xf - -C out
echo "Build artifacts written to out/"
-17
View File
@@ -1,17 +0,0 @@
#!/bin/sh
set -e
# Apple container CLI on macOS, Docker CE on Linux.
case "$(uname -s)" in
Darwin) CONTAINER_RUNTIME=container ;;
Linux) CONTAINER_RUNTIME=docker ;;
*) printf 'ERROR: unsupported platform %s\n' "$(uname -s)" >&2; exit 1 ;;
esac
mkdir -p out
$CONTAINER_RUNTIME build --build-arg HOST_UID=$(id -u) --build-arg HOST_GID=$(id -g) -t vpn-router-builder "$@" .
# Artifacts are streamed out as a tar archive, so no bind mount is needed.
$CONTAINER_RUNTIME run --rm vpn-router-builder tar -cf - -C /out . | tar -xf - -C out
echo "Build artifacts written to out/"
+425 -50
View File
@@ -1,3 +1,311 @@
linux-manager (0.41.0-1) unstable; urgency=medium
feature: the package is linux-manager, the Ubuntu Linux Manager, whose
router and VPN are components switched on in the configuration.
* The package, its paths, units, commands, Python modules, debconf
questions, LDAP base DN dc=linux-manager,dc=local, schema, database and
metric names are linux-manager instead of vpn-router; the Manager's unit
is linux-manager-ui.service, vpn-config-edit is
linux-manager-config-edit and vre is lme.
* The Manager is the Ubuntu Linux Manager, its first page Server.
* router.enabled and vpn.enabled, false by default, switch the router and
the VPN; the debconf questions linux-manager/router_enabled and
vpn_enabled set them, and only the questions of the components on are
asked.
* With the router off the server needs only interfaces.external, which is
also the internal side of the listen choices, and site_to_site must be
empty; with the VPN off p2s, openvpn and wireguard must be off.
* With the router off and the VPN on, the VPN pools are masqueraded out of
the external interface, so the clients reach local.cidrs through NAT.
* linux-manager-setup writes /etc/sysctl.d/99-linux-manager.conf while the
router or the VPN is on and removes it while both are off; the package
ships no sysctl file.
* The firewall is enabled once any service of the core is configured; the
IPsec rules are written only while site-to-site or IKEv2 is ready, and
MSS clamping while the router or the VPN is on; strongSwan is stopped
while neither needs it.
* The P2S CA is created while the VPN is on and the site-to-site CA while
the router is on; switching the VPN on issues the missing certificates
of the VPN users.
* While the VPN is off, a VPN user is created without a certificate, and
WireGuard peers and the VPN profiles are refused.
* The Manager hides the Remote Access page while the VPN is off, and shows
only the Network tab of the Connectivity page while the router is off.
* The Network tab lists every interface but loopback by name, with its
type, external when it holds the IPv4 default route and internal
otherwise, state, addresses and MAC address; the internal gateway and the
local CIDRs are in a Router panel shown while the router is on. /config
has the interfaces in network.interfaces.
* The LDAP schema is under 1.3.6.1.4.1.67380.10.1.2, the arc listed in
OID-REGISTRY.txt.
* Ubuntu 26.04 (resolute) is the only supported release: the build image
and the published distribution are resolute, the Azure and GCP examples
install from resolute, and the Azure example boots the 26.04 image.
* The examples switch the router and the VPN on.
* A NAT rule that the configuration no longer needs is removed from the
running firewall at once, not only at the next boot.
* The last enabled member of admins can no longer be disabled, through
the API, the Manager or linux-manager-user (409, status 4).
* The Users, Groups, Service accounts, Web Sites and Files pages keep a
refused change's message above the list after they read it again.
* tests/remote/run.py tests the components on Ubuntu 26.04 instances of an
LXD remote, and tests/remote/vm.py starts and removes one server
VM there to work with, reached through ports 7080, 7081 and 2222 (SSH)
of the LXD host, by default at the address of the remote's URL.
* tests/remote/lxd-project.py creates and removes a project for manual
tests with its own OVN networks, a WAN and two LANs, and a signed apt
repository; tests/remote/dev-build.py publishes numbered dev builds, version
<upstream>~devN-<revision>, to it, and vm.py installs from it there.
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Sat, 21 Nov 2026 23:39:03 +0100
vpn-router (0.40.0-1) unstable; urgency=medium
feature: the package installs the Router Manager, which is no longer a
package of its own.
* The Router Manager, vpn-router-manager.service, is installed and started
with the router; its programs are in /usr/lib/vpn-router, its application
and help in /usr/share/vpn-router/manager.
* Its settings are /etc/vpn-router/manager.json, its database password
/etc/vpn-router/manager-database.secret and the state of its upgrades
/var/lib/vpn-router/upgrade.json and system-upgrade.json.
* The debconf questions vpn-router/admin_username and admin_password give
its first administrator, written to /etc/vpn-router/manager-admin.json.
* The Overview tab and the upgrade helper know the package vpn-router
alone.
* The bootstrap page on port 17081 shows the manager's service and log;
port 17080 and vpn-router-manager-bootstrap are gone.
* The status has no manager_version; the manager's unit is always among
the services.
* The package cannot be installed beside the package vpn-router-manager,
which has to be removed first.
* A purge also removes the manager's settings, its database and role, and
the record of its upgrades.
* build.sh, publish.sh and the Dockerfile are in the repository root, the
build also builds the application in web/, and CONTAINER_RUNTIME chooses
the container runtime.
* The README and the ADR call the product that deploys the router Cloud
Manager, also where they named manage-ui, and the docstring of the GCP
platform names Google Cloud.
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Fri, 13 Nov 2026 20:15:15 +0100
vpn-router (0.39.1-1) unstable; urgency=medium
fix: the web service starts again after its first start with 0.39.0, and the
name server's domains can be changed from any process.
* The schemas of the web sites and of the name server can be applied any
number of times, so the web service no longer keeps retrying with
'function "site_put" already exists' and never listening, and
vpn-router-site and setup no longer fail the same way.
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Fri, 06 Nov 2026 20:16:57 +0100
vpn-router (0.39.0-2) unstable; urgency=medium
update: vpn-router is published in the koszewscy package registry.
* publish.sh uploads to the owner koszewscy unless OWNER names another.
* repo_url of the Azure and GCP examples defaults to
https://gitea.koszewscy.waw.pl/api/packages/koszewscy/debian.
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Fri, 06 Nov 2026 20:08:29 +0100
vpn-router (0.39.0-1) unstable; urgency=medium
feature: the name server's domains have wan.local_fqdn and dns.secondary as
their name servers, and a site with dns router gets records of the router's
public, internal or external address.
* The SOA and the first NS of every domain are wan.local_fqdn, and the
second NS is dns.secondary, a host name, whose addresses are notified
of every change and may transfer the domains; dns.nameserver is gone.
* wan.local_fqdn gets no record in the domains; its record is kept in
the DNS of its own domain.
* A site has dns, user (the default), served elsewhere, or router, whose
host names get an A or AAAA record in their closest domain, with
dns_address public (wan.public_address, new), internal or external.
dns.addresses, the domains' site records switch and the per-domain
secondaries with their routes are gone; dns.nameserver and
dns.addresses of an earlier configuration are left out when read.
* A site may use wan.local_fqdn as a host name on a port of its own.
* GET /dns answers nameserver, wan.local_fqdn, and public_address,
which PUT /dns changes.
* The summaries of the changelog entries are wrapped at 80 columns, and
release.py joins the lines of a summary into the commit message.
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Fri, 30 Oct 2026 21:30:01 +0100
vpn-router (0.38.1-1) unstable; urgency=medium
fix: /https/settings shows the https section and switches https.acme_staging,
for the Router Manager.
* GET /https/settings shows the https section, and PUT /https/settings
changes acme_staging; the other settings of the section stay with
vpn-config-edit.
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Fri, 30 Oct 2026 02:43:22 +0100
vpn-router (0.38.0-1) unstable; urgency=medium
feature: https.acme_staging takes Caddy's certificates from Let's Encrypt's
staging environment.
* While https.acme_staging is true, Caddy's only CA is Let's Encrypt's
staging environment, for the router's services and the web sites
with tls acme; its certificates are not trusted by browsers.
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Fri, 30 Oct 2026 02:34:01 +0100
vpn-router (0.37.3-1) unstable; urgency=medium
fix: an answer whose client went away ends quietly, and a shell whose client
left before it opened is stopped.
* The web service, the metrics service and the bootstrap server stop
writing an answer when its client closes the connection, and a site
file whose download is left no longer gets a second answer.
* A shell whose client closed the connection before it opened is
stopped and its end is logged, instead of running on.
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Fri, 30 Oct 2026 02:03:26 +0100
vpn-router (0.37.2-1) unstable; urgency=medium
fix: the web service, the metrics service and the bootstrap server log a
failure in one line instead of a traceback, and nothing when a client goes
away.
* A request whose client closed the connection is no longer logged.
* Any other failure of a request, a thread or the service is logged in
one line with the exception's type and message.
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Fri, 30 Oct 2026 01:59:19 +0100
vpn-router (0.37.1-1) unstable; urgency=medium
fix: /config answers the OpenID Provider's state and issuer.
* /config has oidc: enabled, and issuer, the issuer URL, null while the
provider is disabled.
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Fri, 30 Oct 2026 01:53:55 +0100
vpn-router (0.37.0-1) unstable; urgency=medium
feature: the OpenID Provider has the client credentials and device
authorization grants, public clients, and service accounts as resources with
roles and scopes in JWT access tokens.
* A client may have the grants client_credentials, by which a
confidential client gets an access token as its own service account,
and the device code (RFC 8628): POST /oidc/device_authorization gives
a device code and a user code, the user enters the code on
/oidc/device, signs in and allows or denies the device, and the device
polls /oidc/token. A public client has no secret.
* An http loopback redirect URI matches with any port.
* Every service account is a resource: it defines scopes, which clients
ask for as <resource>:<scope> when their client scopes allow them, and
roles, assigned to groups, users and service accounts. A token request
names its resource with resource (RFC 8707) or by its scopes.
* Access tokens are RS256 JWTs of RFC 9068 for the resource, or the
client, with client_id, scope, roles and the claims of the scopes; ID
tokens and /oidc/userinfo carry the user's roles of the client.
* vpn-router-user takes --grant, --public-client, --client-scope and
--resource-scope; the new vpn-router-role manages the roles; and
/service-accounts takes and answers grants, public_client,
client_scopes, resource_scopes and roles.
* Changing the password of a service account, disabling, unlocking or
removing it through /service-accounts revokes its tokens.
* POST /oidc/device and /oidc/device_authorization are rate limited as
/oidc/login is.
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Fri, 30 Oct 2026 00:57:11 +0100
vpn-router (0.36.0-1) unstable; urgency=medium
feature: a shell opens with one of the user's passkeys in place of the
password and the TOTP code.
* POST /shell/passkey begins a proof for one of the signed-in user's own
passkeys, with user verification; POST /shell takes the browser's
answer as {"passkey": {"id", "credential"}} in place of the password
and the code, and gives the ticket of the shell.
* A passkey that is not accepted is refused with 403. The shell still
needs the user's TOTP enrolment, which its login session and sudo in
it need.
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Thu, 29 Oct 2026 23:09:43 +0100
vpn-router (0.35.2-2) unstable; urgency=medium
update: the package is published for Ubuntu 26.04 (resolute) as well as 24.04
(noble).
* publish.sh uploads the package to the distributions noble and resolute
of the repository, component main, skipping a distribution that has
the version already; DISTRIBUTIONS names others. A router on Ubuntu
26.04 installs vpn-router from the distribution resolute.
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Thu, 29 Oct 2026 21:41:32 +0100
vpn-router (0.35.2-1) unstable; urgency=medium
fix: the web service answers /, /status and /status/full again while IKEv2
runs.
* The status of IKEv2 clients is collected again; the requests that
collect the status were dropped with a NameError since 0.35.1.
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Thu, 29 Oct 2026 19:03:16 +0100
vpn-router (0.35.1-1) unstable; urgency=medium
update: the router web service is split into the package vpnrouter_web, with
one module per family of paths.
* /usr/lib/vpn-router/web initializes and serves; the package
/usr/lib/vpn-router/vpnrouter_web/ answers the requests, with a module
for the status, the sign-in, the identities, the connections, backups,
the PKI, logs, web sites, DNS, metrics and the shell. The paths, the
answers and the log messages are the same.
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Thu, 29 Oct 2026 18:20:50 +0100
vpn-router (0.35.0-1) unstable; urgency=medium
feature: vre opens files from the Router Manager's shell in the editor of its
Shell page.
* vpn-router-editor FILE..., or vre, in the shell of the Router Manager
opens the files in the editor of the Shell page and returns at once;
outside that shell it exits with status 2.
* GET, PUT and DELETE /shell/editor/<session>/<id> read, save and close
a file opened so, as the shell's console account; a save is refused
with 409 when the file changed since it was read.
* The editor takes regular files of UTF-8 text up to 1 MiB, at most 20
per shell, while the shell lives.
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Thu, 29 Oct 2026 18:02:27 +0100
vpn-router (0.34.0-1) unstable; urgency=medium
feature: the web service shows and changes the settings of the metrics service
at /metrics/settings, and OpenVPN starts on a new router, as setup installs
CRLs that revoke nothing.
* GET and PUT /metrics/settings show and change the metrics section;
otlp_password is never shown, an empty one keeps it, and an empty
otlp_username removes it.
* A CRL without revoked certificates is installed for strongSwan and
OpenVPN.
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Wed, 28 Oct 2026 22:00:00 +0100
vpn-router (0.33.3-1) unstable; urgency=medium
fix: vpn-router-setup runs on a new installation once libpq can be found.
@@ -21,7 +329,8 @@ vpn-router (0.33.2-1) unstable; urgency=medium
vpn-router (0.33.1-1) unstable; urgency=medium
fix: the name server is the primary of any number of domains, each with secondary name servers and a switch for the records of the sites.
fix: the name server is the primary of any number of domains, each with
secondary name servers and a switch for the records of the sites.
* The domains, their secondaries and manual records are kept in the
vpn-router-dns database and changed through /dns/domains; dns.zone is
@@ -39,7 +348,8 @@ vpn-router (0.33.1-1) unstable; urgency=medium
vpn-router (0.33.0-1) unstable; urgency=medium
feature: road-warrior clients resolve the domains of the new setting p2s.dns_domains with the router's resolver.
feature: road-warrior clients resolve the domains of the new setting
p2s.dns_domains with the router's resolver.
* p2s.dns_domains lists up to 16 domains; setup pushes them to IKEv2
clients as INTERNAL_DNS_DOMAIN (RFC 8598, attribute 25) of the
@@ -51,7 +361,8 @@ vpn-router (0.33.0-1) unstable; urgency=medium
vpn-router (0.32.1-1) unstable; urgency=medium
fix: a VPN user downloads the own OpenVPN client profile through /account, with the current password and code.
fix: a VPN user downloads the own OpenVPN client profile through /account,
with the current password and code.
* POST /account/profile answers the user's OpenVPN client profile; it
needs the current password and, when enrolled, a code.
@@ -60,7 +371,8 @@ vpn-router (0.32.1-1) unstable; urgency=medium
vpn-router (0.32.0-1) unstable; urgency=medium
feature: the web service gives Apple VPN profiles that set up IKEv2 road-warrior access on macOS, iOS and iPadOS.
feature: the web service gives Apple VPN profiles that set up IKEv2
road-warrior access on macOS, iOS and iPadOS.
* POST /users/<name>/apple-profile and POST /account/apple-profile answer
an unsigned .mobileconfig with the P2S CA and the root CA, the user's
@@ -74,7 +386,9 @@ vpn-router (0.32.0-1) unstable; urgency=medium
vpn-router (0.31.3-1) unstable; urgency=medium
fix: WireGuard clients may use the router's DNS resolver, which runs while WireGuard does, and /remote-access names its addresses for their configuration.
fix: WireGuard clients may use the router's DNS resolver, which runs while
WireGuard does, and /remote-access names its addresses for their
configuration.
* The firewall opens port 53 of the remote access clients' resolver for
the WireGuard network, as it does for the IKEv2 and OpenVPN pools.
@@ -88,7 +402,8 @@ vpn-router (0.31.3-1) unstable; urgency=medium
vpn-router (0.31.2-1) unstable; urgency=medium
fix: dnsmasq answers the remote access clients' DNS queries, so they no longer get the router's own host name with all its addresses from systemd-resolved.
fix: dnsmasq answers the remote access clients' DNS queries, so they no longer
get the router's own host name with all its addresses from systemd-resolved.
* vpn-router-resolver.service runs dnsmasq on the addresses of
p2s.dns_listen, forwarding to systemd-resolved's stub on 127.0.0.53;
@@ -103,7 +418,9 @@ vpn-router (0.31.2-1) unstable; urgency=medium
vpn-router (0.31.1-1) unstable; urgency=medium
fix: the remote access clients' resolver answers on the internal address by default, chosen by p2s.dns_listen, so the name server can take port 53 of the external addresses.
fix: the remote access clients' resolver answers on the internal address by
default, chosen by p2s.dns_listen, so the name server can take port 53 of the
external addresses.
* The new setting p2s.dns_listen chooses the addresses of the resolver the
road-warrior and OpenVPN clients are given: internal, the default,
@@ -116,7 +433,9 @@ vpn-router (0.31.1-1) unstable; urgency=medium
vpn-router (0.31.0-1) unstable; urgency=medium
feature: Caddy serves static web sites from folders on the router, defined in their own database, with files uploaded through the web service or WebDAV, and an optional PowerDNS name server answers for a zone of the sites.
feature: Caddy serves static web sites from folders on the router, defined in
their own database, with files uploaded through the web service or WebDAV, and
an optional PowerDNS name server answers for a zone of the sites.
* The new database vpn-router-sites holds the web sites: host names,
port, interfaces, TLS (none, Let's Encrypt, an issuing CA or a
@@ -152,7 +471,8 @@ vpn-router (0.31.0-1) unstable; urgency=medium
vpn-router (0.30.0-1) unstable; urgency=medium
feature: the web service serves the last lines of the journal of the router's units and of the log files web.log_files lists, at /logs.
feature: the web service serves the last lines of the journal of the router's
units and of the log files web.log_files lists, at /logs.
* GET /logs lists the units whose journal and the files whose lines it
serves; GET /logs/journal?unit=&lines= and /logs/file?path=&lines=
@@ -165,7 +485,8 @@ vpn-router (0.30.0-1) unstable; urgency=medium
vpn-router (0.29.0-1) unstable; urgency=medium
feature: vpn-config-edit set and remove take --restart, which applies the change with vpn-router-setup and waits for it.
feature: vpn-config-edit set and remove take --restart, which applies the
change with vpn-router-setup and waits for it.
* vpn-config-edit set --restart and remove --restart restart
vpn-router-setup after writing the configuration and wait until it has
@@ -176,7 +497,8 @@ vpn-router (0.29.0-1) unstable; urgency=medium
vpn-router (0.28.0-1) unstable; urgency=medium
feature: bash completes the vpn-* commands, their options and the names they take.
feature: bash completes the vpn-* commands, their options and the names they
take.
* Bash completes the commands, options and arguments of vpn-router-user,
vpn-router-group, vpn-router-token, vpn-config-edit, vpn-router-backup,
@@ -189,7 +511,9 @@ vpn-router (0.28.0-1) unstable; urgency=medium
vpn-router (0.27.0-1) unstable; urgency=medium
feature: the web service is an OpenID Provider, which signs the directory's users in to other systems, the clients being service accounts with redirect URIs.
feature: the web service is an OpenID Provider, which signs the directory's
users in to other systems, the clients being service accounts with redirect
URIs.
* oidc.enabled turns on the OpenID Provider at the issuer
https://<wan.local_fqdn>:<https.web_port>, with discovery, JWKS,
@@ -218,7 +542,8 @@ vpn-router (0.27.0-1) unstable; urgency=medium
vpn-router (0.26.2-1) unstable; urgency=medium
fix: a service account may be named by a GUID in lowercase, as the Router Manager proposes for a new one.
fix: a service account may be named by a GUID in lowercase, as the Router
Manager proposes for a new one.
* vpn-router-user add --service and POST /service-accounts take a GUID
in lowercase as the name, besides a user name; users and groups keep
@@ -228,7 +553,8 @@ vpn-router (0.26.2-1) unstable; urgency=medium
vpn-router (0.26.1-1) unstable; urgency=medium
fix: the package installs vpnrouter_passkeys, without which the web service and vpn-router-user do not start.
fix: the package installs vpnrouter_passkeys, without which the web service
and vpn-router-user do not start.
* /usr/lib/vpn-router/vpnrouter_passkeys.py is part of the package.
@@ -236,7 +562,8 @@ vpn-router (0.26.1-1) unstable; urgency=medium
vpn-router (0.26.0-1) unstable; urgency=medium
feature: users sign in to the Router Manager with passkeys, and console login needs a TOTP token, so a user without one is refused the shell, SSH and sudo.
feature: users sign in to the Router Manager with passkeys, and console login
needs a TOTP token, so a user without one is refused the shell, SSH and sudo.
* Console login is switched on only for a user enrolled for TOTP. A user
whose console login is on without a token is refused POST /shell, the
@@ -260,7 +587,8 @@ vpn-router (0.26.0-1) unstable; urgency=medium
vpn-router (0.25.4-1) unstable; urgency=medium
fix: a shell opens without the password and code again while the user's last use of a shell is within web.shell_auth_timeout minutes.
fix: a shell opens without the password and code again while the user's last
use of a shell is within web.shell_auth_timeout minutes.
* New setting web.shell_auth_timeout, 15 minutes by default, 0 to 1440:
POST /shell without a password gives a ticket while the user's last
@@ -271,7 +599,8 @@ vpn-router (0.25.4-1) unstable; urgency=medium
vpn-router (0.25.3-1) unstable; urgency=medium
fix: /config gives the state of caddy.service with the installed Caddy, for the Router Manager's Web page.
fix: /config gives the state of caddy.service with the installed Caddy, for
the Router Manager's Web page.
* caddy of /config is {"version", "source", "service"}.
@@ -279,7 +608,9 @@ vpn-router (0.25.3-1) unstable; urgency=medium
vpn-router (0.25.2-1) unstable; urgency=medium
fix: wireguard.address must be the router's own address in its network, not the network or broadcast address, from which the peers' addresses are told apart.
fix: wireguard.address must be the router's own address in its network, not
the network or broadcast address, from which the peers' addresses are told
apart.
* A wireguard.address that is the network or the broadcast address of a
network larger than /31, such as 192.168.232.0/24, is refused with the
@@ -289,7 +620,8 @@ vpn-router (0.25.2-1) unstable; urgency=medium
vpn-router (0.25.1-1) unstable; urgency=medium
fix: /remote-access gives the router's WireGuard public key, endpoint and allowed IPs, which a peer's configuration needs.
fix: /remote-access gives the router's WireGuard public key, endpoint and
allowed IPs, which a peer's configuration needs.
* GET /remote-access answers wireguard_server: {"public_key",
"endpoint", "allowed_ips"}.
@@ -298,7 +630,8 @@ vpn-router (0.25.1-1) unstable; urgency=medium
vpn-router (0.25.0-1) unstable; urgency=medium
feature: a VPN user has WireGuard peers, each a device with its public key and IPs, kept in the directory and written into wg0.conf by setup.
feature: a VPN user has WireGuard peers, each a device with its public key and
IPs, kept in the directory and written into wg0.conf by setup.
* PUT /users/<name> takes wireguard_peers, [{"public_key", "ips"}],
which replaces the user's peers; keys are unique and IPs may not
@@ -314,7 +647,8 @@ vpn-router (0.25.0-1) unstable; urgency=medium
vpn-router (0.24.1-1) unstable; urgency=medium
fix: the status gives the number of connected OpenVPN clients at most 5 seconds old instead of up to a minute.
fix: the status gives the number of connected OpenVPN clients at most 5
seconds old instead of up to a minute.
* The OpenVPN server writes its status file every 5 seconds, so a
refresh of the status shows clients that connected or left since.
@@ -323,7 +657,9 @@ vpn-router (0.24.1-1) unstable; urgency=medium
vpn-router (0.24.0-1) unstable; urgency=medium
feature: /remote-access of the web service shows and changes the settings of IKEv2 road-warrior, OpenVPN and WireGuard access and whether each runs, for the Router Manager's Remote access page.
feature: /remote-access of the web service shows and changes the settings of
IKEv2 road-warrior, OpenVPN and WireGuard access and whether each runs, for
the Router Manager's Remote access page.
* GET /remote-access answers the p2s, openvpn and wireguard sections,
which of them setup runs, and wan.local_fqdn; PUT
@@ -335,7 +671,9 @@ vpn-router (0.24.0-1) unstable; urgency=medium
vpn-router (0.23.2-1) unstable; urgency=medium
fix: the shell of a user's own POSIX account opens on the first setup that sets up console login, and the journal records each refused shell with the reason.
fix: the shell of a user's own POSIX account opens on the first setup that
sets up console login, and the journal records each refused shell with the
reason.
* Setup restarts vpn-router-web when it adds ldap to the name service
switch, so the web service finds the POSIX accounts of the directory.
@@ -346,7 +684,9 @@ vpn-router (0.23.2-1) unstable; urgency=medium
vpn-router (0.23.1-1) unstable; urgency=medium
fix: the shell runs as the console account of each member of admins, a local account or the user's own POSIX account in the directory, which also signs in with SSH and sudo, instead of one web.shell_user.
fix: the shell runs as the console account of each member of admins, a local
account or the user's own POSIX account in the directory, which also signs in
with SSH and sudo, instead of one web.shell_user.
* vpn-router-user console <name> --posix | --account LOCAL | --off, and
PUT /users/<name>/console with a token of vpn-router-token, switch a
@@ -364,7 +704,9 @@ vpn-router (0.23.1-1) unstable; urgency=medium
vpn-router (0.23.0-1) unstable; urgency=medium
feature: web.shell lets /shell of the web service open a login shell of web.shell_user over a WebSocket for the Router Manager's Shell page, after the user signs in again.
feature: web.shell lets /shell of the web service open a login shell of
web.shell_user over a WebSocket for the Router Manager's Shell page, after the
user signs in again.
* web.shell (false), web.shell_user and web.shell_idle_minutes (15).
* POST /shell takes the password and the TOTP code and answers a
@@ -376,7 +718,8 @@ vpn-router (0.23.0-1) unstable; urgency=medium
vpn-router (0.22.1-1) unstable; urgency=medium
fix: the status gives the number of site-to-site connections set up and established, not each connection by name.
fix: the status gives the number of site-to-site connections set up and
established, not each connection by name.
* site_to_site of the status document is {"connections", "established"},
and the status page shows "Site to site: N of M established".
@@ -385,7 +728,9 @@ vpn-router (0.22.1-1) unstable; urgency=medium
vpn-router (0.22.0-1) unstable; urgency=medium
feature: pki.extended_key_usages adds custom extended key usages by their OIDs, which templates give by name, and a template's policies put certificate policy OIDs into its certificates.
feature: pki.extended_key_usages adds custom extended key usages by their
OIDs, which templates give by name, and a template's policies put certificate
policy OIDs into its certificates.
* pki.extended_key_usages maps a name to the OID of a custom extended
key usage; a template's extended_key_usage takes the built-in and the
@@ -399,7 +744,8 @@ vpn-router (0.22.0-1) unstable; urgency=medium
vpn-router (0.21.5-1) unstable; urgency=medium
fix: /status/full of the status web service gives an authenticated client the whole status, whatever web.status_content says.
fix: /status/full of the status web service gives an authenticated client the
whole status, whatever web.status_content says.
* /status/full takes the token of /config and answers the status
document as web.status_content full shows it.
@@ -408,7 +754,8 @@ vpn-router (0.21.5-1) unstable; urgency=medium
vpn-router (0.21.4-1) unstable; urgency=medium
fix: vpn-router-setup, vpn-router-ca and /ca read a CA's key and CRL from the directory each as itself, instead of failing on a CA with a CRL.
fix: vpn-router-setup, vpn-router-ca and /ca read a CA's key and CRL from the
directory each as itself, instead of failing on a CA with a CRL.
* A CA's CRL was read as its key and its key as its CRL, so every
operation on a CA with a CRL failed with "Could not deserialize key
@@ -418,7 +765,10 @@ vpn-router (0.21.4-1) unstable; urgency=medium
vpn-router (0.21.3-1) unstable; urgency=medium
fix: the PKI is kept in ou=pki of the router's directory instead of files under /etc/vpn-router/pki, and the router's site-to-site certificate names only the identities its connections present, and /ca/bundle gives the certificate bundle of a CA.
fix: the PKI is kept in ou=pki of the router's directory instead of files
under /etc/vpn-router/pki, and the router's site-to-site certificate names
only the identities its connections present, and /ca/bundle gives the
certificate bundle of a CA.
* The CAs with their certificates, keys, CSRs and CRLs, and every
certificate issued with its revocation, are entries of the
@@ -442,7 +792,8 @@ vpn-router (0.21.3-1) unstable; urgency=medium
vpn-router (0.21.2-1) unstable; urgency=medium
fix: pki.root_key_warning silences the warning of vpn-router-ca status and /ca about the root key on the router.
fix: pki.root_key_warning silences the warning of vpn-router-ca status and /ca
about the root key on the router.
* pki.root_key_warning, true by default, set to false silences the
warning that the root key is on the router.
@@ -451,7 +802,8 @@ vpn-router (0.21.2-1) unstable; urgency=medium
vpn-router (0.21.1-1) unstable; urgency=medium
fix: vpn-router-ca status and /ca warn about a CRL the router signs only once vpn-router-pki.timer has not renewed it in time.
fix: vpn-router-ca status and /ca warn about a CRL the router signs only once
vpn-router-pki.timer has not renewed it in time.
* A CRL whose CA key is on the router is reported once past half of its
validity, when the timer should have signed it again; a root CRL
@@ -461,7 +813,10 @@ vpn-router (0.21.1-1) unstable; urgency=medium
vpn-router (0.21.0-1) unstable; urgency=medium
feature: the router has a two-level CA with an optional offline root, issuing CAs for P2S, site-to-site and general use, templates, published certificates and CRLs and an ACME server, site-to-site connections authenticate with certificates, and setup installs Caddy from Ubuntu or Caddy's repository.
feature: the router has a two-level CA with an optional offline root, issuing
CAs for P2S, site-to-site and general use, templates, published certificates
and CRLs and an ACME server, site-to-site connections authenticate with
certificates, and setup installs Caddy from Ubuntu or Caddy's repository.
* The root CA signs the issuing CAs p2s, which issues the router's VPN
server certificate and the users' certificates, s2s, which issues
@@ -500,7 +855,9 @@ vpn-router (0.21.0-1) unstable; urgency=medium
vpn-router (0.20.0-1) unstable; urgency=medium
feature: the router makes scheduled backups to a local folder, copied to the cloud with rclone, or to a Proxmox Backup Server, and vpn-router-backup restores them.
feature: the router makes scheduled backups to a local folder, copied to the
cloud with rclone, or to a Proxmox Backup Server, and vpn-router-backup
restores them.
* The backup section of vpn-router.json switches on backups and sets
the schedule (hourly, daily or weekly), the target (local or pbs),
@@ -533,7 +890,8 @@ vpn-router (0.20.0-1) unstable; urgency=medium
vpn-router (0.19.1-3) unstable; urgency=medium
update: ADR-24 names Ubuntu's apt-daily timer, not a cron job, as what updates the package lists.
update: ADR-24 names Ubuntu's apt-daily timer, not a cron job, as what updates
the package lists.
* ADR-24 says that Ubuntu's apt-daily timer updates the package lists
twice a day, and that apt-daily or unattended-upgrades may hold the
@@ -552,7 +910,8 @@ vpn-router (0.19.1-2) unstable; urgency=medium
vpn-router (0.19.1-1) unstable; urgency=medium
fix: /metrics/history gives the availability of each site-to-site connection as the percentage of time it was up in slots of the view.
fix: /metrics/history gives the availability of each site-to-site connection
as the percentage of time it was up in slots of the view.
* site_to_site_up of /metrics/history divides the view into slots of 5
minutes for an hour, 1 hour for a day, 1 day for a month and 1 month
@@ -563,7 +922,8 @@ vpn-router (0.19.1-1) unstable; urgency=medium
vpn-router (0.19.0-1) unstable; urgency=medium
feature: service accounts have a display name and a description, which the router sets for its internal ones.
feature: service accounts have a display name and a description, which the
router sets for its internal ones.
* vpnRouterAccount of the package's schema also takes displayName and
description; vpn-router-setup updates the schema of an existing
@@ -581,7 +941,9 @@ vpn-router (0.19.0-1) unstable; urgency=medium
vpn-router (0.18.3-1) unstable; urgency=medium
fix: internal service accounts are the router's own, of which the metrics service's is the one at present, and the documentation says what a service account is.
fix: internal service accounts are the router's own, of which the metrics
service's is the one at present, and the documentation says what a service
account is.
* /service-accounts, /groups and the internal field of the answers
take every internal service account, the accounts the router creates
@@ -596,7 +958,8 @@ vpn-router (0.18.3-1) unstable; urgency=medium
vpn-router (0.18.2-1) unstable; urgency=medium
fix: service accounts are managed at /service-accounts, not /users, and the internal one of the metrics service is only listed.
fix: service accounts are managed at /service-accounts, not /users, and the
internal one of the metrics service is only listed.
* /service-accounts, for members of admins and admin tokens, lists the
service accounts, adds one with a password and an optional e-mail
@@ -613,7 +976,9 @@ vpn-router (0.18.2-1) unstable; urgency=medium
vpn-router (0.18.1-1) unstable; urgency=medium
fix: vpn-router-setup owns only the creation of the metrics service's account, which is marked internal, and administrators add service accounts with vpn-router-user add --service and /users.
fix: vpn-router-setup owns only the creation of the metrics service's account,
which is marked internal, and administrators add service accounts with
vpn-router-user add --service and /users.
* POST /users takes "service": true and adds a service account, which
cannot sign in to /auth; vpn-router-user add takes --service.
@@ -637,7 +1002,8 @@ vpn-router (0.18.0-2) unstable; urgency=medium
vpn-router (0.18.0-1) unstable; urgency=medium
feature: site_to_site holds any number of named site-to-site connections, which /site-to-site lists, adds, changes and removes.
feature: site_to_site holds any number of named site-to-site connections,
which /site-to-site lists, adds, changes and removes.
* site_to_site maps a connection name to addrs, id, cidrs, local_cidrs,
local_id_mode, psk and psk_file; it replaces remote and
@@ -677,7 +1043,8 @@ vpn-router (0.18.0-1) unstable; urgency=medium
vpn-router (0.17.0-1) unstable; urgency=medium
feature: /config shows the router's PostgreSQL clusters and their settings, and /postgresql their databases with their sizes.
feature: /config shows the router's PostgreSQL clusters and their settings,
and /postgresql their databases with their sizes.
* /config has a postgresql section: each cluster of pg_lsclusters with
its version, name, port, status and data directory, and the main
@@ -689,7 +1056,8 @@ vpn-router (0.17.0-1) unstable; urgency=medium
vpn-router (0.16.0-1) unstable; urgency=medium
feature: the router keeps its metrics in PostgreSQL, serves them for an OpenTelemetry Collector and pushes them over OTLP.
feature: the router keeps its metrics in PostgreSQL, serves them for an
OpenTelemetry Collector and pushes them over OTLP.
* vpn-router-metrics takes a sample every minute: CPU, memory, free
space of /, bytes per second on the external and the internal
@@ -741,7 +1109,8 @@ vpn-router (0.14.1-1) unstable; urgency=medium
vpn-router (0.14.0-1) unstable; urgency=medium
feature: the status page shows the Client IP, and web.status_content sets how much it shows.
feature: the status page shows the Client IP, and web.status_content sets how
much it shows.
* / and /status show the Client IP, the address the request came from:
the last X-Forwarded-For entry for a request from a trusted proxy, so
@@ -756,7 +1125,8 @@ vpn-router (0.14.0-1) unstable; urgency=medium
vpn-router (0.13.0-1) unstable; urgency=medium
feature: Caddy serves the status page and the Router Manager over HTTPS with a certificate from Let's Encrypt.
feature: Caddy serves the status page and the Router Manager over HTTPS with a
certificate from Let's Encrypt.
* The package depends on caddy. While https.enabled is true, the
default, and wan.local_fqdn is set, vpn-router-setup writes
@@ -779,7 +1149,8 @@ vpn-router (0.13.0-1) unstable; urgency=medium
vpn-router (0.12.3-1) unstable; urgency=medium
fix: the web service listens on the internal address, the external address or both.
fix: the web service listens on the internal address, the external address or
both.
* web.listen in vpn-router.json sets the interfaces the web service
listens on: internal, external or both, the default. It is read
@@ -865,7 +1236,8 @@ vpn-router (0.9.2-1) unstable; urgency=medium
vpn-router (0.9.1-1) unstable; urgency=medium
fix: the web service waits for the PostgreSQL driver instead of failing at start.
fix: the web service waits for the PostgreSQL driver instead of failing at
start.
* The web service imports psycopg as the first step of its
initialization, retrying every 15 seconds and logging the failure while
@@ -895,7 +1267,8 @@ vpn-router (0.9.0-1) unstable; urgency=medium
vpn-router (0.8.0-1) unstable; urgency=medium
feature: the web service creates its key and database itself and shows its starting phase.
feature: the web service creates its key and database itself and shows its
starting phase.
* vpn-router-web makes its signing key and creates the refresh tokens'
database on its first run, retrying every 15 seconds until PostgreSQL
@@ -910,7 +1283,8 @@ vpn-router (0.8.0-1) unstable; urgency=medium
vpn-router (0.7.1-1) unstable; urgency=medium
fix: a failing user directory or token database no longer leaves the tunnels down.
fix: a failing user directory or token database no longer leaves the tunnels
down.
* vpn-router-setup brings up strongSwan and WireGuard before it provisions
the user directory and the token database. A failure of either no
@@ -920,7 +1294,8 @@ vpn-router (0.7.1-1) unstable; urgency=medium
vpn-router (0.7.0-1) unstable; urgency=medium
feature: users sign in to the web service, and only members of admins manage the router.
feature: users sign in to the web service, and only members of admins manage
the router.
* POST /auth signs a user in with the password, and the TOTP code for an
enrolled user, and answers a 5 minute access token and an 8 hour refresh
+68 -38
View File
@@ -2,7 +2,7 @@
set -e
. /usr/share/debconf/confmodule
# An upgrade (configure with the old version) asks nothing; vpn-router.json holds the
# An upgrade (configure with the old version) asks nothing; linux-manager.json holds the
# configuration, and dpkg-reconfigure asks the questions again.
if [ "$1" = "configure" ] && [ -n "$2" ]; then
exit 0
@@ -12,59 +12,89 @@ fi
# preseeded completes without prompting. The defaults are platform "none" and
# mode "manual", which together mean: install the files, configure nothing.
db_input high vpn-router/platform || true
# The Manager's first administrator, asked whatever the platform is; the password only with a name.
db_input medium linux-manager/admin_username || true
db_go || true
db_get vpn-router/platform
db_get linux-manager/admin_username
if [ -n "$RET" ]; then
db_input medium linux-manager/admin_password || true
db_go || true
fi
db_input high linux-manager/platform || true
db_go || true
db_get linux-manager/platform
if [ "$RET" = "none" ]; then
# Configuration is deferred. Asking anything else would collect answers
# that nothing is going to apply.
exit 0
fi
db_input high vpn-router/mode || true
db_input high linux-manager/mode || true
db_input high linux-manager/router_enabled || true
db_input high linux-manager/vpn_enabled || true
db_go || true
db_get linux-manager/router_enabled
ROUTER="$RET"
db_get linux-manager/vpn_enabled
VPN="$RET"
# The mode says how much the operator supplies, so it decides which interface
# and address questions are worth asking.
db_get vpn-router/mode
case "$RET" in
manual)
db_input high vpn-router/external_interface || true
db_input high vpn-router/internal_interface || true
db_input high vpn-router/int_addr || true
db_input high vpn-router/int_gateway_ip || true
;;
interfaces)
db_input high vpn-router/external_interface || true
db_input high vpn-router/internal_interface || true
;;
auto)
# and address questions are worth asking; without the router the server has
# only the external interface.
db_get linux-manager/mode
MODE="$RET"
case "$MODE" in
manual|interfaces)
db_input high linux-manager/external_interface || true
;;
esac
if [ "$ROUTER" = "true" ]; then
case "$MODE" in
manual)
db_input high linux-manager/internal_interface || true
db_input high linux-manager/int_addr || true
db_input high linux-manager/int_gateway_ip || true
;;
interfaces)
db_input high linux-manager/internal_interface || true
;;
esac
fi
db_go || true
db_input high vpn-router/local_fqdn || true
db_input high vpn-router/local_id_mode || true
db_input high vpn-router/local_cidrs || true
db_input high vpn-router/remote_addrs || true
db_input high vpn-router/remote_id || true
db_input high vpn-router/remote_cidrs || true
db_input high vpn-router/psk || true
db_input high vpn-router/pki_root_name || true
db_input high vpn-router/p2s_enabled || true
db_input high vpn-router/wg_enabled || true
db_input high linux-manager/local_fqdn || true
if [ "$ROUTER" = "true" ] || [ "$VPN" = "true" ]; then
db_input high linux-manager/local_cidrs || true
fi
if [ "$ROUTER" = "true" ]; then
db_input high linux-manager/local_id_mode || true
db_input high linux-manager/remote_addrs || true
db_input high linux-manager/remote_id || true
db_input high linux-manager/remote_cidrs || true
db_input high linux-manager/psk || true
fi
db_input high linux-manager/pki_root_name || true
if [ "$VPN" = "true" ]; then
db_input high linux-manager/p2s_enabled || true
db_input high linux-manager/wg_enabled || true
fi
db_go || true
db_get vpn-router/p2s_enabled
if [ "$RET" = "true" ]; then
db_input high vpn-router/p2s_address_pool || true
db_go || true
fi
if [ "$VPN" = "true" ]; then
db_get linux-manager/p2s_enabled
if [ "$RET" = "true" ]; then
db_input high linux-manager/p2s_address_pool || true
db_go || true
fi
db_get vpn-router/wg_enabled
if [ "$RET" = "true" ]; then
db_input high vpn-router/wg_address || true
db_input high vpn-router/wg_listen_port || true
db_go || true
db_get linux-manager/wg_enabled
if [ "$RET" = "true" ]; then
db_input high linux-manager/wg_address || true
db_input high linux-manager/wg_listen_port || true
db_go || true
fi
fi
+35 -27
View File
@@ -1,4 +1,4 @@
Source: vpn-router
Source: linux-manager
Section: net
Priority: optional
Maintainer: Sławomir Koszewski <slawek@koszewscy.waw.pl>
@@ -6,7 +6,7 @@ Build-Depends: debhelper-compat (= 13)
Standards-Version: 4.6.2
Rules-Requires-Root: no
Package: vpn-router
Package: linux-manager
Architecture: all
Depends: ${misc:Depends},
strongswan-swanctl (>= 5.0),
@@ -28,6 +28,7 @@ Depends: ${misc:Depends},
python3-psycopg,
python3-psutil,
python3-prometheus-client,
python3-requests,
postgresql,
slapd,
nslcd,
@@ -46,31 +47,38 @@ Suggests: caddy,
pdns-server,
pdns-backend-pgsql,
proxmox-backup-client-static
Description: Linux VPN router with IPSec, IKEv2, OpenVPN and WireGuard support
Configures a Linux host or virtual machine as a VPN router: site-to-site
IKEv2 IPSec (strongSwan swanctl) with pre-shared keys or certificates,
optional IKEv2 access using EAP-TLS, optional OpenVPN with a
certificate and a password per user, kept in a local LDAP directory, optional
WireGuard, and the routing, NAT, MSS clamping and firewall rules a router
needs. Its two-level CA, whose root may be kept offline, has issuing CAs for
the VPN clients, for site-to-site and for general use, and publishes their
certificates and CRLs. Caddy, installed by the package from the Ubuntu
archive or from Caddy's repository, serves HTTPS and static web sites, whose
files are uploaded through the web service or WebDAV, an optional PowerDNS
name server answers for a zone of the sites, and an optional OpenID
Provider signs the directory's users in to other systems. It keeps metrics of the
router in PostgreSQL and serves them for an OpenTelemetry Collector, and
makes scheduled backups to a local folder, copied to the cloud with rclone,
or to a Proxmox Backup Server. Nothing in the package is specific to a cloud
provider.
Description: Ubuntu Linux Manager: directory, PKI, web, optional router and VPN
Configures a Linux host or virtual machine as a managed server: a local LDAP
directory of users, groups and service accounts, and the firewall rules of
its services. Two components are switched on in the configuration. The
router routes between two interfaces and runs site-to-site IKEv2 IPSec
(strongSwan swanctl) with pre-shared keys or certificates. The VPN offers
IKEv2 access using EAP-TLS, OpenVPN with a certificate and a password per
user, and WireGuard; without the router its clients reach the server's
networks through NAT on one interface. Its two-level CA, whose root may be
kept offline, has issuing CAs for the VPN clients, for site-to-site and for
general use, and publishes their certificates and CRLs. Caddy, installed by
the package from the Ubuntu archive or from Caddy's repository, serves HTTPS
and static web sites, whose files are uploaded through the web service or
WebDAV, an optional PowerDNS name server answers for a zone of the sites,
and an optional OpenID Provider signs the directory's users in to other
systems. It keeps metrics of the server in PostgreSQL and serves them for
an OpenTelemetry Collector, and makes scheduled backups to a local folder,
copied to the cloud with rclone, or to a Proxmox Backup Server. Nothing in
the package is specific to a cloud provider.
.
Settings live in /etc/vpn-router/vpn-router.json, which is created on first
install and owned by the administrator afterwards. Editing it and restarting
vpn-router-setup applies the change; the same service reconciles the system
at every boot. It is built on two inputs, the external and internal interface
names; the addresses on them are read from the system rather than configured
again, and nothing is auto-detected.
The Manager, a web application served by the server, shows the
server's state and manages its users, connections, certificates, web sites,
name server, backups and packages. Users sign in through the server's web
service; their sessions are kept in PostgreSQL.
.
Settings live in /etc/linux-manager/linux-manager.json, which is created on
first install and owned by the administrator afterwards. Editing it and
restarting linux-manager-setup applies the change; the same service
reconciles the system at every boot. It is built on the interface names,
the external one and, with the router, the internal one; the addresses on
them are read from the system rather than configured again.
.
Platform-specific additions are provided by modules under
/usr/lib/vpn-router, selected explicitly by the platform setting. Modules for
Azure and GCP are included.
/usr/lib/linux-manager, selected explicitly by the platform setting.
Modules for Azure and GCP are included.
+1 -1
View File
@@ -1,5 +1,5 @@
Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Upstream-Name: vpn-router
Upstream-Name: linux-manager
Upstream-Contact: Sławomir Koszewski <slawek@koszewscy.waw.pl>
Files: *
+6 -5
View File
@@ -1,5 +1,6 @@
etc/vpn-router
usr/lib/vpn-router
usr/lib/vpn-router/vpnrouter_platforms
usr/share/vpn-router/templates
var/lib/vpn-router/sites
etc/linux-manager
usr/lib/linux-manager
usr/lib/linux-manager/linuxmanager_platforms
usr/share/linux-manager/templates
usr/share/linux-manager/manager/web
var/lib/linux-manager/sites
+49 -40
View File
@@ -1,41 +1,50 @@
src/etc/sysctl.d/99-vpn-router.conf etc/sysctl.d/
src/usr/lib/vpn-router/vpnrouter.py usr/lib/vpn-router/
src/usr/lib/vpn-router/vpnrouter_ldap.py usr/lib/vpn-router/
src/usr/lib/vpn-router/vpnrouter_net.py usr/lib/vpn-router/
src/usr/lib/vpn-router/vpnrouter_serve.py usr/lib/vpn-router/
src/usr/lib/vpn-router/vpnrouter_users.py usr/lib/vpn-router/
src/usr/lib/vpn-router/vpnrouter_passkeys.py usr/lib/vpn-router/
src/usr/lib/vpn-router/vpnrouter_auth.py usr/lib/vpn-router/
src/usr/lib/vpn-router/vpnrouter_oidc.py usr/lib/vpn-router/
src/usr/lib/vpn-router/vpnrouter_metrics.py usr/lib/vpn-router/
src/usr/lib/vpn-router/vpnrouter_status.py usr/lib/vpn-router/
src/usr/lib/vpn-router/vpnrouter_backup.py usr/lib/vpn-router/
src/usr/lib/vpn-router/vpnrouter_pki.py usr/lib/vpn-router/
src/usr/lib/vpn-router/vpnrouter_sites.py usr/lib/vpn-router/
src/usr/lib/vpn-router/vpnrouter_dns.py usr/lib/vpn-router/
src/usr/lib/vpn-router/install-packages usr/lib/vpn-router/
src/usr/lib/vpn-router/webdav-auth usr/lib/vpn-router/
src/usr/lib/vpn-router/openvpn-same-user usr/lib/vpn-router/
src/usr/lib/vpn-router/openvpn-issuer usr/lib/vpn-router/
src/usr/sbin/vpn-router-user usr/sbin/
src/usr/sbin/vpn-router-group usr/sbin/
src/usr/sbin/vpn-router-token usr/sbin/
src/usr/sbin/vpn-config-edit usr/sbin/
src/usr/sbin/vpn-router-backup usr/sbin/
src/usr/sbin/vpn-router-ca usr/sbin/
src/usr/sbin/vpn-router-caddy usr/sbin/
src/usr/sbin/vpn-router-site usr/sbin/
src/usr/lib/vpn-router/configure usr/lib/vpn-router/
src/usr/lib/vpn-router/docker-rules usr/lib/vpn-router/
src/usr/lib/vpn-router/setup usr/lib/vpn-router/
src/usr/lib/vpn-router/web usr/lib/vpn-router/
src/usr/lib/vpn-router/metrics usr/lib/vpn-router/
src/usr/lib/vpn-router/bootstrap usr/lib/vpn-router/
src/usr/lib/vpn-router/wait-for-libpq usr/lib/vpn-router/
src/usr/lib/vpn-router/generate-config usr/lib/vpn-router/
src/usr/lib/vpn-router/vpnrouter_platforms/*.py usr/lib/vpn-router/vpnrouter_platforms/
src/usr/share/vpn-router/templates/* usr/share/vpn-router/templates/
src/usr/share/vpn-router/sql/* usr/share/vpn-router/sql/
src/usr/lib/linux-manager/linuxmanager.py usr/lib/linux-manager/
src/usr/lib/linux-manager/linuxmanager_ldap.py usr/lib/linux-manager/
src/usr/lib/linux-manager/linuxmanager_net.py usr/lib/linux-manager/
src/usr/lib/linux-manager/linuxmanager_serve.py usr/lib/linux-manager/
src/usr/lib/linux-manager/linuxmanager_users.py usr/lib/linux-manager/
src/usr/lib/linux-manager/linuxmanager_passkeys.py usr/lib/linux-manager/
src/usr/lib/linux-manager/linuxmanager_auth.py usr/lib/linux-manager/
src/usr/lib/linux-manager/linuxmanager_oidc.py usr/lib/linux-manager/
src/usr/lib/linux-manager/linuxmanager_metrics.py usr/lib/linux-manager/
src/usr/lib/linux-manager/linuxmanager_status.py usr/lib/linux-manager/
src/usr/lib/linux-manager/linuxmanager_backup.py usr/lib/linux-manager/
src/usr/lib/linux-manager/linuxmanager_pki.py usr/lib/linux-manager/
src/usr/lib/linux-manager/linuxmanager_sites.py usr/lib/linux-manager/
src/usr/lib/linux-manager/linuxmanager_dns.py usr/lib/linux-manager/
src/usr/lib/linux-manager/install-packages usr/lib/linux-manager/
src/usr/lib/linux-manager/webdav-auth usr/lib/linux-manager/
src/usr/lib/linux-manager/openvpn-same-user usr/lib/linux-manager/
src/usr/lib/linux-manager/openvpn-issuer usr/lib/linux-manager/
src/usr/lib/linux-manager/editor-file usr/lib/linux-manager/
src/usr/bin/linux-manager-editor usr/bin/
src/usr/sbin/linux-manager-user usr/sbin/
src/usr/sbin/linux-manager-group usr/sbin/
src/usr/sbin/linux-manager-role usr/sbin/
src/usr/sbin/linux-manager-token usr/sbin/
src/usr/sbin/linux-manager-config-edit usr/sbin/
src/usr/sbin/linux-manager-backup usr/sbin/
src/usr/sbin/linux-manager-ca usr/sbin/
src/usr/sbin/linux-manager-caddy usr/sbin/
src/usr/sbin/linux-manager-site usr/sbin/
src/usr/lib/linux-manager/configure usr/lib/linux-manager/
src/usr/lib/linux-manager/docker-rules usr/lib/linux-manager/
src/usr/lib/linux-manager/setup usr/lib/linux-manager/
src/usr/lib/linux-manager/web usr/lib/linux-manager/
src/usr/lib/linux-manager/linuxmanager_web/*.py usr/lib/linux-manager/linuxmanager_web/
src/usr/lib/linux-manager/metrics usr/lib/linux-manager/
src/usr/lib/linux-manager/bootstrap usr/lib/linux-manager/
src/usr/lib/linux-manager/wait-for-libpq usr/lib/linux-manager/
src/usr/lib/linux-manager/generate-config usr/lib/linux-manager/
src/usr/lib/linux-manager/manager usr/lib/linux-manager/
src/usr/lib/linux-manager/linuxmanager_ui.py usr/lib/linux-manager/
src/usr/lib/linux-manager/upgrade usr/lib/linux-manager/
src/usr/lib/linux-manager/linuxmanager_platforms/*.py usr/lib/linux-manager/linuxmanager_platforms/
src/usr/share/linux-manager/templates/* usr/share/linux-manager/templates/
src/usr/share/linux-manager/sql/* usr/share/linux-manager/sql/
src/usr/share/bash-completion/completions/*.bash usr/share/bash-completion/completions/
src/usr/lib/systemd/system/caddy.service.d/vpn-router.conf usr/lib/systemd/system/caddy.service.d/
src/usr/share/doc/vpn-router/vpn-router.json.example usr/share/doc/vpn-router/
src/usr/lib/systemd/system/caddy.service.d/linux-manager.conf usr/lib/systemd/system/caddy.service.d/
src/usr/share/doc/linux-manager/linux-manager.json.example usr/share/doc/linux-manager/
src/usr/share/doc/linux-manager/manager.json.example usr/share/doc/linux-manager/
web/dist/* usr/share/linux-manager/manager/web/
docs/* usr/share/linux-manager/manager/docs/
+1
View File
@@ -0,0 +1 @@
usr/bin/linux-manager-editor usr/bin/lme
@@ -1,11 +1,11 @@
[Unit]
Description=VPN Router backup
Documentation=file:/usr/share/doc/vpn-router/README.md
Description=Linux Manager backup
Documentation=file:/usr/share/doc/linux-manager/README.md
After=network-online.target slapd.service postgresql.service
Wants=network-online.target
[Service]
Type=oneshot
ExecStart=/usr/sbin/vpn-router-backup run
ExecStart=/usr/sbin/linux-manager-backup run
Nice=10
IOSchedulingClass=idle
@@ -0,0 +1,10 @@
[Unit]
Description=Linux Manager backup schedule
Documentation=file:/usr/share/doc/linux-manager/README.md
[Timer]
OnCalendar=daily
Persistent=true
[Install]
WantedBy=timers.target
@@ -0,0 +1,14 @@
[Unit]
Description=Linux Manager bootstrap diagnostics page
Documentation=file:/usr/share/doc/linux-manager/README.md
# Never enabled: linux-manager-setup starts it until a setup has succeeded, and stops it after that.
[Service]
Type=simple
ExecStart=/usr/bin/python3 /usr/lib/linux-manager/bootstrap
Restart=on-failure
RestartSec=5
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
@@ -0,0 +1,10 @@
[Unit]
Description=Linux Manager Caddy installation
Documentation=file:/usr/share/doc/linux-manager/README.md
After=network-online.target
Wants=network-online.target
[Service]
Type=oneshot
ExecStart=/usr/sbin/linux-manager-caddy apply
TimeoutStartSec=20min
@@ -0,0 +1,14 @@
[Unit]
Description=Linux Manager rules for Docker published ports
Documentation=file:/usr/share/doc/linux-manager/README.md
After=docker.service
PartOf=docker.service
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/lib/linux-manager/docker-rules apply
ExecStop=/usr/lib/linux-manager/docker-rules remove
[Install]
WantedBy=docker.service
@@ -0,0 +1,10 @@
[Unit]
Description=Linux Manager installation of the packages of %i
Documentation=file:/usr/share/doc/linux-manager/README.md
After=network-online.target
Wants=network-online.target
[Service]
Type=oneshot
ExecStart=/usr/lib/linux-manager/install-packages %i
TimeoutStartSec=20min
@@ -0,0 +1,22 @@
[Unit]
Description=Linux Manager metrics service
Documentation=file:/usr/share/doc/linux-manager/README.md
After=network-online.target postgresql.service linux-manager-web.service
Wants=network-online.target
[Service]
Type=simple
ExecStartPre=/usr/lib/linux-manager/wait-for-libpq
ExecStart=/usr/bin/python3 /usr/lib/linux-manager/metrics
TimeoutStartSec=5min
Restart=on-failure
RestartSec=15
NoNewPrivileges=yes
ProtectSystem=strict
RuntimeDirectory=linux-manager
RuntimeDirectoryPreserve=yes
ProtectHome=yes
PrivateTmp=yes
[Install]
WantedBy=multi-user.target
@@ -0,0 +1,7 @@
[Unit]
Description=Linux Manager PKI maintenance
Documentation=file:/usr/share/doc/linux-manager/README.md
[Service]
Type=oneshot
ExecStart=/usr/sbin/linux-manager-ca maintain
@@ -0,0 +1,10 @@
[Unit]
Description=Linux Manager PKI maintenance schedule
Documentation=file:/usr/share/doc/linux-manager/README.md
[Timer]
OnCalendar=daily
Persistent=true
[Install]
WantedBy=timers.target
@@ -1,13 +1,13 @@
[Unit]
Description=VPN Router DNS resolver of the remote access clients
Documentation=file:/usr/share/doc/vpn-router/README.md
Description=Linux Manager DNS resolver of the remote access clients
Documentation=file:/usr/share/doc/linux-manager/README.md
After=network-online.target systemd-resolved.service
Wants=network-online.target
ConditionPathExists=/etc/vpn-router/dnsmasq.conf
ConditionPathExists=/etc/linux-manager/dnsmasq.conf
[Service]
Type=simple
ExecStart=/usr/sbin/dnsmasq --keep-in-foreground --pid-file= --log-facility=- --conf-file=/etc/vpn-router/dnsmasq.conf
ExecStart=/usr/sbin/dnsmasq --keep-in-foreground --pid-file= --log-facility=- --conf-file=/etc/linux-manager/dnsmasq.conf
Restart=on-failure
RestartSec=5
@@ -1,13 +1,13 @@
[Unit]
Description=VPN Router Setup
Documentation=file:/usr/share/doc/vpn-router/README.md
Description=Linux Manager Setup
Documentation=file:/usr/share/doc/linux-manager/README.md
Wants=network-online.target postgresql.service
After=network-online.target docker.service postgresql.service
[Service]
Type=oneshot
ExecStartPre=/usr/lib/vpn-router/wait-for-libpq
ExecStart=/usr/lib/vpn-router/setup
ExecStartPre=/usr/lib/linux-manager/wait-for-libpq
ExecStart=/usr/lib/linux-manager/setup
RemainAfterExit=yes
[Install]
@@ -0,0 +1,21 @@
[Unit]
Description=Ubuntu Linux Manager web application
Documentation=file:/usr/share/doc/linux-manager/README.md
After=network-online.target postgresql.service
Wants=network-online.target
[Service]
Type=simple
ExecStartPre=/usr/lib/linux-manager/wait-for-libpq
ExecStart=/usr/bin/python3 /usr/lib/linux-manager/manager
TimeoutStartSec=5min
Restart=on-failure
RestartSec=15
NoNewPrivileges=yes
ProtectSystem=strict
ReadWritePaths=-/etc/linux-manager
ProtectHome=yes
PrivateTmp=yes
[Install]
WantedBy=multi-user.target
@@ -0,0 +1,23 @@
[Unit]
Description=Linux Manager status web service
Documentation=file:/usr/share/doc/linux-manager/README.md
After=network-online.target postgresql.service
Wants=network-online.target
[Service]
Type=simple
ExecStartPre=/usr/lib/linux-manager/wait-for-libpq
ExecStart=/usr/bin/python3 /usr/lib/linux-manager/web
TimeoutStartSec=5min
Restart=on-failure
RestartSec=15
NoNewPrivileges=yes
ProtectSystem=strict
RuntimeDirectory=linux-manager
RuntimeDirectoryPreserve=yes
ReadWritePaths=-/etc/linux-manager -/etc/swanctl/x509crl -/etc/openvpn/server -/var/lib/linux-manager/pki-publish -/var/lib/linux-manager/sites
ProtectHome=yes
PrivateTmp=yes
[Install]
WantedBy=multi-user.target
@@ -0,0 +1,21 @@
[Unit]
Description=Linux Manager WebDAV of the web sites' folders
Documentation=file:/usr/share/doc/linux-manager/README.md
After=network-online.target slapd.service postgresql.service
Wants=network-online.target
[Service]
Type=simple
ExecStart=/usr/bin/rclone serve webdav --addr 127.0.0.1:7446 --auth-proxy /usr/lib/linux-manager/webdav-auth --config /dev/null --cache-dir /var/cache/linux-manager-webdav --vfs-cache-mode writes --umask 022
Restart=on-failure
RestartSec=15
NoNewPrivileges=yes
ProtectSystem=strict
ReadWritePaths=/var/lib/linux-manager/sites
CacheDirectory=linux-manager-webdav
RuntimeDirectory=linux-manager-webdav
ProtectHome=yes
PrivateTmp=yes
[Install]
WantedBy=multi-user.target
+59 -43
View File
@@ -16,67 +16,83 @@ answer() {
case "$1" in
configure)
# --- Read the installer's answers ---
answer vpn-router/platform; VPN_ROUTER_PLATFORM="$ANSWER"
answer vpn-router/mode; VPN_ROUTER_MODE="$ANSWER"
answer vpn-router/int_addr; VPN_ROUTER_INT_ADDR="$ANSWER"
answer vpn-router/external_interface; VPN_ROUTER_EXTERNAL_INTERFACE="$ANSWER"
answer vpn-router/internal_interface; VPN_ROUTER_INTERNAL_INTERFACE="$ANSWER"
answer vpn-router/local_fqdn; VPN_ROUTER_LOCAL_FQDN="$ANSWER"
answer vpn-router/local_id_mode; VPN_ROUTER_LOCAL_ID_MODE="$ANSWER"
answer vpn-router/local_cidrs; VPN_ROUTER_LOCAL_CIDRS="$ANSWER"
answer vpn-router/int_gateway_ip; VPN_ROUTER_INT_GATEWAY_IP="$ANSWER"
answer vpn-router/remote_addrs; VPN_ROUTER_REMOTE_ADDRS="$ANSWER"
answer vpn-router/remote_id; VPN_ROUTER_REMOTE_ID="$ANSWER"
answer vpn-router/remote_cidrs; VPN_ROUTER_REMOTE_CIDRS="$ANSWER"
answer vpn-router/psk; VPN_ROUTER_PSK="$ANSWER"
answer vpn-router/p2s_enabled; VPN_ROUTER_P2S_ENABLED="$ANSWER"
answer vpn-router/p2s_address_pool; VPN_ROUTER_P2S_ADDRESS_POOL="$ANSWER"
answer vpn-router/pki_root_name; VPN_ROUTER_PKI_ROOT_NAME="$ANSWER"
answer vpn-router/wg_enabled; VPN_ROUTER_WG_ENABLED="$ANSWER"
answer vpn-router/wg_address; VPN_ROUTER_WG_ADDRESS="$ANSWER"
answer vpn-router/wg_listen_port; VPN_ROUTER_WG_LISTEN_PORT="$ANSWER"
answer linux-manager/platform; LINUX_MANAGER_PLATFORM="$ANSWER"
answer linux-manager/mode; LINUX_MANAGER_MODE="$ANSWER"
answer linux-manager/router_enabled; LINUX_MANAGER_ROUTER_ENABLED="$ANSWER"
answer linux-manager/vpn_enabled; LINUX_MANAGER_VPN_ENABLED="$ANSWER"
answer linux-manager/int_addr; LINUX_MANAGER_INT_ADDR="$ANSWER"
answer linux-manager/external_interface; LINUX_MANAGER_EXTERNAL_INTERFACE="$ANSWER"
answer linux-manager/internal_interface; LINUX_MANAGER_INTERNAL_INTERFACE="$ANSWER"
answer linux-manager/local_fqdn; LINUX_MANAGER_LOCAL_FQDN="$ANSWER"
answer linux-manager/local_id_mode; LINUX_MANAGER_LOCAL_ID_MODE="$ANSWER"
answer linux-manager/local_cidrs; LINUX_MANAGER_LOCAL_CIDRS="$ANSWER"
answer linux-manager/int_gateway_ip; LINUX_MANAGER_INT_GATEWAY_IP="$ANSWER"
answer linux-manager/remote_addrs; LINUX_MANAGER_REMOTE_ADDRS="$ANSWER"
answer linux-manager/remote_id; LINUX_MANAGER_REMOTE_ID="$ANSWER"
answer linux-manager/remote_cidrs; LINUX_MANAGER_REMOTE_CIDRS="$ANSWER"
answer linux-manager/psk; LINUX_MANAGER_PSK="$ANSWER"
answer linux-manager/p2s_enabled; LINUX_MANAGER_P2S_ENABLED="$ANSWER"
answer linux-manager/p2s_address_pool; LINUX_MANAGER_P2S_ADDRESS_POOL="$ANSWER"
answer linux-manager/pki_root_name; LINUX_MANAGER_PKI_ROOT_NAME="$ANSWER"
answer linux-manager/wg_enabled; LINUX_MANAGER_WG_ENABLED="$ANSWER"
answer linux-manager/wg_address; LINUX_MANAGER_WG_ADDRESS="$ANSWER"
answer linux-manager/wg_listen_port; LINUX_MANAGER_WG_LISTEN_PORT="$ANSWER"
export VPN_ROUTER_PLATFORM VPN_ROUTER_MODE VPN_ROUTER_INT_ADDR \
VPN_ROUTER_EXTERNAL_INTERFACE \
VPN_ROUTER_INTERNAL_INTERFACE VPN_ROUTER_LOCAL_FQDN \
VPN_ROUTER_LOCAL_ID_MODE VPN_ROUTER_LOCAL_CIDRS \
VPN_ROUTER_INT_GATEWAY_IP \
VPN_ROUTER_REMOTE_ADDRS VPN_ROUTER_REMOTE_ID \
VPN_ROUTER_REMOTE_CIDRS VPN_ROUTER_PSK \
VPN_ROUTER_P2S_ENABLED VPN_ROUTER_P2S_ADDRESS_POOL \
VPN_ROUTER_PKI_ROOT_NAME VPN_ROUTER_WG_ENABLED \
VPN_ROUTER_WG_ADDRESS VPN_ROUTER_WG_LISTEN_PORT
export LINUX_MANAGER_PLATFORM LINUX_MANAGER_MODE LINUX_MANAGER_INT_ADDR \
LINUX_MANAGER_ROUTER_ENABLED LINUX_MANAGER_VPN_ENABLED \
LINUX_MANAGER_EXTERNAL_INTERFACE \
LINUX_MANAGER_INTERNAL_INTERFACE LINUX_MANAGER_LOCAL_FQDN \
LINUX_MANAGER_LOCAL_ID_MODE LINUX_MANAGER_LOCAL_CIDRS \
LINUX_MANAGER_INT_GATEWAY_IP \
LINUX_MANAGER_REMOTE_ADDRS LINUX_MANAGER_REMOTE_ID \
LINUX_MANAGER_REMOTE_CIDRS LINUX_MANAGER_PSK \
LINUX_MANAGER_P2S_ENABLED LINUX_MANAGER_P2S_ADDRESS_POOL \
LINUX_MANAGER_PKI_ROOT_NAME LINUX_MANAGER_WG_ENABLED \
LINUX_MANAGER_WG_ADDRESS LINUX_MANAGER_WG_LISTEN_PORT
# --- Hand a candidate configuration to ucf ---
# ucf compares it against the file in /etc and decides what to do about
# local changes, prompting through debconf only for a real conflict.
# This is what makes dpkg-reconfigure apply without destroying edits.
CANDIDATE="$(mktemp)"
/usr/lib/vpn-router/generate-config "$CANDIDATE"
ucf --three-way --debconf-ok "$CANDIDATE" /etc/vpn-router/vpn-router.json
ucfr vpn-router /etc/vpn-router/vpn-router.json
chmod 0600 /etc/vpn-router/vpn-router.json
/usr/lib/linux-manager/generate-config "$CANDIDATE"
ucf --three-way --debconf-ok "$CANDIDATE" /etc/linux-manager/linux-manager.json
ucfr linux-manager /etc/linux-manager/linux-manager.json
chmod 0600 /etc/linux-manager/linux-manager.json
rm -f "$CANDIDATE"
# The key now lives in the configuration file; do not keep a copy. An
# empty answer means "leave alone" next time, so clearing it here does
# not blank the key on the next dpkg-reconfigure.
db_set vpn-router/psk ""
db_set linux-manager/psk ""
# Apply the sysctl drop-in shipped by this package so it takes effect
# without waiting for a reboot.
sysctl --system >/dev/null
# The Manager's administrator account for linux-manager-setup to create, when a name and a
# password were given; the answers are cleared so that a later run does not ask for it again.
db_get linux-manager/admin_username
ADMIN_USERNAME="$RET"
db_get linux-manager/admin_password
ADMIN_PASSWORD="$RET"
if [ -n "$ADMIN_USERNAME" ] && [ -n "$ADMIN_PASSWORD" ]; then
(umask 077; ADMIN_USERNAME="$ADMIN_USERNAME" ADMIN_PASSWORD="$ADMIN_PASSWORD" python3 -c \
'import json, os, sys; json.dump({"username": os.environ["ADMIN_USERNAME"], "password": os.environ["ADMIN_PASSWORD"]}, sys.stdout)' \
> /etc/linux-manager/manager-admin.json)
elif [ -n "$ADMIN_USERNAME" ]; then
echo 'linux-manager: no administrator password was given; no Ubuntu Linux Manager administrator is created' >&2
fi
db_set linux-manager/admin_username ""
db_set linux-manager/admin_password ""
# The firewall is not touched here. Enabling it, allowing SSH and
# setting the forward policy all belong to vpn-router-setup, which does
# setting the forward policy all belong to linux-manager-setup, which does
# them only once the configuration says this host is a router.
# The setup and web services wait for libpq in the linker cache, which
# dpkg refreshes after this script, so they are started without waiting
# for them.
# The setup, web and manager services wait for libpq in the linker cache,
# which dpkg refreshes after this script, so they are started without
# waiting for them.
if [ -d /run/systemd/system ]; then
systemctl --no-block restart vpn-router-setup.service >/dev/null 2>&1 || true
systemctl --no-block restart vpn-router-web.service >/dev/null 2>&1 || true
systemctl --no-block restart linux-manager-setup.service >/dev/null 2>&1 || true
systemctl --no-block restart linux-manager-web.service >/dev/null 2>&1 || true
systemctl --no-block restart linux-manager-ui.service >/dev/null 2>&1 || true
fi
;;
esac
+42 -34
View File
@@ -37,48 +37,56 @@ case "$1" in
# Let ucf forget the file before it is removed, or a reinstall finds a
# stale hash and declines to lay the file down again.
if command -v ucf >/dev/null 2>&1; then
ucf --purge /etc/vpn-router/vpn-router.json
ucf --purge /etc/linux-manager/linux-manager.json
fi
if command -v ucfr >/dev/null 2>&1; then
ucfr --purge vpn-router /etc/vpn-router/vpn-router.json
ucfr --purge linux-manager /etc/linux-manager/linux-manager.json
fi
rm -f /etc/swanctl/conf.d/site-to-site.conf \
/etc/swanctl/conf.d/road-warrior.conf \
/etc/systemd/resolved.conf.d/p2s-forwarder.conf \
/etc/vpn-router/dnsmasq.conf \
/etc/linux-manager/dnsmasq.conf \
/etc/wireguard/wg0.conf \
/etc/openvpn/server/vpn-router.conf \
/etc/openvpn/server/vpn-router-ldap.conf \
/etc/caddy/vpn-router.Caddyfile \
/etc/powerdns/pdns.d/vpn-router.conf \
/etc/vpn-router/vpn-router.json \
/etc/vpn-router/web.key \
/etc/vpn-router/oidc.key \
/etc/vpn-router/metrics.secret \
/etc/systemd/system/vpn-router-backup.timer.d/schedule.conf \
/etc/sudoers.d/vpn-router \
/etc/ssh/sshd_config.d/vpn-router.conf \
/var/lib/vpn-router/setup.done \
/var/lib/vpn-router/backup.json
rmdir --ignore-fail-on-non-empty /etc/systemd/system/vpn-router-backup.timer.d 2>/dev/null || true
# The name service no longer looks up the console accounts of the router's directory.
/etc/openvpn/server/linux-manager.conf \
/etc/openvpn/server/linux-manager-ldap.conf \
/etc/caddy/linux-manager.Caddyfile \
/etc/sysctl.d/99-linux-manager.conf \
/etc/powerdns/pdns.d/linux-manager.conf \
/etc/linux-manager/linux-manager.json \
/etc/linux-manager/web.key \
/etc/linux-manager/oidc.key \
/etc/linux-manager/metrics.secret \
/etc/linux-manager/manager.json \
/etc/linux-manager/manager-admin.json \
/var/lib/linux-manager/upgrade.json \
/var/lib/linux-manager/upgrade.log \
/var/lib/linux-manager/system-upgrade.json \
/var/lib/linux-manager/system-upgrade.log \
/etc/systemd/system/linux-manager-backup.timer.d/schedule.conf \
/etc/sudoers.d/linux-manager \
/etc/ssh/sshd_config.d/linux-manager.conf \
/var/lib/linux-manager/setup.done \
/var/lib/linux-manager/backup.json
rmdir --ignore-fail-on-non-empty /etc/systemd/system/linux-manager-backup.timer.d 2>/dev/null || true
# The name service no longer looks up the console accounts of the server's directory.
sed -i '/^passwd:/s/ ldap\b//' /etc/nsswitch.conf
rm -rf /var/lib/vpn-router/backup/staging /var/lib/vpn-router/backup/restore \
/var/lib/vpn-router/pki-publish /etc/caddy/vpn-router-pki /etc/caddy/vpn-router-sites \
/etc/openvpn/server/vpn-router-ca.pem /etc/openvpn/server/vpn-router-crl.pem
rmdir --ignore-fail-on-non-empty /var/lib/vpn-router/backup /var/lib/vpn-router 2>/dev/null || true
rm -rf /var/lib/linux-manager/backup/staging /var/lib/linux-manager/backup/restore \
/var/lib/linux-manager/pki-publish /etc/caddy/linux-manager-pki /etc/caddy/linux-manager-sites \
/etc/openvpn/server/linux-manager-ca.pem /etc/openvpn/server/linux-manager-crl.pem
rmdir --ignore-fail-on-non-empty /var/lib/linux-manager/backup /var/lib/linux-manager 2>/dev/null || true
# The refresh tokens' database and role, when the package created them.
if [ -f /etc/vpn-router/database.secret ] && command -v psql >/dev/null 2>&1; then
runuser -u postgres -- psql -X -q -c 'DROP DATABASE IF EXISTS "vpn-router"' || true
runuser -u postgres -- psql -X -q -c 'DROP ROLE IF EXISTS "vpn-router"' || true
if [ -f /etc/linux-manager/database.secret ] && command -v psql >/dev/null 2>&1; then
runuser -u postgres -- psql -X -q -c 'DROP DATABASE IF EXISTS "linux-manager"' || true
runuser -u postgres -- psql -X -q -c 'DROP ROLE IF EXISTS "linux-manager"' || true
fi
rm -f /etc/vpn-router/database.secret
rm -f /etc/linux-manager/database.secret
# The databases and roles of the web sites and of the name server, when the package created them.
for database in vpn-router-sites vpn-router-dns; do
secret="/etc/vpn-router/${database#vpn-router-}-database.secret"
# The databases and roles of the web sites, of the name server and of the Manager's sessions,
# when the package created them.
for database in linux-manager-sites linux-manager-dns linux-manager-ui; do
secret="/etc/linux-manager/${database#linux-manager-}-database.secret"
if [ -f "$secret" ] && command -v psql >/dev/null 2>&1; then
runuser -u postgres -- psql -X -q -c "DROP DATABASE IF EXISTS \"$database\"" || true
runuser -u postgres -- psql -X -q -c "DROP ROLE IF EXISTS \"$database\"" || true
@@ -86,14 +94,14 @@ case "$1" in
rm -f "$secret"
done
# Key material, users and content are not package state: /etc/vpn-router/pki,
# /etc/vpn-router/ldap, /etc/vpn-router/rclone.conf, the user directory
# Key material, users and content are not package state: /etc/linux-manager/pki,
# /etc/linux-manager/ldap, /etc/linux-manager/rclone.conf, the user directory
# in slapd, the WireGuard key pair, everything under /etc/swanctl, the
# files of the web sites in /var/lib/vpn-router/sites, the backups and
# files of the web sites in /var/lib/linux-manager/sites, the backups and
# what a restore replaced are left in place. Caddy, rclone and PowerDNS
# stay installed, so Caddy's repository, which vpn-router-caddy
# stay installed, so Caddy's repository, which linux-manager-caddy
# configures, stays too.
rmdir --ignore-fail-on-non-empty /etc/vpn-router 2>/dev/null || true
rmdir --ignore-fail-on-non-empty /etc/linux-manager 2>/dev/null || true
;;
esac
+6 -6
View File
@@ -6,22 +6,22 @@ case "$1" in
# Stop offering the tunnels, but leave strongSwan itself enabled: it is
# a shared service and may be in use by something other than this
# package. The connection files are removed on purge.
# The site-to-site connections are the names of site_to_site in vpn-router.json.
# The site-to-site connections are the names of site_to_site in linux-manager.json.
if [ -d /run/systemd/system ]; then
connections="$(python3 -c 'import json, sys; print(" ".join(json.load(open(sys.argv[1])).get("site_to_site", {})))' \
/etc/vpn-router/vpn-router.json 2>/dev/null || true)"
/etc/linux-manager/linux-manager.json 2>/dev/null || true)"
for connection in $connections; do
swanctl --terminate --ike "$connection" >/dev/null 2>&1 || true
done
swanctl --terminate --ike road-warrior >/dev/null 2>&1 || true
fi
# vpn-router-setup enabled wg-quick@wg0, openvpn-server@vpn-router,
# caddy, vpn-router-metrics, the backup timer and the PKI timer, so the
# linux-manager-setup enabled wg-quick@wg0, openvpn-server@linux-manager,
# caddy, linux-manager-metrics, the backup timer and the PKI timer, so the
# package takes them down again. Use the debhelper wrappers rather than
# systemctl, so the script behaves on a machine without systemd.
for unit in wg-quick@wg0.service openvpn-server@vpn-router.service caddy.service vpn-router-metrics.service \
vpn-router-backup.timer vpn-router-pki.timer; do
for unit in wg-quick@wg0.service openvpn-server@linux-manager.service caddy.service linux-manager-metrics.service \
linux-manager-backup.timer linux-manager-pki.timer; do
if command -v deb-systemd-invoke >/dev/null 2>&1; then
deb-systemd-invoke stop "$unit" >/dev/null 2>&1 || true
fi
+15 -11
View File
@@ -2,15 +2,19 @@
%:
dh $@
override_dh_auto_build:
test -f web/dist/index.html
override_dh_installsystemd:
dh_installsystemd --name=vpn-router-setup --no-start
dh_installsystemd --name=vpn-router-docker
dh_installsystemd --name=vpn-router-web --no-start
dh_installsystemd --name=vpn-router-metrics --no-enable --no-start
dh_installsystemd --name=vpn-router-bootstrap --no-enable --no-start
dh_installsystemd --name=vpn-router-backup --no-enable --no-start
dh_installsystemd --name=vpn-router-pki --no-enable --no-start
dh_installsystemd --name=vpn-router-caddy --no-enable --no-start
dh_installsystemd --name=vpn-router-webdav --no-enable --no-start
dh_installsystemd --name=vpn-router-resolver --no-enable --no-start
dh_installsystemd --name=vpn-router-install@ --no-enable --no-start
dh_installsystemd --name=linux-manager-setup --no-start
dh_installsystemd --name=linux-manager-ui --no-start
dh_installsystemd --name=linux-manager-docker
dh_installsystemd --name=linux-manager-web --no-start
dh_installsystemd --name=linux-manager-metrics --no-enable --no-start
dh_installsystemd --name=linux-manager-bootstrap --no-enable --no-start
dh_installsystemd --name=linux-manager-backup --no-enable --no-start
dh_installsystemd --name=linux-manager-pki --no-enable --no-start
dh_installsystemd --name=linux-manager-caddy --no-enable --no-start
dh_installsystemd --name=linux-manager-webdav --no-enable --no-start
dh_installsystemd --name=linux-manager-resolver --no-enable --no-start
dh_installsystemd --name=linux-manager-install@ --no-enable --no-start
+60 -26
View File
@@ -1,4 +1,4 @@
Template: vpn-router/platform
Template: linux-manager/platform
Type: select
Choices: none, generic, azure, gcp
Default: none
@@ -7,13 +7,31 @@ Description: Platform:
additions, such as the provider DNS resolver for IKEv2 clients.
.
none defers configuration entirely: the package installs its files and
changes nothing on the machine. Choose it when the router will be configured
changes nothing on the machine. Choose it when the server will be configured
later by hand or by a configuration-management tool, then set the platform in
/etc/vpn-router/vpn-router.json when you are ready.
/etc/linux-manager/linux-manager.json when you are ready.
.
generic configures the router with no platform-specific additions.
generic configures the server with no platform-specific additions.
Template: vpn-router/mode
Template: linux-manager/admin_username
Type: string
Default:
Description: Name of the Manager's administrator account:
The Manager can have a first administrator: a Manager-only account
of the server's user directory, a member of the group admins, which may
sign in to the Manager but has no VPN access. linux-manager-setup
creates it.
.
Leave this empty to create no account and no group.
Template: linux-manager/admin_password
Type: password
Description: Password of the administrator account:
The password of the administrator account, 8 to 128 characters. It is
passed to linux-manager-setup in a file readable by root only, which is removed
once the account exists, and not kept by debconf.
Template: linux-manager/mode
Type: select
Choices: manual, interfaces, auto
Default: manual
@@ -31,7 +49,23 @@ Description: Network configuration source:
table and their addresses read from the system. Convenient, but it can pick
the wrong interface and configure the machine incorrectly.
Template: vpn-router/external_interface
Template: linux-manager/router_enabled
Type: boolean
Default: false
Description: Enable the router?
If enabled, the server routes between its external and internal
interfaces and runs the site-to-site connections. Without the router the
server needs one interface, and the VPN clients reach the server's
networks through NAT from its address.
Template: linux-manager/vpn_enabled
Type: boolean
Default: false
Description: Enable the VPN?
If enabled, IKEv2, OpenVPN and WireGuard access can be switched on for
the users. Without the VPN none of them runs.
Template: linux-manager/external_interface
Type: string
Default:
Description: External network interface:
@@ -39,21 +73,21 @@ Description: External network interface:
ens4. The addresses strongSwan binds on are read from it, so they are never
configured separately.
Template: vpn-router/internal_interface
Template: linux-manager/internal_interface
Type: string
Default:
Description: Internal network interface:
Name of the interface facing the protected network, for example eth1 or
ens5. Routes for the local subnets are applied to it.
Template: vpn-router/int_addr
Template: linux-manager/int_addr
Type: string
Default:
Description: Internal interface address:
This host's own address on the internal network. Asked only in manual mode;
in the other modes it is read from the internal interface.
Template: vpn-router/local_fqdn
Template: linux-manager/local_fqdn
Type: string
Default:
Description: Local router FQDN:
@@ -61,7 +95,7 @@ Description: Local router FQDN:
router.example.com). Used as the IKEv2 server identity and
certificate common name.
Template: vpn-router/local_id_mode
Template: linux-manager/local_id_mode
Type: select
Choices: fqdn, public_ip, internal_ip
Default: fqdn
@@ -75,7 +109,7 @@ Description: IKE local identity mode:
.
internal_ip - use the internal NIC address.
Template: vpn-router/local_cidrs
Template: linux-manager/local_cidrs
Type: string
Default:
Description: Local subnet CIDR(s):
@@ -83,7 +117,7 @@ Description: Local subnet CIDR(s):
advertised into the first site-to-site connection (for example
10.0.0.0/24 or 10.0.0.0/24,10.0.1.0/24).
Template: vpn-router/int_gateway_ip
Template: linux-manager/int_gateway_ip
Type: string
Default:
Description: Internal network gateway address:
@@ -91,55 +125,55 @@ Description: Internal network gateway address:
local subnets listed above. Asked only in manual mode; in the other modes it
is derived from the internal interface.
Template: vpn-router/remote_addrs
Template: linux-manager/remote_addrs
Type: string
Default:
Description: Remote site WAN IP address(es):
Comma-separated list of remote gateway addresses or FQDNs for the
first site-to-site connection.
Template: vpn-router/remote_id
Template: linux-manager/remote_id
Type: string
Default:
Description: Remote site IKE identity:
IKE identity of the remote peer of the first site-to-site connection,
without a leading @.
Template: vpn-router/remote_cidrs
Template: linux-manager/remote_cidrs
Type: string
Default:
Description: Remote subnet CIDR(s):
Comma-separated list of remote subnet CIDRs reachable through the
first site-to-site connection (for example 192.168.0.0/24).
Template: vpn-router/psk
Template: linux-manager/psk
Type: password
Description: Pre-shared key (PSK):
Pre-shared key of the first site-to-site connection. Must match the value
configured on the remote peer. Stored in the configuration file and cleared
from the debconf database after installation.
Template: vpn-router/p2s_enabled
Template: linux-manager/p2s_enabled
Type: boolean
Default: false
Description: Enable IKEv2 access?
If enabled, this router accepts IKEv2 EAP-TLS connections from individual
clients, with certificates of the router's P2S issuing CA.
clients, with certificates of the server's P2S issuing CA.
Template: vpn-router/p2s_address_pool
Template: linux-manager/p2s_address_pool
Type: string
Default:
Description: IKEv2 address pool:
CIDR block assigned to IKEv2 clients (for example 172.16.0.0/24).
Template: vpn-router/pki_root_name
Template: linux-manager/pki_root_name
Type: string
Default: VPN Router Root CA
Default: Linux Manager Root CA
Description: Root CA name:
Common name of the root certificate authority the router creates in
/etc/vpn-router/pki/root. Ignored when a root certificate is placed there.
Common name of the root certificate authority the server creates in
/etc/linux-manager/pki/root. Ignored when a root certificate is placed there.
Template: vpn-router/wg_enabled
Template: linux-manager/wg_enabled
Type: boolean
Default: false
Description: Enable WireGuard?
@@ -147,14 +181,14 @@ Description: Enable WireGuard?
the matching firewall rule is installed. Peers are added by editing
/etc/wireguard/wg0.conf.
Template: vpn-router/wg_address
Template: linux-manager/wg_address
Type: string
Default:
Description: WireGuard interface address:
Address and prefix length for the wg0 interface (for example
192.168.200.1/24).
Template: vpn-router/wg_listen_port
Template: linux-manager/wg_listen_port
Type: string
Default: 51820
Description: WireGuard listen port:
@@ -1,10 +0,0 @@
[Unit]
Description=VPN Router backup schedule
Documentation=file:/usr/share/doc/vpn-router/README.md
[Timer]
OnCalendar=daily
Persistent=true
[Install]
WantedBy=timers.target
@@ -1,14 +0,0 @@
[Unit]
Description=VPN Router bootstrap diagnostics page
Documentation=file:/usr/share/doc/vpn-router/README.md
# Never enabled: vpn-router-setup starts it until a setup has succeeded, and stops it after that.
[Service]
Type=simple
ExecStart=/usr/bin/python3 /usr/lib/vpn-router/bootstrap
Restart=on-failure
RestartSec=5
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
@@ -1,10 +0,0 @@
[Unit]
Description=VPN Router Caddy installation
Documentation=file:/usr/share/doc/vpn-router/README.md
After=network-online.target
Wants=network-online.target
[Service]
Type=oneshot
ExecStart=/usr/sbin/vpn-router-caddy apply
TimeoutStartSec=20min
@@ -1,14 +0,0 @@
[Unit]
Description=VPN Router rules for Docker published ports
Documentation=file:/usr/share/doc/vpn-router/README.md
After=docker.service
PartOf=docker.service
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/lib/vpn-router/docker-rules apply
ExecStop=/usr/lib/vpn-router/docker-rules remove
[Install]
WantedBy=docker.service
@@ -1,10 +0,0 @@
[Unit]
Description=VPN Router installation of the packages of %i
Documentation=file:/usr/share/doc/vpn-router/README.md
After=network-online.target
Wants=network-online.target
[Service]
Type=oneshot
ExecStart=/usr/lib/vpn-router/install-packages %i
TimeoutStartSec=20min
@@ -1,22 +0,0 @@
[Unit]
Description=VPN Router metrics service
Documentation=file:/usr/share/doc/vpn-router/README.md
After=network-online.target postgresql.service vpn-router-web.service
Wants=network-online.target
[Service]
Type=simple
ExecStartPre=/usr/lib/vpn-router/wait-for-libpq
ExecStart=/usr/bin/python3 /usr/lib/vpn-router/metrics
TimeoutStartSec=5min
Restart=on-failure
RestartSec=15
NoNewPrivileges=yes
ProtectSystem=strict
RuntimeDirectory=vpn-router
RuntimeDirectoryPreserve=yes
ProtectHome=yes
PrivateTmp=yes
[Install]
WantedBy=multi-user.target
@@ -1,7 +0,0 @@
[Unit]
Description=VPN Router PKI maintenance
Documentation=file:/usr/share/doc/vpn-router/README.md
[Service]
Type=oneshot
ExecStart=/usr/sbin/vpn-router-ca maintain
@@ -1,10 +0,0 @@
[Unit]
Description=VPN Router PKI maintenance schedule
Documentation=file:/usr/share/doc/vpn-router/README.md
[Timer]
OnCalendar=daily
Persistent=true
[Install]
WantedBy=timers.target
@@ -1,23 +0,0 @@
[Unit]
Description=VPN Router status web service
Documentation=file:/usr/share/doc/vpn-router/README.md
After=network-online.target postgresql.service
Wants=network-online.target
[Service]
Type=simple
ExecStartPre=/usr/lib/vpn-router/wait-for-libpq
ExecStart=/usr/bin/python3 /usr/lib/vpn-router/web
TimeoutStartSec=5min
Restart=on-failure
RestartSec=15
NoNewPrivileges=yes
ProtectSystem=strict
RuntimeDirectory=vpn-router
RuntimeDirectoryPreserve=yes
ReadWritePaths=-/etc/vpn-router -/etc/swanctl/x509crl -/etc/openvpn/server -/var/lib/vpn-router/pki-publish -/var/lib/vpn-router/sites
ProtectHome=yes
PrivateTmp=yes
[Install]
WantedBy=multi-user.target
@@ -1,21 +0,0 @@
[Unit]
Description=VPN Router WebDAV of the web sites' folders
Documentation=file:/usr/share/doc/vpn-router/README.md
After=network-online.target slapd.service postgresql.service
Wants=network-online.target
[Service]
Type=simple
ExecStart=/usr/bin/rclone serve webdav --addr 127.0.0.1:7446 --auth-proxy /usr/lib/vpn-router/webdav-auth --config /dev/null --cache-dir /var/cache/vpn-router-webdav --vfs-cache-mode writes --umask 022
Restart=on-failure
RestartSec=15
NoNewPrivileges=yes
ProtectSystem=strict
ReadWritePaths=/var/lib/vpn-router/sites
CacheDirectory=vpn-router-webdav
RuntimeDirectory=vpn-router-webdav
ProtectHome=yes
PrivateTmp=yes
[Install]
WantedBy=multi-user.target
-40
View File
@@ -1,40 +0,0 @@
#!/bin/sh
set -e
# Apple container CLI on macOS, Docker CE on Linux.
case "$(uname -s)" in
Darwin) CONTAINER_RUNTIME=container ;;
Linux) CONTAINER_RUNTIME=docker ;;
*) printf 'ERROR: unsupported platform %s\n' "$(uname -s)" >&2; exit 1 ;;
esac
OWNER="${OWNER:-slawek}"
SERVER_URL="${SERVER_URL:-https://gitea.koszewscy.waw.pl}"
PACKAGES_TOKEN="${PACKAGES_TOKEN:-$(cat ~/.gitea-packages-token)}"
# Reads the changelog copied into the builder image, so no bind mount is needed.
changelog_field() {
$CONTAINER_RUNTIME run --rm vpn-router-builder \
dpkg-parsechangelog --show-field "$1" -l /build/src/debian/changelog
}
VERSION="$(changelog_field Version)"
PACKAGE="$(changelog_field Source)"
DEB="out/${PACKAGE}_${VERSION}_all.deb"
if [ ! -f "$DEB" ]; then
printf 'ERROR: %s does not exist. Run ./build.sh first.\n' "$DEB" >&2
exit 1
fi
STATUS="$(curl -s -o /dev/null -w '%{http_code}' --user "$OWNER:$PACKAGES_TOKEN" \
"$SERVER_URL/api/v1/packages/$OWNER/debian/$PACKAGE/$VERSION")"
case "$STATUS" in
200) printf '%s %s is already published, skipping.\n' "$PACKAGE" "$VERSION"; exit 0 ;;
404) ;;
*) printf 'ERROR: version check returned HTTP %s.\n' "$STATUS" >&2; exit 1 ;;
esac
curl --fail-with-body --user "$OWNER:$PACKAGES_TOKEN" \
--upload-file "$DEB" \
"$SERVER_URL/api/packages/$OWNER/debian/pool/noble/main/upload"
@@ -1,6 +0,0 @@
# Enable forwarding
net.ipv4.ip_forward = 1
# Disable strict rp_filter that breaks asymmetric forwarding on multi-NIC routers
net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.default.rp_filter = 0
+71
View File
@@ -0,0 +1,71 @@
#!/usr/bin/python3
"""Open files in the editor of the Manager's Shell page, from the shell of that page; lme is its short name.
Each file is named to the server's web service over SOCKET with the shell's session,
$LINUX_MANAGER_SHELL, which the web service sets in the shells it starts; the Shell page then shows
the file in a tab of its editor, where it is read and saved. The command returns at once. A
relative path is taken from the current directory, and a file that does not exist is created when
it is saved. The exit status is 0 when every file was opened, 1 when the web service refused one,
with its reason, and 2 outside the Manager's shell or for an invalid request. -h or --help
shows the usage.
lme FILE...
"""
import argparse
import json
import os
import socket
import sys
#: The socket of the server's web service that opens files in the editor of a shell.
SOCKET = '/run/linux-manager/editor.sock'
#: Seconds to wait for the web service's answer.
TIMEOUT = 10
class Parser(argparse.ArgumentParser):
"""An argument parser whose errors end with status 2."""
def error(self, message):
print(f'{self.prog}: {message}; see {self.prog} --help', file=sys.stderr)
sys.exit(2)
def opened(session, path):
"""Ask the web service to open the file in the editor of the session; None, or the reason it refused."""
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as connection:
connection.settimeout(TIMEOUT)
connection.connect(SOCKET)
connection.sendall(json.dumps({'session': session, 'path': path}).encode() + b'\n')
answer = json.loads(connection.makefile('rb').readline() or b'{}')
return None if answer.get('ok') is True else str(answer.get('error', 'no answer'))
def main(argv):
prog = os.path.basename(sys.argv[0])
parser = Parser(prog=prog, description="Open files in the editor of the Manager's Shell page.")
parser.add_argument('files', nargs='+', metavar='FILE', help='a file to open')
options = parser.parse_args(argv)
session = os.environ.get('LINUX_MANAGER_SHELL', '')
if not session:
print(f'{prog}: {prog} works only in the shell of the Manager', file=sys.stderr)
return 2
status = 0
for name in options.files:
path = os.path.abspath(os.path.expanduser(name))
try:
reason = opened(session, path)
except (OSError, ValueError) as exc:
reason = f'the web service cannot be reached: {exc}'
if reason:
print(f'{prog}: {path}: {reason}', file=sys.stderr)
status = 1
else:
print(f'Opened {path} in the editor of the Shell page.')
return status
if __name__ == '__main__':
sys.exit(main(sys.argv[1:]))
+72
View File
@@ -0,0 +1,72 @@
#!/usr/bin/python3
"""Serve diagnostics of the server's first setup on PORT, with the Python standard library alone.
Run by linux-manager-bootstrap.service, which linux-manager-setup starts until a setup has succeeded and
then stops. Every request on any address answers, as plain text, the output of COMMANDS run at that
moment: the state of the last setup run, the state of the services, the addresses, and the end of
the logs.
"""
import subprocess
import sys
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
PORT = 17081
#: The title of each section and its command.
COMMANDS = (
('Setup state', ['cat', '/run/linux-manager/setup.json']),
('Services', ['systemctl', 'is-active', 'linux-manager-setup', 'linux-manager-web', 'linux-manager-ui', 'slapd',
'postgresql', 'strongswan', 'openvpn-server@linux-manager', 'wg-quick@wg0']),
('Failed units', ['systemctl', '--failed', '--no-legend', '--plain']),
('Addresses', ['ip', '-brief', 'address']),
('Routes', ['ip', 'route']),
('cloud-init', ['cloud-init', 'status', '--long']),
('linux-manager-setup log', ['journalctl', '--unit', 'linux-manager-setup', '--lines', '60', '--no-pager', '--output', 'short-iso']),
('linux-manager-web log', ['journalctl', '--unit', 'linux-manager-web', '--lines', '30', '--no-pager', '--output', 'short-iso']),
('linux-manager-ui log', ['journalctl', '--unit', 'linux-manager-ui', '--lines', '30', '--no-pager', '--output',
'short-iso']),
)
def report():
"""The output of every command of COMMANDS under its title."""
sections = []
for title, command in COMMANDS:
try:
result = subprocess.run(command, capture_output=True, text=True, timeout=10)
text = (result.stdout + result.stderr).rstrip()
except (OSError, subprocess.TimeoutExpired) as exc:
text = f'cannot run {command[0]}: {exc}'
sections.append(f'== {title} ==\n{text}\n')
return '\n'.join(sections)
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
body = report().encode()
try:
self.send_response(200)
self.send_header('Content-Type', 'text/plain; charset=utf-8')
self.send_header('Content-Length', str(len(body)))
self.send_header('Cache-Control', 'no-store')
self.send_header('Refresh', '10')
self.end_headers()
self.wfile.write(body)
except ConnectionError:
# The client went away before the answer was written.
self.close_connection = True
def failed(request, client_address):
"""Log a failed request in one line, and nothing when the client went away."""
failure = sys.exc_info()[1]
if not isinstance(failure, ConnectionError):
print(f'a request from {client_address[0]} failed: {type(failure).__name__}: {failure}', file=sys.stderr, flush=True)
if __name__ == '__main__':
sys.excepthook = lambda kind, value, _: print(f'failed: {kind.__name__}: {value}', file=sys.stderr, flush=True)
server = ThreadingHTTPServer(('', PORT), Handler)
server.handle_error = failed
server.serve_forever()
@@ -1,5 +1,5 @@
#!/usr/bin/python3
"""Render configuration files and provision PKI from vpn-router.json.
"""Render configuration files and provision PKI from linux-manager.json.
Run directly to regenerate them without starting, stopping or reloading
anything. This still writes to /etc/swanctl, /etc/systemd/resolved.conf.d and
@@ -13,11 +13,11 @@ import sys
sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
import vpnrouter
import linuxmanager
def main():
vpnrouter.configure()
linuxmanager.configure()
return 0
@@ -5,14 +5,14 @@ Usage: docker-rules apply|remove
Docker evaluates DOCKER-USER in FORWARD before its own rules, which makes it the
place to filter traffic to containers. The rules live in their own chain,
VPN-ROUTER-DOCKER, entered by one jump at the top of DOCKER-USER, so anything
LINUX-MANAGER-DOCKER, entered by one jump at the top of DOCKER-USER, so anything
else in DOCKER-USER is left alone. Traffic arriving on the external interface is
accepted when it is IPsec protected, when it is TCP to docker.exposed_ports, or
when it belongs to an established connection; all other traffic from the
external interface is dropped. Traffic from other interfaces is not affected.
docker.mode decides whether this runs: auto applies the rules when the Docker
service is installed, on requires it, off removes the rules. The vpn-router-docker
service is installed, on requires it, off removes the rules. The linux-manager-docker
unit runs apply after docker.service starts and remove when it stops.
"""
@@ -22,10 +22,10 @@ import sys
sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
import vpnrouter
from vpnrouter import fail, warn
import linuxmanager
from linuxmanager import fail, warn
CHAIN = 'VPN-ROUTER-DOCKER'
CHAIN = 'LINUX-MANAGER-DOCKER'
USER_CHAIN = 'DOCKER-USER'
@@ -61,7 +61,7 @@ def apply(cfg):
if not docker_installed():
if mode == 'on':
fail('docker.mode is on but the docker service is not installed; '
'install Docker before vpn-router')
'install Docker before linux-manager')
return
if iptables('-S', USER_CHAIN, check=False).returncode != 0:
@@ -72,7 +72,7 @@ def apply(cfg):
'the rules are applied when it starts')
return
ctx = vpnrouter.build_context(cfg)
ctx = linuxmanager.build_context(cfg)
wan = ctx['wan_iface']
if not wan:
fail('the external interface is not known; set interfaces.external '
@@ -99,13 +99,13 @@ def main():
remove()
return 0
cfg = vpnrouter.load_config()
if vpnrouter.deferred(cfg):
cfg = linuxmanager.load_config()
if linuxmanager.deferred(cfg):
return 0
try:
vpnrouter.validate(cfg)
linuxmanager.validate(cfg)
apply(cfg)
except vpnrouter.ConfigError as exc:
except linuxmanager.ConfigError as exc:
fail(str(exc))
return 0
+82
View File
@@ -0,0 +1,82 @@
#!/usr/bin/python3
"""Read or write a file of the editor of the Manager's Shell page, as the user it runs as.
The web service runs it with systemd-run --uid=<console account> --pipe, outside its sandbox, so the
file is read and written with the rights of the shell's user. The version of a content is the
hexadecimal SHA-256 of its bytes; a missing file has the version of the empty content.
editor-file read PATH print the content of the file; nothing for a missing file
editor-file write PATH VERSION replace the content with standard input when the file still has
VERSION, writing in place, so its owner, mode and links stay;
a missing file is created
Only regular files of at most MAX_BYTES bytes of UTF-8 text are read and written. The exit status
is 0 on success, 2 for a file that is refused, with the reason on standard error, 3 when the file
does not have VERSION any more, and 1 for any other failure.
"""
import hashlib
import os
import sys
#: The largest file the editor opens, in bytes.
MAX_BYTES = 1024 * 1024
class Refused(Exception):
"""A file the editor does not open or write; status is the exit status."""
def __init__(self, message, status=2):
super().__init__(message)
self.status = status
def text(data, path):
"""The bytes when they are UTF-8 text of at most MAX_BYTES; Refused otherwise."""
if len(data) > MAX_BYTES:
raise Refused(f'{path} is larger than {MAX_BYTES // 1024} KiB')
try:
data.decode('utf-8')
except UnicodeDecodeError:
raise Refused(f'{path} is not UTF-8 text')
return data
def current(path):
"""The content of the file, b'' when it does not exist; Refused for a file that is not a regular one."""
if not os.path.lexists(path):
return b''
if not os.path.isfile(path):
raise Refused(f'{path} is not a regular file')
with open(path, 'rb') as handle:
return text(handle.read(MAX_BYTES + 1), path)
def main(argv):
if len(argv) < 2 or (argv[0], len(argv)) not in (('read', 2), ('write', 3)):
print('usage: editor-file read PATH | editor-file write PATH VERSION', file=sys.stderr)
return 2
path = argv[1]
try:
if argv[0] == 'read':
sys.stdout.buffer.write(current(path))
return 0
content = text(sys.stdin.buffer.read(MAX_BYTES + 1), path)
if hashlib.sha256(current(path)).hexdigest() != argv[2]:
raise Refused(f'{path} changed since it was read', 3)
with open(path, 'r+b' if os.path.lexists(path) else 'xb') as handle:
handle.write(content)
handle.truncate()
handle.flush()
os.fsync(handle.fileno())
except Refused as refused:
print(refused, file=sys.stderr)
return refused.status
except OSError as exc:
print(f'{exc.filename or path}: {exc.strerror}', file=sys.stderr)
return 1
return 0
if __name__ == '__main__':
sys.exit(main(sys.argv[1:]))
@@ -1,25 +1,25 @@
#!/usr/bin/python3
"""Write a candidate vpn-router.json to the path given on the command line.
"""Write a candidate linux-manager.json to the path given on the command line.
Called from postinst, which hands the result to ucf(1). ucf compares it with
/etc/vpn-router/vpn-router.json and decides what to do about local changes, so
/etc/linux-manager/linux-manager.json and decides what to do about local changes, so
this script never writes to /etc and never has to worry about destroying an
administrator's edits.
The candidate is the current /etc/vpn-router/vpn-router.json as it is, or the
The candidate is the current /etc/linux-manager/linux-manager.json as it is, or the
shipped defaults when there is none, with every debconf answer that was given
written over it. Without an answer that changes it, the candidate is the current
file byte for byte, so
an upgrade or a reconfiguration that changes nothing gives ucf nothing to ask;
defaults the file leaves out are applied by vpnrouter.load_config() when the
defaults the file leaves out are applied by linuxmanager.load_config() when the
file is read. An empty answer means "leave alone", which is why clearing the
pre-shared key from the debconf database after installation does not blank it
on the next dpkg-reconfigure.
Answers arrive as VPN_ROUTER_* environment variables and are converted to the
Answers arrive as LINUX_MANAGER_* environment variables and are converted to the
type of the setting they fill: comma-separated lists become arrays, "true" and
"false" become booleans and digits become integers. The site-to-site answers
fill the connection vpnrouter.FIRST_CONNECTION of site_to_site; the local
fill the connection linuxmanager.FIRST_CONNECTION of site_to_site; the local
subnets fill both its local_cidrs and local.cidrs.
"""
@@ -29,31 +29,33 @@ import sys
sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
import vpnrouter
import linuxmanager
FIRST = ('site_to_site', vpnrouter.FIRST_CONNECTION)
FIRST = ('site_to_site', linuxmanager.FIRST_CONNECTION)
# environment variable -> the settings it fills, each (section, option) or (site_to_site, name, option)
ANSWERS = {
'VPN_ROUTER_PLATFORM': (('general', 'platform'),),
'VPN_ROUTER_MODE': (('general', 'mode'),),
'VPN_ROUTER_INT_ADDR': (('local', 'int_addr'),),
'VPN_ROUTER_EXTERNAL_INTERFACE': (('interfaces', 'external'),),
'VPN_ROUTER_INTERNAL_INTERFACE': (('interfaces', 'internal'),),
'VPN_ROUTER_LOCAL_FQDN': (('wan', 'local_fqdn'),),
'VPN_ROUTER_LOCAL_ID_MODE': ((*FIRST, 'local_id_mode'),),
'VPN_ROUTER_LOCAL_CIDRS': (('local', 'cidrs'), (*FIRST, 'local_cidrs')),
'VPN_ROUTER_INT_GATEWAY_IP': (('local', 'int_gateway_ip'),),
'VPN_ROUTER_REMOTE_ADDRS': ((*FIRST, 'addrs'),),
'VPN_ROUTER_REMOTE_ID': ((*FIRST, 'id'),),
'VPN_ROUTER_REMOTE_CIDRS': ((*FIRST, 'cidrs'),),
'VPN_ROUTER_PSK': ((*FIRST, 'psk'),),
'VPN_ROUTER_P2S_ENABLED': (('p2s', 'enabled'),),
'VPN_ROUTER_P2S_ADDRESS_POOL': (('p2s', 'address_pool'),),
'VPN_ROUTER_PKI_ROOT_NAME': (('pki', 'root_name'),),
'VPN_ROUTER_WG_ENABLED': (('wireguard', 'enabled'),),
'VPN_ROUTER_WG_ADDRESS': (('wireguard', 'address'),),
'VPN_ROUTER_WG_LISTEN_PORT': (('wireguard', 'listen_port'),),
'LINUX_MANAGER_PLATFORM': (('general', 'platform'),),
'LINUX_MANAGER_MODE': (('general', 'mode'),),
'LINUX_MANAGER_ROUTER_ENABLED': (('router', 'enabled'),),
'LINUX_MANAGER_VPN_ENABLED': (('vpn', 'enabled'),),
'LINUX_MANAGER_INT_ADDR': (('local', 'int_addr'),),
'LINUX_MANAGER_EXTERNAL_INTERFACE': (('interfaces', 'external'),),
'LINUX_MANAGER_INTERNAL_INTERFACE': (('interfaces', 'internal'),),
'LINUX_MANAGER_LOCAL_FQDN': (('wan', 'local_fqdn'),),
'LINUX_MANAGER_LOCAL_ID_MODE': ((*FIRST, 'local_id_mode'),),
'LINUX_MANAGER_LOCAL_CIDRS': (('local', 'cidrs'), (*FIRST, 'local_cidrs')),
'LINUX_MANAGER_INT_GATEWAY_IP': (('local', 'int_gateway_ip'),),
'LINUX_MANAGER_REMOTE_ADDRS': ((*FIRST, 'addrs'),),
'LINUX_MANAGER_REMOTE_ID': ((*FIRST, 'id'),),
'LINUX_MANAGER_REMOTE_CIDRS': ((*FIRST, 'cidrs'),),
'LINUX_MANAGER_PSK': ((*FIRST, 'psk'),),
'LINUX_MANAGER_P2S_ENABLED': (('p2s', 'enabled'),),
'LINUX_MANAGER_P2S_ADDRESS_POOL': (('p2s', 'address_pool'),),
'LINUX_MANAGER_PKI_ROOT_NAME': (('pki', 'root_name'),),
'LINUX_MANAGER_WG_ENABLED': (('wireguard', 'enabled'),),
'LINUX_MANAGER_WG_ADDRESS': (('wireguard', 'address'),),
'LINUX_MANAGER_WG_LISTEN_PORT': (('wireguard', 'listen_port'),),
}
@@ -64,18 +66,18 @@ def main():
output = sys.argv[1]
try:
current = vpnrouter.CONFIG_FILE.read_text(encoding='utf-8')
current = linuxmanager.CONFIG_FILE.read_text(encoding='utf-8')
except FileNotFoundError:
current = None
if current is None:
cfg = vpnrouter.load_config()
cfg = linuxmanager.load_config()
else:
try:
cfg = json.loads(current)
except ValueError as exc:
vpnrouter.fail(f'cannot parse {vpnrouter.CONFIG_FILE}: {exc}')
linuxmanager.fail(f'cannot parse {linuxmanager.CONFIG_FILE}: {exc}')
if not isinstance(cfg, dict):
vpnrouter.fail(f'cannot parse {vpnrouter.CONFIG_FILE}: the top level is not an object')
linuxmanager.fail(f'cannot parse {linuxmanager.CONFIG_FILE}: the top level is not an object')
answered = False
for variable, paths in ANSWERS.items():
@@ -84,7 +86,7 @@ def main():
continue
for path in paths:
# A value that does not convert is kept as a string for validate() to report.
default = vpnrouter.CONNECTION[path[2]] if path[0] == 'site_to_site' else vpnrouter.SCHEMA[path[0]][path[1]]
default = linuxmanager.CONNECTION[path[2]] if path[0] == 'site_to_site' else linuxmanager.SCHEMA[path[0]][path[1]]
value = answer
if isinstance(default, bool):
value = value == 'true'
@@ -95,7 +97,7 @@ def main():
target = cfg
for key in path[:-1]:
if not isinstance(target.setdefault(key, {}), dict):
vpnrouter.fail(f'cannot parse {vpnrouter.CONFIG_FILE}: {".".join(path[:-1])} is not an object')
linuxmanager.fail(f'cannot parse {linuxmanager.CONFIG_FILE}: {".".join(path[:-1])} is not an object')
target = target[key]
if target.get(path[-1]) != value:
target[path[-1]] = value
@@ -1,7 +1,7 @@
#!/usr/bin/python3
"""Install the packages of a feature from the Ubuntu archive, then restart vpn-router-setup without waiting.
"""Install the packages of a feature from the Ubuntu archive, then restart linux-manager-setup without waiting.
Run by vpn-router-install@<feature>.service, which vpn-router-setup starts when the feature is on and its packages are
Run by linux-manager-install@<feature>.service, which linux-manager-setup starts when the feature is on and its packages are
missing: webdav (rclone) and dns (PowerDNS with its gpgsql backend). Recommended packages are left out, so PowerDNS
comes without its bind backend. A failure is a message on standard error and the exit status 1.
@@ -28,7 +28,7 @@ def main():
print(f'$ {" ".join(args)}', flush=True)
if subprocess.run(args, env=environment).returncode != 0:
sys.exit(f'install-packages: {args[0]} failed')
subprocess.run(['systemctl', 'restart', '--no-block', 'vpn-router-setup.service'])
subprocess.run(['systemctl', 'restart', '--no-block', 'linux-manager-setup.service'])
return 0
@@ -1,4 +1,4 @@
"""Shared library for the vpn-router package.
"""Shared library for the linux-manager package.
Everything derivable from the configuration file alone, with no running
network required. Used by both 'configure' and 'setup'.
@@ -23,39 +23,41 @@ LIB_DIR = pathlib.Path(__file__).resolve().parent
if str(LIB_DIR) not in sys.path:
sys.path.insert(0, str(LIB_DIR))
import vpnrouter_ldap
import vpnrouter_pki
import vpnrouter_platforms
import vpnrouter_status
from vpnrouter_net import DEFAULT_LISTEN, LISTEN_CHOICES, LOOPBACK, ConfigError, detect_interfaces, interface_addresses, ip_output
import linuxmanager_ldap
import linuxmanager_pki
import linuxmanager_platforms
import linuxmanager_status
from linuxmanager_net import DEFAULT_LISTEN, LISTEN_CHOICES, LOOPBACK, ConfigError, detect_interfaces, interface_addresses, ip_output
CONFIG_FILE = pathlib.Path('/etc/vpn-router/vpn-router.json')
CONFIG_LOCK = pathlib.Path('/run/vpn-router/config.lock')
CONFIG_FILE = pathlib.Path('/etc/linux-manager/linux-manager.json')
CONFIG_LOCK = pathlib.Path('/run/linux-manager/config.lock')
#: What setup resolved and runs, for the web service (setup's refresh_web).
WEB_STATE = pathlib.Path('/run/vpn-router/web.json')
TEMPLATE_DIR = pathlib.Path('/usr/share/vpn-router/templates')
WEB_STATE = pathlib.Path('/run/linux-manager/web.json')
TEMPLATE_DIR = pathlib.Path('/usr/share/linux-manager/templates')
SWANCTL_CONF_D = pathlib.Path('/etc/swanctl/conf.d')
SITE_TO_SITE = SWANCTL_CONF_D / 'site-to-site.conf'
ROAD_WARRIOR = SWANCTL_CONF_D / 'road-warrior.conf'
#: The drop-in of systemd-resolved that earlier versions wrote for the remote access clients; removed when found.
P2S_FORWARDER = pathlib.Path('/etc/systemd/resolved.conf.d/p2s-forwarder.conf')
#: The configuration of dnsmasq, the resolver of the remote access clients (vpn-router-resolver.service).
RESOLVER_CONF = pathlib.Path('/etc/vpn-router/dnsmasq.conf')
#: The configuration of dnsmasq, the resolver of the remote access clients (linux-manager-resolver.service).
RESOLVER_CONF = pathlib.Path('/etc/linux-manager/dnsmasq.conf')
WG_CONF = pathlib.Path('/etc/wireguard/wg0.conf')
#: The router's WireGuard public key, which setup writes with its private key.
#: The server's WireGuard public key, which setup writes with its private key.
WG_PUBLIC_KEY = pathlib.Path('/etc/wireguard/wg0.pub')
#: The lines around the [Peer] sections of the users' WireGuard peers in WG_CONF, which setup writes from the directory.
WG_PEERS_BEGIN = "# BEGIN WireGuard peers of the users of the router's directory"
WG_PEERS_END = "# END WireGuard peers of the users of the router's directory"
OPENVPN_CONF = pathlib.Path('/etc/openvpn/server/vpn-router.conf')
OPENVPN_LDAP = pathlib.Path('/etc/openvpn/server/vpn-router-ldap.conf')
WG_PEERS_BEGIN = "# BEGIN WireGuard peers of the users of the server's directory"
WG_PEERS_END = "# END WireGuard peers of the users of the server's directory"
OPENVPN_CONF = pathlib.Path('/etc/openvpn/server/linux-manager.conf')
OPENVPN_LDAP = pathlib.Path('/etc/openvpn/server/linux-manager-ldap.conf')
# section -> option -> default; the type of the default is the required type.
SCHEMA = {
'general': {'platform': 'none', 'mode': 'manual'},
'router': {'enabled': False},
'vpn': {'enabled': False},
'interfaces': {'external': '', 'internal': ''},
'wan': {'local_fqdn': ''},
'wan': {'local_fqdn': '', 'public_address': ''},
'local': {'cidrs': [], 'int_addr': '', 'int_gateway_ip': ''},
'site_to_site': {},
'p2s': {'enabled': False, 'address_pool': '', 'dns_listen': 'internal', 'dns_domains': []},
@@ -66,19 +68,20 @@ SCHEMA = {
'status_content': 'basic', 'shell': False, 'shell_idle_minutes': 15, 'shell_auth_timeout': 15,
'log_files': []},
'ssh': {'ldap_login': False},
'https': {'enabled': True, 'email': '', 'web_port': 7444, 'manager_port': 7443, 'caddy_source': 'ubuntu'},
'https': {'enabled': True, 'email': '', 'web_port': 7444, 'manager_port': 7443, 'caddy_source': 'ubuntu',
'acme_staging': False},
'oidc': {'enabled': False},
'sites': {'enabled': False, 'upload_max_mb': 512, 'webdav': False, 'webdav_port': 7445},
'dns': {'enabled': False, 'nameserver': '', 'addresses': [], 'listen': 'external', 'ttl': 300},
'pki': {'root_name': 'VPN Router Root CA', 'root_days': 7300, 'ca_days': 3650, 'key': 'rsa-4096',
'crl_days': 7, 'root_crl_days': 180, 'publish': ['http'], 'p2s_name': 'VPN Router P2S CA',
's2s_name': 'VPN Router Site-to-Site CA', 'issuing_cas': {}, 'templates': {},
'dns': {'enabled': False, 'secondary': '', 'listen': 'external', 'ttl': 300},
'pki': {'root_name': 'Linux Manager Root CA', 'root_days': 7300, 'ca_days': 3650, 'key': 'rsa-4096',
'crl_days': 7, 'root_crl_days': 180, 'publish': ['http'], 'p2s_name': 'Linux Manager VPN CA',
's2s_name': 'Linux Manager Site-to-Site CA', 'issuing_cas': {}, 'templates': {},
'extended_key_usages': {}, 'root_key_warning': True},
'metrics': {'enabled': True, 'listen': 'internal', 'port': 9464, 'https_port': 9465, 'service_account': 'metrics',
'otlp_endpoint': '', 'otlp_username': '', 'otlp_password': '',
'keep_details': '24 hours', 'keep_hourly': '1 month', 'keep_daily': '3 months',
'keep_monthly': '1 year', 'keep_yearly': '10 years'},
'backup': {'enabled': False, 'schedule': 'daily', 'target': 'local', 'path': '/var/backups/vpn-router',
'backup': {'enabled': False, 'schedule': 'daily', 'target': 'local', 'path': '/var/backups/linux-manager',
'keep_hourly': 24, 'keep_daily': 7, 'keep_weekly': 4, 'keep_monthly': 12, 'keep_yearly': 0,
'metrics': False, 'rclone_remote': '', 'rclone_mode': 'sync',
'pbs_repository': '', 'pbs_password': '', 'pbs_fingerprint': '', 'pbs_namespace': '',
@@ -143,27 +146,33 @@ WEBDAV_LOOPBACK_PORT = 7446
#: The largest sites.upload_max_mb, 10 GiB.
UPLOAD_MAX_MB = 10240
#: Settings that no longer exist, section -> options, left out when the file is read so that a file of an earlier
#: version loads: the name server is wan.local_fqdn and a site chooses the address of its records.
RETIRED = {'dns': ('nameserver', 'addresses')}
#: The shortest and the longest dns.ttl, in seconds.
DNS_TTL_RANGE = (60, 86400)
#: A DNS name without a wildcard, in lowercase, of dns.nameserver and the domains of the name server.
#: A DNS name without a wildcard, in lowercase, of the name servers and the domains of the name server.
ZONE_NAME = re.compile(r'^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?(\.[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?)+$')
#: A domain of p2s.dns_domains, one label or more, and the most domains.
DNS_DOMAIN = re.compile(r'^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?(\.[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?)*$')
DNS_DOMAINS_MAX = 16
#: The ports of the bootstrap services: the Router Manager's and the router's.
BOOTSTRAP_PORTS = (17080, 17081)
#: The ports of the bootstrap service.
BOOTSTRAP_PORTS = (17081,)
#: The Router Manager, vpn-router-manager, when it is installed, its settings and its port when they set none.
MANAGER = pathlib.Path('/usr/lib/vpn-router-manager/manager')
MANAGER_CONFIG = pathlib.Path('/etc/vpn-router-manager/manager.json')
#: The settings of the Manager, and its port when they set none.
MANAGER_CONFIG = pathlib.Path('/etc/linux-manager/manager.json')
MANAGER_PORT = 7080
#: Where vpn-router-setup installs Caddy from, https.caddy_source: the Ubuntu archive or Caddy's own repository.
#: Where linux-manager-setup installs Caddy from, https.caddy_source: the Ubuntu archive or Caddy's own repository.
CADDY_SOURCES = ('ubuntu', 'upstream')
#: The ACME directory of Let's Encrypt's staging environment, Caddy's only CA while https.acme_staging is true.
ACME_STAGING_CA = 'https://acme-staging-v02.api.letsencrypt.org/directory'
#: Caddy's own repository, which https.caddy_source upstream configures: the source, its signing key and their URLs.
CADDY_APT_SOURCE = pathlib.Path('/etc/apt/sources.list.d/caddy-stable.sources')
CADDY_KEYRING = pathlib.Path('/etc/apt/keyrings/caddy-stable.asc')
@@ -219,7 +228,7 @@ PKI_TEMPLATES = {
'client': {'extended_key_usage': ['client_auth'], 'subject': 'name', 'sans': ['dns', 'email']},
}
#: The templates only the router issues: its VPN server certificate and the users' certificates.
#: The templates only the server issues: its VPN server certificate and the users' certificates.
ROUTER_TEMPLATES = ('vpn-server', 'vpn-user')
#: The built-in issuing CAs: name -> the pki option of the common name and the templates the CA issues.
@@ -271,12 +280,12 @@ PLATFORM_NONE = 'none'
def warn(message):
print(f'vpn-router: warning: {message}', file=sys.stderr)
print(f'linux-manager: warning: {message}', file=sys.stderr)
def fail(message):
"""End with status 1; SystemExit carries the message, which Python prints to standard error."""
sys.exit(f'vpn-router: error: {message}')
sys.exit(f'linux-manager: error: {message}')
def read_settings(path=None):
@@ -296,6 +305,9 @@ def read_settings(path=None):
for section, options in data.items():
if not isinstance(options, dict):
raise ConfigError(f'cannot parse {path}: {section} is not an object')
for section, options in RETIRED.items():
for option in options:
data.get(section, {}).pop(option, None)
return data
@@ -340,7 +352,7 @@ def save_config(data):
The caller holds config_lock.
"""
descriptor, name = tempfile.mkstemp(dir=CONFIG_FILE.parent, prefix='.vpn-router.json.')
descriptor, name = tempfile.mkstemp(dir=CONFIG_FILE.parent, prefix='.linux-manager.json.')
try:
with os.fdopen(descriptor, 'w', encoding='utf-8') as handle:
handle.write(json.dumps(data, indent=2) + '\n')
@@ -419,7 +431,7 @@ def validate(cfg):
_check_types(cfg)
platform = _get(cfg, 'general', 'platform')
installed = [PLATFORM_NONE] + vpnrouter_platforms.available()
installed = [PLATFORM_NONE] + linuxmanager_platforms.available()
if platform and platform not in installed:
raise ConfigError(
f'general.platform: {platform!r} is not installed, expected one of '
@@ -430,6 +442,7 @@ def validate(cfg):
raise ConfigError(
f'general.mode: {mode!r} is not one of {", ".join(MODES)}')
router = _get(cfg, 'router', 'enabled')
if not deferred(cfg):
required = {
'manual': (('interfaces', 'external'), ('interfaces', 'internal'),
@@ -437,11 +450,21 @@ def validate(cfg):
'interfaces': (('interfaces', 'external'), ('interfaces', 'internal')),
'auto': (),
}[mode or 'manual']
# Without the router the server has one interface, interfaces.external.
if not router:
required = tuple(item for item in required if item == ('interfaces', 'external'))
for section, option in required:
if not _get(cfg, section, option):
raise ConfigError(
f'{section}.{option} is required in mode {mode or "manual"}')
if cfg['site_to_site'] and not router:
raise ConfigError('site_to_site: site-to-site connections need router.enabled')
if not _get(cfg, 'vpn', 'enabled'):
for system in ('p2s', 'wireguard', 'openvpn'):
if _get(cfg, system, 'enabled'):
raise ConfigError(f'{system}.enabled needs vpn.enabled')
identities = [('wan.local_fqdn', _get(cfg, 'wan', 'local_fqdn'))]
for name, connection in cfg['site_to_site'].items():
prefix = f'site_to_site.{name}'
@@ -458,7 +481,7 @@ def validate(cfg):
_check_networks(_clean(connection['local_cidrs']), f'{prefix}.local_cidrs')
if connection['auth'] not in CONNECTION_AUTH:
raise ConfigError(f'{prefix}.auth: {connection["auth"]!r} is not one of {", ".join(CONNECTION_AUTH)}')
if connection['remote_ca'].strip() and not vpnrouter_pki.is_ca_certificate(connection['remote_ca']):
if connection['remote_ca'].strip() and not linuxmanager_pki.is_ca_certificate(connection['remote_ca']):
raise ConfigError(f'{prefix}.remote_ca is not the PEM certificate of a CA')
for option, value in identities:
@@ -485,7 +508,7 @@ def validate(cfg):
# In auto mode the names are worked out later, so there is nothing to check.
if mode != 'auto':
for option in ('external', 'internal'):
for option in ('external', 'internal') if router else ('external',):
name = _get(cfg, 'interfaces', option)
if name and not (pathlib.Path('/sys/class/net') / name).exists():
raise ConfigError(
@@ -498,7 +521,7 @@ def validate(cfg):
except ValueError:
raise ConfigError(
f'wireguard.address: {wg_address!r} is not an address with prefix')
# The router's own address on wg0, from which the peers' addresses of the same network are told apart.
# The server's own address on wg0, from which the peers' addresses of the same network are told apart.
network = interface.network
if network.num_addresses > 2 and interface.ip in (network.network_address, network.broadcast_address):
raise ConfigError(
@@ -577,14 +600,17 @@ def validate(cfg):
raise ConfigError('sites.webdav: WebDAV is served by Caddy at https://<wan.local_fqdn>:<sites.webdav_port>, '
'which needs https.enabled and wan.local_fqdn')
_check_address(_get(cfg, 'wan', 'public_address'), 'wan.public_address')
dns = cfg['dns']
if dns['enabled']:
if not ZONE_NAME.match(dns['nameserver'].strip()):
raise ConfigError(f'dns.nameserver: {dns["nameserver"]!r} is not a DNS name in lowercase')
if not _get_list(cfg, 'dns', 'addresses'):
raise ConfigError('dns.enabled needs dns.addresses, the public addresses of the name server and the sites')
for item in _get_list(cfg, 'dns', 'addresses'):
_check_address(item, 'dns.addresses')
if dns['enabled'] and not (ZONE_NAME.match(_get(cfg, 'wan', 'local_fqdn'))
and not re.search(r'\.[0-9]+$', _get(cfg, 'wan', 'local_fqdn'))):
raise ConfigError('dns.enabled needs wan.local_fqdn, a DNS name in lowercase: the primary name server of the '
'domains')
secondary = _get(cfg, 'dns', 'secondary')
if secondary and not ZONE_NAME.match(secondary):
raise ConfigError(f'dns.secondary: {secondary!r} is not a DNS name in lowercase')
if secondary and secondary == _get(cfg, 'wan', 'local_fqdn'):
raise ConfigError('dns.secondary: the secondary name server is not wan.local_fqdn, the primary')
if dns['listen'] not in LISTEN_CHOICES:
raise ConfigError(f'dns.listen: {dns["listen"]!r} is not one of {", ".join(LISTEN_CHOICES)}')
if not DNS_TTL_RANGE[0] <= dns['ttl'] <= DNS_TTL_RANGE[1]:
@@ -656,7 +682,7 @@ def validate(cfg):
raise ConfigError(f'{prefix}.{option}: {item!r} is not one of {", ".join(allowed)}')
for item in template['policies']:
if not OID.match(item):
raise ConfigError(f'{prefix}.policies: {item!r} is not an OID in dotted form, such as 1.3.6.1.4.1.99999.1')
raise ConfigError(f'{prefix}.policies: {item!r} is not an OID in dotted form, such as 1.3.6.1.4.1.32473.1')
if not template['key_usage']:
raise ConfigError(f'{prefix}.key_usage needs at least one usage')
if template['subject'] not in TEMPLATE_SUBJECTS:
@@ -831,7 +857,7 @@ def pki_base_urls(cfg):
def manager_port():
"""The port of the Router Manager: port in its settings, or MANAGER_PORT when it is not set or not a port."""
"""The port of the Manager: port in its settings, or MANAGER_PORT when it is not set or not a port."""
try:
port = json.loads(MANAGER_CONFIG.read_text(encoding='utf-8')).get('port')
except (OSError, ValueError, AttributeError):
@@ -858,16 +884,16 @@ def resolve_psk(connection, name):
def platform_module(cfg):
"""Import the configured platform module, falling back to the default."""
name = _get(cfg, 'general', 'platform') or vpnrouter_platforms.DEFAULT
name = _get(cfg, 'general', 'platform') or linuxmanager_platforms.DEFAULT
if name == PLATFORM_NONE:
# Not normally reached: configure() returns before this when deferred.
name = vpnrouter_platforms.DEFAULT
name = linuxmanager_platforms.DEFAULT
try:
return vpnrouter_platforms.load(name)
return linuxmanager_platforms.load(name)
except ModuleNotFoundError:
warn(f'platform {name!r} is not installed, using '
f'{vpnrouter_platforms.DEFAULT}')
return vpnrouter_platforms.load(vpnrouter_platforms.DEFAULT)
f'{linuxmanager_platforms.DEFAULT}')
return linuxmanager_platforms.load(linuxmanager_platforms.DEFAULT)
def _join(*groups):
@@ -912,20 +938,27 @@ def resolve_interfaces(cfg):
What is read from the system and what is taken verbatim depends entirely on
general.mode. In manual mode nothing below touches the system at all.
Without the router the server has one interface: interfaces.external is also
the internal side, with its first address and no gateway.
"""
mode = _get(cfg, 'general', 'mode') or 'manual'
router = _get(cfg, 'router', 'enabled')
ext_iface = _get(cfg, 'interfaces', 'external')
int_iface = _get(cfg, 'interfaces', 'internal')
int_addr = _get(cfg, 'local', 'int_addr')
gateway = _get(cfg, 'local', 'int_gateway_ip')
if mode == 'auto':
ext_iface, int_iface = detect_interfaces()
ext_iface, int_iface = detect_interfaces(internal=router)
local_addrs = interface_addresses(ext_iface, 'interfaces.external')
if ext_iface and not local_addrs:
warn(f'{ext_iface} has no IPv4 address yet')
if not router:
return ext_iface, ext_iface, local_addrs, local_addrs[0] if local_addrs else '', ''
if mode == 'manual':
return ext_iface, int_iface, local_addrs, int_addr, gateway
@@ -944,7 +977,7 @@ def build_context(cfg, public_ip=None):
site_to_site connections in mode public_ip; setup resolves it once the network is up and
renders again, and without it such a connection is not ready.
"""
extras = vpnrouter_platforms.context(platform_module(cfg))
extras = linuxmanager_platforms.context(platform_module(cfg))
ext_iface, int_iface, local_addrs, int_addr, gateway = resolve_interfaces(cfg)
@@ -976,8 +1009,8 @@ def build_context(cfg, public_ip=None):
'local_id': identities.get(id_mode, ''),
'auth': auth,
'psk': resolve_psk(connection, name) if auth == 'psk' else '',
'remote_cacert': vpnrouter_pki.remote_ca_file(name) if connection['remote_ca'].strip()
else vpnrouter_pki.SWANCTL_CA_FILES['s2s'],
'remote_cacert': linuxmanager_pki.remote_ca_file(name) if connection['remote_ca'].strip()
else linuxmanager_pki.SWANCTL_CA_FILES['s2s'],
}
required = ('remote_addrs', 'remote_id', 'remote_cidrs', 'local_id', *(('psk',) if auth == 'psk' else ()))
entry['ready'] = bool(local_addrs and connection_cidrs and all(entry[key] for key in required))
@@ -986,6 +1019,8 @@ def build_context(cfg, public_ip=None):
return {
'platform': _get(cfg, 'general', 'platform') or PLATFORM_NONE,
'mode': _get(cfg, 'general', 'mode') or 'manual',
'router_enabled': _get(cfg, 'router', 'enabled'),
'vpn_enabled': _get(cfg, 'vpn', 'enabled'),
'wan_iface': ext_iface,
'int_iface': int_iface,
'local_addrs': ', '.join(local_addrs),
@@ -1020,20 +1055,21 @@ def build_context(cfg, public_ip=None):
'openvpn_protocol': _get(cfg, 'openvpn', 'protocol'),
'openvpn_device': OPENVPN_DEVICE,
'openvpn_ldap_conf': str(OPENVPN_LDAP),
'openvpn_status': vpnrouter_status.OPENVPN_STATUS,
'openvpn_status_seconds': vpnrouter_status.OPENVPN_STATUS_SECONDS,
'openvpn_status': linuxmanager_status.OPENVPN_STATUS,
'openvpn_status_seconds': linuxmanager_status.OPENVPN_STATUS_SECONDS,
'p2s_dns_list': _split_list(_join(resolver_addrs, _split_list(extras['EXTRA_P2S_DNS']))),
'openvpn_ca': str(vpnrouter_pki.OPENVPN_CA),
'openvpn_crl': str(vpnrouter_pki.OPENVPN_CRL),
'server_cert': str(vpnrouter_pki.VPN_SERVER_CERT),
'server_key': str(vpnrouter_pki.VPN_SERVER_KEY),
'openvpn_ca': str(linuxmanager_pki.OPENVPN_CA),
'openvpn_crl': str(linuxmanager_pki.OPENVPN_CRL),
'server_cert': str(linuxmanager_pki.VPN_SERVER_CERT),
'server_key': str(linuxmanager_pki.VPN_SERVER_KEY),
'server_certificate': False,
'ldap_uri': vpnrouter_ldap.URI,
'ldap_users': vpnrouter_ldap.USERS,
'ldap_uri': linuxmanager_ldap.URI,
'ldap_users': linuxmanager_ldap.USERS,
'web_enabled': _get(cfg, 'web', 'enabled'),
'web_port': _get(cfg, 'web', 'port'),
'https_enabled': _get(cfg, 'https', 'enabled'),
'https_email': _get(cfg, 'https', 'email'),
'https_acme_ca': ACME_STAGING_CA if _get(cfg, 'https', 'acme_staging') else '',
'https_web_port': _get(cfg, 'https', 'web_port'),
'https_manager_port': _get(cfg, 'https', 'manager_port'),
'pki_publish': _get_list(cfg, 'pki', 'publish'),
@@ -1052,28 +1088,38 @@ def build_context(cfg, public_ip=None):
def s2s_ready(ctx):
"""True when any site_to_site connection is ready."""
return any(connection['ready'] for connection in ctx['connections'])
"""True when the router is on and any site_to_site connection is ready."""
return bool(ctx['router_enabled'] and any(connection['ready'] for connection in ctx['connections']))
def routing_ready(ctx):
"""Routing and NAT need both interfaces named, plus the subnets involved."""
return all([ctx['local_cidrs'], ctx['int_gateway_ip'],
ctx['int_iface'], ctx['wan_iface']])
"""Routing and NAT need the router on, both interfaces named, plus the subnets involved."""
return bool(ctx['router_enabled'] and all([ctx['local_cidrs'], ctx['int_gateway_ip'],
ctx['int_iface'], ctx['wan_iface']]))
def p2s_ready(ctx):
"""IKEv2 access needs its settings and the router's VPN server certificate, which the P2S CA issues."""
return bool(ctx['p2s_enabled'] and ctx['p2s_address_pool'] and ctx['local_fqdn'] and ctx['server_certificate'])
"""IKEv2 access needs the VPN on, its settings and the server's VPN server certificate, which the P2S CA issues."""
return bool(ctx['vpn_enabled'] and ctx['p2s_enabled'] and ctx['p2s_address_pool'] and ctx['local_fqdn']
and ctx['server_certificate'])
def wg_ready(ctx):
return bool(ctx['wg_enabled'] and ctx['wg_address'])
"""WireGuard needs the VPN on and its settings."""
return bool(ctx['vpn_enabled'] and ctx['wg_enabled'] and ctx['wg_address'])
def openvpn_ready(ctx):
"""OpenVPN needs its settings and the router's VPN server certificate, which the P2S CA issues."""
return bool(ctx['openvpn_enabled'] and ctx['openvpn_pool'] and ctx['local_fqdn'] and ctx['server_certificate'])
"""OpenVPN needs the VPN on, its settings and the server's VPN server certificate, which the P2S CA issues."""
return bool(ctx['vpn_enabled'] and ctx['openvpn_enabled'] and ctx['openvpn_pool'] and ctx['local_fqdn']
and ctx['server_certificate'])
def vpn_pools(ctx):
"""The address pools of the ready remote access systems: IKEv2, OpenVPN and WireGuard."""
return [pool for ready, pool in ((p2s_ready(ctx), ctx['p2s_address_pool']),
(openvpn_ready(ctx), ctx['openvpn_pool']),
(wg_ready(ctx), ctx['wg_network'])) if ready]
def https_ready(ctx):
@@ -1087,7 +1133,7 @@ def metrics_ready(ctx):
def webdav_ready(ctx):
"""WebDAV of the sites' folders is served by Caddy, on the router's own name, while it is on and HTTPS is ready."""
"""WebDAV of the sites' folders is served by Caddy, on the server's own name, while it is on and HTTPS is ready."""
return bool(ctx['webdav_enabled'] and https_ready(ctx))
@@ -1208,12 +1254,12 @@ def configure(cfg=None, public_ip=None):
changed = set()
# Without the directory, which holds the PKI, the copies of the last run stay and what needs none is configured.
with vpnrouter_pki.locked():
with linuxmanager_pki.locked():
try:
notices = vpnrouter_pki.provision(cfg, ctx)
if vpnrouter_pki.distribute(cfg):
notices = linuxmanager_pki.provision(cfg, ctx)
if linuxmanager_pki.distribute(cfg):
changed.add('swanctl')
except vpnrouter_pki.Failure as failure:
except linuxmanager_pki.Failure as failure:
fail(str(failure))
except ldap.LDAPError as exc:
notices = [f'the PKI is left as it is: the directory cannot be reached: {exc}']
@@ -1223,14 +1269,14 @@ def configure(cfg=None, public_ip=None):
ctx['wg_peers'] = None
if wg_ready(ctx):
try:
ctx['wg_peers'] = vpnrouter_ldap.wireguard_peers()
ctx['wg_peers'] = linuxmanager_ldap.wireguard_peers()
except ldap.LDAPError as exc:
warn(f'the WireGuard peers of the users are left as they are: the directory cannot be reached: {exc}')
# What needs the router's own certificates.
ctx['server_certificate'] = vpnrouter_pki.VPN_SERVER_CERT.is_file()
# What needs the server's own certificates.
ctx['server_certificate'] = linuxmanager_pki.VPN_SERVER_CERT.is_file()
for connection in ctx['connections']:
if connection['auth'] == 'certificate':
connection['ready'] = connection['ready'] and vpnrouter_pki.S2S_CERT.is_file()
connection['ready'] = connection['ready'] and linuxmanager_pki.S2S_CERT.is_file()
env = _jinja_env()
@@ -1247,7 +1293,7 @@ def configure(cfg=None, public_ip=None):
changed.add('swanctl')
# The resolver of the remote access clients is dnsmasq on the addresses of p2s.dns_listen; systemd-resolved keeps
# its loopback stub only, so its records of the router's own name stay on the router.
# its loopback stub only, so its records of the server's own name stay on the server.
if remove(P2S_FORWARDER):
changed.add('resolved')
if (p2s_ready(ctx) or openvpn_ready(ctx) or wg_ready(ctx)) and ctx['p2s_resolver_addrs']:
@@ -1,16 +1,16 @@
"""The tokens of the router's web service, for the web service and vpn-router-token.
"""The tokens of the server's web service, for the web service and linux-manager-token.
Tokens are HS256 JWTs signed with WEB_KEY, a key the web service makes on its first run on the
router and that never leaves it. /auth issues an access token naming the user in sub, valid ACCESS_SECONDS, and
a refresh token valid REFRESH_SECONDS; vpn-router-token issues tokens with the admin claim and
a refresh token valid REFRESH_SECONDS; linux-manager-token issues tokens with the admin claim and
any lifetime. A token without the admin claim must expire within MAX_LIFETIME seconds.
Every failure of the database raises DatabaseError, the only database exception callers see.
Refresh tokens are random strings kept in PostgreSQL as their SHA-256, in the database of
web.database, by default the vpn-router database on this host that provision() creates on the
web.database, by default the linux-manager database on this host that provision() creates on the
web service's first run, with the password in DATABASE_SECRET. A refresh token is used once: refreshing replaces it.
The same database holds the state of the OpenID Provider (vpnrouter_oidc), whose tables and procedures OIDC_SQL
The same database holds the state of the OpenID Provider (linuxmanager_oidc), whose tables and procedures OIDC_SQL
defines; the first connection of a process loads it. revoke() removes a user's refresh tokens and OpenID Provider
codes, tokens and sessions.
"""
@@ -27,14 +27,14 @@ import time
import psycopg
WEB_KEY = pathlib.Path('/etc/vpn-router/web.key')
DATABASE_SECRET = pathlib.Path('/etc/vpn-router/database.secret')
OIDC_SQL = pathlib.Path('/usr/share/vpn-router/sql/oidc.sql')
WEB_KEY = pathlib.Path('/etc/linux-manager/web.key')
DATABASE_SECRET = pathlib.Path('/etc/linux-manager/database.secret')
OIDC_SQL = pathlib.Path('/usr/share/linux-manager/sql/oidc.sql')
#: The role and the database provision() creates when web.database is not set.
DATABASE = 'vpn-router'
DATABASE = 'linux-manager'
#: The user operations of vpnrouter_users after which the user's refresh tokens are revoked.
#: The user operations of linuxmanager_users after which the user's refresh tokens are revoked.
REVOKING = ('passwd', 'disable', 'unlock', 'remove')
#: Seconds an access token of /auth and a refresh token are valid.
@@ -122,7 +122,7 @@ def admin_token(seconds):
"""A token with the admin claim, valid for the seconds given."""
secret = key()
if not secret:
raise OSError(f'{WEB_KEY} does not exist; vpn-router-web makes it when it starts')
raise OSError(f'{WEB_KEY} does not exist; linux-manager-web makes it when it starts')
now = int(time.time())
return {'token': sign({'admin': True, 'iat': now, 'exp': now + seconds}, secret), 'expires': now + seconds}
@@ -1,9 +1,9 @@
"""The backups of the router, for vpn-router-backup and the web service.
"""The backups of the server, for linux-manager-backup and the web service.
A backup is a consistent copy of the state that setup cannot make again, staged in STAGING: the
trees of TREES, the folders of the web sites among them, the files of site_to_site psk_file outside them under files/,
the directory as slapcat writes it (ldap/vpn-router.ldif), with backup.metrics the database as pg_dump writes it
(postgresql/vpn-router.dump), the databases of the web sites and of the name server (DUMPS) once they exist, and
the directory as slapcat writes it (ldap/linux-manager.ldif), with backup.metrics the database as pg_dump writes it
(postgresql/linux-manager.dump), the databases of the web sites and of the name server (DUMPS) once they exist, and
manifest.json. The trees and the directory are copied while
the configuration lock and the users lock are held, so the configuration, the PKI and the users
match; pg_dump reads a snapshot of its own.
@@ -32,39 +32,38 @@ import subprocess
import tempfile
import time
import vpnrouter
import vpnrouter_auth
import vpnrouter_dns
import vpnrouter_ldap
import vpnrouter_sites
import vpnrouter_users
import linuxmanager
import linuxmanager_auth
import linuxmanager_dns
import linuxmanager_ldap
import linuxmanager_sites
import linuxmanager_users
STATE = pathlib.Path('/var/lib/vpn-router/backup.json')
WORK_DIR = pathlib.Path('/var/lib/vpn-router/backup')
STATE = pathlib.Path('/var/lib/linux-manager/backup.json')
WORK_DIR = pathlib.Path('/var/lib/linux-manager/backup')
STAGING = WORK_DIR / 'staging'
RESTORE_DIR = WORK_DIR / 'restore'
#: What a restore replaces is kept here, in a directory named after the time of the restore.
BEFORE_RESTORE = WORK_DIR / 'before-restore'
RUN_LOCK = pathlib.Path('/run/vpn-router/backup.lock')
RCLONE_CONFIG = pathlib.Path('/etc/vpn-router/rclone.conf')
RUN_LOCK = pathlib.Path('/run/linux-manager/backup.lock')
RCLONE_CONFIG = pathlib.Path('/etc/linux-manager/rclone.conf')
PBS_CLIENT = pathlib.Path('/usr/bin/proxmox-backup-client')
UNIT = 'vpn-router-backup.service'
TIMER = 'vpn-router-backup.timer'
UNIT = 'linux-manager-backup.service'
TIMER = 'linux-manager-backup.timer'
#: The state a backup copies: directories and files, each where it is on the system.
TREES = ('/etc/vpn-router', '/etc/wireguard', '/etc/ufw', '/etc/default/ufw', '/etc/vpn-router-manager',
str(vpnrouter_sites.SITES_DIR))
TREES = ('/etc/linux-manager', '/etc/wireguard', '/etc/ufw', '/etc/default/ufw', str(linuxmanager_sites.SITES_DIR))
#: The databases a backup always dumps, each while it exists: the dump -> the database and its password file.
DUMPS = {'postgresql/vpn-router-sites.dump': (vpnrouter_sites.DATABASE, vpnrouter_sites.DATABASE_SECRET),
'postgresql/vpn-router-dns.dump': (vpnrouter_dns.DATABASE, vpnrouter_dns.DATABASE_SECRET)}
DUMPS = {'postgresql/linux-manager-sites.dump': (linuxmanager_sites.DATABASE, linuxmanager_sites.DATABASE_SECRET),
'postgresql/linux-manager-dns.dump': (linuxmanager_dns.DATABASE, linuxmanager_dns.DATABASE_SECRET)}
#: The name of a local archive: the host and the UTC time of the backup.
ARCHIVE = re.compile(r'^vpn-router-(?P<host>[A-Za-z0-9][A-Za-z0-9.-]*)-(?P<stamp>\d{8}T\d{6}Z)\.tar\.zst$')
ARCHIVE = re.compile(r'^linux-manager-(?P<host>[A-Za-z0-9][A-Za-z0-9.-]*)-(?P<stamp>\d{8}T\d{6}Z)\.tar\.zst$')
STAMP = '%Y%m%dT%H%M%SZ'
LDIF = 'ldap/vpn-router.ldif'
DUMP = 'postgresql/vpn-router.dump'
LDIF = 'ldap/linux-manager.ldif'
DUMP = 'postgresql/linux-manager.dump'
PBS_ARCHIVE = 'router.pxar'
#: The period of each retention setting, as strftime formats of local time; weeks are ISO weeks.
@@ -72,10 +71,10 @@ PERIODS = {'keep_hourly': '%Y-%m-%d %H', 'keep_daily': '%Y-%m-%d', 'keep_weekly'
'keep_yearly': '%Y'}
#: The units a restore stops while it replaces the directory and the database, and the ones it restarts after setup.
RESTORE_STOPS = ('vpn-router-web.service', 'vpn-router-metrics.service', 'vpn-router-webdav.service', 'pdns.service',
RESTORE_STOPS = ('linux-manager-web.service', 'linux-manager-metrics.service', 'linux-manager-webdav.service', 'pdns.service',
'slapd.service')
RESTORE_RESTARTS = ('wg-quick@wg0.service', 'openvpn-server@vpn-router.service', 'strongswan.service',
'vpn-router-manager.service')
RESTORE_RESTARTS = ('wg-quick@wg0.service', 'openvpn-server@linux-manager.service', 'strongswan.service',
'linux-manager-ui.service')
class Failure(Exception):
@@ -209,11 +208,11 @@ def psk_files(cfg):
def database_args(cfg):
"""The command prefix and the database argument of pg_dump and pg_restore: postgres and vpn-router, or web.database."""
"""The command prefix and the database argument of pg_dump and pg_restore: postgres and linux-manager, or web.database."""
configured = cfg['web']['database'].strip()
if configured:
return [], [f'--dbname={configured}']
return ['runuser', '-u', 'postgres', '--'], [f'--dbname={vpnrouter_auth.DATABASE}']
return ['runuser', '-u', 'postgres', '--'], [f'--dbname={linuxmanager_auth.DATABASE}']
def stage(cfg):
@@ -222,15 +221,15 @@ def stage(cfg):
STAGING.mkdir(parents=True, mode=0o700)
trees = [tree for tree in TREES if os.path.exists(tree)]
files = psk_files(cfg)
vpnrouter_users.LOCK_FILE.parent.mkdir(parents=True, exist_ok=True)
with vpnrouter.config_lock(), open(vpnrouter_users.LOCK_FILE, 'w') as users:
linuxmanager_users.LOCK_FILE.parent.mkdir(parents=True, exist_ok=True)
with linuxmanager.config_lock(), open(linuxmanager_users.LOCK_FILE, 'w') as users:
fcntl.flock(users, fcntl.LOCK_EX)
command(['cp', '-a', '--parents', *trees, str(STAGING)])
if files:
(STAGING / 'files').mkdir()
command(['cp', '-a', '--parents', *files, str(STAGING / 'files')])
(STAGING / LDIF).parent.mkdir()
command(['slapcat', '-b', vpnrouter_ldap.BASE, '-l', str(STAGING / LDIF)])
command(['slapcat', '-b', linuxmanager_ldap.BASE, '-l', str(STAGING / LDIF)])
items = trees + [f'files{name}' for name in files] + [LDIF]
dumps = {**({DUMP: database_args(cfg)} if cfg['backup']['metrics'] else {}),
**{item: (['runuser', '-u', 'postgres', '--'], [f'--dbname={name}'])
@@ -242,7 +241,7 @@ def stage(cfg):
if result.returncode != 0:
raise Failure(f'pg_dump for {item} failed: {result.stderr.decode(errors="replace").strip()}')
items.append(item)
version = subprocess.run(['dpkg-query', '-W', '-f=${Version}', 'vpn-router'], capture_output=True, text=True)
version = subprocess.run(['dpkg-query', '-W', '-f=${Version}', 'linux-manager'], capture_output=True, text=True)
manifest = {'host': host(), 'time': now(), 'version': version.stdout.strip() or None,
'schedule': cfg['backup']['schedule'], 'items': items}
(STAGING / 'manifest.json').write_text(json.dumps(manifest, indent=2) + '\n', encoding='utf-8')
@@ -279,12 +278,12 @@ def cloud(cfg):
if not RCLONE_CONFIG.is_file():
raise Failure(f'{RCLONE_CONFIG} does not exist; make the remote with rclone config --config {RCLONE_CONFIG}')
command([rclone, backup['rclone_mode'], backup['path'], backup['rclone_remote'], '--config', str(RCLONE_CONFIG),
'--include', f'vpn-router-{host()}-*.tar.zst'])
'--include', f'linux-manager-{host()}-*.tar.zst'])
def run():
"""Make a backup to backup.target as the configuration says, recording it in STATE; the recorded state."""
cfg = vpnrouter.load_config()
cfg = linuxmanager.load_config()
backup = cfg['backup']
started = time.time()
record = {'result': 'running', 'started': now(), 'finished': None, 'target': backup['target'], 'archive': None,
@@ -304,7 +303,7 @@ def run():
else:
path = pathlib.Path(backup['path'])
path.mkdir(parents=True, mode=0o700, exist_ok=True)
name = f'vpn-router-{host()}-{time.strftime(STAMP, time.gmtime(started))}.tar.zst'
name = f'linux-manager-{host()}-{time.strftime(STAMP, time.gmtime(started))}.tar.zst'
partial = path / f'{name}.partial'
command(['tar', '--zstd', '-cf', str(partial), '-C', str(STAGING), '.'])
partial.chmod(0o600)
@@ -312,7 +311,7 @@ def run():
record['archive'] = name
record['size'] = (path / name).stat().st_size
found = archives(path)
keep = kept(found, {option: backup[option] for option in vpnrouter.backup_keeps(backup['schedule'])})
keep = kept(found, {option: backup[option] for option in linuxmanager.backup_keeps(backup['schedule'])})
for item in found:
if item['name'] not in keep:
(path / item['name']).unlink()
@@ -354,7 +353,7 @@ def extract(cfg, source, pbs):
try:
return json.loads((RESTORE_DIR / 'manifest.json').read_text(encoding='utf-8'))
except (OSError, ValueError):
raise Failure(f'{source} is not a backup of vpn-router-backup: it has no manifest.json')
raise Failure(f'{source} is not a backup of linux-manager-backup: it has no manifest.json')
def put_back(relative, target, saved):
@@ -375,7 +374,7 @@ def restore(source, pbs, confirm):
are replaced, and setup applies the configuration afterwards. Returns the directory of what was replaced, or None
when confirm declined.
"""
cfg = vpnrouter.load_config()
cfg = linuxmanager.load_config()
with run_lock():
try:
manifest = extract(cfg, source, pbs)
@@ -391,22 +390,22 @@ def restore(source, pbs, confirm):
elif item.startswith('files/'):
put_back(item, item[len('files'):], saved)
if LDIF in manifest['items']:
directory = vpnrouter_ldap.DB_DIRECTORY
directory = linuxmanager_ldap.DB_DIRECTORY
if directory.exists():
shutil.move(directory, saved / 'ldap')
directory.mkdir(mode=0o700)
command(['slapadd', '-b', vpnrouter_ldap.BASE, '-l', str(RESTORE_DIR / LDIF)])
command(['slapadd', '-b', linuxmanager_ldap.BASE, '-l', str(RESTORE_DIR / LDIF)])
command(['chown', '-R', 'openldap:openldap', str(directory)])
subprocess.run(['systemctl', 'start', 'slapd.service'], capture_output=True)
restored = vpnrouter.load_config()
restored = linuxmanager.load_config()
if not restored['web']['database'].strip():
vpnrouter_auth.reset_password(vpnrouter_auth.DATABASE, vpnrouter_auth.DATABASE_SECRET)
vpnrouter_auth.provision(restored)
linuxmanager_auth.reset_password(linuxmanager_auth.DATABASE, linuxmanager_auth.DATABASE_SECRET)
linuxmanager_auth.provision(restored)
dumps = {DUMP: database_args(restored)}
for item, (name, secret) in DUMPS.items():
if item in manifest['items']:
vpnrouter_auth.reset_password(name, secret)
vpnrouter_auth.provision_database(name, secret)
linuxmanager_auth.reset_password(name, secret)
linuxmanager_auth.provision_database(name, secret)
dumps[item] = (['runuser', '-u', 'postgres', '--'], [f'--dbname={name}'])
for item, (prefix, database) in dumps.items():
if item not in manifest['items']:
@@ -416,8 +415,8 @@ def restore(source, pbs, confirm):
stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
if result.returncode != 0:
raise Failure(f'pg_restore of {item} failed: {result.stderr.decode(errors="replace").strip()}')
subprocess.run(['systemctl', 'start', 'vpn-router-web.service'], capture_output=True)
command(['systemctl', 'restart', 'vpn-router-setup.service'])
subprocess.run(['systemctl', 'start', 'linux-manager-web.service'], capture_output=True)
command(['systemctl', 'restart', 'linux-manager-setup.service'])
for unit in RESTORE_RESTARTS:
subprocess.run(['systemctl', 'try-restart', unit], capture_output=True)
subprocess.run(['ufw', 'reload'], capture_output=True)
@@ -1,17 +1,17 @@
"""The router's authoritative name server, PowerDNS with its gpgsql backend, for vpn-router-setup and the web service.
"""The server's authoritative name server, PowerDNS with its gpgsql backend, for linux-manager-setup and the web service.
The name server is the primary of its domains and answers on the addresses of the interfaces of dns.listen. The
domains, their secondary name servers and the records an administrator added (manual records) are in the
vpn-router-dns database, which provision() creates with its role and the password in DATABASE_SECRET, and fills with
the schema of pdns-backend-pgsql, PDNS_SCHEMA; DNS_SQL defines the router's own tables and procedures, and the first
connection of a process loads it.
domains and the records an administrator added (manual records) are in the linux-manager-dns database, which
provision() creates with its role and the password in DATABASE_SECRET, and fills with the schema of
pdns-backend-pgsql, PDNS_SCHEMA; DNS_SQL defines the server's own tables and procedures, and the first connection of a
process loads it.
apply() writes every domain as a primary zone: its SOA with dns.nameserver, its NS, dns.nameserver and the name
servers of its secondaries, the NS of the domains directly below it, A and AAAA records of dns.addresses for its apex,
for those name servers inside it that are the router, and, when the domain has site records, for every host name of an
enabled site whose closest domain it is, and the manual records, but those of a name and type the records before them
have. A secondary's address is notified of every change (ALSO-NOTIFY) and may transfer the zone (ALLOW-AXFR-FROM); the
serial of the SOA grows only when the records change.
apply() writes every domain as a primary zone: its SOA with wan.local_fqdn as the primary name server, its NS,
wan.local_fqdn and dns.secondary, the NS of the domains directly below it, an A or AAAA record for every host name of
an enabled site with dns router whose closest domain it is, with the server's address the site chooses, and the manual
records, but those of a name and type the records before them have. wan.local_fqdn gets no record in the zones: its
record is kept where its own domain is. The addresses dns.secondary resolves to are notified of every change
(ALSO-NOTIFY) and may transfer the zones (ALLOW-AXFR-FROM); the serial of the SOA grows only when the records change.
Every failure of the database raises DatabaseError; a request that cannot be done raises Failure.
"""
@@ -20,23 +20,24 @@ import contextlib
import ipaddress
import pathlib
import re
import socket
import psycopg
from psycopg.rows import dict_row
import vpnrouter
import vpnrouter_auth
import linuxmanager
import linuxmanager_auth
DATABASE = 'vpn-router-dns'
DATABASE_SECRET = pathlib.Path('/etc/vpn-router/dns-database.secret')
DNS_SQL = pathlib.Path('/usr/share/vpn-router/sql/dns.sql')
DATABASE = 'linux-manager-dns'
DATABASE_SECRET = pathlib.Path('/etc/linux-manager/dns-database.secret')
DNS_SQL = pathlib.Path('/usr/share/linux-manager/sql/dns.sql')
#: The schema of PowerDNS's gpgsql backend, which pdns-backend-pgsql installs.
PDNS_SCHEMA = pathlib.Path('/usr/share/pdns-backend-pgsql/schema/schema.pgsql.sql')
#: The settings of PowerDNS that vpn-router-setup writes, read after /etc/powerdns/pdns.conf; the server, its unit and
#: The settings of PowerDNS that linux-manager-setup writes, read after /etc/powerdns/pdns.conf; the server, its unit and
#: the group it runs as.
PDNS_CONF = pathlib.Path('/etc/powerdns/pdns.d/vpn-router.conf')
PDNS_CONF = pathlib.Path('/etc/powerdns/pdns.d/linux-manager.conf')
PDNS_SERVER = pathlib.Path('/usr/sbin/pdns_server')
UNIT = 'pdns.service'
GROUP = 'pdns'
@@ -44,9 +45,8 @@ GROUP = 'pdns'
#: The types of the manual records.
TYPES = ('A', 'AAAA', 'CNAME', 'TXT', 'MX', 'SRV', 'CAA')
#: The most domains, secondaries of a domain, and manual records of a domain.
#: The most domains, and manual records of a domain.
DOMAINS_MAX = 100
SECONDARIES_MAX = 8
MANUAL_MAX = 1000
#: The refresh, retry and expire times of the SOA, in seconds.
@@ -73,8 +73,7 @@ class DatabaseError(Exception):
class Failure(Exception):
"""A request that cannot be done; status is 2 invalid, 3 no such domain, secondary or record, 4 the domain
exists."""
"""A request that cannot be done; status is 2 invalid, 3 no such domain or record, 4 the domain exists."""
def __init__(self, message, status=2):
super().__init__(message)
@@ -84,9 +83,9 @@ class Failure(Exception):
def provision():
"""Create the role, its password file and the database of the name server when they are missing, and PowerDNS's
tables in it; DatabaseError while pdns-backend-pgsql is not installed."""
vpnrouter_auth.provision_database(DATABASE, DATABASE_SECRET)
linuxmanager_auth.provision_database(DATABASE, DATABASE_SECRET)
try:
with psycopg.connect(vpnrouter_auth.local_connection(DATABASE, DATABASE_SECRET), autocommit=True) as connection:
with psycopg.connect(linuxmanager_auth.local_connection(DATABASE, DATABASE_SECRET), autocommit=True) as connection:
if connection.execute("SELECT to_regclass('public.domains')").fetchone()[0] is None:
connection.execute(PDNS_SCHEMA.read_text(encoding='utf-8'))
except OSError as exc:
@@ -97,12 +96,12 @@ def provision():
@contextlib.contextmanager
def database():
"""A connection to the database of the name server, with the router's tables and procedures, closed at the end;
"""A connection to the database of the name server, with the server's tables and procedures, closed at the end;
DatabaseError on any failure, also before provision() made it."""
global _loaded
try:
schema = None if _loaded else DNS_SQL.read_text(encoding='utf-8')
target = vpnrouter_auth.local_connection(DATABASE, DATABASE_SECRET)
target = linuxmanager_auth.local_connection(DATABASE, DATABASE_SECRET)
except OSError as exc:
raise DatabaseError(f'the database of the name server is not ready: {exc}') from exc
try:
@@ -189,15 +188,15 @@ def closest(zones, name):
def check_domain(name):
"""The domain name in lowercase without a final dot; Failure when it is not a DNS name."""
if not isinstance(name, str) or not vpnrouter.ZONE_NAME.match(name.strip().lower().rstrip('.')):
if not isinstance(name, str) or not linuxmanager.ZONE_NAME.match(name.strip().lower().rstrip('.')):
raise Failure(f'{name!r} is not a domain name')
return name.strip().lower().rstrip('.')
def check_ttl(ttl):
if ttl is not None and (isinstance(ttl, bool) or not isinstance(ttl, int) or
not vpnrouter.DNS_TTL_RANGE[0] <= ttl <= vpnrouter.DNS_TTL_RANGE[1]):
raise Failure(f'ttl must be {vpnrouter.DNS_TTL_RANGE[0]} to {vpnrouter.DNS_TTL_RANGE[1]} seconds')
not linuxmanager.DNS_TTL_RANGE[0] <= ttl <= linuxmanager.DNS_TTL_RANGE[1]):
raise Failure(f'ttl must be {linuxmanager.DNS_TTL_RANGE[0]} to {linuxmanager.DNS_TTL_RANGE[1]} seconds')
return ttl
@@ -211,13 +210,10 @@ def domain(connection, name):
def listing():
"""Every domain with its secondaries, its manual records and the records the name server answers for it."""
"""Every domain with its manual records and the records the name server answers for it."""
with database() as connection:
secondaries = connection.execute('SELECT * FROM dns_secondaries_list()').fetchall()
manual = connection.execute('SELECT * FROM dns_records_list()').fetchall()
return [{**row,
'secondaries': [{key: item[key] for key in ('nameserver', 'address')}
for item in secondaries if item['domain'] == row['name']],
'manual': [{key: item[key] for key in ('name', 'type', 'content', 'ttl')}
for item in manual if item['domain'] == row['name']],
'records': connection.execute('SELECT * FROM dns_served(%s)', (row['name'],)).fetchall()}
@@ -225,71 +221,52 @@ def listing():
def domain_add(body):
"""Add the domain {name, sites, ttl?}; the domain."""
if not isinstance(body, dict) or not isinstance(body.get('sites', False), bool):
raise Failure('a domain is an object of name, sites and ttl')
name, sites, ttl = check_domain(body.get('name')), body.get('sites', False), check_ttl(body.get('ttl'))
"""Add the domain {name, ttl?}; the domain."""
if not isinstance(body, dict) or not set(body) <= {'name', 'ttl'}:
raise Failure('a domain is an object of name and ttl')
name, ttl = check_domain(body.get('name')), check_ttl(body.get('ttl'))
with database() as connection:
names = [row['name'] for row in connection.execute('SELECT * FROM dns_domains_list()').fetchall()]
if name in names:
raise Failure(f'the domain {name} exists', 4)
if len(names) >= DOMAINS_MAX:
raise Failure(f'the name server has {DOMAINS_MAX} domains')
connection.execute('CALL dns_domain_put(%s, %s, %s)', (name, sites, ttl))
return {'name': name, 'sites': sites, 'ttl': ttl}
connection.execute('CALL dns_domain_put(%s, %s)', (name, ttl))
return {'name': name, 'ttl': ttl}
def domain_change(name, body):
"""Change the sites and ttl of the domain; the domain."""
if not isinstance(body, dict) or not set(body) <= {'sites', 'ttl'} or \
not isinstance(body.get('sites', False), bool):
raise Failure('a change of a domain is an object of sites and ttl')
"""Change the ttl of the domain; the domain."""
if not isinstance(body, dict) or not set(body) <= {'ttl'}:
raise Failure('a change of a domain is an object of ttl')
with database() as connection:
row = domain(connection, name)
sites, ttl = body.get('sites', row['sites']), check_ttl(body.get('ttl', row['ttl']))
connection.execute('CALL dns_domain_put(%s, %s, %s)', (name, sites, ttl))
return {'name': name, 'sites': sites, 'ttl': ttl}
ttl = check_ttl(body.get('ttl', row['ttl']))
connection.execute('CALL dns_domain_put(%s, %s)', (name, ttl))
return {'name': name, 'ttl': ttl}
def domain_remove(name):
"""Remove the domain with its secondaries and manual records."""
"""Remove the domain with its manual records."""
with database() as connection:
domain(connection, name)
connection.execute('CALL dns_domain_remove(%s)', (name,))
return {'name': name}
def secondary_add(name, body):
"""Add the secondary name server {nameserver, address} of the domain, or set its address; the secondary."""
if not isinstance(body, dict):
raise Failure('a secondary is an object of nameserver and address')
nameserver = body.get('nameserver')
if not isinstance(nameserver, str) or not vpnrouter.ZONE_NAME.match(nameserver.strip().lower().rstrip('.')):
raise Failure(f'{nameserver!r} is not a name server name')
nameserver = nameserver.strip().lower().rstrip('.')
def secondary_addresses(cfg):
"""The addresses dns.secondary resolves to, which are notified of the zones' changes and may transfer them; none,
with a warning, while it is not set or does not resolve."""
secondary = cfg['dns']['secondary'].strip()
if not secondary:
linuxmanager.warn('dns.secondary is not set: the domains have no secondary name server')
return []
try:
address = str(ipaddress.ip_address(str(body.get('address', '')).strip()))
except ValueError as exc:
raise Failure(f'{body.get("address")!r} is not an IP address') from exc
with database() as connection:
domain(connection, name)
others = [row for row in connection.execute('SELECT * FROM dns_secondaries_list()').fetchall()
if row['domain'] == name and row['nameserver'] != nameserver]
if len(others) >= SECONDARIES_MAX:
raise Failure(f'the domain {name} has {SECONDARIES_MAX} secondaries')
connection.execute('CALL dns_secondary_put(%s, %s, %s)', (name, nameserver, address))
return {'domain': name, 'nameserver': nameserver, 'address': address}
def secondary_remove(name, nameserver):
"""Remove the secondary name server of the domain."""
with database() as connection:
domain(connection, name)
if not any(row['domain'] == name and row['nameserver'] == nameserver
for row in connection.execute('SELECT * FROM dns_secondaries_list()').fetchall()):
raise Failure(f'no secondary {nameserver} of {name}', 3)
connection.execute('CALL dns_secondary_remove(%s, %s)', (name, nameserver))
return {'domain': name, 'nameserver': nameserver}
found = socket.getaddrinfo(secondary, 53, proto=socket.IPPROTO_UDP)
except OSError as exc:
linuxmanager.warn(f'dns.secondary {secondary} does not resolve, so it may not transfer the domains: {exc}')
return []
return list(dict.fromkeys(str(ipaddress.ip_address(item[4][0])) for item in found))
def add(cfg, name, record):
@@ -332,25 +309,20 @@ def remove(name, record_name, kind, content):
return {'name': full, 'type': kind, 'removed': len(found)}
def records(cfg, row, zones, secondaries, hosts, manual_records):
def records(cfg, row, zones, hosts, manual_records):
"""The records of the domain row besides its SOA, as (name, type, content, ttl): the NS of the domain and of the
domains directly below it, the A and AAAA records of dns.addresses for its apex, dns.nameserver when it is inside
and, with site records, each host name whose closest domain it is, the glue of the secondaries inside it, and its
manual records. secondaries maps a domain to its secondaries."""
domains directly below it, wan.local_fqdn and dns.secondary, the A or AAAA record of each host name of hosts, pairs
of a host name and an address, whose closest domain it is, and its manual records."""
dns = cfg['dns']
zone, nameserver, ttl = row['name'], dns['nameserver'].strip(), row['ttl'] or dns['ttl']
addresses = [ipaddress.ip_address(item.strip()) for item in dns['addresses'] if item.strip()]
zone, ttl = row['name'], row['ttl'] or dns['ttl']
servers = [cfg['wan']['local_fqdn'].strip(), *([dns['secondary'].strip()] if dns['secondary'].strip() else [])]
children = [name for name in zones
if name != zone and closest([item for item in zones if item != name], name) == zone]
found = [(name, 'NS', server, ttl) for name in [zone, *children]
for server in [nameserver, *(item['nameserver'] for item in secondaries.get(name, []))]]
for name in dict.fromkeys([zone, *([nameserver] if in_zone(zone, nameserver) else []),
*(host for host in hosts if row['sites'] and closest(zones, host) == zone)]):
found += [(name, 'A' if address.version == 4 else 'AAAA', str(address), ttl) for address in addresses]
for item in (item for name in [zone, *children] for item in secondaries.get(name, [])):
if in_zone(zone, item['nameserver']) and item['nameserver'] != nameserver:
address = ipaddress.ip_address(item['address'])
found.append((item['nameserver'], 'A' if address.version == 4 else 'AAAA', str(address), ttl))
found = [(name, 'NS', server, ttl) for name in [zone, *children] for server in servers]
for host, address in hosts:
if closest(zones, host) == zone:
address = ipaddress.ip_address(address)
found.append((host, 'A' if address.version == 4 else 'AAAA', str(address), ttl))
taken = {(name, kind) for name, kind, _, _ in found}
found += [(item['name'], item['type'], item['content'], item['ttl']) for item in manual_records
if (item['name'], item['type']) not in taken]
@@ -358,7 +330,8 @@ def records(cfg, row, zones, secondaries, hosts, manual_records):
def apply(cfg, hosts):
"""Make the name server the primary of every domain with records(), and of none while dns.enabled is false."""
"""Make the name server the primary of every domain with records(), hosts being the pairs of a host name and an
address of the sites with dns router, and of none while dns.enabled is false."""
with database() as connection:
if not cfg['dns']['enabled']:
connection.execute('CALL dns_retain(%s::text[])', ([],))
@@ -366,20 +339,21 @@ def apply(cfg, hosts):
dns = cfg['dns']
domains = connection.execute('SELECT * FROM dns_domains_list()').fetchall()
zones = [row['name'] for row in domains]
secondaries, manual = {}, {}
for item in connection.execute('SELECT * FROM dns_secondaries_list()').fetchall():
secondaries.setdefault(item['domain'], []).append(item)
for host, _ in hosts:
if closest(zones, host) is None:
linuxmanager.warn(f'{host} is in none of the domains, so the name server has no record of it')
manual = {}
for item in connection.execute('SELECT * FROM dns_records_list()').fetchall():
manual.setdefault(item['domain'], []).append(item)
transfers = secondary_addresses(cfg) if domains else []
for row in domains:
zone, ttl = row['name'], row['ttl'] or dns['ttl']
rows = records(cfg, row, zones, secondaries, hosts, manual.get(zone, []))
soa = f'{dns["nameserver"].strip()} hostmaster.{zone} {{serial}} {SOA_TIMES} {ttl}'
rows = records(cfg, row, zones, hosts, manual.get(zone, []))
soa = f'{cfg["wan"]["local_fqdn"].strip()} hostmaster.{zone} {{serial}} {SOA_TIMES} {ttl}'
connection.execute('CALL dns_apply(%s, %s, %s, %s::text[], %s::text[], %s::text[], %s::integer[], '
'%s::text[])',
(zone, soa, ttl, [item[0] for item in rows], [item[1] for item in rows],
[item[2] for item in rows], [item[3] for item in rows],
[item['address'] for item in secondaries.get(zone, [])]))
[item[2] for item in rows], [item[3] for item in rows], transfers))
connection.execute('CALL dns_retain(%s::text[])', (zones,))
@@ -388,7 +362,7 @@ def config_text(addresses):
addresses it answers on."""
password = DATABASE_SECRET.read_text(encoding='utf-8').strip()
return '\n'.join([
'# Written by vpn-router-setup from /etc/vpn-router/vpn-router.json; edits are overwritten.',
'# Written by linux-manager-setup from /etc/linux-manager/linux-manager.json; edits are overwritten.',
'launch=gpgsql',
'gpgsql-host=localhost',
f'gpgsql-dbname={DATABASE}',
@@ -1,6 +1,6 @@
"""The router's user directory in the local slapd.
"""The server's user directory in the local slapd.
The package keeps its own database, dc=vpn-router,dc=local in /var/lib/ldap/vpn-router,
The package keeps its own database, dc=linux-manager,dc=local in /var/lib/ldap/linux-manager,
next to the one slapd's installation creates. Root manages it through the local socket
with SASL EXTERNAL: the database's root DN is root's peer-credential DN, so there is no
administrator password. Users and groups are changed by the user-manager service account,
@@ -14,10 +14,13 @@ cn=totp,ou=policies holds the TOTP parameters the token of every user points to.
A user with console login on its own account is also a posixAccount, with a uidNumber from
CONSOLE_UID_MIN and the gidNumber of the local group CONSOLE_GROUP; nslcd serves those with
vpnRouterConsoleLogin TRUE to the system over the local socket.
linuxManagerConsoleLogin TRUE to the system over the local socket.
ou=pki holds the router's PKI (vpnrouter_pki): an entry per CA, and under it an entry per
ou=pki holds the server's PKI (linuxmanager_pki): an entry per CA, and under it an entry per
certificate the CA issued. Private keys are readable by root alone, the database's root DN.
ou=roles holds the roles of the resources of the OpenID Provider: an entry per role,
cn=<resource>:<role>, whose members are the groups, users and service accounts it is assigned to.
"""
import os
@@ -33,19 +36,20 @@ import ldap.sasl
URI = 'ldapi:///'
#: The local socket ldapi:/// names, which slapd listens on.
SLAPD_SOCKET = '/var/run/slapd/ldapi'
BASE = 'dc=vpn-router,dc=local'
BASE = 'dc=linux-manager,dc=local'
USERS = f'ou=users,{BASE}'
GROUPS = f'ou=groups,{BASE}'
SERVICES = f'ou=services,{BASE}'
POLICIES = f'ou=policies,{BASE}'
PKI = f'ou=pki,{BASE}'
ROLES = f'ou=roles,{BASE}'
DEFAULT_POLICY = f'cn=default,{POLICIES}'
TOTP_PARAMS = f'cn=totp,{POLICIES}'
MANAGER = f'cn=user-manager,{SERVICES}'
ROOT = 'gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth'
DB_DIRECTORY = pathlib.Path('/var/lib/ldap/vpn-router')
SECRET_DIR = pathlib.Path('/etc/vpn-router/ldap')
DB_DIRECTORY = pathlib.Path('/var/lib/ldap/linux-manager')
SECRET_DIR = pathlib.Path('/etc/linux-manager/ldap')
MANAGER_SECRET = SECRET_DIR / 'user-manager.secret'
SLAPD_DEFAULTS = pathlib.Path('/etc/default/slapd')
SLAPD_SERVICES = 'SLAPD_SERVICES="ldapi:///"'
@@ -60,70 +64,86 @@ LOCKOUT_SECONDS = 900
#: The overlays of the database, in the order they are configured.
OVERLAYS = ('ppolicy', 'otp')
#: The package's own schema, under an OID of the UUID arc 2.25 (ITU-T X.667): attribute types
#: under .1 and object classes under .2. vpnRouterAccount, auxiliary, adds properties of the
#: package to a user of either kind: mail, for a Manager-only account, vpnRouterScope,
#: The package's own schema, under the LDAP arc of the Linux Manager OID arc (OID-REGISTRY.txt):
#: attribute types under .1 and object classes under .2. linuxManagerAccount, auxiliary, adds properties of the
#: package to a user of either kind: mail, for a Manager-only account, linuxManagerScope,
#: authorization scopes, of which service marks a service account, displayName and
#: description, the labels of a service account, and vpnRouterConsoleLogin and
#: vpnRouterConsoleAccount, whether the user may use a console and the local account it runs as,
#: without which it runs as the user's own posixAccount, and vpnRouterWireGuardPeer, each a
#: WireGuard peer of the user as "<public key> <IP>[,<IP>...]", and vpnRouterPasskey, each a
#: passkey of the user as JSON (vpnrouter_passkeys). A service account with vpnRouterRedirectUri
#: is a client of the OpenID Provider (vpnrouter_oidc): the URIs it is sent back to after a sign-in
#: and, vpnRouterPostLogoutRedirectUri, after a sign-out, and vpnRouterAllowedGroup, the DNs of the
#: groups whose members may sign in to it. vpnRouterAuthority is a CA of the PKI: its
#: description, the labels of a service account, and linuxManagerConsoleLogin and
#: linuxManagerConsoleAccount, whether the user may use a console and the local account it runs as,
#: without which it runs as the user's own posixAccount, and linuxManagerWireGuardPeer, each a
#: WireGuard peer of the user as "<public key> <IP>[,<IP>...]", and linuxManagerPasskey, each a
#: passkey of the user as JSON (linuxmanager_passkeys). A service account with linuxManagerRedirectUri
#: is a client of the OpenID Provider (linuxmanager_oidc): the URIs it is sent back to after a sign-in
#: and, linuxManagerPostLogoutRedirectUri, after a sign-out, and linuxManagerAllowedGroup, the DNs of the
#: groups whose members may sign in to it; linuxManagerGrantType, the further grants it may use,
#: linuxManagerPublicClient, whether it has no secret, and linuxManagerClientScope, the scopes of
#: resources it may ask for, as <resource>:<scope>. A service account with linuxManagerResourceScope
#: is a resource of the OpenID Provider with those scopes. linuxManagerRole is a role of a resource,
#: named cn=<resource>:<role>, with the DNs of the groups and accounts it is assigned to in
#: member. linuxManagerAuthority is a CA of the PKI: its
#: certificate and CRL (cACertificate and certificateRevocationList of RFC 4523), its key, the
#: CSR it waits with for the offline root, the number of its last CRL, where the CRL came from,
#: and when it was retired. vpnRouterCertificate is a certificate a CA issued, named by its
#: serial number: the certificate, the key the router keeps for its own and the users'
#: and when it was retired. linuxManagerCertificate is a certificate a CA issued, named by its
#: serial number: the certificate, the key the server keeps for its own and the users'
#: certificates, its kind, template, names and expiry, and its revocation.
SCHEMA_OID = '2.25.98477352595735834532404256531620013001'
SCHEMA_NAME = 'vpn-router'
SCHEMA_OID = '1.3.6.1.4.1.67380.10.1.2'
SCHEMA_NAME = 'linux-manager'
_STRING = 'EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{256} SINGLE-VALUE'
_TIME = 'EQUALITY generalizedTimeMatch ORDERING generalizedTimeOrderingMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.24 SINGLE-VALUE'
_OCTETS = 'EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 SINGLE-VALUE'
SCHEMA_ATTRIBUTE_TYPES = (
f"( {SCHEMA_OID}.1.1 NAME 'vpnRouterScope' DESC 'VPN router: an authorization scope of the account' "
f"( {SCHEMA_OID}.1.1 NAME 'linuxManagerScope' DESC 'Linux Manager: an authorization scope of the account' "
"EQUALITY caseIgnoreMatch SUBSTR caseIgnoreSubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{256} )",
f"( {SCHEMA_OID}.1.2 NAME 'vpnRouterPrivateKey' DESC 'VPN router: a private key, PEM' {_OCTETS} )",
f"( {SCHEMA_OID}.1.3 NAME 'vpnRouterCertificateRequest' DESC 'VPN router: the CSR of a CA, PEM' {_OCTETS} )",
f"( {SCHEMA_OID}.1.4 NAME 'vpnRouterCRLNumber' DESC 'VPN router: the number of the last CRL of a CA' "
f"( {SCHEMA_OID}.1.2 NAME 'linuxManagerPrivateKey' DESC 'Linux Manager: a private key, PEM' {_OCTETS} )",
f"( {SCHEMA_OID}.1.3 NAME 'linuxManagerCertificateRequest' DESC 'Linux Manager: the CSR of a CA, PEM' {_OCTETS} )",
f"( {SCHEMA_OID}.1.4 NAME 'linuxManagerCRLNumber' DESC 'Linux Manager: the number of the last CRL of a CA' "
"EQUALITY integerMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.27 SINGLE-VALUE )",
f"( {SCHEMA_OID}.1.5 NAME 'vpnRouterCRLSource' DESC 'VPN router: signed or uploaded, where the CRL of a CA came from' {_STRING} )",
f"( {SCHEMA_OID}.1.6 NAME 'vpnRouterRetired' DESC 'VPN router: when a CA was retired' {_TIME} )",
f"( {SCHEMA_OID}.1.7 NAME 'vpnRouterKind' DESC 'VPN router: ca, router, user or external, the kind of a certificate' {_STRING} )",
f"( {SCHEMA_OID}.1.8 NAME 'vpnRouterTemplate' DESC 'VPN router: the template of a certificate' {_STRING} )",
f"( {SCHEMA_OID}.1.9 NAME 'vpnRouterSubjectAltName' DESC 'VPN router: a subject alternative name of a certificate' "
f"( {SCHEMA_OID}.1.5 NAME 'linuxManagerCRLSource' DESC 'Linux Manager: signed or uploaded, where the CRL of a CA came from' {_STRING} )",
f"( {SCHEMA_OID}.1.6 NAME 'linuxManagerRetired' DESC 'Linux Manager: when a CA was retired' {_TIME} )",
f"( {SCHEMA_OID}.1.7 NAME 'linuxManagerKind' DESC 'Linux Manager: ca, router, user or external, the kind of a certificate' {_STRING} )",
f"( {SCHEMA_OID}.1.8 NAME 'linuxManagerTemplate' DESC 'Linux Manager: the template of a certificate' {_STRING} )",
f"( {SCHEMA_OID}.1.9 NAME 'linuxManagerSubjectAltName' DESC 'Linux Manager: a subject alternative name of a certificate' "
"EQUALITY caseIgnoreMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{256} )",
f"( {SCHEMA_OID}.1.10 NAME 'vpnRouterNotAfter' DESC 'VPN router: when a certificate expires' {_TIME} )",
f"( {SCHEMA_OID}.1.11 NAME 'vpnRouterRevoked' DESC 'VPN router: when a certificate was revoked' {_TIME} )",
f"( {SCHEMA_OID}.1.12 NAME 'vpnRouterRevocationReason' DESC 'VPN router: why a certificate was revoked' {_STRING} )",
f"( {SCHEMA_OID}.1.13 NAME 'vpnRouterConsoleLogin' DESC 'VPN router: whether the user may use a console' "
f"( {SCHEMA_OID}.1.10 NAME 'linuxManagerNotAfter' DESC 'Linux Manager: when a certificate expires' {_TIME} )",
f"( {SCHEMA_OID}.1.11 NAME 'linuxManagerRevoked' DESC 'Linux Manager: when a certificate was revoked' {_TIME} )",
f"( {SCHEMA_OID}.1.12 NAME 'linuxManagerRevocationReason' DESC 'Linux Manager: why a certificate was revoked' {_STRING} )",
f"( {SCHEMA_OID}.1.13 NAME 'linuxManagerConsoleLogin' DESC 'Linux Manager: whether the user may use a console' "
"EQUALITY booleanMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.7 SINGLE-VALUE )",
f"( {SCHEMA_OID}.1.14 NAME 'vpnRouterConsoleAccount' DESC 'VPN router: the local account the console of the user runs as' "
f"( {SCHEMA_OID}.1.14 NAME 'linuxManagerConsoleAccount' DESC 'Linux Manager: the local account the console of the user runs as' "
f"{_STRING} )",
f"( {SCHEMA_OID}.1.15 NAME 'vpnRouterWireGuardPeer' DESC 'VPN router: a WireGuard peer of the user, its public key and IPs' "
f"( {SCHEMA_OID}.1.15 NAME 'linuxManagerWireGuardPeer' DESC 'Linux Manager: a WireGuard peer of the user, its public key and IPs' "
"EQUALITY caseExactMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{1024} )",
f"( {SCHEMA_OID}.1.16 NAME 'vpnRouterPasskey' DESC 'VPN router: a passkey of the user, its credential as JSON' "
f"( {SCHEMA_OID}.1.16 NAME 'linuxManagerPasskey' DESC 'Linux Manager: a passkey of the user, its credential as JSON' "
"EQUALITY caseExactMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{4096} )",
f"( {SCHEMA_OID}.1.17 NAME 'vpnRouterRedirectUri' DESC 'VPN router: a redirect URI of an OpenID Connect client' "
f"( {SCHEMA_OID}.1.17 NAME 'linuxManagerRedirectUri' DESC 'Linux Manager: a redirect URI of an OpenID Connect client' "
"EQUALITY caseExactMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{1024} )",
f"( {SCHEMA_OID}.1.18 NAME 'vpnRouterPostLogoutRedirectUri' DESC 'VPN router: a post-logout redirect URI of an "
f"( {SCHEMA_OID}.1.18 NAME 'linuxManagerPostLogoutRedirectUri' DESC 'Linux Manager: a post-logout redirect URI of an "
"OpenID Connect client' EQUALITY caseExactMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{1024} )",
f"( {SCHEMA_OID}.1.19 NAME 'vpnRouterAllowedGroup' DESC 'VPN router: a group whose members may sign in to an "
f"( {SCHEMA_OID}.1.19 NAME 'linuxManagerAllowedGroup' DESC 'Linux Manager: a group whose members may sign in to an "
"OpenID Connect client' EQUALITY distinguishedNameMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.12 )",
f"( {SCHEMA_OID}.1.20 NAME 'linuxManagerGrantType' DESC 'Linux Manager: a grant an OpenID Connect client may use besides "
"the authorization code' EQUALITY caseExactMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{64} )",
f"( {SCHEMA_OID}.1.21 NAME 'linuxManagerPublicClient' DESC 'Linux Manager: whether an OpenID Connect client has no secret' "
"EQUALITY booleanMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.7 SINGLE-VALUE )",
f"( {SCHEMA_OID}.1.22 NAME 'linuxManagerClientScope' DESC 'Linux Manager: a scope of a resource an OpenID Connect client "
"may ask for' EQUALITY caseExactMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{128} )",
f"( {SCHEMA_OID}.1.23 NAME 'linuxManagerResourceScope' DESC 'Linux Manager: a scope of a resource of the OpenID Provider' "
"EQUALITY caseExactMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.15{64} )",
)
SCHEMA_OBJECT_CLASSES = (
f"( {SCHEMA_OID}.2.1 NAME 'vpnRouterAccount' DESC 'VPN router: properties of a user or a Manager-only account' "
"SUP top AUXILIARY MAY ( mail $ vpnRouterScope $ displayName $ description $ vpnRouterConsoleLogin $ "
"vpnRouterConsoleAccount $ vpnRouterWireGuardPeer $ vpnRouterPasskey $ vpnRouterRedirectUri $ "
"vpnRouterPostLogoutRedirectUri $ vpnRouterAllowedGroup ) )",
f"( {SCHEMA_OID}.2.2 NAME 'vpnRouterAuthority' DESC 'VPN router: a CA of the PKI' SUP top STRUCTURAL MUST cn "
"MAY ( cACertificate $ certificateRevocationList $ vpnRouterPrivateKey $ vpnRouterCertificateRequest $ "
"vpnRouterCRLNumber $ vpnRouterCRLSource $ vpnRouterRetired ) )",
f"( {SCHEMA_OID}.2.3 NAME 'vpnRouterCertificate' DESC 'VPN router: a certificate a CA of the PKI issued' SUP top "
"STRUCTURAL MUST serialNumber MAY ( cn $ userCertificate $ vpnRouterPrivateKey $ vpnRouterKind $ vpnRouterTemplate $ "
"vpnRouterSubjectAltName $ vpnRouterNotAfter $ vpnRouterRevoked $ vpnRouterRevocationReason ) )",
f"( {SCHEMA_OID}.2.1 NAME 'linuxManagerAccount' DESC 'Linux Manager: properties of a user or a Manager-only account' "
"SUP top AUXILIARY MAY ( mail $ linuxManagerScope $ displayName $ description $ linuxManagerConsoleLogin $ "
"linuxManagerConsoleAccount $ linuxManagerWireGuardPeer $ linuxManagerPasskey $ linuxManagerRedirectUri $ "
"linuxManagerPostLogoutRedirectUri $ linuxManagerAllowedGroup $ linuxManagerGrantType $ linuxManagerPublicClient $ "
"linuxManagerClientScope $ linuxManagerResourceScope ) )",
f"( {SCHEMA_OID}.2.2 NAME 'linuxManagerAuthority' DESC 'Linux Manager: a CA of the PKI' SUP top STRUCTURAL MUST cn "
"MAY ( cACertificate $ certificateRevocationList $ linuxManagerPrivateKey $ linuxManagerCertificateRequest $ "
"linuxManagerCRLNumber $ linuxManagerCRLSource $ linuxManagerRetired ) )",
f"( {SCHEMA_OID}.2.3 NAME 'linuxManagerCertificate' DESC 'Linux Manager: a certificate a CA of the PKI issued' SUP top "
"STRUCTURAL MUST serialNumber MAY ( cn $ userCertificate $ linuxManagerPrivateKey $ linuxManagerKind $ linuxManagerTemplate $ "
"linuxManagerSubjectAltName $ linuxManagerNotAfter $ linuxManagerRevoked $ linuxManagerRevocationReason ) )",
f"( {SCHEMA_OID}.2.4 NAME 'linuxManagerRole' DESC 'Linux Manager: a role of a resource of the OpenID Provider' SUP top "
"STRUCTURAL MUST cn MAY ( description $ member ) )",
)
#: The TOTP parameters: digits, the OID of HMAC-SHA1, seconds per time step, and the time steps accepted before and after now.
@@ -136,22 +156,23 @@ TOTP_WINDOW = 1
DISABLED = '000001010000Z'
#: The indexes of the database.
INDEXES = ('objectClass eq', 'uid eq', 'uidNumber eq', 'vpnRouterConsoleLogin eq')
INDEXES = ('objectClass eq', 'uid eq', 'uidNumber eq', 'linuxManagerConsoleLogin eq')
#: The POSIX accounts of console login (vpnrouter_users.console): the lowest uidNumber, which nslcd's nss_min_uid
#: The POSIX accounts of console login (linuxmanager_users.console): the lowest uidNumber, which nslcd's nss_min_uid
#: matches, the local group that is their primary group and grants sudo, the shell and the parent of their homes.
CONSOLE_UID_MIN = 20000
CONSOLE_GROUP = 'vpn-router-console'
CONSOLE_GROUP = 'linux-manager-console'
CONSOLE_SHELL = '/bin/bash'
CONSOLE_HOMES = '/home'
#: The ACLs of the database, in order.
ACCESS = (
'to attrs=vpnRouterPrivateKey by * none',
'to attrs=linuxManagerPrivateKey by * none',
f'to attrs=userPassword by dn.exact="{MANAGER}" =wx by self =wx by anonymous auth by * none',
f'to attrs=oathSecret by dn.exact="{MANAGER}" =w by * none',
f'to dn.subtree="{USERS}" by dn.exact="{MANAGER}" manage by * read',
f'to dn.subtree="{GROUPS}" by dn.exact="{MANAGER}" manage by * read',
f'to dn.subtree="{ROLES}" by dn.exact="{MANAGER}" manage by * read',
'to * by * read',
)
@@ -311,13 +332,13 @@ def wireguard_peers():
connection = root()
try:
found = connection.search_s(USERS, ldap.SCOPE_ONELEVEL,
f'(&(vpnRouterWireGuardPeer=*)(!(pwdAccountLockedTime={DISABLED})))',
['uid', 'vpnRouterWireGuardPeer'])
f'(&(linuxManagerWireGuardPeer=*)(!(pwdAccountLockedTime={DISABLED})))',
['uid', 'linuxManagerWireGuardPeer'])
finally:
connection.unbind_s()
peers = []
for _, attributes in found:
for value in attributes['vpnRouterWireGuardPeer']:
for value in attributes['linuxManagerWireGuardPeer']:
key, _, ips = value.decode().partition(' ')
peers.append((attributes['uid'][0].decode(), key, ips.split(',')))
return sorted(peers)
@@ -332,8 +353,8 @@ def provision():
_ensure_password_hash(connection)
database = _ensure_database(connection)
_ensure_overlays(connection, database)
_ensure_entry(connection, BASE, {'objectClass': [b'dcObject', b'organization'], 'dc': [b'vpn-router'], 'o': [b'vpn-router']})
for unit in (USERS, GROUPS, SERVICES, POLICIES, PKI):
_ensure_entry(connection, BASE, {'objectClass': [b'dcObject', b'organization'], 'dc': [b'linux-manager'], 'o': [b'linux-manager']})
for unit in (USERS, GROUPS, SERVICES, POLICIES, PKI, ROLES):
_ensure_entry(connection, unit, {'objectClass': [b'organizationalUnit'], 'ou': [unit.split(',')[0][3:].encode()]})
_ensure_entry(connection, DEFAULT_POLICY, {
'objectClass': [b'organizationalRole', b'pwdPolicy'],
@@ -1,4 +1,4 @@
"""The metrics of the router in PostgreSQL, for the metrics service and the web service.
"""The metrics of the server in PostgreSQL, for the metrics service and the web service.
The metrics service writes one detail row per minute to the four detail tables, of TABLES:
metrics_system (CPU, memory, disk and load averages), metrics_network (bytes per second
@@ -15,7 +15,7 @@ after the last summarized one, so a stopped service loses no summary of what was
collect_garbage() deletes the rows older than the retention of their level, metrics.keep_*;
the service runs it on start and once a day. history() answers the views of the Metrics tab.
The tables are in the database of the refresh tokens (vpnrouter_auth.connection_string) and are
The tables are in the database of the refresh tokens (linuxmanager_auth.connection_string) and are
created when missing. Every failure of the database raises DatabaseError.
"""
@@ -23,7 +23,7 @@ import contextlib
import psycopg
import vpnrouter_auth
import linuxmanager_auth
#: The detail tables: their key columns besides time, and their value columns.
TABLES = {
@@ -86,7 +86,7 @@ def schema():
def database(cfg):
"""A connection to the metrics database, with its tables, closed at the end; DatabaseError on any failure."""
try:
with psycopg.connect(vpnrouter_auth.connection_string(cfg), autocommit=True) as connection:
with psycopg.connect(linuxmanager_auth.connection_string(cfg), autocommit=True) as connection:
for statement in schema():
connection.execute(statement)
yield connection
@@ -1,4 +1,4 @@
"""Interfaces and addresses of the router, read from the system.
"""Interfaces and addresses of the server, read from the system.
Only the Python standard library is used, so the web service and its bootstrap server can use
this module too.
@@ -7,7 +7,7 @@ this module too.
import pathlib
import subprocess
#: The values of web.listen and of the Router Manager's listen: the interfaces a service listens on.
#: The values of web.listen and of the Manager's listen: the interfaces a service listens on.
LISTEN_CHOICES = ('internal', 'external', 'both')
#: The interfaces a service listens on when it is not set.
@@ -30,12 +30,14 @@ def ip_output(*args):
return result.stdout if result.returncode == 0 else ''
def detect_interfaces():
def detect_interfaces(internal=True):
"""Work out the external and internal interface names.
Only reached in auto mode, where the operator has accepted that a wrong
guess is possible. The external interface is whichever one carries the
default route; the internal one is the only other addressed interface.
With internal false only the external interface is looked for, and the
internal name is empty.
"""
tokens = ip_output('route', 'get', '1.1.1.1').split()
external = ''
@@ -45,6 +47,8 @@ def detect_interfaces():
break
if not external:
raise ConfigError('auto mode: cannot determine the external interface')
if not internal:
return external, ''
candidates = [name for name in _addressed_interfaces() if name != external]
if len(candidates) != 1:
@@ -103,7 +107,7 @@ def interface_addresses(name, setting):
def internal_address(mode, internal, int_addr):
"""The router's address on the internal network, as general.mode directs, or '' when it is not known yet.
"""The server's address on the internal network, as general.mode directs, or '' when it is not known yet.
manual takes local.int_addr; interfaces reads the address of interfaces.internal; auto also
works out which interface that is.
@@ -120,7 +124,7 @@ def internal_address(mode, internal, int_addr):
def external_address(mode, external):
"""The router's address on the external interface, as general.mode directs, or '' when it is not known yet.
"""The server's address on the external interface, as general.mode directs, or '' when it is not known yet.
auto works out which interface that is; the other modes read the address of interfaces.external.
"""
@@ -140,25 +144,29 @@ def _section(configuration, name):
def listen_addresses(configuration, choice):
"""The addresses a service listens on for choice of LISTEN_CHOICES, from vpn-router.json read as configuration.
"""The addresses a service listens on for choice of LISTEN_CHOICES, from linux-manager.json read as configuration.
The internal address comes first; an address not known yet is left out. A choice that is not
one of LISTEN_CHOICES is DEFAULT_LISTEN. LOOPBACK is not among them; web_listen adds it.
Without the router (router.enabled false) the external interface is also the internal side.
"""
if choice not in LISTEN_CHOICES:
choice = DEFAULT_LISTEN
general, interfaces, local = (_section(configuration, name) for name in ('general', 'interfaces', 'local'))
general, router, interfaces, local = (_section(configuration, name)
for name in ('general', 'router', 'interfaces', 'local'))
mode = str(general.get('mode') or 'manual')
external = external_address(mode, str(interfaces.get('external') or ''))
addresses = []
if choice in ('internal', 'both'):
addresses.append(internal_address(mode, str(interfaces.get('internal') or ''), str(local.get('int_addr') or '')))
addresses.append(internal_address(mode, str(interfaces.get('internal') or ''), str(local.get('int_addr') or ''))
if router.get('enabled') is True else external)
if choice in ('external', 'both'):
addresses.append(external_address(mode, str(interfaces.get('external') or '')))
return [address for address in addresses if address]
addresses.append(external)
return list(dict.fromkeys(address for address in addresses if address))
def web_listen(configuration, default_port):
"""Where the web service listens, from vpn-router.json read as configuration, as ([(address, port), ...], '').
"""Where the web service listens, from linux-manager.json read as configuration, as ([(address, port), ...], '').
The addresses are those of the interfaces web.listen names (listen_addresses), then
LOOPBACK, the port web.port, else default_port. While web.enabled is false or no address of
@@ -0,0 +1,982 @@
"""The server's OpenID Provider, for the web service, with python3-oauthlib and python3-jwt.
While oidc.enabled is true, the web service answers OpenID Connect at the issuer linuxmanager.oidc_issuer, its HTTPS site
that Caddy serves; the web service answers it only on the loopback address, where Caddy passes the requests. The users
of the server's directory sign in, with the password and the TOTP code or with a passkey (linuxmanager_passkeys), to its
clients, the service accounts with redirect URIs or grants (linuxmanager_users.client_settings): a client's ID is its name
and its secret its password, sent with HTTP Basic authentication or in the body of the token request; a public client
sends only its ID. A client with allowed groups admits only their members; a service account never signs in as a user.
GET /.well-known/openid-configuration the discovery document
GET /oidc/jwks the public key the ID tokens and the access tokens are signed with
GET, POST /oidc/authorize the authorization code flow, PKCE with S256 required; the sign-in page while
the browser has no session, or with prompt=login or an older one than max_age
POST /oidc/login the sign-in page's form: the name, the password and the code, or a passkey
POST /oidc/passkey begin a sign-in with a passkey: {"id", "options"}
POST /oidc/token exchange a code, a refresh token or a device code for tokens, or a client's
credentials for an access token
POST /oidc/device_authorization begin the device authorization grant (RFC 8628): a device code and a user code
GET, POST /oidc/device the page where the user enters the user code, signs in and is asked to allow
the device
POST /oidc/device/decide allow or deny the device
GET, POST /oidc/userinfo the claims of the user an access token was issued for
GET /oidc/logout end the session, then go to a post-logout redirect URI of the client
The grants: authorization_code for a client with redirect URIs, client_credentials, by which a confidential client
gets an access token as itself, and the device code, as the client's grants give them, each with refresh_token but
client_credentials. A client asks for tokens for a resource, a service account, with resource (RFC 8707), its name or
<issuer>/<name>, or by the resource's scopes it asks for, <resource>:<scope>, of its client scopes; without one the
tokens are for the client itself.
ID tokens are RS256 JWTs signed with KEY, an RSA key the web service makes on its first run on the server, named by the
RFC 7638 thumbprint of its public key, for the client. Access tokens are RS256 JWTs of RFC 9068 with the same key, for
the resource or the client: iss, sub, aud, client_id, iat, exp, jti, scope, roles, the claims of the scopes, and of a
user auth_time and amr. sub is the entryUUID of the user's entry, so a user added again under the same name is another
subject. The scopes: openid, required but with client_credentials, which refuses it; profile, preferred_username and
the names; email; groups, the names of the user's groups; their claims are in the ID token, the access token and the
userinfo answer; and the scopes of the resources. roles are the roles of the resource assigned to the user or service
account or a group of it (linuxmanager_users.roles_held), read at every issue; the ID token and the userinfo answer have
those of the client. A code is valid CODE_SECONDS, an access token and an ID token ACCESS_SECONDS, a refresh token
REFRESH_SECONDS, used once, and a device code DEVICE_SECONDS, polled at most every DEVICE_INTERVAL seconds, 5 more after
each poll that came too soon. A session, the cookie COOKIE of the issuer's /oidc/ path, lasts SESSION_SECONDS. Codes,
tokens, device codes and sessions are kept in the database of linuxmanager_auth as their SHA-256
(linuxmanager_auth.OIDC_SQL); linuxmanager_auth.revoke removes a user's or a service account's. A client is registered by an
administrator, so a user is not asked for consent, but for a device, which the user allows on the server's page.
"""
import base64
import collections
import contextlib
import datetime
import hashlib
import http.cookies
import json
import pathlib
import secrets
import time
import types
import urllib.parse
import jinja2
import jwt
import ldap
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric import rsa
from oauthlib.common import Request, add_params_to_uri
from oauthlib.oauth2 import (AccessDeniedError, FatalClientError, InvalidClientError, InvalidGrantError,
InvalidRequestError, InvalidScopeError, OAuth2Error, UnauthorizedClientError,
UnsupportedGrantTypeError)
from oauthlib.oauth2.rfc6749 import utils
from oauthlib.oauth2.rfc6749.grant_types.base import GrantTypeBase
from oauthlib.oauth2.rfc6749.tokens import random_token_generator
from oauthlib.openid import RequestValidator, Server
from oauthlib.openid.connect.core.exceptions import LoginRequired
import linuxmanager
import linuxmanager_auth
import linuxmanager_ldap
import linuxmanager_passkeys
import linuxmanager_users
KEY = pathlib.Path('/etc/linux-manager/oidc.key')
KEY_BITS = 3072
CODE_SECONDS = 60
ACCESS_SECONDS = 300
REFRESH_SECONDS = 8 * 3600
SESSION_SECONDS = 8 * 3600
DEVICE_SECONDS = 600
DEVICE_INTERVAL = 5
#: The grant type of the device authorization grant.
DEVICE_GRANT = 'urn:ietf:params:oauth:grant-type:device_code'
#: A user code: USER_CODE_LENGTH of these letters, shown in two halves.
USER_CODE_LETTERS = 'BCDFGHJKLMNPQRSTVWXZ'
USER_CODE_LENGTH = 8
COOKIE = 'linux_manager_oidc'
#: The claims of each scope but openid, which gives sub.
SCOPE_CLAIMS = {
'profile': ('preferred_username', 'name', 'given_name', 'family_name'),
'email': ('email',),
'groups': ('groups',),
}
SCOPES = ('openid', *SCOPE_CLAIMS)
#: The authentication methods (RFC 8176) of a sign-in with a password, with a password and a TOTP code, and with a
#: passkey, which verifies the user; a service account's client credentials are its password.
PASSWORD_AMR = ['pwd']
OTP_AMR = ['pwd', 'otp']
PASSKEY_AMR = ['hwk', 'user']
SERVICE_AMR = ['pwd']
TEMPLATE = 'oidc-page.html.j2'
#: The headers of the pages: no framing, and nothing loaded but from the page itself and the server.
PAGE_HEADERS = [('Content-Security-Policy', "default-src 'none'; script-src 'unsafe-inline'; style-src 'unsafe-inline'; "
"connect-src 'self'; frame-ancestors 'none'")]
#: The headers of oauthlib's answers that the web service sets itself.
OWN_HEADERS = ('content-type', 'cache-control', 'content-length')
#: A user's sign-in at the OpenID Provider: the user name, when, a datetime, and how (amr).
Session = collections.namedtuple('Session', 'name auth_time amr')
_pages = jinja2.Environment(loader=jinja2.FileSystemLoader(str(linuxmanager.TEMPLATE_DIR)), autoescape=True,
undefined=jinja2.StrictUndefined)
class InvalidTarget(OAuth2Error):
"""A resource that is unknown, or more than one (RFC 8707)."""
error = 'invalid_target'
class AuthorizationPending(OAuth2Error):
"""A device code the user has not yet allowed or denied (RFC 8628)."""
error = 'authorization_pending'
class SlowDown(OAuth2Error):
"""A device code polled sooner than its interval (RFC 8628)."""
error = 'slow_down'
class ExpiredToken(OAuth2Error):
"""A device code that has expired (RFC 8628)."""
error = 'expired_token'
def ensure_key():
"""Make the signing key when it is missing."""
if KEY.is_file() and KEY.stat().st_size:
return
key = rsa.generate_private_key(public_exponent=65537, key_size=KEY_BITS)
KEY.parent.mkdir(parents=True, exist_ok=True)
KEY.touch(mode=0o600)
KEY.chmod(0o600)
KEY.write_bytes(key.private_bytes(serialization.Encoding.PEM, serialization.PrivateFormat.PKCS8,
serialization.NoEncryption()))
def private_key():
return serialization.load_pem_private_key(KEY.read_bytes(), None)
def public_jwk(key):
"""The JWK of the key's public key, its kid the RFC 7638 thumbprint."""
numbers = key.public_key().public_numbers()
jwk = {'e': numbers.e, 'kty': 'RSA', 'n': numbers.n}
jwk = {name: linuxmanager_auth.b64encode(value.to_bytes((value.bit_length() + 7) // 8, 'big')) if isinstance(value, int)
else value for name, value in jwk.items()}
thumbprint = hashlib.sha256(json.dumps(jwk, sort_keys=True, separators=(',', ':')).encode()).digest()
return {**jwk, 'kid': linuxmanager_auth.b64encode(thumbprint), 'use': 'sig', 'alg': 'RS256'}
def sign(claims, headers=None):
"""The claims as an RS256 JWT with the key's kid and the headers given."""
key = private_key()
return jwt.encode(claims, key, algorithm='RS256', headers={'kid': public_jwk(key)['kid'], **(headers or {})})
def discovery(issuer, resource_scopes):
"""The discovery document of OpenID Connect Discovery 1.0, with the device authorization endpoint of RFC 8628;
resource_scopes are the scopes of the resources."""
return {
'issuer': issuer,
'authorization_endpoint': f'{issuer}/oidc/authorize',
'token_endpoint': f'{issuer}/oidc/token',
'device_authorization_endpoint': f'{issuer}/oidc/device_authorization',
'userinfo_endpoint': f'{issuer}/oidc/userinfo',
'jwks_uri': f'{issuer}/oidc/jwks',
'end_session_endpoint': f'{issuer}/oidc/logout',
'scopes_supported': [*SCOPES, *resource_scopes],
'response_types_supported': ['code'],
'response_modes_supported': ['query'],
'grant_types_supported': ['authorization_code', 'refresh_token', 'client_credentials', DEVICE_GRANT],
'subject_types_supported': ['public'],
'id_token_signing_alg_values_supported': ['RS256'],
'token_endpoint_auth_methods_supported': ['client_secret_basic', 'client_secret_post', 'none'],
'code_challenge_methods_supported': ['S256'],
'claims_supported': ['iss', 'sub', 'aud', 'exp', 'iat', 'auth_time', 'nonce', 'amr', 'at_hash', 'client_id',
'scope', 'roles', *(claim for claims in SCOPE_CLAIMS.values() for claim in claims)],
}
@contextlib.contextmanager
def directory():
"""A connection to the server's directory as the user manager account, for reading."""
connection = linuxmanager_ldap.manager()
try:
yield connection
finally:
connection.unbind_s()
def account_of(name):
"""A namespace of the linuxmanager_users.summary of the service account of the name, else None."""
if not isinstance(name, str):
return None
with directory() as connection:
try:
user = linuxmanager_users.summary(linuxmanager_users.entry(connection, name))
except linuxmanager_users.Failure:
return None
return types.SimpleNamespace(**user) if user['service'] else None
def client_of(client_id):
"""The client of the ID, a namespace of its linuxmanager_users.summary with client_id, when it is an enabled service
account with redirect URIs or grants; else None."""
account = account_of(client_id)
if account is None or not account.enabled or not (account.redirect_uris or account.grants):
return None
account.client_id = client_id
return account
def resource_of(name):
"""The resource of the name, a namespace of its linuxmanager_users.summary, when it is an enabled service account that
is not internal; else None."""
account = account_of(name)
return account if account is not None and account.enabled and not account.internal else None
def resource_scopes():
"""The scopes of every resource, <resource>:<scope>, sorted."""
with directory() as connection:
users = linuxmanager_users.list_users(connection)['users']
return sorted(f'{user["name"]}:{scope}' for user in users
if user['service'] and user['enabled'] and not user['internal'] for scope in user['resource_scopes'])
def account_claims(connection, name):
"""The summary of the entry of the name and its claims of every scope, with sub; Failure when there is none."""
user = linuxmanager_users.summary(linuxmanager_users.entry(connection, name))
groups = sorted(linuxmanager_users.groups_of(connection, name))
uuid = connection.search_s(linuxmanager_users.dn_of(name), ldap.SCOPE_BASE, attrlist=['entryUUID'])[0][1]['entryUUID']
full = ' '.join(part for part in (user['first_name'], user['last_name']) if part)
claims = {'sub': uuid[0].decode(), 'preferred_username': name, 'name': full or None, 'given_name': user['first_name'],
'family_name': user['last_name'], 'email': user['email'], 'groups': groups}
return user, {claim: value for claim, value in claims.items() if value is not None}
def subject(name, client):
"""The claims of every scope of the user, with sub, when the user may sign in to the client: an enabled, unlocked
user who is not a service account, a member of one of the client's allowed groups when it has any; else None."""
with directory() as connection:
try:
user, claims = account_claims(connection, name)
except linuxmanager_users.Failure:
return None
if user['service'] or not user['enabled'] or user['locked']:
return None
if client.allowed_groups and not set(client.allowed_groups) & set(claims['groups']):
return None
return claims
def service_subject(name):
"""The claims of every scope of the service account, with sub."""
with directory() as connection:
return account_claims(connection, name)[1]
def roles_at(resource, name):
"""The roles of the resource the user or service account holds."""
with directory() as connection:
return linuxmanager_users.roles_held(connection, resource, name)
def claims_of(person, scopes):
"""The claims of the scopes of a subject's claims (subject)."""
return {'sub': person['sub'], **{claim: person[claim] for scope in scopes for claim in SCOPE_CLAIMS.get(scope, ())
if claim in person}}
def target_of(issuer, resource, scopes):
"""The name of the resource a request with the resource parameter, None when absent, and the scopes names, None
for none; InvalidTarget when they name more than one or one that is not a resource."""
names = {scope.partition(':')[0] for scope in scopes if scope not in SCOPES}
if resource is not None:
names.add(resource[len(issuer) + 1:] if isinstance(resource, str) and resource.startswith(f'{issuer}/') else resource)
if not names:
return None
if len(names) > 1:
raise InvalidTarget(description='the resource and the scopes name more than one resource')
name = names.pop()
if resource_of(name) is None:
raise InvalidTarget(description=f'{name} is not a resource')
return name
def scopes_allowed(client, scopes, target, grant_type):
"""Whether the client may ask for the scopes: openid, required but by client_credentials, which refuses it, the
other scopes of SCOPES, and those of its client scopes that belong to the target."""
for scope in scopes:
if scope not in SCOPES and (scope not in client.client_scopes or scope.partition(':')[0] != target):
return False
return ('openid' not in scopes) if grant_type == 'client_credentials' else ('openid' in scopes)
def same_redirect(registered, given):
"""Whether the redirect URI given is the registered one; an http URI of a loopback host may have any port (RFC 8252
7.3)."""
if registered == given:
return True
try:
wanted, found = urllib.parse.urlsplit(registered), urllib.parse.urlsplit(given)
# A port that is not a number makes .port raise ValueError.
wanted.port, found.port
except (ValueError, TypeError, AttributeError):
return False
return wanted.scheme == 'http' and wanted.hostname in linuxmanager_users.LOOPBACK_HOSTS and not found.fragment and \
(wanted.scheme, wanted.hostname, wanted.path, wanted.query) == (found.scheme, found.hostname, found.path, found.query)
def cookie_value(cookies):
"""The value of the session cookie in a Cookie header, or ''."""
try:
morsel = http.cookies.SimpleCookie(cookies or '').get(COOKIE)
except http.cookies.CookieError:
return ''
return morsel.value if morsel is not None else ''
def session_of(cfg, cookies):
"""The unexpired Session the session cookie in a Cookie header names, or None."""
value = cookie_value(cookies)
if not value:
return None
with linuxmanager_auth.database(cfg) as connection:
row = connection.execute('SELECT * FROM oidc_session_get(%s)', (linuxmanager_auth.digest(value),)).fetchone()
return Session(*row) if row else None
def access_token(cfg, request):
"""The access token of a token request: a JWT of RFC 9068 for the request's target, or the client."""
now = int(time.time())
client_id = request.client.client_id
service = request.grant_type == 'client_credentials'
name = client_id if service else request.user
person = service_subject(client_id) if service else request.oidc_person
target = getattr(request, 'oidc_target', None)
claims = {**claims_of(person, request.scopes), 'iss': linuxmanager.oidc_issuer(cfg), 'aud': target or client_id,
'client_id': client_id, 'iat': now, 'exp': now + ACCESS_SECONDS, 'jti': secrets.token_urlsafe(16),
'scope': ' '.join(request.scopes)}
roles = roles_at(target or client_id, name)
if roles:
claims['roles'] = roles
if not service:
claims.update({'auth_time': int(request.auth_time.timestamp()), 'amr': request.amr})
return sign(claims, {'typ': 'at+jwt'})
class Validator(RequestValidator):
"""The checks and the storage of oauthlib's server for the server's clients, users and database.
The grants are those of validate_grant_type, with PKCE S256 for the authorization code. The values of a code, a
token, a device code or a session found are kept on the request as oidc_code, oidc_scopes and oidc_person, the
resource the tokens are for as oidc_target.
"""
def __init__(self, cfg):
super().__init__()
self.cfg = cfg
def fetch(self, sql, values):
with linuxmanager_auth.database(self.cfg) as connection:
return connection.execute(sql, values).fetchone()
def call(self, *statements):
with linuxmanager_auth.database(self.cfg) as connection:
for sql, values in statements:
connection.execute(sql, values)
def session(self, request):
return session_of(self.cfg, request.headers.get('Cookie'))
def check_target(self, request, stored):
"""Refuse a resource parameter that names another resource than the one the grant was made for."""
resource = getattr(request, 'resource', None)
if 'resource' in request.duplicate_params or \
resource is not None and target_of(linuxmanager.oidc_issuer(self.cfg), resource, []) != stored:
raise InvalidTarget(description='the grant was made for another resource', request=request)
request.oidc_target = stored
# Clients.
def validate_client_id(self, client_id, request, *args, **kwargs):
request.client = client_of(client_id)
return request.client is not None
def validate_redirect_uri(self, client_id, redirect_uri, request, *args, **kwargs):
return any(same_redirect(uri, redirect_uri) for uri in request.client.redirect_uris)
def get_default_redirect_uri(self, client_id, request, *args, **kwargs):
return None
def validate_response_type(self, client_id, response_type, client, request, *args, **kwargs):
return response_type == 'code'
def validate_scopes(self, client_id, scopes, client, request, *args, **kwargs):
"""Whether the client may ask for the scopes; the resource they are for is kept as oidc_target."""
if 'resource' in request.duplicate_params:
raise InvalidTarget(description='give one resource', request=request)
try:
request.oidc_target = target_of(linuxmanager.oidc_issuer(self.cfg), getattr(request, 'resource', None), scopes)
except InvalidTarget as error:
raise InvalidTarget(description=error.description, request=request)
return scopes_allowed(client, scopes, request.oidc_target, request.grant_type)
def get_default_scopes(self, client_id, request, *args, **kwargs):
return [] if request.grant_type == 'client_credentials' else ['openid']
def is_pkce_required(self, client_id, request):
return True
def client_authentication_required(self, request, *args, **kwargs):
"""Whether the client must sign the request with its secret: unless it is a public client named by client_id."""
client = client_of(request.client_id)
return client is None or not client.public_client
def authenticate_client(self, request, *args, **kwargs):
"""Check the client's secret, the password of its service account, given with HTTP Basic authentication, its
ID and secret form-encoded (RFC 6749 2.3.1), or in the body; a public client has none."""
name, secret = request.client_id, request.client_secret
header = request.headers.get('Authorization', '')
if header[:6].lower() == 'basic ':
try:
name, _, secret = base64.b64decode(header[6:].strip(), validate=True).decode().partition(':')
except (ValueError, UnicodeDecodeError):
return False
name, secret = urllib.parse.unquote_plus(name), urllib.parse.unquote_plus(secret)
client = client_of(name)
if client is None or client.public_client:
return False
try:
linuxmanager_users.login(name, secret, None)
except linuxmanager_users.Failure:
return False
request.client = client
return True
def authenticate_client_id(self, client_id, request, *args, **kwargs):
"""Accept a public client by its ID."""
client = client_of(client_id)
if client is None or not client.public_client:
return False
request.client = client
return True
def validate_grant_type(self, client_id, grant_type, client, request, *args, **kwargs):
"""The grants of the client: authorization_code with redirect URIs, client_credentials but for a public client,
the device code, and refresh_token with either of the first and the last."""
grants = set(client.grants) - ({'client_credentials'} if client.public_client else set())
if client.redirect_uris:
grants.add('authorization_code')
if 'device_code' in grants:
grants.add(DEVICE_GRANT)
if grants & {'authorization_code', DEVICE_GRANT}:
grants.add('refresh_token')
return grant_type in grants
def validate_user(self, username, password, client, request, *args, **kwargs):
return False
# Codes.
def save_authorization_code(self, client_id, code, request, *args, **kwargs):
self.call(('CALL oidc_code_save(%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)',
(linuxmanager_auth.digest(code['code']), client_id, request.user, request.redirect_uri, request.scopes,
request.nonce, request.code_challenge, request.code_challenge_method, request.auth_time, request.amr,
getattr(request, 'oidc_target', None), CODE_SECONDS)))
def code(self, code):
row = self.fetch('SELECT * FROM oidc_code_get(%s)', (linuxmanager_auth.digest(code),)) if isinstance(code, str) else None
return dict(zip(('client_id', 'name', 'redirect_uri', 'scopes', 'nonce', 'code_challenge', 'code_challenge_method',
'auth_time', 'amr', 'resource'), row)) if row else None
def get_authorization_code_scopes(self, client_id, code, redirect_uri, request):
found = self.code(code)
return found['scopes'] if found else []
def validate_code(self, client_id, code, client, request, *args, **kwargs):
found = self.code(code)
if found is None or found['client_id'] != client_id:
return False
person = subject(found['name'], client)
if person is None:
return False
self.check_target(request, found['resource'])
request.user, request.scopes, request.auth_time, request.amr = found['name'], found['scopes'], found['auth_time'], found['amr']
request.oidc_code, request.oidc_person = found, person
return True
def get_code_challenge(self, code, request):
return request.oidc_code['code_challenge']
def get_code_challenge_method(self, code, request):
return request.oidc_code['code_challenge_method']
def confirm_redirect_uri(self, client_id, code, redirect_uri, client, request, *args, **kwargs):
return redirect_uri == request.oidc_code['redirect_uri']
def get_authorization_code_nonce(self, client_id, code, redirect_uri, request):
return request.oidc_code['nonce']
def invalidate_authorization_code(self, client_id, code, request, *args, **kwargs):
self.call(('CALL oidc_code_delete(%s)', (linuxmanager_auth.digest(code),)))
# Device codes.
def poll_device(self, code, request):
"""Poll the device code for the request's client: with the user's sign-in on the request when the user allowed
it, else the error RFC 8628 gives."""
row = self.fetch('SELECT * FROM oidc_device_poll(%s)', (linuxmanager_auth.digest(code),)) \
if isinstance(code, str) and code else None
found = dict(zip(('client_id', 'scopes', 'resource', 'state', 'name', 'auth_time', 'amr', 'expired', 'too_soon'),
row)) if row else None
if found is None or found['client_id'] != request.client.client_id:
raise InvalidGrantError(description='the device code is unknown', request=request)
if found['expired']:
raise ExpiredToken(description='the device code has expired', request=request)
if found['state'] == 'pending':
raise (SlowDown if found['too_soon'] else AuthorizationPending)(request=request)
if found['state'] == 'denied':
self.call(('CALL oidc_device_delete(%s)', (linuxmanager_auth.digest(code),)))
raise AccessDeniedError(description='the user denied the device', request=request)
person = subject(found['name'], request.client)
if person is None:
raise AccessDeniedError(description=f'{found["name"]} may not sign in to this client', request=request)
self.check_target(request, found['resource'])
request.user, request.scopes, request.auth_time, request.amr = found['name'], found['scopes'], found['auth_time'], found['amr']
request.oidc_person = person
def delete_device(self, code):
self.call(('CALL oidc_device_delete(%s)', (linuxmanager_auth.digest(code),)))
# Tokens.
def token(self, token, kind):
row = self.fetch('SELECT * FROM oidc_token_get(%s, %s)', (linuxmanager_auth.digest(token), kind)) \
if isinstance(token, str) and token else None
return dict(zip(('client_id', 'name', 'scopes', 'auth_time', 'amr', 'resource'), row)) if row else None
def save_bearer_token(self, token, request, *args, **kwargs):
"""Keep the access token and the refresh token; a refresh token that was exchanged is used up. A token of
client_credentials is kept for its service account."""
if request.grant_type == 'client_credentials':
name, auth_time, amr = request.client.client_id, datetime.datetime.now(datetime.timezone.utc), SERVICE_AMR
else:
name, auth_time, amr = request.user, request.auth_time, request.amr
values = (request.client.client_id, name, request.scopes, auth_time, amr, getattr(request, 'oidc_target', None))
statements = [('CALL oidc_token_save(%s, %s, %s, %s, %s, %s, %s, %s, %s)',
(linuxmanager_auth.digest(token['access_token']), 'access', *values, ACCESS_SECONDS))]
if token.get('refresh_token'):
statements.append(('CALL oidc_token_save(%s, %s, %s, %s, %s, %s, %s, %s, %s)',
(linuxmanager_auth.digest(token['refresh_token']), 'refresh', *values, REFRESH_SECONDS)))
if request.grant_type == 'refresh_token':
statements.append(('CALL oidc_token_delete(%s)', (linuxmanager_auth.digest(request.refresh_token),)))
self.call(*statements)
def validate_refresh_token(self, refresh_token, client, request, *args, **kwargs):
found = self.token(refresh_token, 'refresh')
if found is None or found['client_id'] != client.client_id:
return False
person = subject(found['name'], client)
if person is None:
return False
self.check_target(request, found['resource'])
request.user, request.auth_time, request.amr = found['name'], found['auth_time'], found['amr']
request.oidc_scopes, request.oidc_person = found['scopes'], person
return True
def get_original_scopes(self, refresh_token, request, *args, **kwargs):
return request.oidc_scopes
def validate_bearer_token(self, token, scopes, request):
found = self.token(token, 'access')
if found is None or not set(scopes) <= set(found['scopes']):
return False
client = client_of(found['client_id'])
person = subject(found['name'], client) if client else None
if person is None:
return False
request.client, request.user, request.oidc_scopes, request.oidc_person = client, found['name'], found['scopes'], person
return True
def get_userinfo_claims(self, request):
claims = claims_of(request.oidc_person, request.oidc_scopes)
roles = roles_at(request.client.client_id, request.user)
return {**claims, 'roles': roles} if roles else claims
# OpenID Connect.
def finalize_id_token(self, id_token, token, token_handler, request):
now = int(time.time())
id_token.update(claims_of(request.oidc_person, request.scopes))
id_token.update({'iss': linuxmanager.oidc_issuer(self.cfg), 'exp': now + ACCESS_SECONDS,
'auth_time': int(request.auth_time.timestamp()), 'amr': request.amr})
roles = roles_at(request.client.client_id, request.user)
if roles:
id_token['roles'] = roles
return sign(id_token)
def validate_silent_login(self, request):
session = self.session(request)
return session is not None and subject(session.name, request.client) is not None
def validate_silent_authorization(self, request):
return True
def validate_user_match(self, id_token_hint, scopes, claims, request):
"""Whether the user of the session, when there is one, is the subject of an ID token the server signed."""
session = self.session(request) if id_token_hint else None
if session is None:
return True
try:
hint = jwt.decode(id_token_hint, private_key().public_key(), algorithms=['RS256'],
options={'verify_exp': False, 'verify_aud': False})
except jwt.InvalidTokenError:
return False
person = subject(session.name, request.client)
return person is not None and hint.get('sub') == person['sub']
class DeviceCodeGrant(GrantTypeBase):
"""The token request of the device authorization grant (RFC 8628): a device code the user allowed gives an access
token, a refresh token and, with openid, an ID token; until then the request is answered authorization_pending or
slow_down."""
def create_token_response(self, request, token_handler):
headers = self._get_default_headers()
try:
self.validate_token_request(request)
except OAuth2Error as error:
headers.update(error.headers)
return headers, error.json, error.status_code
token = token_handler.create_token(request, refresh_token=True)
if 'openid' in request.scopes:
token['id_token'] = self.request_validator.finalize_id_token(
{'aud': request.client.client_id, 'iat': int(time.time())}, token, token_handler, request)
self.request_validator.save_token(token, request)
self.request_validator.delete_device(request.device_code)
return headers, json.dumps(token), 200
def validate_token_request(self, request):
if request.grant_type != DEVICE_GRANT:
raise UnsupportedGrantTypeError(request=request)
for param in ('grant_type', 'device_code'):
if param in request.duplicate_params:
raise InvalidRequestError(description=f'Duplicate {param} parameter.', request=request)
request.device_code = getattr(request, 'device_code', None)
if not request.device_code:
raise InvalidRequestError(description='Missing device_code parameter.', request=request)
if self.request_validator.client_authentication_required(request):
if not self.request_validator.authenticate_client(request):
raise InvalidClientError(request=request)
elif not self.request_validator.authenticate_client_id(request.client_id, request):
raise InvalidClientError(request=request)
self.validate_grant_type(request)
self.request_validator.poll_device(request.device_code, request)
def user_code_of(value):
"""The user code a user typed, without the dash and spaces, in capitals; '' when it is not one."""
code = ''.join(character for character in (value or '').upper() if character.isalnum())
return code if len(code) == USER_CODE_LENGTH and all(character in USER_CODE_LETTERS for character in code) else ''
def device_authorization(validator, cfg, text, headers):
"""Answer a device authorization request (RFC 8628 3.1): the client, public by client_id or confidential with its
secret, with the device_code grant, the scopes and the resource; the device code, the user code and the
verification URIs."""
issuer = linuxmanager.oidc_issuer(cfg)
request = Request(f'{issuer}/oidc/device_authorization', 'POST', text, headers)
for param in ('scope', 'resource'):
if param in request.duplicate_params:
raise InvalidRequestError(description=f'Duplicate {param} parameter.', request=request)
if validator.client_authentication_required(request):
if not validator.authenticate_client(request):
raise InvalidClientError(request=request)
elif not validator.authenticate_client_id(request.client_id, request):
raise InvalidClientError(request=request)
if not validator.validate_grant_type(request.client.client_id, DEVICE_GRANT, request.client, request):
raise UnauthorizedClientError(description='the client may not use the device authorization grant', request=request)
scopes = utils.scope_to_list(request.scope) or ['openid']
target = target_of(issuer, getattr(request, 'resource', None), scopes)
if not scopes_allowed(request.client, scopes, target, DEVICE_GRANT):
raise InvalidScopeError(description='the client may not ask for these scopes; openid is required', request=request)
device_code = secrets.token_urlsafe(32)
user_code = ''.join(secrets.choice(USER_CODE_LETTERS) for _ in range(USER_CODE_LENGTH))
with linuxmanager_auth.database(cfg) as connection:
connection.execute('CALL oidc_device_save(%s, %s, %s, %s, %s, %s, %s)',
(linuxmanager_auth.digest(device_code), linuxmanager_auth.digest(user_code), request.client.client_id,
scopes, target, DEVICE_INTERVAL, DEVICE_SECONDS))
shown = f'{user_code[:USER_CODE_LENGTH // 2]}-{user_code[USER_CODE_LENGTH // 2:]}'
return document(200, {'device_code': device_code, 'user_code': shown, 'verification_uri': f'{issuer}/oidc/device',
'verification_uri_complete': f'{issuer}/oidc/device?' + urllib.parse.urlencode({'user_code': shown}),
'expires_in': DEVICE_SECONDS, 'interval': DEVICE_INTERVAL})
def device_of(cfg, user_code):
"""The waiting device code of a user code, {client_id, scopes, resource}, or None."""
code = user_code_of(user_code)
if not code:
return None
with linuxmanager_auth.database(cfg) as connection:
row = connection.execute('SELECT * FROM oidc_device_find(%s)', (linuxmanager_auth.digest(code),)).fetchone()
return dict(zip(('client_id', 'scopes', 'resource'), row)) if row else None
def page(status, cfg, title, message='', error='', form=False, request='', name='', headers=(), action='/oidc/login',
device=False, user_code='', decide=False, items=()):
"""An HTML page of TEMPLATE: a message, an error and items; with form the sign-in form, posted to action with the
authorization request, or with device the user code field, with or without the sign-in fields; with decide the
Allow and Deny buttons of the device of the user code."""
body = _pages.get_template(TEMPLATE).render(title=title, message=message, error=error, form=form, request=request,
name=name, issuer=linuxmanager.oidc_issuer(cfg), action=action, device=device,
user_code=user_code, decide=decide, items=list(items))
return status, 'text/html; charset=utf-8', body.encode(), [*PAGE_HEADERS, *headers]
def document(status, value, headers=()):
return status, 'application/json', json.dumps(value, indent=2).encode() + b'\n', list(headers)
def relayed(headers, body, status):
"""An answer of oauthlib's server, without the headers the web service sets."""
kind = next((value for name, value in headers.items() if name.lower() == 'content-type'), 'application/json')
return status, kind, (body or '').encode(), [(name, value) for name, value in headers.items()
if name.lower() not in OWN_HEADERS]
def redirect(location, headers=()):
return 302, 'text/plain; charset=utf-8', b'', [('Location', location), *headers]
def cookie(value, seconds):
return 'Set-Cookie', f'{COOKIE}={value}; Path=/oidc/; Max-Age={seconds}; Secure; HttpOnly; SameSite=Lax'
def sign_in(form):
"""The name and the amr of a sign-in with the form's name, password and code, or its passkey; Failure when it is
refused, ValueError when the passkey's answer is not JSON."""
if form.get('passkey_id'):
with linuxmanager_users.session() as connection:
name = linuxmanager_passkeys.login_complete(connection, form['passkey_id'], json.loads(form.get('credential', '')))
return name, PASSKEY_AMR
name, code = form.get('name', '').strip(), form.get('code', '').strip()
linuxmanager_users.login(name, form.get('password', ''), code or None)
with directory() as connection:
if linuxmanager_users.service(linuxmanager_users.entry(connection, name)):
raise linuxmanager_users.Failure('the name, the password or the code is wrong', 5)
return name, OTP_AMR if code else PASSWORD_AMR
def start_session(cfg, name, amr):
"""Start a session of the user: a namespace of the Session and the header that sets its cookie."""
value = secrets.token_urlsafe(32)
with linuxmanager_auth.database(cfg) as connection:
connection.execute('CALL oidc_session_save(%s, %s, %s, %s)', (linuxmanager_auth.digest(value), name, amr, SESSION_SECONDS))
return types.SimpleNamespace(session=Session(name, datetime.datetime.now(datetime.timezone.utc), amr),
cookie=cookie(value, SESSION_SECONDS))
def authorize(server, cfg, params, headers, fresh=None, name='', error=''):
"""Answer an authorization request, the query or the form params: the sign-in page while the browser has no
session, or with prompt=login or max_age older than the session, else a redirect to the client with a code.
fresh is the session of a sign-in just made, which counts whatever prompt and max_age say; the redirect then sets
its cookie, fresh.cookie. error is why a sign-in failed: the sign-in page shows it, filled with the name.
"""
uri = f'{linuxmanager.oidc_issuer(cfg)}/oidc/authorize?{params}'
try:
scopes, info = server.validate_authorization_request(uri, 'GET', None, headers)
except FatalClientError as error:
return page(error.status_code, cfg, 'The sign-in request is refused', error=error.description or error.error)
except OAuth2Error as error:
if not error.redirect_uri:
return page(error.status_code, cfg, 'The sign-in request is refused', error=error.description or error.error)
return redirect(error.in_uri(error.redirect_uri))
request = info['request']
title = f'Sign in to {request.client.display_name or request.client.client_id}'
if info.get('code_challenge_method') != 'S256':
return redirect(InvalidRequestError(description='PKCE with code_challenge_method S256 is required',
request=request).in_uri(request.redirect_uri))
if error:
return page(401, cfg, title, error=error, form=True, request=params, name=name)
session = fresh.session if fresh else session_of(cfg, headers.get('Cookie'))
max_age = getattr(request, 'max_age', None)
stale = session is not None and isinstance(max_age, str) and max_age.isdigit() and \
time.time() - session.auth_time.timestamp() > int(max_age)
prompt = info.get('prompt') or set()
if session is None or (not fresh and ('login' in prompt or stale)):
if 'none' in prompt:
return redirect(LoginRequired(request=request).in_uri(request.redirect_uri))
return page(200, cfg, title, form=True, request=params, name=name)
if subject(session.name, request.client) is None:
if fresh:
return redirect(AccessDeniedError(description=f'{session.name} may not sign in to this client',
request=request).in_uri(request.redirect_uri), [fresh.cookie])
return page(403, cfg, title, error=f'{session.name} may not sign in here; sign in as another user.', form=True,
request=params)
answer, _, _ = server.create_authorization_response(uri, 'GET', None, headers, scopes, credentials={
'user': session.name, 'auth_time': session.auth_time, 'amr': session.amr})
return redirect(answer['Location'], [fresh.cookie] if fresh else [])
def login(server, cfg, form, headers):
"""Sign a user in with the sign-in page's form, start the session and go on with its authorization request."""
params = form.get('request', '')
name = form.get('name', '').strip()
try:
name, amr = sign_in(form)
except linuxmanager_users.Failure as failure:
return authorize(server, cfg, params, headers, name=name, error=str(failure))
except ValueError:
return authorize(server, cfg, params, headers, name=name, error='the passkey is not accepted')
return authorize(server, cfg, params, headers, fresh=start_session(cfg, name, amr))
def device_page(cfg, status=200, signed_in=False, user_code='', name='', error='', headers=()):
"""The page where the user enters the user code of a device, with the sign-in fields unless signed_in."""
return page(status, cfg, 'Sign in on a device', message='Enter the code the device shows.', error=error,
form=not signed_in, name=name, headers=headers, action='/oidc/device', device=True, user_code=user_code)
def device(cfg, form, headers):
"""The device page's form: sign in when the browser has no session, then ask the user to allow the device of the
user code."""
user_code, fresh = form.get('user_code', ''), None
session = session_of(cfg, headers.get('Cookie'))
if session is None:
try:
name, amr = sign_in(form)
except linuxmanager_users.Failure as failure:
return device_page(cfg, 401, user_code=user_code, name=form.get('name', '').strip(), error=str(failure))
except ValueError:
return device_page(cfg, 401, user_code=user_code, name=form.get('name', '').strip(),
error='the passkey is not accepted')
fresh = start_session(cfg, name, amr)
session = fresh.session
extra = [fresh.cookie] if fresh else []
found = device_of(cfg, user_code)
client = client_of(found['client_id']) if found else None
if client is None:
return device_page(cfg, 400, signed_in=True, user_code=user_code, error='the code is wrong or has expired',
headers=extra)
shown = client.display_name or client.client_id
if subject(session.name, client) is None:
return device_page(cfg, 403, signed_in=True, user_code=user_code, headers=extra,
error=f'{session.name} may not sign in to {shown}')
items = ([f'resource: {found["resource"]}'] if found['resource'] else []) + [f'scope: {scope}' for scope in found['scopes']]
return page(200, cfg, f'Allow {shown}?', message=f'{shown} asks to sign in as {session.name} on a device.',
items=items, decide=True, user_code=user_code_of(user_code), headers=extra)
def decide(cfg, form, headers):
"""Allow or deny the device of the user code as the user of the session."""
session = session_of(cfg, headers.get('Cookie'))
if session is None:
return device_page(cfg, 401, user_code=form.get('user_code', ''), error='sign in first')
found = device_of(cfg, form.get('user_code', ''))
client = client_of(found['client_id']) if found else None
if client is None or subject(session.name, client) is None:
return device_page(cfg, 400, signed_in=True, user_code=form.get('user_code', ''),
error='the code is wrong or has expired')
allowed = form.get('decision') == 'allow'
with linuxmanager_auth.database(cfg) as connection:
decided = connection.execute('SELECT oidc_device_decide(%s, %s, %s, %s, %s)',
(linuxmanager_auth.digest(user_code_of(form['user_code'])), session.name,
session.auth_time, session.amr, allowed)).fetchone()[0]
if not decided:
return device_page(cfg, 400, signed_in=True, error='the code is wrong or has expired')
shown = client.display_name or client.client_id
if allowed:
return page(200, cfg, 'The device is signed in', message=f'{shown} is signed in as {session.name}. You can return '
'to the device.')
return page(200, cfg, 'The device is denied', message=f'{shown} is not signed in.')
def logout(cfg, query, headers):
"""End the session the cookie names, then go to post_logout_redirect_uri, with state, when it is a post-logout
redirect URI of the client that client_id or the audience of id_token_hint names; else show that the user is
signed out."""
values = dict(urllib.parse.parse_qsl(query))
value = cookie_value(headers.get('Cookie'))
if value:
with linuxmanager_auth.database(cfg) as connection:
connection.execute('CALL oidc_session_delete(%s)', (linuxmanager_auth.digest(value),))
client_id = values.get('client_id')
if not client_id and values.get('id_token_hint'):
try:
audience = jwt.decode(values['id_token_hint'], private_key().public_key(), algorithms=['RS256'],
options={'verify_exp': False, 'verify_aud': False}).get('aud')
except jwt.InvalidTokenError:
audience = None
client_id = audience[0] if isinstance(audience, list) and audience else audience
client = client_of(client_id)
target = values.get('post_logout_redirect_uri')
expired = cookie('', 0)
if client is not None and target in client.post_logout_redirect_uris:
return redirect(add_params_to_uri(target, [('state', values['state'])]) if values.get('state') else target, [expired])
return page(200, cfg, 'Signed out', message='You are signed out of the server.', headers=[expired])
def answer(method, target, headers, body):
"""Answer a request of the OpenID Provider: the HTTP status, the content type, the body and [(header, value)]; 404
while oidc.enabled is false. target is the path with the query, headers a mapping of the request's headers and
body its bytes; HEAD is answered as GET."""
path, _, query = target.partition('?')
method = 'GET' if method == 'HEAD' else method
try:
cfg = linuxmanager.load_config()
if not cfg['oidc']['enabled']:
return 404, 'text/plain; charset=utf-8', b'Not found\n', []
issuer = linuxmanager.oidc_issuer(cfg)
headers = dict(headers.items())
text = body.decode('utf-8', 'replace') if body else ''
validator = Validator(cfg)
server = Server(validator, token_expires_in=ACCESS_SECONDS, token_generator=lambda request: access_token(cfg, request),
refresh_token_generator=random_token_generator)
server.grant_types[DEVICE_GRANT] = DeviceCodeGrant(validator)
if (method, path) == ('GET', '/.well-known/openid-configuration'):
return document(200, discovery(issuer, resource_scopes()))
if (method, path) == ('GET', '/oidc/jwks'):
return document(200, {'keys': [public_jwk(private_key())]})
if path == '/oidc/authorize' and method in ('GET', 'POST'):
return authorize(server, cfg, query if method == 'GET' else text, headers)
if (method, path) == ('POST', '/oidc/login'):
return login(server, cfg, dict(urllib.parse.parse_qsl(text)), headers)
if (method, path) == ('POST', '/oidc/passkey'):
return document(200, linuxmanager_passkeys.login_begin())
if (method, path) == ('POST', '/oidc/token'):
return relayed(*server.create_token_response(f'{issuer}{path}', 'POST', text, headers))
if (method, path) == ('POST', '/oidc/device_authorization'):
return device_authorization(validator, cfg, text, headers)
if (method, path) == ('GET', '/oidc/device'):
signed_in = session_of(cfg, headers.get('Cookie')) is not None
return device_page(cfg, signed_in=signed_in, user_code=dict(urllib.parse.parse_qsl(query)).get('user_code', ''))
if (method, path) == ('POST', '/oidc/device'):
return device(cfg, dict(urllib.parse.parse_qsl(text)), headers)
if (method, path) == ('POST', '/oidc/device/decide'):
return decide(cfg, dict(urllib.parse.parse_qsl(text)), headers)
if path == '/oidc/userinfo' and method in ('GET', 'POST'):
return relayed(*server.create_userinfo_response(f'{issuer}{target}', method, text or None, headers))
if (method, path) == ('GET', '/oidc/logout'):
return logout(cfg, query, headers)
return 404, 'text/plain; charset=utf-8', b'Not found\n', []
except OAuth2Error as error:
return error.status_code, 'application/json', error.json.encode(), list(error.headers.items())
except linuxmanager_users.Failure as failure:
return document(400 if failure.status == 2 else 500, {'error': str(failure)})
except (ldap.LDAPError, OSError) as exc:
return document(500, {'error': f'the OpenID Provider failed: {exc}'})
except linuxmanager_auth.DatabaseError as exc:
return document(500, {'error': str(exc)})
except SystemExit as exc:
return document(500, {'error': str(exc.code)})
except ValueError as exc:
return document(400, {'error': 'invalid_request', 'error_description': str(exc)})
@@ -1,17 +1,18 @@
"""The passkeys of the router's users, for signing in to the Router Manager and the OpenID Provider (vpnrouter_oidc),
with python3-fido2.
"""The passkeys of the server's users, for signing in to the Manager and the OpenID Provider (linuxmanager_oidc),
and for opening a shell, with python3-fido2.
A passkey is a discoverable WebAuthn credential with user verification, of the relying party
wan.local_fqdn, so the browser offers it at the HTTPS addresses on that name.
Its user handle is the user name. Each passkey of a user is a value of vpnRouterPasskey on the
Its user handle is the user name. Each passkey of a user is a value of linuxManagerPasskey on the
user entry: a JSON object {"id", "credential", "label", "created"}, id the credential ID and
credential the attested credential data (with the public key), both base64url. A user has at
most PASSKEYS_MAX passkeys; service accounts have none.
Registration and sign-in are two steps each: begin answers the options for the browser and a
pending id, whose state the web service keeps in memory for PENDING_SECONDS; complete takes the
id and the browser's answer. A request that cannot be done raises vpnrouter_users.Failure, with
the statuses of that module; a sign-in that fails gives status 5 whatever the reason.
Registration, sign-in and the proof of a signed-in user for a shell are two steps each: begin
answers the options for the browser and a pending id, whose state the web service keeps in memory
for PENDING_SECONDS; complete takes the id and the browser's answer. A request that cannot be done
raises linuxmanager_users.Failure, with the statuses of that module; a sign-in or a proof that fails
gives status 5 whatever the reason.
"""
import json
@@ -19,20 +20,14 @@ import secrets
import threading
import time
import fido2.features
import ldap
from fido2.server import Fido2Server
from fido2.utils import websafe_decode, websafe_encode
from fido2.webauthn import (AttestedCredentialData, PublicKeyCredentialRpEntity, PublicKeyCredentialUserEntity,
ResidentKeyRequirement, UserVerificationRequirement)
import vpnrouter
import vpnrouter_users
# The options and the browser's answers are in the JSON mapping of WebAuthn Level 3; python3-fido2 2.0 always uses it.
_json_mapping = getattr(fido2.features, 'webauthn_json_mapping', None)
if _json_mapping is not None:
_json_mapping.enabled = True
import linuxmanager
import linuxmanager_users
#: The most passkeys of one user.
PASSKEYS_MAX = 10
@@ -50,9 +45,9 @@ _lock = threading.Lock()
def server():
"""The Fido2Server of the relying party wan.local_fqdn; passkeys need the name."""
fqdn = vpnrouter.load_config()['wan']['local_fqdn'].strip()
fqdn = linuxmanager.load_config()['wan']['local_fqdn'].strip()
if not fqdn:
raise vpnrouter_users.Failure('passkeys need wan.local_fqdn, the name the Router Manager is reached at', 2)
raise linuxmanager_users.Failure('passkeys need wan.local_fqdn, the name the Manager is reached at', 2)
return Fido2Server(rp=PublicKeyCredentialRpEntity(name=fqdn, id=fqdn))
@@ -64,7 +59,7 @@ def keep(kind, name, state):
for old in [key for key, (*_, expires) in _pending.items() if expires <= now]:
del _pending[old]
if len(_pending) >= PENDING_MAX:
raise vpnrouter_users.Failure('too many passkey requests at once; retry in a moment')
raise linuxmanager_users.Failure('too many passkey requests at once; retry in a moment')
_pending[pending] = (kind, name, state, now + PENDING_SECONDS)
return pending
@@ -81,16 +76,16 @@ def take(kind, name, pending):
def credentials(attributes):
"""The attested credential data of the passkeys of an entry."""
return [AttestedCredentialData(websafe_decode(passkey['credential']))
for _, passkey in vpnrouter_users.passkeys_of(attributes)]
for _, passkey in linuxmanager_users.passkeys_of(attributes)]
def register_begin(connection, name):
"""Begin the registration of a passkey of the user: {"id", "options"}, options those of navigator.credentials.create."""
current = vpnrouter_users.entry(connection, name)
if vpnrouter_users.service(current):
raise vpnrouter_users.Failure(f'{name} is a service account, which has no passkeys', 2)
if len(vpnrouter_users.passkeys_of(current)) >= PASSKEYS_MAX:
raise vpnrouter_users.Failure(f'user {name} has {PASSKEYS_MAX} passkeys, the most; remove one first', 2)
current = linuxmanager_users.entry(connection, name)
if linuxmanager_users.service(current):
raise linuxmanager_users.Failure(f'{name} is a service account, which has no passkeys', 2)
if len(linuxmanager_users.passkeys_of(current)) >= PASSKEYS_MAX:
raise linuxmanager_users.Failure(f'user {name} has {PASSKEYS_MAX} passkeys, the most; remove one first', 2)
options, state = server().register_begin(
PublicKeyCredentialUserEntity(id=name.encode(), name=name, display_name=name),
credentials(current),
@@ -101,44 +96,44 @@ def register_begin(connection, name):
def register_complete(connection, name, pending, response, label):
"""Store the passkey the browser created for a begun registration, with its label; the user (vpnrouter_users.summary)."""
"""Store the passkey the browser created for a begun registration, with its label; the user (linuxmanager_users.summary)."""
state = take('register', name, pending)
if state is None:
raise vpnrouter_users.Failure('the registration is unknown or expired; begin it again', 2)
raise linuxmanager_users.Failure('the registration is unknown or expired; begin it again', 2)
if label is None or (isinstance(label, str) and not label.strip()):
label = 'Passkey'
if not isinstance(label, str) or len(label.strip()) > LABEL_MAX:
raise vpnrouter_users.Failure(f'the label must be text of at most {LABEL_MAX} characters', 2)
raise linuxmanager_users.Failure(f'the label must be text of at most {LABEL_MAX} characters', 2)
if not isinstance(response, dict):
raise vpnrouter_users.Failure('credential must be the answer of navigator.credentials.create as JSON', 2)
raise linuxmanager_users.Failure('credential must be the answer of navigator.credentials.create as JSON', 2)
try:
data = server().register_complete(state, response).credential_data
except (ValueError, KeyError, TypeError) as exc:
raise vpnrouter_users.Failure(f'the passkey is refused: {exc}', 2)
current = vpnrouter_users.entry(connection, name)
raise linuxmanager_users.Failure(f'the passkey is refused: {exc}', 2)
current = linuxmanager_users.entry(connection, name)
value = json.dumps({
'id': websafe_encode(data.credential_id),
'credential': websafe_encode(bytes(data)),
'label': label.strip(),
'created': time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime()),
}, separators=(',', ':'))
changes = [] if 'vpnrouteraccount' in vpnrouter_users.classes(current) else [(ldap.MOD_ADD, 'objectClass', [b'vpnRouterAccount'])]
changes.append((ldap.MOD_ADD, 'vpnRouterPasskey', [value.encode()]))
connection.modify_s(vpnrouter_users.dn_of(name), changes)
return vpnrouter_users.summary(vpnrouter_users.entry(connection, name))
changes = [] if 'linuxmanageraccount' in linuxmanager_users.classes(current) else [(ldap.MOD_ADD, 'objectClass', [b'linuxManagerAccount'])]
changes.append((ldap.MOD_ADD, 'linuxManagerPasskey', [value.encode()]))
connection.modify_s(linuxmanager_users.dn_of(name), changes)
return linuxmanager_users.summary(linuxmanager_users.entry(connection, name))
def remove(connection, name, passkey_id=None):
"""Remove the user's passkey of the id, or all its passkeys without one; the user (vpnrouter_users.summary)."""
current = vpnrouter_users.entry(connection, name)
values = current.get('vpnRouterPasskey', [])
kept = [value for value, passkey in vpnrouter_users.passkeys_of(current)
"""Remove the user's passkey of the id, or all its passkeys without one; the user (linuxmanager_users.summary)."""
current = linuxmanager_users.entry(connection, name)
values = current.get('linuxManagerPasskey', [])
kept = [value for value, passkey in linuxmanager_users.passkeys_of(current)
if passkey_id is not None and passkey['id'] != passkey_id]
if passkey_id is not None and len(kept) == len(vpnrouter_users.passkeys_of(current)):
raise vpnrouter_users.Failure(f'user {name} has no passkey {passkey_id}', 3)
if passkey_id is not None and len(kept) == len(linuxmanager_users.passkeys_of(current)):
raise linuxmanager_users.Failure(f'user {name} has no passkey {passkey_id}', 3)
if len(kept) != len(values):
connection.modify_s(vpnrouter_users.dn_of(name), [(ldap.MOD_REPLACE, 'vpnRouterPasskey', kept)])
return vpnrouter_users.summary(vpnrouter_users.entry(connection, name))
connection.modify_s(linuxmanager_users.dn_of(name), [(ldap.MOD_REPLACE, 'linuxManagerPasskey', kept)])
return linuxmanager_users.summary(linuxmanager_users.entry(connection, name))
def login_begin():
@@ -150,7 +145,7 @@ def login_begin():
def login_complete(connection, pending, response):
"""The name of the user whose passkey signed a begun sign-in, when the user is enabled, not locked and not a
service account."""
refused = vpnrouter_users.Failure('the passkey is not accepted', 5)
refused = linuxmanager_users.Failure('the passkey is not accepted', 5)
state = take('login', None, pending)
if state is None or not isinstance(response, dict):
raise refused
@@ -158,13 +153,13 @@ def login_complete(connection, pending, response):
name = websafe_decode(response['response']['userHandle']).decode()
except (KeyError, TypeError, ValueError):
raise refused
if not vpnrouter_users.NAME.match(name):
if not linuxmanager_users.NAME.match(name):
raise refused
try:
current = vpnrouter_users.entry(connection, name)
except vpnrouter_users.Failure:
current = linuxmanager_users.entry(connection, name)
except linuxmanager_users.Failure:
raise refused
user = vpnrouter_users.summary(current)
user = linuxmanager_users.summary(current)
if user['service'] or not user['enabled'] or user['locked']:
raise refused
try:
@@ -172,3 +167,30 @@ def login_complete(connection, pending, response):
except (ValueError, KeyError, TypeError):
raise refused
return name
def shell_begin(connection, name):
"""Begin the proof of the signed-in user for a shell: {"id", "options"}, options those of navigator.credentials.get
for the user's own passkeys, with user verification."""
own = credentials(linuxmanager_users.entry(connection, name))
if not own:
raise linuxmanager_users.Failure(f'{name} has no passkey; add one on the Account page', 2)
options, state = server().authenticate_begin(own, user_verification=UserVerificationRequirement.REQUIRED)
return {'id': keep('shell', name, state), 'options': dict(options)}
def shell_complete(connection, name, pending, response):
"""Check the browser's answer to a begun proof of the user for a shell: one of the user's own passkeys, with the
user verified, while the user is enabled and not locked."""
refused = linuxmanager_users.Failure('the passkey is not accepted', 5)
state = take('shell', name, pending)
if state is None or not isinstance(response, dict):
raise refused
current = linuxmanager_users.entry(connection, name)
user = linuxmanager_users.summary(current)
if not user['enabled'] or user['locked']:
raise refused
try:
server().authenticate_complete(state, credentials(current), response)
except (ValueError, KeyError, TypeError):
raise refused
@@ -1,18 +1,18 @@
"""The router's PKI, for vpn-router-setup, the web service, vpn-router-ca and the user module.
"""The server's PKI, for linux-manager-setup, the web service, linux-manager-ca and the user module.
The root CA signs the issuing CAs, and only them: p2s, which signs the router's VPN server certificate and the users'
certificates, s2s, which signs the router's site-to-site certificate and those of its peers, and the general-purpose
CAs of pki.issuing_cas. The PKI is kept in ou=pki of the router's directory (vpnrouter_ldap), which root reaches through
the local socket. Each CA is an entry cn=<name> of class vpnRouterAuthority, with its certificate, key, CRL and CRL
number; while the root key is not on the router, a new issuing CA has its key and a CSR and is pending until import_ca
The root CA signs the issuing CAs, and only them: p2s, which signs the server's VPN server certificate and the users'
certificates, s2s, which signs the server's site-to-site certificate and those of its peers, and the general-purpose
CAs of pki.issuing_cas. The PKI is kept in ou=pki of the server's directory (linuxmanager_ldap), which root reaches through
the local socket. Each CA is an entry cn=<name> of class linuxManagerAuthority, with its certificate, key, CRL and CRL
number; while the root key is not on the server, a new issuing CA has its key and a CSR and is pending until import_ca
takes its certificate, signed offline. Each certificate a CA issued is an entry serialNumber=<serial> of class
vpnRouterCertificate under the CA, with its kind, template, names, expiry and revocation; the router's own certificates
and the users' keep their keys there, as the router and the OpenVPN profile need them, and a certificate issued for
linuxManagerCertificate under the CA, with its kind, template, names, expiry and revocation; the server's own certificates
and the users' keep their keys there, as the server and the OpenVPN profile need them, and a certificate issued for
anything else is kept without its key. A user's certificate is also the userCertificate of the user's entry. A retired
CA keeps its entry, renamed cn=<name>-<time> and marked with vpnRouterRetired.
CA keeps its entry, renamed cn=<name>-<time> and marked with linuxManagerRetired.
An existing root CA is supplied by placing its certificate, and its key if the router may use it, in SEED_DIR before
vpn-router-setup runs; provision imports them into the directory and removes the files.
An existing root CA is supplied by placing its certificate, and its key if the server may use it, in SEED_DIR before
linux-manager-setup runs; provision imports them into the directory and removes the files.
The certificates carry the URLs of pki.publish for their issuer's certificate (AIA) and CRL (CRL distribution point);
PUBLISH_DIR holds the DER copies served there. distribute writes the files strongSwan, OpenVPN and the system's trust
@@ -46,17 +46,17 @@ from cryptography.hazmat.primitives.asymmetric import ec, rsa
from cryptography.hazmat.primitives.serialization import pkcs12
from cryptography.x509.oid import AuthorityInformationAccessOID, NameOID
import vpnrouter
import vpnrouter_ldap
import linuxmanager
import linuxmanager_ldap
#: The certificate (cert.pem) and the key (key.pem) of an existing root CA, which provision imports and removes.
SEED_DIR = pathlib.Path('/etc/vpn-router/pki/root')
SEED_DIR = pathlib.Path('/etc/linux-manager/pki/root')
#: The DER copies of the CA certificates and CRLs, <name>.crt and <name>.crl, which the web service and Caddy serve.
PUBLISH_DIR = pathlib.Path('/var/lib/vpn-router/pki-publish')
PUBLISH_DIR = pathlib.Path('/var/lib/linux-manager/pki-publish')
#: The lock of the PKI and of the user module's sessions.
LOCK_FILE = pathlib.Path('/run/vpn-router/users.lock')
LOCK_FILE = pathlib.Path('/run/linux-manager/users.lock')
#: The copies strongSwan reads: one CA certificate per file, as strongSwan reads only the first of a file.
X509CA_DIR = pathlib.Path('/etc/swanctl/x509ca')
@@ -70,21 +70,25 @@ S2S_CERT = X509_DIR / 'site-to-site.pem'
S2S_KEY = PRIVATE_DIR / 'site-to-site.key'
#: What OpenVPN reads: the root and the P2S CA, and their CRLs, which it needs for every CA of a client's chain.
OPENVPN_CA = pathlib.Path('/etc/openvpn/server/vpn-router-ca.pem')
OPENVPN_CRL = pathlib.Path('/etc/openvpn/server/vpn-router-crl.pem')
OPENVPN_CA = pathlib.Path('/etc/openvpn/server/linux-manager-ca.pem')
OPENVPN_CRL = pathlib.Path('/etc/openvpn/server/linux-manager-crl.pem')
#: The root CA in the system's trust store.
TRUST_ANCHOR = pathlib.Path('/usr/local/share/ca-certificates/vpn-router-ca.crt')
TRUST_ANCHOR = pathlib.Path('/usr/local/share/ca-certificates/linux-manager-ca.crt')
#: The router's own certificates: name -> the issuing CA and the template, and the files strongSwan reads.
#: The server's own certificates: name -> the issuing CA and the template, and the files strongSwan reads.
ROUTER_CERTS = {'vpn-server': ('p2s', 'vpn-server'), 'site-to-site': ('s2s', 'site-to-site')}
#: The component, a section of the configuration, that a built-in issuing CA and the server's certificate it issues
#: belong to: provision creates them only while the component's enabled is true.
BUILTIN_COMPONENTS = {'p2s': 'vpn', 's2s': 'router'}
ROUTER_FILES = {'vpn-server': (VPN_SERVER_CERT, VPN_SERVER_KEY), 'site-to-site': (S2S_CERT, S2S_KEY)}
#: The attributes of a CA's entry and of a certificate's entry.
AUTHORITY_ATTRIBUTES = ['cACertificate;binary', 'certificateRevocationList;binary', 'vpnRouterPrivateKey',
'vpnRouterCertificateRequest', 'vpnRouterCRLNumber', 'vpnRouterCRLSource', 'vpnRouterRetired']
CERTIFICATE_ATTRIBUTES = ['serialNumber', 'cn', 'vpnRouterKind', 'vpnRouterTemplate', 'vpnRouterSubjectAltName',
'vpnRouterNotAfter', 'vpnRouterRevoked', 'vpnRouterRevocationReason', 'createTimestamp']
AUTHORITY_ATTRIBUTES = ['cACertificate;binary', 'certificateRevocationList;binary', 'linuxManagerPrivateKey',
'linuxManagerCertificateRequest', 'linuxManagerCRLNumber', 'linuxManagerCRLSource', 'linuxManagerRetired']
CERTIFICATE_ATTRIBUTES = ['serialNumber', 'cn', 'linuxManagerKind', 'linuxManagerTemplate', 'linuxManagerSubjectAltName',
'linuxManagerNotAfter', 'linuxManagerRevoked', 'linuxManagerRevocationReason', 'createTimestamp']
#: The suffix of a retired CA's name: -<time of the retirement>.
RETIRED_SUFFIX = re.compile(r'-\d{14}$')
@@ -127,8 +131,8 @@ def locked():
@contextlib.contextmanager
def directory():
"""A connection to the router's directory as root, closed at the end."""
connection = vpnrouter_ldap.root()
"""A connection to the server's directory as root, closed at the end."""
connection = linuxmanager_ldap.root()
try:
yield connection
finally:
@@ -208,7 +212,7 @@ def first(attributes, name):
def authority_dn(name):
return f'cn={ldap.dn.escape_dn_chars(name)},{vpnrouter_ldap.PKI}'
return f'cn={ldap.dn.escape_dn_chars(name)},{linuxmanager_ldap.PKI}'
def authority(connection, name):
@@ -222,11 +226,11 @@ def authority(connection, name):
'name': name,
'cert': x509.load_der_x509_certificate(cert) if cert else None,
'key': serialization.load_pem_private_key(key, None) if key else None,
'csr': first(attributes, 'vpnRouterCertificateRequest'),
'csr': first(attributes, 'linuxManagerCertificateRequest'),
'crl': x509.load_der_x509_crl(crl) if crl else None,
'crl_number': int(first(attributes, 'vpnRouterCRLNumber') or 0),
'crl_source': (first(attributes, 'vpnRouterCRLSource') or b'').decode() or None,
'retired': from_generalized(first(attributes, 'vpnRouterRetired')),
'crl_number': int(first(attributes, 'linuxManagerCRLNumber') or 0),
'crl_source': (first(attributes, 'linuxManagerCRLSource') or b'').decode() or None,
'retired': from_generalized(first(attributes, 'linuxManagerRetired')),
}
@@ -237,7 +241,7 @@ def update(connection, name, values):
connection.search_s(dn, ldap.SCOPE_BASE, attrlist=['1.1'])
except ldap.NO_SUCH_OBJECT:
connection.add_s(dn, ldap.modlist.addModlist({
'objectClass': [b'vpnRouterAuthority'], 'cn': [name.encode()],
'objectClass': [b'linuxManagerAuthority'], 'cn': [name.encode()],
**{attribute: [value] for attribute, value in values.items() if value is not None}}))
return
connection.modify_s(dn, [(ldap.MOD_REPLACE, attribute, [value] if value is not None else [])
@@ -246,8 +250,8 @@ def update(connection, name, values):
def issuing(connection):
"""The names of the issuing CAs in the directory that are not retired, sorted."""
found = connection.search_s(vpnrouter_ldap.PKI, ldap.SCOPE_ONELEVEL,
'(&(objectClass=vpnRouterAuthority)(!(vpnRouterRetired=*))(!(cn=root)))', ['cn'])
found = connection.search_s(linuxmanager_ldap.PKI, ldap.SCOPE_ONELEVEL,
'(&(objectClass=linuxManagerAuthority)(!(linuxManagerRetired=*))(!(cn=root)))', ['cn'])
return sorted(attributes['cn'][0].decode() for _, attributes in found)
@@ -258,20 +262,24 @@ def certificate_entry(dn, attributes):
return (first(attributes, name) or b'').decode() or None
return {
'serial': value('serialNumber'), 'ca': ldap.dn.str2dn(dn)[1][0][1], 'kind': value('vpnRouterKind'),
'template': value('vpnRouterTemplate'), 'subject': value('cn') or '',
'sans': [item.decode() for item in attributes.get('vpnRouterSubjectAltName', [])],
'serial': value('serialNumber'), 'ca': ldap.dn.str2dn(dn)[1][0][1], 'kind': value('linuxManagerKind'),
'template': value('linuxManagerTemplate'), 'subject': value('cn') or '',
'sans': [item.decode() for item in attributes.get('linuxManagerSubjectAltName', [])],
'created': from_generalized(first(attributes, 'createTimestamp')),
'expires': from_generalized(first(attributes, 'vpnRouterNotAfter')),
'revoked': from_generalized(first(attributes, 'vpnRouterRevoked')), 'reason': value('vpnRouterRevocationReason'),
'expires': from_generalized(first(attributes, 'linuxManagerNotAfter')),
'revoked': from_generalized(first(attributes, 'linuxManagerRevoked')), 'reason': value('linuxManagerRevocationReason'),
'dn': dn,
}
def certificate_entries(connection, base, condition=''):
"""The certificates' entries under base that match the LDAP condition, newest first."""
found = connection.search_s(base, ldap.SCOPE_SUBTREE, f'(&(objectClass=vpnRouterCertificate){condition})',
CERTIFICATE_ATTRIBUTES)
"""The certificates' entries under base that match the LDAP condition, newest first; none when base does not exist,
as for a built-in CA whose component was never on."""
try:
found = connection.search_s(base, ldap.SCOPE_SUBTREE, f'(&(objectClass=linuxManagerCertificate){condition})',
CERTIFICATE_ATTRIBUTES)
except ldap.NO_SUCH_OBJECT:
return []
return sorted((certificate_entry(dn, attributes) for dn, attributes in found),
key=lambda entry: entry['created'] or '', reverse=True)
@@ -280,45 +288,48 @@ def record(connection, cert, ca, kind, template=None, key=None):
"""Add a certificate the CA issued: kind is ca, router, user or external; key is kept for router and user."""
serial = f'{cert.serial_number:X}'
attributes = {
'objectClass': [b'vpnRouterCertificate'], 'serialNumber': [serial.encode()], 'cn': [common_name(cert).encode()],
'userCertificate;binary': [cert_der(cert)], 'vpnRouterKind': [kind.encode()],
'vpnRouterNotAfter': [generalized(utc(cert, 'not_valid_after'))],
'objectClass': [b'linuxManagerCertificate'], 'serialNumber': [serial.encode()], 'cn': [common_name(cert).encode()],
'userCertificate;binary': [cert_der(cert)], 'linuxManagerKind': [kind.encode()],
'linuxManagerNotAfter': [generalized(utc(cert, 'not_valid_after'))],
}
if template:
attributes['vpnRouterTemplate'] = [template.encode()]
attributes['linuxManagerTemplate'] = [template.encode()]
if san_values(cert):
attributes['vpnRouterSubjectAltName'] = [value.encode() for value in san_values(cert)]
attributes['linuxManagerSubjectAltName'] = [value.encode() for value in san_values(cert)]
if key is not None:
attributes['vpnRouterPrivateKey'] = [key_pem(key)]
attributes['linuxManagerPrivateKey'] = [key_pem(key)]
connection.add_s(f'serialNumber={serial},{authority_dn(ca)}', ldap.modlist.addModlist(attributes))
def holders(connection, ca, condition):
"""The certificates of the CA that match the LDAP condition and have their key, not revoked, newest first: [(dn,
cert, key)]."""
found = connection.search_s(
authority_dn(ca), ldap.SCOPE_ONELEVEL,
f'(&(objectClass=vpnRouterCertificate)(vpnRouterPrivateKey=*)(!(vpnRouterRevoked=*)){condition})',
['userCertificate;binary', 'vpnRouterPrivateKey'])
cert, key)]; none when the CA does not exist."""
try:
found = connection.search_s(
authority_dn(ca), ldap.SCOPE_ONELEVEL,
f'(&(objectClass=linuxManagerCertificate)(linuxManagerPrivateKey=*)(!(linuxManagerRevoked=*)){condition})',
['userCertificate;binary', 'linuxManagerPrivateKey'])
except ldap.NO_SUCH_OBJECT:
return []
pairs = [(dn, x509.load_der_x509_certificate(attributes['userCertificate;binary'][0]),
serialization.load_pem_private_key(attributes['vpnRouterPrivateKey'][0], None)) for dn, attributes in found]
serialization.load_pem_private_key(attributes['linuxManagerPrivateKey'][0], None)) for dn, attributes in found]
return sorted(pairs, key=lambda pair: utc(pair[1], 'not_valid_after'), reverse=True)
def router_condition(name):
return f'(vpnRouterKind=router)(vpnRouterTemplate={ROUTER_CERTS[name][1]})'
return f'(linuxManagerKind=router)(linuxManagerTemplate={ROUTER_CERTS[name][1]})'
def user_condition(name):
return f'(vpnRouterKind=user)(cn={ldap.filter.escape_filter_chars(name)})'
return f'(linuxManagerKind=user)(cn={ldap.filter.escape_filter_chars(name)})'
def drop_key(connection, dn):
connection.modify_s(dn, [(ldap.MOD_REPLACE, 'vpnRouterPrivateKey', [])])
connection.modify_s(dn, [(ldap.MOD_REPLACE, 'linuxManagerPrivateKey', [])])
def new_key(kind):
"""A new private key of one of vpnrouter.PKI_KEYS."""
"""A new private key of one of linuxmanager.PKI_KEYS."""
if kind.startswith('rsa-'):
return rsa.generate_private_key(public_exponent=65537, key_size=int(kind[4:]))
return ec.generate_private_key(ec.SECP384R1() if kind == 'ec-p384' else ec.SECP256R1())
@@ -421,7 +432,7 @@ def ca_extensions(path_length):
def general_names(values, allowed):
"""The subject alternative names of values, each a DNS name, an IP address or an e-mail address of the allowed
types (vpnrouter.SAN_TYPES)."""
types (linuxmanager.SAN_TYPES)."""
names = []
for value in values:
if not isinstance(value, str) or not value.strip():
@@ -447,12 +458,12 @@ def general_names(values, allowed):
def leaf_extensions(cfg, template, names):
"""The extensions of a certificate of the template with the subject alternative names: the extended key usages by
their OIDs (vpnrouter.extended_key_usages), and the template's policies as certificate policies."""
usage = x509.KeyUsage(**{option: option in template['key_usage'] for option in vpnrouter.KEY_USAGES},
their OIDs (linuxmanager.extended_key_usages), and the template's policies as certificate policies."""
usage = x509.KeyUsage(**{option: option in template['key_usage'] for option in linuxmanager.KEY_USAGES},
key_cert_sign=False, crl_sign=False, encipher_only=False, decipher_only=False)
extensions = [(x509.BasicConstraints(ca=False, path_length=None), True), (usage, True)]
if template['extended_key_usage']:
oids = vpnrouter.extended_key_usages(cfg)
oids = linuxmanager.extended_key_usages(cfg)
extensions.append((x509.ExtendedKeyUsage([x509.ObjectIdentifier(oids[item])
for item in template['extended_key_usage']]), False))
if template['policies']:
@@ -464,7 +475,7 @@ def leaf_extensions(cfg, template, names):
def check_subject(kind, subject):
"""Check a common name against the subject of a template (vpnrouter.TEMPLATE_SUBJECTS)."""
"""Check a common name against the subject of a template (linuxmanager.TEMPLATE_SUBJECTS)."""
if not isinstance(subject, str) or not subject.strip():
raise Failure('the subject is required', 2)
subject = subject.strip()
@@ -491,7 +502,7 @@ def sign_crl(connection, cfg, name):
.next_update(moment + datetime.timedelta(days=days))
.add_extension(x509.CRLNumber(number), False)
.add_extension(authority_key_id(cert), False))
for revoked_entry in certificate_entries(connection, authority_dn(name), '(vpnRouterRevoked=*)'):
for revoked_entry in certificate_entries(connection, authority_dn(name), '(linuxManagerRevoked=*)'):
if revoked_entry['expires'] and revoked_entry['expires'] <= iso(moment):
continue
revoked = (x509.RevokedCertificateBuilder()
@@ -503,8 +514,8 @@ def sign_crl(connection, cfg, name):
builder = builder.add_revoked_certificate(revoked.build())
crl = builder.sign(key, digest(key))
update(connection, name, {'certificateRevocationList;binary': crl.public_bytes(serialization.Encoding.DER),
'vpnRouterCRLNumber': str(number).encode(),
**({'vpnRouterCRLSource': b'signed'} if name == 'root' else {})})
'linuxManagerCRLNumber': str(number).encode(),
**({'linuxManagerCRLSource': b'signed'} if name == 'root' else {})})
def crl_stale(crl):
@@ -523,7 +534,7 @@ def ready(name):
def refresh_crls(connection, cfg, force=()):
"""Sign a new CRL for each CA with its key on the router whose CRL is stale, or that force names."""
"""Sign a new CRL for each CA with its key on the server whose CRL is stale, or that force names."""
for name in ['root', *issuing(connection)]:
entry = authority(connection, name)
if entry and entry['key'] and entry['cert'] and (name in force or crl_stale(entry['crl'])):
@@ -545,7 +556,7 @@ def is_address(value):
def router_identities(name, ctx):
"""The common name and the subject alternative names of one of the router's certificates (ROUTER_CERTS): the VPN
"""The common name and the subject alternative names of one of the server's certificates (ROUTER_CERTS): the VPN
server certificate names wan.local_fqdn, and the site-to-site certificate also the local identity of each
site_to_site connection with certificate authentication, the public or the internal address of its
local_id_mode; None while such an identity is not known yet."""
@@ -564,7 +575,7 @@ def router_identities(name, ctx):
def ensure_router_cert(connection, cfg, ctx, name, urls):
"""Issue one of the router's certificates when it has none, or its current one is due for renewal, not issued by
"""Issue one of the server's certificates when it has none, or its current one is due for renewal, not issued by
its CA's current key, or named otherwise than router_identities says; the older ones lose their keys. Nothing is
issued while an identity it needs is not known yet."""
ca, template_name = ROUTER_CERTS[name]
@@ -580,10 +591,10 @@ def ensure_router_cert(connection, cfg, ctx, name, urls):
if not renewal_due(cert) and issued_by(cert, issuer['cert']) and common_name(cert) == subject and \
wanted == set(san_values(cert)):
return
template = vpnrouter.pki_templates(cfg)[template_name]
template = linuxmanager.pki_templates(cfg)[template_name]
key = new_key(template['key'])
cert = sign(subject, key.public_key(), issuer['cert'], issuer['key'], template['days'],
leaf_extensions(cfg, template, general_names(sans, vpnrouter.SAN_TYPES)), urls, ca)
leaf_extensions(cfg, template, general_names(sans, linuxmanager.SAN_TYPES)), urls, ca)
record(connection, cert, ca, 'router', template_name, key)
for dn, _, _ in current:
drop_key(connection, dn)
@@ -602,7 +613,7 @@ def import_seed(connection):
raise Failure(f'{cert_file} is not a CA certificate')
if key is not None and not same_key(key.public_key(), cert.public_key()):
raise Failure(f'{key_file} is not the key of {cert_file}')
update(connection, 'root', {'cACertificate;binary': cert_der(cert), 'vpnRouterPrivateKey': key_pem(key) if key else None})
update(connection, 'root', {'cACertificate;binary': cert_der(cert), 'linuxManagerPrivateKey': key_pem(key) if key else None})
for path in (key_file, cert_file):
remove(path)
for folder in (SEED_DIR, SEED_DIR.parent):
@@ -615,14 +626,20 @@ def provision(cfg, ctx):
"""Create what the configuration needs and is missing, and keep it current; the notices to report.
The root is imported from SEED_DIR, or created when it has neither certificate nor seed. Each issuing CA without a
certificate is signed by the root key, or, while the root key is not on the router, gets its key and a CSR and is
certificate is signed by the root key, or, while the root key is not on the server, gets its key and a CSR and is
pending. An issuing CA that pki.issuing_cas no longer names is retired, and its certificate revoked on the root CRL.
Stale CRLs are signed again, the router's certificates issued or renewed (ensure_router_cert), and the copies of
PUBLISH_DIR written.
Stale CRLs are signed again, the server's certificates issued or renewed (ensure_router_cert), and the copies of
PUBLISH_DIR written. A built-in CA and the server's certificate it issues are created only while their component
(BUILTIN_COMPONENTS) is on; switching the component off keeps them. While the VPN is on, each VPN user without a
certificate gets one.
"""
pki = cfg['pki']
urls = vpnrouter.pki_base_urls(cfg)
urls = linuxmanager.pki_base_urls(cfg)
notices = []
def component_on(ca):
return ca not in BUILTIN_COMPONENTS or bool(cfg[BUILTIN_COMPONENTS[ca]]['enabled'])
with directory() as connection:
root = authority(connection, 'root')
if root is None or root['cert'] is None:
@@ -631,24 +648,24 @@ def provision(cfg, ctx):
if not import_seed(connection):
root_key = new_key(pki['key'])
root_cert = sign(pki['root_name'], root_key.public_key(), None, root_key, pki['root_days'], ca_extensions(1))
update(connection, 'root', {'cACertificate;binary': cert_der(root_cert), 'vpnRouterPrivateKey': key_pem(root_key)})
update(connection, 'root', {'cACertificate;binary': cert_der(root_cert), 'linuxManagerPrivateKey': key_pem(root_key)})
root = authority(connection, 'root')
root_cert, root_key = root['cert'], root['key']
authorities = vpnrouter.issuing_cas(cfg)
authorities = linuxmanager.issuing_cas(cfg)
force = set()
for name, settings in authorities.items():
entry = authority(connection, name)
if entry is not None and entry['cert'] is not None:
if (entry is not None and entry['cert'] is not None) or not component_on(name):
continue
key = entry['key'] if entry is not None and entry['key'] is not None else None
if key is None:
key = new_key(pki['key'])
update(connection, name, {'vpnRouterPrivateKey': key_pem(key)})
update(connection, name, {'linuxManagerPrivateKey': key_pem(key)})
if root_key is not None:
cert = sign(settings['name'].strip(), key.public_key(), root_cert, root_key, pki['ca_days'],
ca_extensions(0), urls, 'root')
update(connection, name, {'cACertificate;binary': cert_der(cert), 'vpnRouterCertificateRequest': None})
update(connection, name, {'cACertificate;binary': cert_der(cert), 'linuxManagerCertificateRequest': None})
record(connection, cert, 'root', 'ca')
force.add(name)
continue
@@ -658,9 +675,9 @@ def provision(cfg, ctx):
.add_extension(x509.BasicConstraints(ca=True, path_length=0), True)
.add_extension(CA_USAGE, True)
.sign(key, digest(key)))
update(connection, name, {'vpnRouterCertificateRequest': request.public_bytes(serialization.Encoding.PEM)})
notices.append(f'issuing CA {name} waits for its certificate: sign its CSR (vpn-router-ca ca csr {name}) with '
f'the root CA and import it with vpn-router-ca ca import {name}')
update(connection, name, {'linuxManagerCertificateRequest': request.public_bytes(serialization.Encoding.PEM)})
notices.append(f'issuing CA {name} waits for its certificate: sign its CSR (linux-manager-ca ca csr {name}) with '
f'the root CA and import it with linux-manager-ca ca import {name}')
for name in issuing(connection):
if name in authorities:
@@ -670,21 +687,28 @@ def provision(cfg, ctx):
if entry['cert'] is not None:
serial = f'{entry["cert"].serial_number:X}'
for revoked in certificate_entries(connection, authority_dn('root'),
f'(serialNumber={serial})(!(vpnRouterRevoked=*))'):
connection.modify_s(revoked['dn'], [(ldap.MOD_REPLACE, 'vpnRouterRevoked', [generalized(moment)]),
(ldap.MOD_REPLACE, 'vpnRouterRevocationReason', [b'cessation_of_operation'])])
f'(serialNumber={serial})(!(linuxManagerRevoked=*))'):
connection.modify_s(revoked['dn'], [(ldap.MOD_REPLACE, 'linuxManagerRevoked', [generalized(moment)]),
(ldap.MOD_REPLACE, 'linuxManagerRevocationReason', [b'cessation_of_operation'])])
force.add('root')
if root_key is None:
notices.append(f'issuing CA {name} was retired: revoke its certificate on the root CRL offline and '
'upload the CRL with vpn-router-ca root import-crl')
'upload the CRL with linux-manager-ca root import-crl')
retired = f'{name}-{moment.strftime("%Y%m%d%H%M%S")}'
connection.rename_s(authority_dn(name), f'cn={retired}')
update(connection, retired, {'vpnRouterRetired': generalized(moment), 'vpnRouterPrivateKey': None})
update(connection, retired, {'linuxManagerRetired': generalized(moment), 'linuxManagerPrivateKey': None})
refresh_crls(connection, cfg, force)
for name in ROUTER_CERTS:
ensure_router_cert(connection, cfg, ctx, name, urls)
if component_on(ROUTER_CERTS[name][0]):
ensure_router_cert(connection, cfg, ctx, name, urls)
publish(connection)
p2s = authority(connection, 'p2s')
missing = connection.search_s(linuxmanager_ldap.USERS, ldap.SCOPE_ONELEVEL,
'(&(objectClass=inetOrgPerson)(mail=*)(!(userCertificate=*)))', ['uid', 'mail']) \
if component_on('p2s') and p2s and p2s['cert'] and p2s['key'] and not p2s['retired'] else []
for _, attributes in missing:
issue_user(cfg, attributes['uid'][0].decode(), attributes['mail'][0].decode())
return notices
@@ -722,7 +746,7 @@ def install_crls(connection):
crls = {}
for name, file in SWANCTL_CA_FILES.items():
entry = authority(connection, name)
crls[name] = entry['crl'].public_bytes(serialization.Encoding.PEM) if entry and entry['crl'] else None
crls[name] = entry['crl'].public_bytes(serialization.Encoding.PEM) if entry and entry['crl'] is not None else None
changed |= write(X509CRL_DIR / file, crls[name], 0o644) if crls[name] else remove(X509CRL_DIR / file)
if crls['root'] and crls['p2s']:
changed |= write(OPENVPN_CRL, crls['root'] + crls['p2s'], 0o644)
@@ -738,7 +762,7 @@ def reload_crls(connection):
def distribute(cfg):
"""Write the certificates, keys and CRLs strongSwan, OpenVPN and the system's trust store read; True on change.
strongSwan gets the root, p2s and s2s, the remote CA of each site_to_site connection that has one, the router's
strongSwan gets the root, p2s and s2s, the remote CA of each site_to_site connection that has one, the server's
certificates and keys, and the CRLs; OpenVPN the root and p2s in one file, and their CRLs (install_crls).
"""
changed = False
@@ -800,7 +824,7 @@ def authority_files(name):
def issuer_of(connection, cfg, ca, template):
"""The entry of the issuing CA and the template, checked to be one the CA issues."""
authorities = vpnrouter.issuing_cas(cfg)
authorities = linuxmanager.issuing_cas(cfg)
if ca not in authorities:
raise Failure(f'no issuing CA {ca}', 3)
if template not in authorities[ca]['templates']:
@@ -808,19 +832,19 @@ def issuer_of(connection, cfg, ca, template):
entry = authority(connection, ca)
if not entry or entry['cert'] is None or entry['key'] is None:
raise Failure(f'issuing CA {ca} has no certificate yet', 4)
return entry, vpnrouter.pki_templates(cfg)[template]
return entry, linuxmanager.pki_templates(cfg)[template]
def issue(cfg, ca, template, csr=None, subject=None, sans=None, password=None):
"""Issue a certificate of the template from the issuing CA for something other than the router.
"""Issue a certificate of the template from the issuing CA for something other than the server.
With a CSR, the certificate is for the CSR's key, common name and subject alternative names, and the answer is
{serial, certificate, chain} in PEM. Without one, the router makes a key of the template for the subject and sans,
{serial, certificate, chain} in PEM. Without one, the server makes a key of the template for the subject and sans,
and the answer is {serial, pfx}: a PKCS#12 file in base64 with the key, the certificate and the chain, protected
by the password; the key is not kept. The templates of vpnrouter.ROUTER_TEMPLATES are not issued here.
by the password; the key is not kept. The templates of linuxmanager.ROUTER_TEMPLATES are not issued here.
"""
if template in vpnrouter.ROUTER_TEMPLATES:
raise Failure(f'{template} certificates are issued by the router only', 2)
if template in linuxmanager.ROUTER_TEMPLATES:
raise Failure(f'{template} certificates are issued by the server only', 2)
with directory() as connection:
issuer, values = issuer_of(connection, cfg, ca, template)
key = None
@@ -847,7 +871,7 @@ def issue(cfg, ca, template, csr=None, subject=None, sans=None, password=None):
names = general_names(([subject] if values['subject'] == 'dns' else []) + list(sans),
set(values['sans']) | ({'dns'} if values['subject'] == 'dns' else set()))
cert = sign(subject, public_key, issuer['cert'], issuer['key'], values['days'], leaf_extensions(cfg, values, names),
vpnrouter.pki_base_urls(cfg), ca)
linuxmanager.pki_base_urls(cfg), ca)
record(connection, cert, ca, 'external', template)
root = authority(connection, 'root')['cert']
serial = f'{cert.serial_number:X}'
@@ -864,7 +888,7 @@ def issue(cfg, ca, template, csr=None, subject=None, sans=None, password=None):
def user_dn(name):
return f'uid={ldap.dn.escape_dn_chars(name)},{vpnrouter_ldap.USERS}'
return f'uid={ldap.dn.escape_dn_chars(name)},{linuxmanager_ldap.USERS}'
def issue_user(cfg, name, email):
@@ -874,8 +898,8 @@ def issue_user(cfg, name, email):
issuer, values = issuer_of(connection, cfg, 'p2s', 'vpn-user')
key = new_key(values['key'])
cert = sign(name, key.public_key(), issuer['cert'], issuer['key'], values['days'],
leaf_extensions(cfg, values, general_names([name, email], vpnrouter.SAN_TYPES)),
vpnrouter.pki_base_urls(cfg), 'p2s')
leaf_extensions(cfg, values, general_names([name, email], linuxmanager.SAN_TYPES)),
linuxmanager.pki_base_urls(cfg), 'p2s')
for dn, _, _ in holders(connection, 'p2s', user_condition(name)):
drop_key(connection, dn)
record(connection, cert, 'p2s', 'user', 'vpn-user', key)
@@ -898,12 +922,12 @@ def revoke_entries(connection, cfg, entries, reason):
for name in names:
current = authority(connection, name)
if not current or current['key'] is None:
raise Failure(f'the key of {name} is not on the router, so it cannot sign its CRL', 4)
raise Failure(f'the key of {name} is not on the server, so it cannot sign its CRL', 4)
moment = generalized(now())
for entry in entries:
connection.modify_s(entry['dn'], [(ldap.MOD_REPLACE, 'vpnRouterRevoked', [moment]),
(ldap.MOD_REPLACE, 'vpnRouterRevocationReason', [reason.encode()]),
(ldap.MOD_REPLACE, 'vpnRouterPrivateKey', [])])
connection.modify_s(entry['dn'], [(ldap.MOD_REPLACE, 'linuxManagerRevoked', [moment]),
(ldap.MOD_REPLACE, 'linuxManagerRevocationReason', [reason.encode()]),
(ldap.MOD_REPLACE, 'linuxManagerPrivateKey', [])])
for name in names:
sign_crl(connection, cfg, name)
publish(connection)
@@ -913,7 +937,7 @@ def revoke_entries(connection, cfg, entries, reason):
def revoke_user(cfg, name):
"""Revoke the user's certificates, and remove the userCertificate of the user's entry."""
with directory() as connection:
entries = certificate_entries(connection, authority_dn('p2s'), f'{user_condition(name)}(!(vpnRouterRevoked=*))')
entries = certificate_entries(connection, authority_dn('p2s'), f'{user_condition(name)}(!(linuxManagerRevoked=*))')
if entries:
revoke_entries(connection, cfg, entries, 'cessation_of_operation')
with contextlib.suppress(ldap.NO_SUCH_OBJECT):
@@ -921,10 +945,10 @@ def revoke_user(cfg, name):
def revoke(cfg, serial, reason=None):
"""Revoke a certificate the router issued, by its serial number, for one of REASONS.
"""Revoke a certificate the server issued, by its serial number, for one of REASONS.
A user's certificate is revoked by removing the user, and an issuing CA by removing it from pki.issuing_cas. A
revoked router certificate loses its key, so that vpn-router-setup issues a new one; the answer says so with
revoked router certificate loses its key, so that linux-manager-setup issues a new one; the answer says so with
reissue.
"""
reason = reason or 'unspecified'
@@ -933,7 +957,7 @@ def revoke(cfg, serial, reason=None):
if not isinstance(serial, str) or not SERIAL.match(serial):
raise Failure('the serial number must be hexadecimal', 2)
with directory() as connection:
found = certificate_entries(connection, vpnrouter_ldap.PKI, f'(serialNumber={serial.upper().lstrip("0")})')
found = certificate_entries(connection, linuxmanager_ldap.PKI, f'(serialNumber={serial.upper().lstrip("0")})')
if not found:
raise Failure(f'no certificate with the serial number {serial}', 3)
entry = found[0]
@@ -949,10 +973,10 @@ def revoke(cfg, serial, reason=None):
def certificates():
"""The certificates the router issued, newest first, each with state: valid, expired or revoked."""
"""The certificates the server issued, newest first, each with state: valid, expired or revoked."""
moment = iso(now())
with directory() as connection:
entries = certificate_entries(connection, vpnrouter_ldap.PKI)
entries = certificate_entries(connection, linuxmanager_ldap.PKI)
return [{**{key: value for key, value in entry.items() if key != 'dn'},
'ca': RETIRED_SUFFIX.sub('', entry['ca']),
'state': 'revoked' if entry['revoked'] else 'expired' if (entry['expires'] or '') <= moment else 'valid'}
@@ -961,13 +985,13 @@ def certificates():
def csr(cfg, name):
"""The CSR of a pending issuing CA and the openssl extension section to sign it with: {name, csr, extensions}."""
if name not in vpnrouter.issuing_cas(cfg):
if name not in linuxmanager.issuing_cas(cfg):
raise Failure(f'no issuing CA {name}', 3)
with directory() as connection:
entry = authority(connection, name)
if not entry or entry['cert'] is not None or entry['csr'] is None:
raise Failure(f'issuing CA {name} is not waiting for a certificate', 4)
urls = vpnrouter.pki_base_urls(cfg)
urls = linuxmanager.pki_base_urls(cfg)
lines = ['[ issuing_ca ]', 'basicConstraints = critical, CA:TRUE, pathlen:0', 'keyUsage = critical, keyCertSign, cRLSign',
'subjectKeyIdentifier = hash', 'authorityKeyIdentifier = keyid:always']
if urls:
@@ -979,14 +1003,14 @@ def csr(cfg, name):
def import_ca(cfg, name, text):
"""Take the certificate of a pending issuing CA, signed by the root offline: it must be a CA certificate the root
signed for the key of the CA's CSR."""
if name not in vpnrouter.issuing_cas(cfg):
if name not in linuxmanager.issuing_cas(cfg):
raise Failure(f'no issuing CA {name}', 3)
with directory() as connection:
entry = authority(connection, name)
if entry is not None and entry['cert'] is not None:
raise Failure(f'issuing CA {name} has its certificate', 4)
if entry is None or entry['key'] is None:
raise Failure(f'issuing CA {name} has no key; vpn-router-setup makes it with the CSR', 3)
raise Failure(f'issuing CA {name} has no key; linux-manager-setup makes it with the CSR', 3)
try:
cert = x509.load_pem_x509_certificate(text.strip().encode() if isinstance(text, str) else b'')
except ValueError:
@@ -997,7 +1021,7 @@ def import_ca(cfg, name, text):
raise Failure(f'the certificate is not for the key of the CSR of {name}', 2)
if not is_ca(cert):
raise Failure('the certificate is not a CA certificate', 2)
update(connection, name, {'cACertificate;binary': cert_der(cert), 'vpnRouterCertificateRequest': None})
update(connection, name, {'cACertificate;binary': cert_der(cert), 'linuxManagerCertificateRequest': None})
record(connection, cert, 'root', 'ca')
sign_crl(connection, cfg, name)
publish(connection)
@@ -1022,7 +1046,7 @@ def import_root_crl(data):
if root['crl'] is not None and utc(crl, 'last_update') < utc(root['crl'], 'last_update'):
raise Failure('the CRL is older than the current one', 2)
update(connection, 'root', {'certificateRevocationList;binary': crl.public_bytes(serialization.Encoding.DER),
'vpnRouterCRLSource': b'uploaded'})
'linuxManagerCRLSource': b'uploaded'})
publish(connection)
reload_crls(connection)
following = utc(crl, 'next_update')
@@ -1036,7 +1060,7 @@ def export_root_key(passphrase):
with directory() as connection:
root = authority(connection, 'root')
if root is None or root['key'] is None:
raise Failure('the root key is not on the router', 3)
raise Failure('the root key is not on the server', 3)
return key_pem(root['key'], passphrase.encode())
@@ -1044,8 +1068,8 @@ def remove_root_key():
with directory() as connection:
root = authority(connection, 'root')
if root is None or root['key'] is None:
raise Failure('the root key is not on the router', 3)
update(connection, 'root', {'vpnRouterPrivateKey': None})
raise Failure('the root key is not on the server', 3)
update(connection, 'root', {'linuxManagerPrivateKey': None})
def import_root_key(data, passphrase=None):
@@ -1060,12 +1084,12 @@ def import_root_key(data, passphrase=None):
raise Failure('there is no root CA', 3)
if not same_key(key.public_key(), root['cert'].public_key()):
raise Failure('the key is not the key of the root CA', 2)
update(connection, 'root', {'vpnRouterPrivateKey': key_pem(key)})
update(connection, 'root', {'linuxManagerPrivateKey': key_pem(key)})
def maintain(cfg):
"""Sign the stale CRLs again, publish and install them; True when one of the router's certificates is due for
renewal, which vpn-router-setup does."""
"""Sign the stale CRLs again, publish and install them; True when one of the server's certificates is due for
renewal, which linux-manager-setup does."""
with directory() as connection:
refresh_crls(connection, cfg)
publish(connection)
@@ -1081,14 +1105,14 @@ def acme_url(cfg, name):
def status(cfg):
"""The root, the issuing CAs, the retired CAs and the warnings, as /ca and vpn-router-ca status show them; the root
key on the router is a warning while pki.root_key_warning is true."""
"""The root, the issuing CAs, the retired CAs and the warnings, as /ca and linux-manager-ca status show them; the root
key on the server is a warning while pki.root_key_warning is true."""
moment = now()
soon = moment + datetime.timedelta(days=WARN_DAYS)
warnings = []
def crl_of(entry):
"""The next update of the CA's CRL; a CRL the router signs warns once past half of its validity, which the timer
"""The next update of the CA's CRL; a CRL the server signs warns once past half of its validity, which the timer
renews before, and one signed offline WARN_DAYS before it expires."""
crl = entry['crl']
following = utc(crl, 'next_update') if crl is not None else None
@@ -1100,10 +1124,10 @@ def status(cfg):
with directory() as connection:
root_entry = authority(connection, 'root')
entries = {name: authority(connection, name) for name in vpnrouter.issuing_cas(cfg)}
retired_found = connection.search_s(vpnrouter_ldap.PKI, ldap.SCOPE_ONELEVEL,
'(&(objectClass=vpnRouterAuthority)(vpnRouterRetired=*))',
['cn', 'vpnRouterRetired', 'cACertificate;binary'])
entries = {name: authority(connection, name) for name in linuxmanager.issuing_cas(cfg)}
retired_found = connection.search_s(linuxmanager_ldap.PKI, ldap.SCOPE_ONELEVEL,
'(&(objectClass=linuxManagerAuthority)(linuxManagerRetired=*))',
['cn', 'linuxManagerRetired', 'cACertificate;binary'])
router_certs = [current[0][1] for current in (holders(connection, ROUTER_CERTS[name][0], router_condition(name))
for name in ROUTER_CERTS) if current]
@@ -1114,15 +1138,15 @@ def status(cfg):
'crl_next_update': crl_of(root_entry) if key or root_entry['crl'] is not None else None,
'crl_source': root_entry['crl_source']}
if key and cfg['pki']['root_key_warning']:
warnings.append('the root key is on the router; export it with vpn-router-ca root export-key and remove it '
'with vpn-router-ca root remove-key to keep the root offline')
warnings.append('the root key is on the server; export it with linux-manager-ca root export-key and remove it '
'with linux-manager-ca root remove-key to keep the root offline')
elif not key and root_entry['crl'] is None:
warnings.append('the root has no CRL; sign one offline and upload it with vpn-router-ca root import-crl')
warnings.append('the root has no CRL; sign one offline and upload it with linux-manager-ca root import-crl')
if utc(cert, 'not_valid_after') <= soon:
warnings.append(f'the root certificate expires on {root["expires"]}')
authorities = []
for name, settings in vpnrouter.issuing_cas(cfg).items():
for name, settings in linuxmanager.issuing_cas(cfg).items():
entry = entries[name]
cert = entry['cert'] if entry else None
kind = 'active' if cert is not None else 'pending' if entry and entry['csr'] is not None else 'missing'
@@ -1144,11 +1168,11 @@ def status(cfg):
der = first(attributes, 'cACertificate;binary')
serial = f'{x509.load_der_x509_certificate(der).serial_number:X}' if der else None
name = RETIRED_SUFFIX.sub('', attributes['cn'][0].decode())
retired.append({'name': name, 'retired': from_generalized(first(attributes, 'vpnRouterRetired')), 'serial': serial})
retired.append({'name': name, 'retired': from_generalized(first(attributes, 'linuxManagerRetired')), 'serial': serial})
root_crl = root_entry['crl'] if root_entry else None
if serial and root_crl is not None and root_crl.get_revoked_certificate_by_serial_number(int(serial, 16)) is None:
warnings.append(f'the root CRL does not revoke retired issuing CA {name}')
for cert in router_certs:
if utc(cert, 'not_valid_after') <= soon:
warnings.append(f'the router certificate {common_name(cert)} expires on {iso(utc(cert, "not_valid_after"))}')
warnings.append(f'the server certificate {common_name(cert)} expires on {iso(utc(cert, "not_valid_after"))}')
return {'root': root, 'authorities': authorities, 'retired': retired, 'warnings': warnings}
@@ -1,4 +1,4 @@
"""Platform modules for vpn-router.
"""Platform modules for linux-manager.
A platform module supplies the values and behaviour that only make sense on one
kind of host. The core is complete without any of them; a module only adds
@@ -4,8 +4,8 @@ No platform-specific configuration is currently required.
The Azure platform resolver 168.63.129.16 is deliberately *not* advertised to
IKEv2 clients and *not* added to the traffic selectors. Clients are
given the router's addresses of p2s.dns_listen as their DNS servers, where dnsmasq
forwards to the router's own systemd-resolved. That resolves
given the server's addresses of p2s.dns_listen as their DNS servers, where dnsmasq
forwards to the server's own systemd-resolved. That resolves
Azure-internal names without routing the wireserver address through the tunnel
and without a second resolver in the pool.
"""
@@ -0,0 +1,8 @@
"""Google Cloud.
No platform-specific configuration is currently required.
As on Azure, IKEv2 clients are given the server itself as their DNS
server and the server forwards through its own systemd-resolved, so the
metadata resolver does not belong in the pool or in the traffic selectors.
"""
@@ -4,6 +4,7 @@ Only the Python standard library is used, so the bootstrap server, which must ru
is broken, can use it.
"""
import sys
import threading
import time
from http.server import ThreadingHTTPServer
@@ -14,9 +15,21 @@ def serve(wanted, handler, log, interval):
wanted answers ([(address, port), ...], '') or ([], the reason not to listen). A server whose
place is no longer wanted is stopped, and a new one listens at each new place; a place that
cannot be bound is tried again the next round. Every new message is logged with log. Does
not return.
cannot be bound is tried again the next round. Every new message is logged with log. A request
whose client went away is not logged; any other exception a request, a thread or the process
does not handle is logged with log in one line, without a traceback. Does not return.
"""
def failed(request, client_address):
failure = sys.exc_info()[1]
if not isinstance(failure, ConnectionError):
log(f'a request from {client_address[0]} failed: {type(failure).__name__}: {failure}')
def thread_failed(hook):
if not issubclass(hook.exc_type, (SystemExit, ConnectionError)):
log(f'{hook.thread.name if hook.thread else "a thread"} failed: {hook.exc_type.__name__}: {hook.exc_value}')
threading.excepthook = thread_failed
sys.excepthook = lambda kind, value, _: log(f'failed: {kind.__name__}: {value}')
servers, reported = {}, None
while True:
targets, reason = wanted()
@@ -34,6 +47,7 @@ def serve(wanted, handler, log, interval):
except OSError as exc:
messages.append(f'cannot listen on {target[0]}:{target[1]}: {exc}')
continue
server.handle_error = failed
thread = threading.Thread(target=server.serve_forever, name=f'server {target[0]}', daemon=True)
thread.start()
servers[target] = (server, thread)
@@ -1,16 +1,20 @@
"""The static web sites Caddy serves, for the web service, vpn-router-site, vpn-router-setup and webdav-auth.
"""The static web sites Caddy serves, for the web service, linux-manager-site, linux-manager-setup and webdav-auth.
A site is a row of the sites table in the vpn-router-sites database, which the web service creates on its first run
A site is a row of the sites table in the linux-manager-sites database, which the web service creates on its first run
(provision) with its role and the password in DATABASE_SECRET; SITES_SQL defines the table and its procedures, and the
first connection of a process loads it. A site has a name, which also names its folder SITES_DIR/<name>, the host names
Caddy answers on its port (hosts, the names TLS clients send as SNI; none answers any name), the interfaces whose
firewall opens the port (listen), and TLS: none (plain HTTP), acme (a certificate of Let's Encrypt), ca (certificates
Caddy issues with one of pki.issuing_cas) or file (a certificate and key given with the site). Caddy serves the files of
the folder, index_file for a folder, and with spa index_file of the site for any path without a file. editors names
the group whose members change the site's files besides the admins, through the web service and WebDAV.
the group whose members change the site's files besides the admins, through the web service and WebDAV. dns says who
serves the records of its host names: user, a DNS provider outside the server, or router, the server's name server,
which gives each of them in the closest of its domains an A or AAAA record of the server's address of dns_address:
public (wan.public_address), internal or external. A site may use wan.local_fqdn, on a port of its own; its record
stays where the domain of wan.local_fqdn is.
The sites are served while sites.enabled is true; vpn-router-setup writes Caddy's configuration and opens the ports. A
site is changed only through this module, which checks it (check), and its callers then restart vpn-router-setup.
The sites are served while sites.enabled is true; linux-manager-setup writes Caddy's configuration and opens the ports. A
site is changed only through this module, which checks it (check), and its callers then restart linux-manager-setup.
The files of a site are read and written by path relative to its folder; resolve() refuses a path that leaves it.
receive() writes a file from a stream, extract() a zip archive into a folder; both stop at sites.upload_max_mb.
@@ -34,17 +38,17 @@ from psycopg.rows import dict_row
from cryptography import x509
from cryptography.hazmat.primitives import serialization
import vpnrouter
import vpnrouter_auth
import vpnrouter_pki
import vpnrouter_users
import linuxmanager
import linuxmanager_auth
import linuxmanager_pki
import linuxmanager_users
DATABASE = 'vpn-router-sites'
DATABASE_SECRET = pathlib.Path('/etc/vpn-router/sites-database.secret')
SITES_SQL = pathlib.Path('/usr/share/vpn-router/sql/sites.sql')
DATABASE = 'linux-manager-sites'
DATABASE_SECRET = pathlib.Path('/etc/linux-manager/sites-database.secret')
SITES_SQL = pathlib.Path('/usr/share/linux-manager/sql/sites.sql')
#: The folders of the sites, SITES_DIR/<name>.
SITES_DIR = pathlib.Path('/var/lib/vpn-router/sites')
SITES_DIR = pathlib.Path('/var/lib/linux-manager/sites')
#: The name of a site, which also names its folder.
NAME = re.compile(r'^[a-z][a-z0-9-]{0,30}$')
@@ -52,9 +56,14 @@ NAME = re.compile(r'^[a-z][a-z0-9-]{0,30}$')
#: A file name of index_file.
INDEX = re.compile(r'^[A-Za-z0-9_-][A-Za-z0-9._-]{0,63}$')
LISTEN = vpnrouter.LISTEN_CHOICES
LISTEN = linuxmanager.LISTEN_CHOICES
TLS = ('none', 'acme', 'ca', 'file')
#: Who serves the records of a site's host names: user, elsewhere, or router, the server's name server, with the
#: server's address of DNS_ADDRESSES.
DNS = ('user', 'router')
DNS_ADDRESSES = ('public', 'internal', 'external')
#: The most host names of one site.
HOSTS_MAX = 16
@@ -66,7 +75,8 @@ CHUNK = 65536
#: The settings of a site and their defaults; the type of a default is the required type.
SETTINGS = {'enabled': True, 'hosts': [], 'port': 443, 'listen': 'external', 'tls': 'acme', 'ca': '', 'certificate': '',
'private_key': '', 'index_file': 'index.html', 'spa': False, 'editors': ''}
'private_key': '', 'index_file': 'index.html', 'spa': False, 'editors': '', 'dns': 'user',
'dns_address': 'public'}
#: Whether this process has loaded SITES_SQL.
_loaded = False
@@ -86,7 +96,7 @@ class Failure(Exception):
def provision():
"""Create the role, its password file and the database of the sites when they are missing."""
vpnrouter_auth.provision_database(DATABASE, DATABASE_SECRET)
linuxmanager_auth.provision_database(DATABASE, DATABASE_SECRET)
@contextlib.contextmanager
@@ -96,7 +106,7 @@ def database():
global _loaded
try:
schema = None if _loaded else SITES_SQL.read_text(encoding='utf-8')
target = vpnrouter_auth.local_connection(DATABASE, DATABASE_SECRET)
target = linuxmanager_auth.local_connection(DATABASE, DATABASE_SECRET)
except OSError as exc:
raise DatabaseError(f'the database of the sites is not ready: {exc}') from exc
try:
@@ -162,14 +172,14 @@ def rename_group(old, new):
def taken_ports(cfg):
"""The TCP ports of the router's own services, port -> what has it, which no site may take."""
"""The TCP ports of the server's own services, port -> what has it, which no site may take."""
taken = {22: 'SSH', cfg['web']['port']: 'web.port', cfg['https']['web_port']: 'https.web_port',
cfg['https']['manager_port']: 'https.manager_port', cfg['metrics']['port']: 'metrics.port',
cfg['metrics']['https_port']: 'metrics.https_port', vpnrouter.manager_port(): 'the Router Manager',
cfg['sites']['webdav_port']: 'sites.webdav_port', vpnrouter.WEBDAV_LOOPBACK_PORT: 'rclone\'s WebDAV',
cfg['metrics']['https_port']: 'metrics.https_port', linuxmanager.manager_port(): 'the Manager',
cfg['sites']['webdav_port']: 'sites.webdav_port', linuxmanager.WEBDAV_LOOPBACK_PORT: 'rclone\'s WebDAV',
53: 'DNS'}
for port in vpnrouter.BOOTSTRAP_PORTS:
taken[port] = 'the bootstrap services'
for port in linuxmanager.BOOTSTRAP_PORTS:
taken[port] = 'the bootstrap service'
if cfg['openvpn']['protocol'] == 'tcp':
taken[cfg['openvpn']['port']] = 'openvpn.port'
for item in cfg['docker']['exposed_ports']:
@@ -190,9 +200,9 @@ def check_certificate(certificate, private_key, hosts):
key = serialization.load_pem_private_key(private_key.encode(), password=None)
except (ValueError, TypeError):
raise Failure('private_key must be an unencrypted PEM private key', 2)
if not vpnrouter_pki.same_key(chain[0].public_key(), key.public_key()):
if not linuxmanager_pki.same_key(chain[0].public_key(), key.public_key()):
raise Failure('private_key is not the key of the certificate', 2)
names = {value.partition(':')[2] for value in vpnrouter_pki.san_values(chain[0]) if not value.startswith('email:')}
names = {value.partition(':')[2] for value in linuxmanager_pki.san_values(chain[0]) if not value.startswith('email:')}
missing = [host for host in hosts if host not in names and f'*.{host.partition(".")[2]}' not in names]
if missing:
raise Failure(f'the certificate does not name {", ".join(missing)}', 2)
@@ -200,7 +210,7 @@ def check_certificate(certificate, private_key, hosts):
def check(cfg, sites, site):
"""Refuse a site, a dict of SETTINGS with its name, that is invalid or collides with the other sites (sites, rows of
the table) or with the router's own ports."""
the table) or with the server's own ports."""
name = site['name']
if not isinstance(name, str) or not NAME.match(name):
raise Failure('a site name starts with a lowercase letter and has at most 31 lowercase letters, digits and '
@@ -208,15 +218,24 @@ def check(cfg, sites, site):
for option, default in SETTINGS.items():
value = site[option]
if type(value) is not type(default) or (isinstance(value, list) and not all(isinstance(item, str) for item in value)):
raise Failure(f'{option} must be {vpnrouter.TYPE_NAMES[type(default)]}', 2)
raise Failure(f'{option} must be {linuxmanager.TYPE_NAMES[type(default)]}', 2)
hosts = site['hosts']
if len(hosts) > HOSTS_MAX or len(set(hosts)) != len(hosts):
raise Failure(f'hosts must be at most {HOSTS_MAX} different host names', 2)
for host in hosts:
if not (vpnrouter_pki.DNS_NAME.match(host) or vpnrouter_pki.is_address(host)):
if not (linuxmanager_pki.DNS_NAME.match(host) or linuxmanager_pki.is_address(host)):
raise Failure(f'hosts: {host!r} is not a host name in lowercase or an IP address', 2)
if host == cfg['wan']['local_fqdn'].strip():
raise Failure(f'hosts: {host} is wan.local_fqdn, the name of the router\'s own services', 2)
if site['dns'] not in DNS:
raise Failure(f'dns: {site["dns"]!r} is not one of {", ".join(DNS)}', 2)
if site['dns_address'] not in DNS_ADDRESSES:
raise Failure(f'dns_address: {site["dns_address"]!r} is not one of {", ".join(DNS_ADDRESSES)}', 2)
if site['dns'] == 'router':
local_fqdn = cfg['wan']['local_fqdn'].strip()
if any(linuxmanager_pki.is_address(host) or host.startswith('*.') or host == local_fqdn for host in hosts):
raise Failure('dns router takes host names only, without wildcards and other than wan.local_fqdn, whose '
'record is kept where its domain is', 2)
if site['dns_address'] == 'public' and not cfg['wan']['public_address'].strip():
raise Failure('dns_address public needs wan.public_address', 2)
port = site['port']
if not 1 <= port <= 65535:
raise Failure(f'port: {port} is not a port', 2)
@@ -230,10 +249,10 @@ def check(cfg, sites, site):
raise Failure(f'tls: {tls!r} is not one of {", ".join(TLS)}', 2)
if tls != 'none' and not hosts:
raise Failure(f'tls {tls} needs hosts, the names of the certificate', 2)
if tls == 'acme' and any(host.startswith('*.') or vpnrouter_pki.is_address(host) for host in hosts):
if tls == 'acme' and any(host.startswith('*.') or linuxmanager_pki.is_address(host) for host in hosts):
raise Failure('tls acme takes host names only, without wildcards', 2)
if tls == 'ca':
authorities = {key: value for key, value in vpnrouter.issuing_cas(cfg).items() if not value['builtin']}
authorities = {key: value for key, value in linuxmanager.issuing_cas(cfg).items() if not value['builtin']}
if site['ca'] not in authorities:
raise Failure(f'ca: {site["ca"]!r} is not one of pki.issuing_cas', 2)
if tls == 'file':
@@ -241,10 +260,10 @@ def check(cfg, sites, site):
if not INDEX.match(site['index_file']):
raise Failure('index_file must be a file name', 2)
if site['editors']:
with vpnrouter_users.session() as connection:
with linuxmanager_users.session() as connection:
try:
vpnrouter_users.group_entry(connection, site['editors'])
except vpnrouter_users.Failure:
linuxmanager_users.group_entry(connection, site['editors'])
except linuxmanager_users.Failure:
raise Failure(f'editors: group {site["editors"]} does not exist', 2)
if not site['enabled']:
return
@@ -285,7 +304,7 @@ def save(cfg, name, values, create):
if site['tls'] != 'ca':
site['ca'] = ''
check(cfg, rows, site)
connection.execute('CALL site_put(%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)',
connection.execute(f'CALL site_put({", ".join(["%s"] * (len(SETTINGS) + 1))})',
tuple(site[key] for key in ('name', *SETTINGS)))
ensure_folders([name])
return shown(get(connection, name))
@@ -1,4 +1,4 @@
"""What the router's VPN systems report, for the web service and the metrics service.
"""What the server's VPN systems report, for the web service and the metrics service.
Each reader asks the running system once and answers None when it cannot be asked: swanctl for
the IKE SAs and the state and traffic of the site-to-site connections, wg for the WireGuard
@@ -10,7 +10,7 @@ import time
COMMAND_TIMEOUT = 5
OPENVPN_STATUS = '/run/openvpn-server/status-vpn-router.log'
OPENVPN_STATUS = '/run/openvpn-server/status-linux-manager.log'
#: Seconds between two writes of OPENVPN_STATUS by the OpenVPN server, whose configuration sets it (OpenVPN's default is 60).
OPENVPN_STATUS_SECONDS = 5
@@ -0,0 +1,212 @@
"""The settings of the Manager and its listening loop.
The settings file (CONFIG_FILE unless another is given) sets, all optional: router_url, database,
listen, bind, port, terraform_dir and iac_tool. bind is the one address to listen on. Without
router_url, terraform_dir names the Terraform root module whose output router_web_url gives the
URL of the server's web service, read with iac_tool, terraform (the default) or tofu; dev.sh uses
it to run the manager on a workstation. A setting that is not given comes from the server's
configuration, ROUTER_CONFIG_FILE: without bind the manager listens on the addresses of the
interfaces listen names, internal, external or both (the default), and on the loopback address,
where the server's HTTPS proxy calls it, router_url is the first address and web.port of the
server's web service, and database is the DATABASE database on this host, with the password in
DATABASE_SECRET.
"""
import json
import os
import subprocess
import sys
import threading
import time
import urllib.parse
from http.server import ThreadingHTTPServer
import linuxmanager_net
CONFIG_FILE = '/etc/linux-manager/manager.json'
ROUTER_CONFIG_FILE = '/etc/linux-manager/linux-manager.json'
DATABASE_SECRET = '/etc/linux-manager/manager-database.secret'
#: The role and the database the manager creates on the server.
DATABASE = 'linux-manager-ui'
#: The port when port is not set.
DEFAULT_PORT = 7080
#: The port of the server's web service when web.port is not set.
ROUTER_PORT = 7081
#: The HTTPS port of the server's web service, on its HTTPS proxy, when https.web_port is not set.
ROUTER_HTTPS_PORT = 7444
#: Seconds the Terraform outputs are kept; terraform output is not cheap.
OUTPUTS_SECONDS = 30
_outputs = {}
_outputs_lock = threading.Lock()
def read_json(path):
"""The object in a JSON file, or {} when the file is missing or holds no object."""
try:
with open(path, encoding='utf-8') as handle:
data = json.load(handle)
except (OSError, ValueError):
return {}
return data if isinstance(data, dict) else {}
def text_setting(configuration, name):
"""A string setting, or '' when it is missing or not a string."""
value = configuration.get(name)
return value.strip() if isinstance(value, str) else ''
def port_setting(value, default):
"""A port number, or the default when the value is not one."""
return value if isinstance(value, int) and not isinstance(value, bool) and 1 <= value <= 65535 else default
def terraform_outputs(directory, tool):
"""The outputs of the Terraform root directory read with the tool, terraform or tofu, kept for OUTPUTS_SECONDS, or
{} when they cannot be read."""
with _outputs_lock:
cached = _outputs.get(directory)
if cached and time.monotonic() - cached[0] < OUTPUTS_SECONDS:
return cached[1]
try:
result = subprocess.run([tool, f'-chdir={directory}', 'output', '-json', '-no-color'],
capture_output=True, text=True, timeout=60,
env=dict(os.environ, TF_IN_AUTOMATION='1'))
outputs = json.loads(result.stdout) if result.returncode == 0 else {}
except (OSError, subprocess.TimeoutExpired, ValueError):
outputs = {}
if not isinstance(outputs, dict):
outputs = {}
_outputs[directory] = (time.monotonic(), outputs)
return outputs
def output_value(outputs, name):
"""The value of a Terraform output, or None when it is missing."""
output = outputs.get(name)
return output.get('value') if isinstance(output, dict) else None
def listen(config_file):
"""Where the manager listens, read now: ([(address, port), ...], ''), or ([], the reason) while no address is known."""
configuration = read_json(config_file)
bind = text_setting(configuration, 'bind')
port = port_setting(configuration.get('port'), DEFAULT_PORT)
if bind:
return [(bind, port)], ''
addresses = linuxmanager_net.listen_addresses(read_json(ROUTER_CONFIG_FILE), text_setting(configuration, 'listen'))
if not addresses:
return [], "the server's addresses to listen on are not known yet"
return [(address, port) for address in [*addresses, linuxmanager_net.LOOPBACK]], ''
def settings(config_file):
"""The settings read now: router_url and database, with the server's defaults for those not set."""
configuration = read_json(config_file)
result = {name: text_setting(configuration, name) for name in ('router_url', 'database')}
directory = text_setting(configuration, 'terraform_dir')
if directory and not result['router_url']:
tool = 'tofu' if text_setting(configuration, 'iac_tool') == 'tofu' else 'terraform'
url = output_value(terraform_outputs(os.path.expanduser(directory), tool), 'router_web_url')
if isinstance(url, str):
result['router_url'] = url
if os.path.exists(ROUTER_CONFIG_FILE):
router = read_json(ROUTER_CONFIG_FILE)
if not result['database']:
try:
with open(DATABASE_SECRET, encoding='utf-8') as handle:
password = handle.read().strip()
except OSError:
password = ''
# The password, of URL-safe characters, is written unquoted.
if password:
result['database'] = f'host=localhost dbname={DATABASE} user={DATABASE} password={password}'
if not result['router_url']:
targets, _ = linuxmanager_net.web_listen(router, ROUTER_PORT)
if targets:
result['router_url'] = f'http://{targets[0][0]}:{targets[0][1]}'
result['router_url'] = result['router_url'].rstrip('/')
return result
def status_url(config_file, host, address, https):
"""The URL of the server's status page for a browser that reached the manager as host on its address.
On the server, while neither router_url nor terraform_dir is set, it has host, without its
port, so the link works by whatever name or address the browser used: with https, a call
through the server's HTTPS proxy, https and https.web_port; else, while the server's web
service listens on address, http and web.port. Otherwise it is router_url, or '' without one.
"""
configuration = read_json(config_file)
try:
hostname = urllib.parse.urlsplit(f'//{host}').hostname
except ValueError:
hostname = None
if (hostname and os.path.exists(ROUTER_CONFIG_FILE)
and not text_setting(configuration, 'router_url') and not text_setting(configuration, 'terraform_dir')):
name = f'[{hostname}]' if ':' in hostname else hostname
router = read_json(ROUTER_CONFIG_FILE)
if https:
options = router.get('https') if isinstance(router.get('https'), dict) else {}
return f'https://{name}:{port_setting(options.get("web_port"), ROUTER_HTTPS_PORT)}'
targets, _ = linuxmanager_net.web_listen(router, ROUTER_PORT)
ports = [port for listened, port in targets if listened == address]
if ports:
return f'http://{name}:{ports[0]}'
return settings(config_file)['router_url']
def serve(wanted, handler, log, interval):
"""Work out where to listen every interval seconds with wanted, and run an HTTP server of handler at each place.
wanted answers ([(address, port), ...], '') or ([], the reason not to listen). A server whose
place is no longer wanted is stopped, and a new one listens at each new place; a place that
cannot be bound is tried again the next round. Every new message is logged with log. A request
whose client went away is not logged; any other exception a request, a thread or the process
does not handle is logged with log in one line, without a traceback. Does not return.
"""
def failed(request, client_address):
failure = sys.exc_info()[1]
if not isinstance(failure, ConnectionError):
log(f'a request from {client_address[0]} failed: {type(failure).__name__}: {failure}')
def thread_failed(hook):
if not issubclass(hook.exc_type, (SystemExit, ConnectionError)):
log(f'{hook.thread.name if hook.thread else "a thread"} failed: {hook.exc_type.__name__}: {hook.exc_value}')
threading.excepthook = thread_failed
sys.excepthook = lambda kind, value, _: log(f'failed: {kind.__name__}: {value}')
servers, reported = {}, None
while True:
targets, reason = wanted()
for target in [target for target in servers if target not in targets]:
server, thread = servers.pop(target)
server.shutdown()
server.server_close()
thread.join()
messages = [reason] if reason else []
for target in targets:
if target in servers:
continue
try:
server = ThreadingHTTPServer(target, handler)
except OSError as exc:
messages.append(f'cannot listen on {target[0]}:{target[1]}: {exc}')
continue
server.handle_error = failed
thread = threading.Thread(target=server.serve_forever, name=f'server {target[0]}', daemon=True)
thread.start()
servers[target] = (server, thread)
if servers:
messages.append('serving on ' + ', '.join(f'http://{address}:{port}' for address, port in servers))
message = '; '.join(messages)
if message and message != reported:
log(message)
reported = message
time.sleep(interval)
@@ -0,0 +1,101 @@
"""The server's web service, run by linux-manager-web.service; the entry point is /usr/lib/linux-manager/web.
Pages:
/ the status for people: plain text for curl and wget, a styled page
(Tailwind CSS from a CDN) for a browser
/status the status as JSON
/status/full
the whole status as JSON, whatever web.status_content says, for the Manager
/config the configuration without its secrets, and what the server resolved and
runs, with the installed Caddy and where it comes from, as JSON
/users the users in the server's directory, whether or not a P2S system is enabled: list,
add, change, set the password, disable, enable, remove, the OpenVPN client
profile, and TOTP enrolment (see users()); service accounts are not among them
/service-accounts
the service accounts: list, add, change the e-mail address, set the password,
disable, enable, unlock and remove; the internal ones are only listed (see
service_accounts())
/groups the groups of the server's directory: list, add, rename or describe, remove, and
add and take out members (see groups())
/account the account of the signed-in user: its details, its password, its TOTP
enrolment and its passkeys (see account())
/site-to-site
the site-to-site connections of the configuration: list, add, change and remove,
applied by linux-manager-setup (see site_to_site())
/backup the backup settings, the last run and the local archives: show, change and run
now (see backup())
/remote-access
the settings of IKEv2, OpenVPN and WireGuard access and whether each
runs: show and change, applied by linux-manager-setup (see remote_access())
/ca the PKI: its settings, the root and issuing CAs, the certificates issued; change the
settings, issue and revoke certificates, and take the certificate of an issuing CA
and a root CRL signed offline (see ca())
/pki/<name>.crt, /pki/<name>.crl
the DER certificate and CRL of the root (root) and of each issuing CA, which the
certificates name in their AIA and CRL distribution points; no authentication
/shell while web.shell is true, a login shell of the user's console account on a terminal, for
a member of admins whose console login is on and who has a TOTP token: POST signs the user in again,
with the password and the TOTP code or with a passkey whose proof POST /shell/passkey begins,
unless the user used a shell within web.shell_auth_timeout minutes, for a one-time ticket
(see shell_ticket() and shell_passkey()), GET with it is a WebSocket
that carries the terminal (see Handler.shell()); /shell/editor/<session>/<id> reads, saves and
closes a file the shell opened with linux-manager-editor (lme) over EDITOR_SOCKET (see
shell_editor() and editor_request())
/auth sign-in with a user's password and TOTP code or with a passkey, refresh and sign-out
(see auth())
/metrics/history
the metrics the metrics service keeps, for a view: hour, day, month or year (see
metrics_history())
/metrics/settings
the settings of the metrics service: show and change, applied by linux-manager-setup (see
metrics_settings())
/logs the last lines of the journal of the server's units and of the files web.log_files lists
(see logs())
/sites the web sites Caddy serves: the settings of sites, list, add, change and remove (see sites()),
and the files of each: list, download, upload, extract a zip archive, create a folder, rename and
remove (see Handler.site_files()); a member of a site's editors group may use the site's files
/dns the primary name server of the server's domains: its settings, and the domains: add, change and
remove, each with the records it serves and its manual records (see dns())
/postgresql
the databases of each PostgreSQL cluster on the server with their sizes, as JSON (see
postgresql_databases()); /config has the clusters' settings
/.well-known/openid-configuration, /oidc/...
while oidc.enabled is true, the OpenID Provider (linuxmanager_oidc), answered only on the
loopback address, where Caddy passes the requests of the HTTPS site; POST /oidc/login counts
towards the limit of /auth calls
/status/full, /config, /metrics/history, /metrics/settings, /postgresql, /users, /service-accounts, /groups, /site-to-site, /backup, /remote-access, /ca, /logs, /sites, /dns, /shell and /account need an HS256 JWT in an Authorization: Bearer header signed
with linuxmanager_auth.WEB_KEY (see linuxmanager_auth.verify). A token of /auth names a user in sub;
/account serves that user, and the other paths need a member of linuxmanager_users.ADMIN_GROUP. A
token of linux-manager-token has the admin claim and names no user; it may call every path but
/account, and only such a token may change ADMIN_GROUP: create, rename to it, change, remove it,
add or take out its members, and remove one of its members, and switch a user's console login.
These paths and /auth are not
served (404) while the key does not exist. /auth but its sign-out and the beginning of a passkey
sign-in is limited to AUTH_LIMIT calls per client
address in AUTH_WINDOW seconds; the address of a call from one of web.trusted_proxies, by
default the address the call came in on, is the last entry of its X-Forwarded-For header.
/ and /status show only the status, as much as web.status_content says (see shown): basic, the
default, the overall status, the time and the client's address (its address as for the rate limit
below); standard also the package version and the number of active and other services; full
whether each service runs, whether Docker is installed, how many of the site-to-site connections are established
(site_to_site: connections and established, null when swanctl cannot be asked), and IKEv2, OpenVPN
and WireGuard. No configuration data, key or connection name is shown, and no address but the client's.
Modules, one per family of paths, with what they share in common:
common the paths of the configuration and of setup's state, the body limit, the HTTP statuses of the
errors of the server's modules, reading the configuration and the state, changing a section, log
status /, /status, /status/full, /config and /postgresql, with the cache of the status
auth /auth, its rate limit, and the rules of who may call a protected path (refusal)
identities /users, /service-accounts, /groups and /account
connectivity /site-to-site and /remote-access
backup /backup
pki /ca, and the files /pki/ serves
logs /logs
sites /sites, with the files of each site
dns /dns
metrics /metrics/history and /metrics/settings
shell /shell, its WebSocket, and the editor of the shell with its socket
handler Handler, which reads a request, checks its token and passes it to the module of its path
"""
@@ -0,0 +1,130 @@
"""/auth, the sign-in, refresh and sign-out of the web service, its rate limit per client address, and the rules of
who may call a protected path (refusal)."""
import collections
import threading
import time
import ldap
import linuxmanager
import linuxmanager_auth
import linuxmanager_passkeys
import linuxmanager_sites
import linuxmanager_users
from .common import USER_STATUS
from .sites import SITE_FILES
#: The calls of /auth allowed per client address in AUTH_WINDOW seconds.
AUTH_LIMIT = 10
AUTH_WINDOW = 60
#: The times of the recent /auth calls of each client address.
_auth_calls = collections.defaultdict(collections.deque)
_auth_lock = threading.Lock()
def changes_admins(connection, method, parts, body):
"""Whether a /users, /service-accounts or /groups request creates, renames to, changes or removes ADMIN_GROUP, or
removes a member of it."""
admins = linuxmanager_users.ADMIN_GROUP
if parts[0] == 'groups':
if method == 'POST' and len(parts) == 1:
return body.get('name') == admins
if method != 'GET' and len(parts) > 1 and parts[1] == admins:
return True
return method == 'PUT' and len(parts) == 2 and body.get('name') == admins
if parts[0] in ('users', 'service-accounts') and method == 'DELETE' and len(parts) == 2:
return admins in linuxmanager_users.groups_of(connection, parts[1])
return False
def refusal(claims, method, path, body):
"""Why the holder of the claims may not make the request to a protected path, or '' when it may.
The files of a site may also be read and changed by a member of the site's editors group.
"""
name = claims.get('sub')
if path == '/account' or path.startswith('/account/'):
return '' if isinstance(name, str) else 'the token names no user'
if claims.get('admin') is True:
return ''
if not isinstance(name, str):
return 'the token names no user'
admins = linuxmanager_users.ADMIN_GROUP
with linuxmanager_users.session() as connection:
if not linuxmanager_users.is_admin(connection, name):
files = SITE_FILES.match(path)
if files and files.group(1) in linuxmanager_sites.editable(linuxmanager_users.groups_of(connection, name)):
return ''
return f'user {name} is not a member of {admins}'
parts = path.strip('/').split('/')
if changes_admins(connection, method, parts, body if isinstance(body, dict) else {}):
return f'only a token of linux-manager-token may change the group {admins} or remove its members'
if parts[0] == 'users' and len(parts) == 3 and parts[2] == 'console' and method != 'GET':
return 'only a token of linux-manager-token may change console login'
return ''
def auth(path, body):
"""Answer a POST to /auth with linuxmanager_users and linuxmanager_auth: the HTTP status and the JSON document.
POST /auth sign in: {"name", "password", "code"?}; the tokens and the user's name
POST /auth/passkey begin a sign-in with a passkey: {"id", "options"} for navigator.credentials.get
POST /auth/passkey/complete
{"id", "credential"}, credential the browser's answer; the tokens and the user's name
POST /auth/refresh {"refresh_token"}; new tokens, the refresh token used up
POST /auth/logout {"refresh_token"}; the refresh token revoked
"""
if not isinstance(body, dict):
return 400, {'error': 'the body must be a JSON object'}
try:
cfg = linuxmanager.load_config()
if path == '/auth':
linuxmanager_users.login(body.get('name'), body.get('password'), body.get('code'))
with linuxmanager_users.session() as connection:
if linuxmanager_users.service(linuxmanager_users.entry(connection, body['name'])):
raise linuxmanager_users.Failure('the name, the password or the code is wrong', 5)
return 200, {**linuxmanager_auth.issue(cfg, body['name']), 'name': body['name']}
if path == '/auth/passkey':
return 200, linuxmanager_passkeys.login_begin()
if path == '/auth/passkey/complete':
with linuxmanager_users.session() as connection:
name = linuxmanager_passkeys.login_complete(connection, body.get('id'), body.get('credential'))
return 200, {**linuxmanager_auth.issue(cfg, name), 'name': name}
name = linuxmanager_auth.take(cfg, body.get('refresh_token'))
if path == '/auth/logout':
return 200, {'revoked': name is not None}
if name is None:
return 401, {'error': 'the refresh token is not valid'}
with linuxmanager_users.session() as connection:
user = linuxmanager_users.summary(linuxmanager_users.entry(connection, name))
if not user['enabled'] or user['locked']:
return 401, {'error': f'user {name} is disabled or locked'}
return 200, linuxmanager_auth.issue(cfg, name)
except linuxmanager_users.Failure as failure:
return (401 if failure.status == 3 else USER_STATUS.get(failure.status, 500)), {'error': str(failure)}
except (ldap.LDAPError, OSError) as exc:
return 500, {'error': f'the user directory failed: {exc}'}
except linuxmanager_auth.DatabaseError as exc:
return 500, {'error': str(exc)}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
def auth_wait(address):
"""Seconds until the client address may call /auth again, 0 when it may now; a call that may is counted."""
now = time.monotonic()
with _auth_lock:
for idle in [key for key, times in _auth_calls.items() if times[-1] <= now - AUTH_WINDOW]:
del _auth_calls[idle]
calls = _auth_calls[address]
while calls and calls[0] <= now - AUTH_WINDOW:
calls.popleft()
if len(calls) >= AUTH_LIMIT:
return int(calls[0] + AUTH_WINDOW - now) + 1
calls.append(now)
return 0
@@ -0,0 +1,84 @@
"""/backup: the backup settings, the last run and the local archives, and a run now."""
import configparser
import shutil
import linuxmanager
import linuxmanager_backup
from linuxmanager_status import run
def backup_settings(settings):
"""The backup section as /backup shows it: its settings but pbs_password, with pbs_password_set."""
return {**{option: value for option, value in settings.items() if option != 'pbs_password'},
'pbs_password_set': bool(settings['pbs_password'])}
def rclone_remotes():
"""The names of the remotes of linuxmanager_backup.RCLONE_CONFIG; [] when it cannot be read, as when it is encrypted."""
parser = configparser.ConfigParser(interpolation=None)
try:
parser.read(linuxmanager_backup.RCLONE_CONFIG, encoding='utf-8')
except (OSError, configparser.Error):
return []
return parser.sections()
def backup(method, path, body):
"""Answer a /backup request: the HTTP status and the JSON document.
GET /backup {"settings", "state", "archives", "next", "client", "rclone"}: the backup section without
pbs_password, with pbs_password_set; the last run (linuxmanager_backup.state), null before the
first; the local archives of this host, newest first, [{name, time, size}], [] with the pbs
target; the next run of the timer as systemd shows it, null while backups are disabled;
whether proxmox-backup-client is installed; and {"installed", "config", "remotes"} of rclone
PUT /backup change any of the settings of the backup section; an empty or absent pbs_password keeps the
current secret
POST /backup/run start a backup now, without waiting for it; 409 while one runs
A change is made as /site-to-site makes one, and applied by restarting linux-manager-setup without waiting for it.
"""
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
return 400, {'error': 'the body must be a JSON object'}
body = body or {}
if (method, path) == ('POST', '/backup/run'):
if linuxmanager_backup.running():
return 409, {'error': 'a backup is running'}
result = run('systemctl', 'start', '--no-block', linuxmanager_backup.UNIT)
if result is None or result.returncode != 0:
return 500, {'error': f'the backup cannot be started: {result.stderr.strip() if result else "no systemctl"}'}
return 202, {'started': True}
if (method, path) not in (('GET', '/backup'), ('PUT', '/backup')):
return 404, {'error': f'no route {method} {path}'}
try:
if method == 'GET':
settings = linuxmanager.load_config()['backup']
timer = run('systemctl', 'show', '-p', 'NextElapseUSecRealtime', '--value', linuxmanager_backup.TIMER)
next_run = timer.stdout.strip() if timer and timer.returncode == 0 else ''
return 200, {
'settings': backup_settings(settings),
'state': linuxmanager_backup.state() or None,
'archives': linuxmanager_backup.archives(settings['path']) if settings['target'] == 'local' else [],
'next': next_run if settings['enabled'] and next_run not in ('', 'n/a') else None,
'client': linuxmanager_backup.PBS_CLIENT.exists(),
'rclone': {'installed': shutil.which('rclone') is not None,
'config': linuxmanager_backup.RCLONE_CONFIG.is_file(), 'remotes': rclone_remotes()},
}
settings = {option: body[option] for option in linuxmanager.SCHEMA['backup'] if option in body}
if not settings.get('pbs_password'):
settings.pop('pbs_password', None)
with linuxmanager.config_lock():
data = linuxmanager.read_settings()
current = data.get('backup')
data['backup'] = {**(current if isinstance(current, dict) else {}), **settings}
cfg = linuxmanager.with_defaults(data)
linuxmanager.validate(cfg)
linuxmanager.save_config(data)
except linuxmanager.ConfigError as exc:
return 400, {'error': str(exc)}
except OSError as exc:
return 500, {'error': f'the configuration cannot be written: {exc}'}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
run('systemctl', 'restart', '--no-block', 'linux-manager-setup.service')
return 200, {'settings': backup_settings(cfg['backup'])}
@@ -0,0 +1,85 @@
"""What the modules of the web service share: the paths of the configuration and of the state linux-manager-setup writes,
the limit of request bodies, the HTTP statuses of the failures of the server's modules, reading the configuration and
the state, changing a section of the configuration, the state of a unit, and the log."""
import json
import sys
import linuxmanager
import linuxmanager_auth
from linuxmanager_status import run
STATE_FILE = '/run/linux-manager/web.json'
CONFIG_FILE = '/etc/linux-manager/linux-manager.json'
#: The largest request body accepted, in bytes; a CSR or a certificate chain in PEM is a few kilobytes.
BODY_LIMIT = 65536
#: The HTTP status for each status of a linuxmanager_users.Failure; any other is 500.
USER_STATUS = {2: 400, 3: 404, 4: 409, 5: 401}
#: The HTTP status for each status of a linuxmanager_pki.Failure; any other is 500.
PKI_STATUS = {2: 400, 3: 404, 4: 409}
#: The HTTP status for each status of a linuxmanager_sites.Failure; any other is 500.
SITE_STATUS = {2: 400, 3: 404, 4: 409}
#: The bytes in a MiB, the unit of sites.upload_max_mb.
MIB = 1024 * 1024
def read_json(path):
"""The object in a JSON file, or {} when the file is missing or holds no object."""
try:
with open(path, encoding='utf-8') as handle:
data = json.load(handle)
except (OSError, ValueError):
return {}
return data if isinstance(data, dict) else {}
def section(configuration, name):
"""A section of the configuration, or {} when it is missing or not an object."""
value = configuration.get(name)
return value if isinstance(value, dict) else {}
def current():
"""What a request needs, read now: the features and facts setup resolved, and the signing key.
Before setup has run the server has no features, and without a key the protected paths and
/auth are not served.
"""
return dict(read_json(STATE_FILE), key=linuxmanager_auth.key())
def unit_state(unit):
result = run('systemctl', 'is-active', unit)
return result.stdout.strip() if result and result.stdout.strip() else 'unknown'
def section_change(name, body):
"""Change any of the settings of the configuration's section as /backup changes its own; the section, or the HTTP
status and the error document as a tuple."""
settings = {option: body[option] for option in linuxmanager.SCHEMA[name] if option in body}
try:
with linuxmanager.config_lock():
data = linuxmanager.read_settings()
current = data.get(name)
data[name] = {**(current if isinstance(current, dict) else {}), **settings}
cfg = linuxmanager.with_defaults(data)
linuxmanager.validate(cfg)
linuxmanager.save_config(data)
except linuxmanager.ConfigError as exc:
return 400, {'error': str(exc)}
except OSError as exc:
return 500, {'error': f'the configuration cannot be written: {exc}'}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
run('systemctl', 'restart', '--no-block', 'linux-manager-setup.service')
return cfg[name]
def log(message):
print(f'linux-manager-web: {message}', file=sys.stderr, flush=True)
@@ -0,0 +1,155 @@
"""The connections of the server: /site-to-site and /remote-access, the IKEv2, OpenVPN and WireGuard access."""
import ipaddress
import linuxmanager
from linuxmanager_status import established_each, run
from .common import section
#: The settings of a site_to_site connection that /site-to-site adds and changes.
CONNECTION_SETTINGS = ('addrs', 'id', 'cidrs', 'local_cidrs', 'local_id_mode', 'auth', 'psk', 'remote_ca')
#: The sections of the remote access systems /remote-access changes: IKEv2, OpenVPN and WireGuard.
REMOTE_ACCESS = ('p2s', 'openvpn', 'wireguard')
def connection_of(name, connection, count):
"""A site_to_site connection as /site-to-site shows it: its settings but psk, psk_set and established.
count is the number of its established IKE SAs, None when it is not known.
"""
return {'name': name, **{option: value for option, value in connection.items() if option != 'psk'},
'psk_set': bool(connection['psk']), 'established': None if count is None else count > 0}
def site_to_site(method, path, body):
"""Answer a /site-to-site request: the HTTP status and the JSON document.
GET /site-to-site the connections, {"connections": [...]}: each with its settings but psk, with
psk_set, whether psk holds a key, and established, whether its tunnel is up,
null when swanctl cannot be asked
POST /site-to-site add a connection: {"name", "addrs", "id", "cidrs", "local_cidrs", "local_id_mode",
"auth", "psk", "remote_ca"}; psk is required with auth psk, the default, and an
existing name is 409
PUT /site-to-site/<name> change any of the settings of POST but name; an empty or absent psk keeps the
current key, and a psk given replaces psk_file
DELETE /site-to-site/<name> remove the connection
A change is made holding linuxmanager.config_lock, checked as linux-manager-setup checks the configuration (400 with
the reason), written with linuxmanager.save_config, and applied by restarting linux-manager-setup without waiting for it.
"""
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
return 400, {'error': 'the body must be a JSON object'}
body = body or {}
parts = path.strip('/').split('/')
name = parts[1] if len(parts) == 2 else None
if (method, len(parts)) not in (('GET', 1), ('POST', 1), ('PUT', 2), ('DELETE', 2)):
return 404, {'error': f'no route {method} {path}'}
try:
if method == 'GET':
connections = linuxmanager.load_config()['site_to_site']
counts = established_each(connections)
return 200, {'connections': [connection_of(key, value, counts[key]) for key, value in connections.items()]}
settings = {option: body[option] for option in CONNECTION_SETTINGS if option in body}
with linuxmanager.config_lock():
data = linuxmanager.read_settings()
connections = data.setdefault('site_to_site', {})
if method == 'POST':
name = body.get('name')
if not isinstance(name, str) or not name:
return 400, {'error': 'name is required'}
if name in connections:
return 409, {'error': f'connection {name} exists'}
if settings.get('auth', 'psk') == 'psk' and (not isinstance(settings.get('psk'), str) or not settings['psk']):
return 400, {'error': 'psk is required with auth psk'}
connections[name] = settings
elif name not in connections:
return 404, {'error': f'no connection {name}'}
elif method == 'PUT':
connection = dict(connections[name]) if isinstance(connections[name], dict) else {}
if settings.get('psk'):
connection.pop('psk_file', None)
else:
settings.pop('psk', None)
connections[name] = {**connection, **settings}
else:
del connections[name]
cfg = linuxmanager.with_defaults(data)
linuxmanager.validate(cfg)
linuxmanager.save_config(data)
except linuxmanager.ConfigError as exc:
return 400, {'error': str(exc)}
except OSError as exc:
return 500, {'error': f'the configuration cannot be written: {exc}'}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
run('systemctl', 'restart', '--no-block', 'linux-manager-setup.service')
if method == 'DELETE':
return 200, {'name': name, 'removed': True}
return (201 if method == 'POST' else 200), connection_of(name, cfg['site_to_site'][name], None)
def wireguard_server(cfg, state):
"""What a WireGuard peer's configuration needs of the server: {public_key, endpoint, allowed_ips, dns}; public_key is
null before setup made the key pair, endpoint, <wan.local_fqdn>:<wireguard.listen_port>, null without a name,
allowed_ips the networks reached through the tunnel, the server's local CIDRs and the WireGuard network, and dns the
addresses of the remote access clients' resolver, [] while it does not run."""
try:
public_key = linuxmanager.WG_PUBLIC_KEY.read_text(encoding='utf-8').strip() or None
except OSError:
public_key = None
fqdn = cfg['wan']['local_fqdn'].strip()
address = cfg['wireguard']['address'].strip()
networks = section(state, 'network').get('local_cidrs')
allowed = [cidr for cidr in networks if isinstance(cidr, str)] if isinstance(networks, list) else []
if address:
allowed.append(str(ipaddress.ip_interface(address).network))
dns = section(state, 'network').get('client_dns')
return {'public_key': public_key, 'endpoint': f'{fqdn}:{cfg["wireguard"]["listen_port"]}' if fqdn else None,
'allowed_ips': list(dict.fromkeys(allowed)),
'dns': [address for address in dns if isinstance(address, str)] if isinstance(dns, list) else []}
def remote_access(state, method, path, body):
"""Answer a /remote-access request: the HTTP status and the JSON document.
GET /remote-access {"p2s", "openvpn", "wireguard", "running", "local_fqdn", "wireguard_server"}: the
settings of the three remote access systems (REMOTE_ACCESS), whether setup runs
each (running: {p2s, openvpn, wireguard}), wan.local_fqdn, which IKEv2 and OpenVPN
need with the server's VPN server certificate, and what a WireGuard peer's
configuration needs of the server (wireguard_server)
PUT /remote-access/<system> change any of the settings of the system's section; answers the section
A change is made as /site-to-site makes one, and applied by restarting linux-manager-setup without waiting for it.
"""
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
return 400, {'error': 'the body must be a JSON object'}
body = body or {}
parts = path.strip('/').split('/')
try:
if (method, len(parts)) == ('GET', 1):
cfg = linuxmanager.load_config()
return 200, {**{system: cfg[system] for system in REMOTE_ACCESS},
'running': {system: bool(state.get(system)) for system in REMOTE_ACCESS},
'local_fqdn': cfg['wan']['local_fqdn'],
'wireguard_server': wireguard_server(cfg, state)}
if method != 'PUT' or len(parts) != 2 or parts[1] not in REMOTE_ACCESS:
return 404, {'error': f'no route {method} {path}'}
system = parts[1]
settings = {option: body[option] for option in linuxmanager.SCHEMA[system] if option in body}
with linuxmanager.config_lock():
data = linuxmanager.read_settings()
current = data.get(system)
data[system] = {**(current if isinstance(current, dict) else {}), **settings}
cfg = linuxmanager.with_defaults(data)
linuxmanager.validate(cfg)
linuxmanager.save_config(data)
except linuxmanager.ConfigError as exc:
return 400, {'error': str(exc)}
except OSError as exc:
return 500, {'error': f'the configuration cannot be written: {exc}'}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
run('systemctl', 'restart', '--no-block', 'linux-manager-setup.service')
return 200, cfg[system]
@@ -0,0 +1,84 @@
"""/dns: the primary name server of the server's domains."""
import linuxmanager
import linuxmanager_dns
from linuxmanager_status import run
from .common import section_change, unit_state
def dns(method, path, query, body):
"""Answer a /dns request with linuxmanager_dns: the HTTP status and the JSON document.
GET /dns {"settings", "nameserver", "public_address", "installed", "running", "domains",
"problem"}: the dns section of the configuration, wan.local_fqdn, the primary name server
of the domains, wan.public_address, the address of the records of the sites with
dns_address public, whether PowerDNS is installed and runs, the domains, each {"name",
"ttl", "manual", "records"} with its manual records and the records the name server
serves, and why the domains cannot be read, null when they can
PUT /dns change any of the settings of the dns section, and with "public_address"
wan.public_address; {"settings", "public_address"}
POST /dns/domains add a domain: {"name", "ttl"?}; ttl is dns.ttl when null or absent; an existing name is
409
PUT /dns/domains/<domain>
change "ttl" of the domain
DELETE /dns/domains/<domain>
remove the domain with its manual records
POST /dns/domains/<domain>/records
add a manual record: {"name", "type", "content", "ttl"?}; name is relative to the domain,
@ for the domain itself, or absolute with a final dot; ttl is the domain's without it
DELETE /dns/domains/<domain>/records
remove the manual records of name= and type=, with content= only that one
A change is applied by restarting linux-manager-setup without waiting for it.
"""
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
return 400, {'error': 'the body must be a JSON object'}
body = body or {}
parts = path.strip('/').split('/')
route = (method, *parts[:2], *parts[3:4])
try:
cfg = linuxmanager.load_config()
if route == ('GET', 'dns'):
domains, problem = [], None
try:
domains = linuxmanager_dns.listing()
except linuxmanager_dns.DatabaseError as exc:
problem = str(exc)
return 200, {'settings': cfg['dns'], 'nameserver': cfg['wan']['local_fqdn'].strip(),
'public_address': cfg['wan']['public_address'].strip(),
'installed': linuxmanager_dns.PDNS_SERVER.exists(),
'running': unit_state(linuxmanager_dns.UNIT) == 'active', 'domains': domains,
'problem': problem}
if route == ('PUT', 'dns'):
wan = cfg['wan']
if 'public_address' in body:
wan = section_change('wan', {'public_address': body.pop('public_address')})
if isinstance(wan, tuple):
return wan
changed = section_change('dns', body)
if isinstance(changed, tuple):
return changed
return 200, {'settings': changed, 'public_address': wan['public_address'].strip()}
if route == ('POST', 'dns', 'domains') and len(parts) == 2:
document, code = linuxmanager_dns.domain_add(body), 201
elif route == ('PUT', 'dns', 'domains') and len(parts) == 3:
document, code = linuxmanager_dns.domain_change(parts[2], body), 200
elif route == ('DELETE', 'dns', 'domains') and len(parts) == 3:
document, code = linuxmanager_dns.domain_remove(parts[2]), 200
elif route == ('POST', 'dns', 'domains', 'records') and len(parts) == 4:
document, code = linuxmanager_dns.add(cfg, parts[2], body), 201
elif route == ('DELETE', 'dns', 'domains', 'records') and len(parts) == 4:
content = query.get('content', [None])[0]
document = linuxmanager_dns.remove(parts[2], query.get('name', [''])[0], query.get('type', [''])[0], content)
code = 200
else:
return 404, {'error': f'no route {method} {path}'}
except linuxmanager_dns.Failure as failure:
return {3: 404, 4: 409}.get(failure.status, 400), {'error': str(failure)}
except linuxmanager_dns.DatabaseError as exc:
return 500, {'error': str(exc)}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
run('systemctl', 'restart', '--no-block', 'linux-manager-setup.service')
return code, document
@@ -0,0 +1,249 @@
"""The request handler of the web service: it reads a request, checks its token and passes it to the module of its
path (answer)."""
import json
import sys
import urllib.parse
from http.server import BaseHTTPRequestHandler
import ldap
import linuxmanager
import linuxmanager_auth
import linuxmanager_net
import linuxmanager_oidc
import linuxmanager_pki
import linuxmanager_sites
from .auth import auth, auth_wait, refusal
from .backup import backup
from .common import BODY_LIMIT, CONFIG_FILE, current, log, read_json, section
from .connectivity import remote_access, site_to_site
from .dns import dns
from .identities import account, groups, service_accounts, users
from .logs import logs
from .https import https_settings
from .metrics import metrics_history, metrics_settings
from .pki import PUBLISHED, ca
from .shell import EDITOR_BODY_LIMIT, shell, shell_editor, shell_passkey, shell_ticket
from .sites import SITE_FILES, site_files, sites
from .status import cached, details, html_of, postgresql_databases, shown, text_of
class Handler(BaseHTTPRequestHandler):
protocol_version = 'HTTP/1.1'
def claims(self, secret):
"""The claims of the request's token signed with the secret, or None without an acceptable one."""
header = self.headers.get('Authorization', '')
return linuxmanager_auth.verify(header[7:].strip(), secret) if header.startswith('Bearer ') else None
def client(self):
"""The client address: the last X-Forwarded-For entry when the peer is a trusted proxy, else the peer."""
peer = self.client_address[0]
trusted = section(read_json(CONFIG_FILE), 'web').get('trusted_proxies') or [self.server.server_address[0]]
forwarded = self.headers.get('X-Forwarded-For', '')
if isinstance(trusted, list) and peer in trusted and forwarded.strip():
return forwarded.split(',')[-1].strip()
return peer
def refusal(self, claims, path, payload):
"""The HTTP status and the message that refuse a request to a protected path, or None when it may be made."""
try:
reason = refusal(claims, self.command, path, payload)
except (ldap.LDAPError, OSError) as exc:
return 500, f'the user directory failed: {exc}'
except linuxmanager_sites.DatabaseError as exc:
return 500, str(exc)
return (403, reason) if reason else None
def answer(self, send_body):
path = self.path.split('?', 1)[0]
extra = []
self.principal = None
# A body that is not read must not be taken for the next request.
if self.command not in ('GET', 'HEAD'):
self.close_connection = True
directory = next((name for name in ('users', 'service-accounts', 'groups', 'account', 'site-to-site', 'backup',
'ca', 'shell', 'remote-access', 'logs', 'sites', 'dns')
if path == f'/{name}' or path.startswith(f'/{name}/')), None)
protected = path in ('/status/full', '/config', '/metrics/history', '/metrics/settings', '/https/settings',
'/postgresql') or \
directory is not None
signing = path in ('/auth', '/auth/passkey', '/auth/passkey/complete', '/auth/refresh', '/auth/logout')
oidc = (path == '/.well-known/openid-configuration' or path.startswith('/oidc/')) and \
self.server.server_address[0] == linuxmanager_net.LOOPBACK
files = SITE_FILES.match(path) if directory == 'sites' else None
query = urllib.parse.parse_qs(urllib.parse.urlsplit(self.path).query)
state = current() if protected or signing else {}
claims = self.claims(state['key']) if protected and state['key'] else None
# The body of a file upload is read as a stream, after the request is allowed.
payload = self.request_body(EDITOR_BODY_LIMIT if path.startswith('/shell/editor/') else BODY_LIMIT) \
if (protected or signing) and self.command not in ('GET', 'HEAD') and not (files and self.command == 'PUT') \
else None
if claims is not None:
self.principal = 'admin' if claims.get('admin') is True else claims.get('sub')
if (protected or signing) and not state['key']:
body, kind, code = b'Not found\n', 'text/plain; charset=utf-8', 404
elif signing and self.command != 'POST':
body, kind, code = b'Method not allowed\n', 'text/plain; charset=utf-8', 405
elif signing:
wait = auth_wait(self.client()) if path not in ('/auth/logout', '/auth/passkey') else 0
if wait:
code, document = 429, {'error': f'too many sign-in calls; retry in {wait} seconds'}
extra.append(('Retry-After', str(wait)))
else:
code, document = auth(path, payload)
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif protected and claims is None:
body, kind, code = b'Unauthorized\n', 'text/plain; charset=utf-8', 401
extra.append(('WWW-Authenticate', 'Bearer'))
elif protected and (refused := self.refusal(claims, path, payload)):
code, kind = refused[0], 'application/json'
body = json.dumps({'error': refused[1]}, indent=2).encode() + b'\n'
elif path == '/status/full' and self.command in ('GET', 'HEAD'):
document = shown(cached(), 'full', self.client())
body, kind, code = json.dumps(document, indent=2).encode() + b'\n', 'application/json', 200
elif path == '/config' and self.command in ('GET', 'HEAD'):
body, kind, code = json.dumps(details(state), indent=2).encode() + b'\n', 'application/json', 200
elif path == '/postgresql' and self.command in ('GET', 'HEAD'):
body, kind, code = json.dumps(postgresql_databases(), indent=2).encode() + b'\n', 'application/json', 200
elif path == '/metrics/history' and self.command in ('GET', 'HEAD'):
query = urllib.parse.parse_qs(urllib.parse.urlsplit(self.path).query)
code, document = metrics_history(query.get('range', ['hour'])[0])
body, kind = json.dumps(document).encode() + b'\n', 'application/json'
elif path == '/metrics/settings':
code, document = metrics_settings(self.command, payload)
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif path == '/https/settings':
code, document = https_settings(self.command, payload)
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif directory == 'shell' and (path, self.command) == ('/shell', 'GET'):
return shell(self, claims.get('sub'))
elif directory == 'shell' and path.startswith('/shell/editor/'):
code, document = shell_editor(claims.get('sub'), self.command, path.strip('/').split('/'), payload)
body, kind = json.dumps(document).encode() + b'\n', 'application/json'
elif directory == 'shell':
code, document = shell_ticket(claims.get('sub'), payload) if (path, self.command) == ('/shell', 'POST') \
else shell_passkey(claims.get('sub')) if (path, self.command) == ('/shell/passkey', 'POST') \
else (404, {'error': f'no route {self.command} {path}'})
if code != 200:
log(f'shell ticket of {claims.get("sub")} from {self.client()} refused: {document["error"]}')
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif directory == 'account':
code, document = account(claims['sub'], self.command, path, payload)
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif directory == 'site-to-site':
code, document = site_to_site(self.command, path, payload)
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif directory == 'remote-access':
code, document = remote_access(state, self.command, path, payload)
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif directory == 'backup':
code, document = backup(self.command, path, payload)
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif directory == 'ca':
code, document = ca(self.command, path, payload)
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif directory == 'logs':
code, document = logs(self.command, path, query)
body, kind = json.dumps(document).encode() + b'\n', 'application/json'
elif files:
answered = site_files(self, files.group(1), files.group(2) or '', query, payload, send_body)
if answered is None:
return
code, document = answered
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif directory == 'sites':
code, document = sites(self.command, path, query, payload)
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif directory == 'dns':
code, document = dns(self.command, path, query, payload)
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif directory == 'service-accounts':
code, document = service_accounts(self.command, path, payload)
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif directory is not None:
code, document = (users if directory == 'users' else groups)(state, self.command, path, payload)
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif oidc:
wait = auth_wait(self.client()) if self.command == 'POST' and path in (
'/oidc/login', '/oidc/device', '/oidc/device_authorization') else 0
if wait:
code, kind = 429, 'application/json'
body = json.dumps({'error': f'too many sign-in calls; retry in {wait} seconds'}).encode() + b'\n'
extra.append(('Retry-After', str(wait)))
else:
code, kind, body, headers = linuxmanager_oidc.answer(self.command, self.path, self.headers, self.raw_body())
extra += headers
elif self.command not in ('GET', 'HEAD'):
body, kind, code = b'Method not allowed\n', 'text/plain; charset=utf-8', 405
elif path.startswith('/pki/') and linuxmanager.PKI_NAME.match(path[5:-4]) and path[-4:] in PUBLISHED and \
(linuxmanager_pki.PUBLISH_DIR / path[5:]).is_file():
body, kind, code = (linuxmanager_pki.PUBLISH_DIR / path[5:]).read_bytes(), PUBLISHED[path[-4:]], 200
elif path in ('/status', '/'):
document = shown(cached(), section(read_json(CONFIG_FILE), 'web').get('status_content'), self.client())
agent = self.headers.get('User-Agent', '').lower()
if path == '/status':
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif 'curl' in agent or 'wget' in agent:
body, kind = text_of(document).encode(), 'text/plain; charset=utf-8'
else:
body, kind = html_of(document).encode(), 'text/html; charset=utf-8'
code = 200
else:
body, kind, code = b'Not found\n', 'text/plain; charset=utf-8', 404
try:
self.send_response(code)
self.send_header('Content-Type', kind)
self.send_header('Content-Length', str(len(body)))
self.send_header('Cache-Control', 'no-store')
for name, value in extra:
self.send_header(name, value)
self.end_headers()
if send_body:
self.wfile.write(body)
except ConnectionError:
# The client went away before the answer was written.
self.close_connection = True
def raw_body(self):
"""The request body's bytes, b'' without one or over BODY_LIMIT."""
try:
size = int(self.headers.get('Content-Length') or 0)
except ValueError:
return b''
return self.rfile.read(size) if 0 < size <= BODY_LIMIT else b''
def request_body(self, limit=BODY_LIMIT):
"""The request body as JSON, None when there is none, or the string 'invalid', also when it is larger than
limit bytes."""
length = self.headers.get('Content-Length')
if not length:
return None
try:
size = int(length)
if not 0 <= size <= limit:
return 'invalid'
return json.loads(self.rfile.read(size) or b'null')
except ValueError:
return 'invalid'
def do_GET(self):
self.answer(True)
def do_HEAD(self):
self.answer(False)
def do_POST(self):
self.answer(True)
def do_PUT(self):
self.answer(True)
def do_DELETE(self):
self.answer(True)
def log_message(self, format, *args):
principal = getattr(self, 'principal', None) or '-'
sys.stderr.write('%s - %s [%s] %s\n' % (self.client_address[0], principal, self.log_date_time_string(), format % args))
@@ -0,0 +1,33 @@
"""/https/settings: the settings of HTTPS, of which the API changes the CA of Caddy's certificates."""
import linuxmanager
from .common import section_change
#: The settings of the https section PUT /https/settings changes.
CHANGEABLE = ('acme_staging',)
def https_settings(method, body):
"""Answer a /https/settings request: the HTTP status and the JSON document.
GET /https/settings {"settings"}: the https section
PUT /https/settings change acme_staging; the other settings of the section are changed on the server with
linux-manager-config-edit, and a body naming one is 400
A change is applied by restarting linux-manager-setup without waiting for it.
"""
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
return 400, {'error': 'the body must be a JSON object'}
if method in ('GET', 'HEAD'):
try:
return 200, {'settings': linuxmanager.load_config()['https']}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
if method != 'PUT':
return 404, {'error': f'no route {method} /https/settings'}
others = sorted(set(body or {}) - set(CHANGEABLE))
if others:
return 400, {'error': f'https.{", https.".join(others)}: changed on the server with linux-manager-config-edit only'}
changed = section_change('https', body or {})
return changed if isinstance(changed, tuple) else (200, {'settings': changed})
@@ -0,0 +1,299 @@
"""The identities of the server's directory: /users, /service-accounts, /groups and the signed-in user's /account."""
import urllib.parse
import ldap
import linuxmanager
import linuxmanager_auth
import linuxmanager_passkeys
import linuxmanager_users
from .common import USER_STATUS
def users(state, method, path, body):
"""Answer a /users request with linuxmanager_users: the HTTP status and the JSON document.
GET /users the users, and {"enabled", "address"} of wireguard, whose network their
WireGuard peers take their addresses from
POST /users add a user: {"name", "password", "first_name", "last_name", "email"};
without the names a Manager-only account
PUT /users/<name> change any of {"first_name", "last_name", "email", "wireguard_peers"};
wireguard_peers, [{"public_key", "ips"}], replaces the user's peers
PUT /users/<name>/password set the password: {"password": ...}
POST /users/<name>/disable lock the user out of OpenVPN
POST /users/<name>/enable let the user in again
POST /users/<name>/unlock end a lockout after failed logins
DELETE /users/<name> remove the user and revoke the certificate
GET /users/<name>/profile the OpenVPN client profile
POST /users/<name>/apple-profile
the Apple configuration profile of IKEv2 access: {"password"}, which
encrypts the user's key in it; {"name", "profile"}
POST /users/<name>/otp a new TOTP secret and its otpauth URI; nothing is stored
PUT /users/<name>/otp store a TOTP secret: {"secret", "code"}, code a code of the secret
DELETE /users/<name>/otp remove the TOTP token
DELETE /users/<name>/passkeys remove all the user's passkeys
PUT /users/<name>/console switch console login on or off: {"enabled", "account"?}, account a local account,
without which the user's own posixAccount; only a token of linux-manager-token
Service accounts are served by /service-accounts: they are not listed, and a request naming one is 404.
"""
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
return 400, {'error': 'the body must be a JSON object'}
body = body or {}
parts = path.strip('/').split('/')
route = (method, len(parts), parts[2] if len(parts) == 3 else None)
name = parts[1] if len(parts) > 1 else None
actions = {
('GET', 1, None): lambda connection: {
'users': [user for user in linuxmanager_users.list_users(connection)['users'] if not user['service']],
'wireguard': {option: linuxmanager.load_config()['wireguard'][option] for option in ('enabled', 'address')}},
('POST', 1, None): lambda connection: linuxmanager_users.add(connection, body.get('name'), body.get('password'), body),
('PUT', 2, None): lambda connection: linuxmanager_users.update(connection, name, body),
('PUT', 3, 'password'): lambda connection: linuxmanager_users.passwd(connection, name, body.get('password')),
('POST', 3, 'disable'): lambda connection: linuxmanager_users.disable(connection, name),
('POST', 3, 'enable'): lambda connection: linuxmanager_users.enable(connection, name),
('POST', 3, 'unlock'): lambda connection: linuxmanager_users.unlock(connection, name),
('DELETE', 2, None): lambda connection: linuxmanager_users.remove(connection, name),
('GET', 3, 'profile'): lambda connection: linuxmanager_users.profile(connection, name),
('POST', 3, 'apple-profile'): lambda connection: linuxmanager_users.apple_profile(connection, name,
body.get('password')),
('POST', 3, 'otp'): lambda connection: linuxmanager_users.otp_new(connection, name),
('PUT', 3, 'otp'): lambda connection: linuxmanager_users.otp_enrol(connection, name, body.get('secret'), body.get('code')),
('DELETE', 3, 'otp'): lambda connection: linuxmanager_users.otp_remove(connection, name),
('DELETE', 3, 'passkeys'): lambda connection: linuxmanager_passkeys.remove(connection, name),
('PUT', 3, 'console'): lambda connection: linuxmanager_users.console(connection, name, body.get('enabled'), body.get('account')),
}
# The linuxmanager_users operation of each route that may revoke refresh tokens.
operations = {('PUT', 3, 'password'): 'passwd', ('POST', 3, 'disable'): 'disable', ('POST', 3, 'unlock'): 'unlock',
('DELETE', 2, None): 'remove'}
if route not in actions:
return 404, {'error': f'no route {method} {path}'}
try:
with linuxmanager_users.session() as connection:
if name is not None and linuxmanager_users.service(linuxmanager_users.entry(connection, name)):
raise linuxmanager_users.Failure(f'{name} is a service account; see /service-accounts', 3)
document = actions[route](connection)
if operations.get(route) in linuxmanager_auth.REVOKING:
linuxmanager_auth.revoke(linuxmanager.load_config(), name)
except linuxmanager_users.Failure as failure:
return USER_STATUS.get(failure.status, 500), {'error': str(failure)}
except (ldap.LDAPError, OSError) as exc:
return 500, {'error': f'the user directory failed: {exc}'}
except linuxmanager_auth.DatabaseError as exc:
return 500, {'error': str(exc)}
except SystemExit as exc:
# linuxmanager ends with SystemExit, carrying the reason, on a configuration it cannot read.
return 500, {'error': str(exc.code)}
return (201 if route == ('POST', 1, None) else 200), document
def oidc_settings():
"""{"enabled", "issuer"} of the OpenID Provider, issuer null while it is disabled."""
cfg = linuxmanager.load_config()
enabled = cfg['oidc']['enabled']
return {'enabled': enabled, 'issuer': linuxmanager.oidc_issuer(cfg) if enabled else None}
def service_accounts(method, path, body):
"""Answer a /service-accounts request with linuxmanager_users: the HTTP status and the JSON document.
GET /service-accounts the service accounts, each with its roles: {"service_accounts": [...],
"oidc": {"enabled", "issuer"}}, issuer that of the OpenID Provider, null
while it is disabled
POST /service-accounts add one: {"name", "password", "display_name"?, "description"?, "email"?,
"redirect_uris"?, "post_logout_redirect_uris"?, "allowed_groups"?,
"grants"?, "public_client"?, "client_scopes"?, "resource_scopes"?,
"roles"?}
PUT /service-accounts/<name> change any of {"display_name", "description", "email", "redirect_uris",
"post_logout_redirect_uris", "allowed_groups", "grants",
"public_client", "client_scopes", "resource_scopes", "roles"}; ""
removes one of the first three, [] the values of the lists
PUT /service-accounts/<name>/password set the password: {"password": ...}
POST /service-accounts/<name>/disable lock it out
POST /service-accounts/<name>/enable let it in again
POST /service-accounts/<name>/unlock end a lockout after failed logins
DELETE /service-accounts/<name> remove it
A service account with redirect URIs or grants is a client of the OpenID Provider, and every service account a
resource (linuxmanager_users.client_settings); roles, [{"name", "description", "groups", "accounts"}], replaces its
roles (linuxmanager_users.set_roles). A name that is not a service account is 404. The internal service accounts
(linuxmanager_users.internal_accounts), which the server manages, are only listed: every other request on one, and a
role assigned to one, is refused with 403. A password change, disabling, unlocking and removing revoke the tokens
the account was issued.
"""
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
return 400, {'error': 'the body must be a JSON object'}
body = body or {}
parts = path.strip('/').split('/')
route = (method, len(parts), parts[2] if len(parts) == 3 else None)
name = parts[1] if len(parts) > 1 else None
settings = {key: body.get(key) for key in ('email', 'display_name', 'description', *linuxmanager_users.CLIENT_KEYS)}
actions = {
('GET', 1, None): lambda connection: {
'service_accounts': [linuxmanager_users.with_roles(connection, user)
for user in linuxmanager_users.list_users(connection)['users'] if user['service']],
'oidc': oidc_settings()},
('POST', 1, None): lambda connection: linuxmanager_users.add(connection, body.get('name'), body.get('password'), settings,
service=True),
('PUT', 2, None): lambda connection: linuxmanager_users.update_service(connection, name, settings),
('PUT', 3, 'password'): lambda connection: linuxmanager_users.passwd(connection, name, body.get('password')),
('POST', 3, 'disable'): lambda connection: linuxmanager_users.disable(connection, name),
('POST', 3, 'enable'): lambda connection: linuxmanager_users.enable(connection, name),
('POST', 3, 'unlock'): lambda connection: linuxmanager_users.unlock(connection, name),
('DELETE', 2, None): lambda connection: linuxmanager_users.remove(connection, name),
}
# The linuxmanager_users operation of each route that may revoke tokens.
operations = {('PUT', 3, 'password'): 'passwd', ('POST', 3, 'disable'): 'disable', ('POST', 3, 'unlock'): 'unlock',
('DELETE', 2, None): 'remove'}
if route not in actions:
return 404, {'error': f'no route {method} {path}'}
internal = linuxmanager_users.internal_accounts()
if name is not None and name in internal:
return 403, {'error': f'{name} is an internal service account, which the server manages'}
roles = settings['roles'] if isinstance(settings['roles'], list) else []
assigned = sorted(set(internal).intersection(account for role in roles if isinstance(role, dict)
for account in (role.get('accounts') or [])
if isinstance(account, str)))
if assigned:
return 403, {'error': f'{", ".join(assigned)}: internal service accounts, which the server manages, are not '
'assigned roles'}
try:
with linuxmanager_users.session() as connection:
if name is not None and not linuxmanager_users.service(linuxmanager_users.entry(connection, name)):
raise linuxmanager_users.Failure(f'{name} is not a service account', 3)
document = actions[route](connection)
if operations.get(route) in linuxmanager_auth.REVOKING:
linuxmanager_auth.revoke(linuxmanager.load_config(), name)
except linuxmanager_users.Failure as failure:
return USER_STATUS.get(failure.status, 500), {'error': str(failure)}
except (ldap.LDAPError, OSError) as exc:
return 500, {'error': f'the user directory failed: {exc}'}
except linuxmanager_auth.DatabaseError as exc:
return 500, {'error': str(exc)}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
return (201 if route == ('POST', 1, None) else 200), document
def groups(state, method, path, body):
"""Answer a /groups request with linuxmanager_users: the HTTP status and the JSON document.
GET /groups the groups with their members
POST /groups add a group: {"name", "members": [...], "description"?}
PUT /groups/<name> rename it or change the description: {"name"?, "description"?}
DELETE /groups/<name> remove the group; its members are kept
POST /groups/<name>/members add a member: {"user": ...}
DELETE /groups/<name>/members/<user> take a member out; the last member cannot leave
An internal service account (linuxmanager_users.internal_accounts) is not put in or taken out of a group: such a
request is refused with 403.
"""
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
return 400, {'error': 'the body must be a JSON object'}
body = body or {}
parts = path.strip('/').split('/')
name = parts[1] if len(parts) > 1 else None
route = (method, len(parts), parts[2] if len(parts) > 2 else None)
members = body.get('members') if route == ('POST', 1, None) else [body.get('user')] if route == ('POST', 3, 'members') \
else [parts[3]] if route == ('DELETE', 4, 'members') else []
internal = sorted(set(linuxmanager_users.internal_accounts()).intersection(
member for member in (members if isinstance(members, list) else []) if isinstance(member, str)))
if internal:
return 403, {'error': f'{", ".join(internal)}: internal service accounts, which the server manages, are not group members'}
actions = {
('GET', 1, None): lambda connection: linuxmanager_users.list_groups(connection),
('POST', 1, None): lambda connection: linuxmanager_users.group_add(
connection, body.get('name'), body.get('description'), body.get('members')),
('PUT', 2, None): lambda connection: linuxmanager_users.group_update(
connection, name, body.get('name'), body.get('description')),
('DELETE', 2, None): lambda connection: linuxmanager_users.group_remove(connection, name),
('POST', 3, 'members'): lambda connection: linuxmanager_users.group_member_add(connection, name, body.get('user')),
('DELETE', 4, 'members'): lambda connection: linuxmanager_users.group_member_remove(connection, name, parts[3]),
}
if route not in actions:
return 404, {'error': f'no route {method} {path}'}
try:
with linuxmanager_users.session() as connection:
document = actions[route](connection)
except linuxmanager_users.Failure as failure:
return USER_STATUS.get(failure.status, 500), {'error': str(failure)}
except (ldap.LDAPError, OSError) as exc:
return 500, {'error': f'the user directory failed: {exc}'}
return (201 if route == ('POST', 1, None) else 200), document
def account(name, method, path, body):
"""Answer an /account request of the user named by the token with linuxmanager_users: the HTTP status and the JSON document.
GET /account the user, with admin, whether the user is a member of ADMIN_GROUP
PUT /account/password set the password: {"current_password", "current_code"?, "password"}
POST /account/otp a new TOTP secret and its otpauth URI; nothing is stored
PUT /account/otp store a TOTP secret: {"current_password", "current_code"?, "secret", "code"},
code a code of the secret
POST /account/passkeys begin the registration of a passkey: {"current_password", "current_code"?};
{"id", "options"} for navigator.credentials.create
PUT /account/passkeys store the passkey: {"id", "credential", "label"?}, credential the browser's answer
DELETE /account/passkeys/<id>
remove a passkey
POST /account/profile the user's OpenVPN client profile: {"current_password", "current_code"?}; {"name",
"profile"}
POST /account/apple-profile
the user's Apple configuration profile of IKEv2 access:
{"current_password", "current_code"?, "password"}, password encrypting the user's key in
it; {"name", "profile"}
A change but the removal of a passkey, and the profiles, need the current password and, for an enrolled user, a
current code; a wrong
one is 403 and counts as a failed login. A password change revokes the user's refresh tokens.
"""
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
return 400, {'error': 'the body must be a JSON object'}
body = body or {}
route = (method, path.rstrip('/'))
removing = method == 'DELETE' and path.startswith('/account/passkeys/') and len(path.strip('/').split('/')) == 3
if not removing and route not in (('GET', '/account'), ('PUT', '/account/password'), ('POST', '/account/otp'),
('PUT', '/account/otp'), ('POST', '/account/passkeys'), ('PUT', '/account/passkeys'),
('POST', '/account/profile'), ('POST', '/account/apple-profile')):
return 404, {'error': f'no route {method} {path}'}
try:
if route == ('PUT', '/account/password'):
linuxmanager_users.check_password(body.get('password'))
if route in (('PUT', '/account/password'), ('PUT', '/account/otp'), ('POST', '/account/passkeys'),
('POST', '/account/profile'), ('POST', '/account/apple-profile')):
linuxmanager_users.login(name, body.get('current_password'), body.get('current_code'))
with linuxmanager_users.session() as connection:
if route == ('GET', '/account'):
document = {**linuxmanager_users.summary(linuxmanager_users.entry(connection, name)),
'admin': linuxmanager_users.is_admin(connection, name)}
elif route == ('PUT', '/account/password'):
document = linuxmanager_users.passwd(connection, name, body['password'])
elif route == ('POST', '/account/otp'):
document = linuxmanager_users.otp_new(connection, name)
elif route == ('POST', '/account/profile'):
document = linuxmanager_users.profile(connection, name)
elif route == ('POST', '/account/apple-profile'):
document = linuxmanager_users.apple_profile(connection, name, body.get('password'))
elif route == ('POST', '/account/passkeys'):
document = linuxmanager_passkeys.register_begin(connection, name)
elif route == ('PUT', '/account/passkeys'):
document = linuxmanager_passkeys.register_complete(connection, name, body.get('id'), body.get('credential'),
body.get('label'))
elif removing:
document = linuxmanager_passkeys.remove(connection, name, urllib.parse.unquote(path.strip('/').split('/')[2]))
else:
document = linuxmanager_users.otp_enrol(connection, name, body.get('secret'), body.get('code'))
if route == ('PUT', '/account/password'):
linuxmanager_auth.revoke(linuxmanager.load_config(), name)
except linuxmanager_users.Failure as failure:
if failure.status == 5:
return 403, {'error': 'the current password or code is wrong'}
return USER_STATUS.get(failure.status, 500), {'error': str(failure)}
except (ldap.LDAPError, OSError) as exc:
return 500, {'error': f'the user directory failed: {exc}'}
except linuxmanager_auth.DatabaseError as exc:
return 500, {'error': str(exc)}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
return 200, document
@@ -0,0 +1,75 @@
"""/logs: the last lines of the journal of the server's units and of the files web.log_files lists."""
import os
import subprocess
import linuxmanager
#: The units whose journal /logs/journal shows, besides the whole journal: the package's own and the services it runs.
JOURNAL_UNITS = ('linux-manager-setup', 'linux-manager-web', 'linux-manager-metrics', 'linux-manager-pki', 'linux-manager-backup',
'linux-manager-caddy', 'linux-manager-ui', 'caddy', 'slapd', 'nslcd', 'postgresql', 'strongswan',
'wg-quick@wg0', 'openvpn-server@linux-manager', 'ssh', 'linux-manager-resolver', 'linux-manager-webdav', 'pdns')
#: The most lines /logs answers, the number it answers without lines, and the most bytes it reads from the end of a file.
LOG_LINES_MAX = 1000
LOG_LINES_DEFAULT = 200
LOG_BYTES_MAX = 1024 * 1024
def tail(path, count):
"""The last count lines of a file, read from at most LOG_BYTES_MAX bytes at its end, invalid UTF-8 replaced."""
with open(path, 'rb') as handle:
size = handle.seek(0, os.SEEK_END)
start = max(0, size - LOG_BYTES_MAX)
handle.seek(start)
lines = handle.read().decode('utf-8', 'replace').splitlines()
# A read that starts inside the file starts inside a line, which is left out.
return (lines[1:] if start and lines else lines)[-count:]
def logs(method, path, query):
"""Answer a /logs request: the HTTP status and the JSON document.
GET /logs {"units": JOURNAL_UNITS, "files": web.log_files, "max_lines": LOG_LINES_MAX}
GET /logs/journal {"lines": [...]}: the last lines of the journal, of the unit unit= names, one of
JOURNAL_UNITS, or of the whole journal without it
GET /logs/file {"lines": [...]}: the last lines of the file path= names, one of web.log_files
lines= is the number of lines, 1 to LOG_LINES_MAX, LOG_LINES_DEFAULT without it.
"""
if method not in ('GET', 'HEAD'):
return 405, {'error': f'{method} is not allowed on /logs'}
try:
files = [item.strip() for item in linuxmanager.load_config()['web']['log_files'] if item.strip()]
except SystemExit as exc:
return 500, {'error': str(exc.code)}
if path == '/logs':
return 200, {'units': list(JOURNAL_UNITS), 'files': files, 'max_lines': LOG_LINES_MAX}
text = query.get('lines', [str(LOG_LINES_DEFAULT)])[0]
if not text.isdigit() or not 1 <= int(text) <= LOG_LINES_MAX:
return 400, {'error': f'lines must be a whole number from 1 to {LOG_LINES_MAX}'}
count = int(text)
if path == '/logs/journal':
unit = query.get('unit', [''])[0]
if unit and unit not in JOURNAL_UNITS:
return 404, {'error': f'{unit} is not one of the units whose journal /logs shows'}
try:
result = subprocess.run(['journalctl', '--no-pager', '--quiet', '--output=short-iso', f'--lines={count}',
*([f'--unit={unit}.service'] if unit else [])], capture_output=True, text=True,
errors='replace', timeout=30)
except (OSError, subprocess.TimeoutExpired) as exc:
return 500, {'error': f'journalctl cannot be run: {exc}'}
if result.returncode != 0:
return 500, {'error': result.stderr.strip() or f'journalctl ended with status {result.returncode}'}
return 200, {'lines': result.stdout.splitlines()[-count:]}
if path == '/logs/file':
name = query.get('path', [''])[0]
if name not in files:
return 404, {'error': f'{name!r} is not one of web.log_files'}
try:
return 200, {'lines': tail(name, count)}
except OSError as exc:
return (404 if isinstance(exc, FileNotFoundError) else 500), {'error': f'{name} cannot be read: {exc.strerror}'}
return 404, {'error': f'no route {method} {path}'}
@@ -0,0 +1,55 @@
"""/metrics/history and /metrics/settings: the metrics the metrics service keeps, and its settings."""
import linuxmanager
import linuxmanager_metrics
from .common import section_change
def metrics_history(view):
"""Answer GET /metrics/history?range=<view> with linuxmanager_metrics.history: the HTTP status and the JSON document.
view is hour, the default, day, month or year (linuxmanager_metrics.RANGES).
"""
if view not in linuxmanager_metrics.RANGES:
return 400, {'error': f'range must be one of {", ".join(linuxmanager_metrics.RANGES)}'}
try:
with linuxmanager_metrics.database(linuxmanager.load_config()) as connection:
return 200, linuxmanager_metrics.history(connection, view)
except linuxmanager_metrics.DatabaseError as exc:
return 500, {'error': str(exc)}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
def metrics_shown(settings):
"""The metrics section as /metrics/settings shows it: its settings but otlp_password, with otlp_password_set."""
return {**{option: value for option, value in settings.items() if option != 'otlp_password'},
'otlp_password_set': bool(settings['otlp_password'])}
def metrics_settings(method, body):
"""Answer a /metrics/settings request: the HTTP status and the JSON document.
GET /metrics/settings {"settings"}: the metrics section without otlp_password, with otlp_password_set
PUT /metrics/settings change any of the settings of the metrics section; an empty or absent otlp_password keeps
the current secret, and an empty otlp_username removes it
A change is applied by restarting linux-manager-setup without waiting for it.
"""
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
return 400, {'error': 'the body must be a JSON object'}
if method in ('GET', 'HEAD'):
try:
return 200, {'settings': metrics_shown(linuxmanager.load_config()['metrics'])}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
if method != 'PUT':
return 404, {'error': f'no route {method} /metrics/settings'}
body = dict(body or {})
if not body.get('otlp_password'):
body.pop('otlp_password', None)
if body.get('otlp_username') == '':
body['otlp_password'] = ''
changed = section_change('metrics', body)
return changed if isinstance(changed, tuple) else (200, {'settings': metrics_shown(changed)})
@@ -0,0 +1,90 @@
"""The PKI: /ca, and the certificates and CRLs /pki/ serves without authentication."""
import ldap
import linuxmanager
import linuxmanager_pki
from linuxmanager_status import run
from .common import PKI_STATUS
#: A file /pki/ serves: the certificate or the CRL of a CA, and its content type.
PUBLISHED = {'.crt': 'application/pkix-cert', '.crl': 'application/pkix-crl'}
def ca(method, path, body):
"""Answer a /ca request with linuxmanager_pki: the HTTP status and the JSON document.
GET /ca {"settings", "templates", "extended_key_usages", "status",
"certificates"}: the pki section, the templates with the built-in
ones, the extended key usages, each {oid, builtin}, the root, the
issuing CAs, the retired CAs and the warnings (linuxmanager_pki.status),
and the certificates issued, newest first
PUT /ca change any of the settings of the pki section
POST /ca/certificates issue a certificate: {"ca", "template", "csr"} answers {"serial",
"certificate", "chain"}; {"ca", "template", "subject", "sans",
"password"} answers {"serial", "pfx"}, a PKCS#12 file in base64
POST /ca/certificates/<serial>/revoke revoke a certificate: {"reason"?}
GET /ca/bundle/<name> the PEM certificates a client trusts the CA with: {"name", "pem"}; for
root the root alone, for an issuing CA the issuing CA and the root
GET /ca/issuing/<name>/csr the CSR of an issuing CA that waits for the offline root, with the
openssl extensions to sign it with: {"name", "csr", "extensions"}
PUT /ca/issuing/<name>/certificate take its certificate signed by the root: {"certificate"}
PUT /ca/root/crl take a root CRL signed offline: {"crl"}
A change of the settings is made as /site-to-site makes one; it, a revoked router certificate and an imported
issuing CA certificate are applied by restarting linux-manager-setup without waiting for it.
"""
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
return 400, {'error': 'the body must be a JSON object'}
body = body or {}
parts = path.strip('/').split('/')
route = (method, *parts[1:])
try:
cfg = linuxmanager.load_config()
if len(route) == 3 and route[:2] == ('GET', 'bundle'):
return 200, linuxmanager_pki.bundle(route[2])
if route == ('GET',):
return 200, {'settings': cfg['pki'], 'templates': linuxmanager.pki_templates(cfg),
'extended_key_usages': {name: {'oid': oid, 'builtin': name in linuxmanager.EXTENDED_KEY_USAGES}
for name, oid in linuxmanager.extended_key_usages(cfg).items()},
'status': linuxmanager_pki.status(cfg), 'certificates': linuxmanager_pki.certificates()}
if route == ('PUT',):
settings = {option: body[option] for option in linuxmanager.SCHEMA['pki'] if option in body}
with linuxmanager.config_lock():
data = linuxmanager.read_settings()
current = data.get('pki')
data['pki'] = {**(current if isinstance(current, dict) else {}), **settings}
cfg = linuxmanager.with_defaults(data)
linuxmanager.validate(cfg)
linuxmanager.save_config(data)
run('systemctl', 'restart', '--no-block', 'linux-manager-setup.service')
return 200, {'settings': cfg['pki'], 'templates': linuxmanager.pki_templates(cfg)}
with linuxmanager_pki.locked():
if route == ('POST', 'certificates'):
return 201, linuxmanager_pki.issue(cfg, body.get('ca'), body.get('template'), csr=body.get('csr'),
subject=body.get('subject'), sans=body.get('sans'),
password=body.get('password'))
if len(route) == 4 and route[:2] == ('POST', 'certificates') and route[3] == 'revoke':
document = linuxmanager_pki.revoke(cfg, route[2], body.get('reason'))
elif len(route) == 4 and route[:2] == ('GET', 'issuing') and route[3] == 'csr':
return 200, linuxmanager_pki.csr(cfg, route[2])
elif len(route) == 4 and route[:2] == ('PUT', 'issuing') and route[3] == 'certificate':
document = linuxmanager_pki.import_ca(cfg, route[2], body.get('certificate'))
elif route == ('PUT', 'root', 'crl'):
return 200, linuxmanager_pki.import_root_crl(body.get('crl'))
else:
return 404, {'error': f'no route {method} {path}'}
except linuxmanager.ConfigError as exc:
return 400, {'error': str(exc)}
except linuxmanager_pki.Failure as failure:
return PKI_STATUS.get(failure.status, 500), {'error': str(failure)}
except ldap.LDAPError as exc:
return 500, {'error': f'the directory, which holds the PKI, failed: {exc}'}
except OSError as exc:
return 500, {'error': f'the PKI cannot be changed: {exc}'}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
if route[1] == 'issuing' or document.get('reissue'):
run('systemctl', 'restart', '--no-block', 'linux-manager-setup.service')
return 200, document
@@ -0,0 +1,512 @@
"""/shell: the ticket (shell_ticket), with the passkey proof it may take (shell_passkey), the WebSocket of the terminal
(shell), and the editor of the shell: the socket linux-manager-editor names files on (editor_listen) and /shell/editor/...
(shell_editor)."""
import base64
import fcntl
import hashlib
import json
import os
import pwd
import secrets
import select
import socket
import struct
import subprocess
import termios
import threading
import time
import urllib.parse
import ldap
import linuxmanager
import linuxmanager_passkeys
import linuxmanager_users
from linuxmanager_status import run
from .common import BODY_LIMIT, MIB, USER_STATUS, log
#: Seconds a ticket of POST /shell may open a shell.
SHELL_TICKET_SECONDS = 30
#: Seconds between the checks of an open shell's user, which end the shell once its console is refused.
SHELL_CHECK_SECONDS = 60
#: The largest WebSocket message accepted from a client, in bytes.
MESSAGE_LIMIT = 65536
#: The socket linux-manager-editor (lme) names files on to open them in the editor of a shell (see editor_listen()).
EDITOR_SOCKET = '/run/linux-manager/editor.sock'
#: The helper that reads and writes a file of the editor as the shell's user.
EDITOR_FILE = '/usr/lib/linux-manager/editor-file'
#: The largest file the editor opens, in bytes, as editor-file allows; a body of PUT /shell/editor/... may hold its
#: content escaped in JSON.
EDITOR_MAX_BYTES = MIB
EDITOR_BODY_LIMIT = 2 * EDITOR_MAX_BYTES + BODY_LIMIT
#: The most files a shell has open in its editor at once.
EDITOR_FILES_MAX = 20
#: Seconds a reader of the editor socket and a run of editor-file may take.
EDITOR_TIMEOUT = 30
#: The GUID of RFC 6455 that the Sec-WebSocket-Accept of a handshake is made with.
WEBSOCKET_GUID = '258EAFA5-E914-47DA-95CA-C5AB0DC85B11'
#: WebSocket opcodes.
TEXT, BINARY, CLOSE, PING, PONG = 1, 2, 8, 9, 10
#: The unused tickets of POST /shell: ticket -> (user, monotonic time of expiry).
_shell_tickets = {}
_shell_lock = threading.Lock()
#: The last use of a shell of each user, a sign-in of POST /shell or a keystroke: user -> monotonic time.
_shell_used = {}
#: The open shells by their session, which each shell has in LINUX_MANAGER_SHELL: session -> {"user", "account", "uid",
#: "send", the function that sends a WebSocket frame to its client, "files": {file id: path}}.
_shell_sessions = {}
def shell_ticket(name, body):
"""Answer POST /shell: the HTTP status and the JSON document.
The user named by the token signs in again with {"password", "code"}, its password and TOTP code, or with
{"passkey": {"id", "credential"}}, the answer of the browser to a proof begun by POST /shell/passkey (see
shell_passkey) with one of its own passkeys and the user verified. A wrong password or code counts as a failed
login; it and a passkey that is not accepted are 403. Without either, the user is not asked to sign in while its
last use of a shell (see _shell_used) is less than web.shell_auth_timeout minutes ago. The answer is {"ticket",
"expires_in"}: a ticket that opens one shell (GET /shell) for the same user within SHELL_TICKET_SECONDS, or a null
ticket when the user must sign in. 403 while web.shell is false, the user's console login is not on or the user has
no TOTP token.
"""
if not isinstance(name, str):
return 403, {'error': 'the shell needs a token of a signed-in user'}
if not isinstance(body, dict):
return 400, {'error': 'the body must be a JSON object'}
try:
options = linuxmanager.load_config()['web']
if not options['shell']:
return 403, {'error': 'the shell is disabled; linux-manager-config-edit set web.shell true enables it'}
refusal = console_account(name)[1]
if refusal:
return 403, {'error': refusal}
if 'passkey' in body:
passkey = body['passkey'] if isinstance(body['passkey'], dict) else {}
with linuxmanager_users.session() as connection:
linuxmanager_passkeys.shell_complete(connection, name, passkey.get('id'), passkey.get('credential'))
elif 'password' in body:
linuxmanager_users.login(name, body.get('password'), body.get('code'))
else:
with _shell_lock:
used = _shell_used.get(name)
if used is None or time.monotonic() - used >= options['shell_auth_timeout'] * 60:
return 200, {'ticket': None, 'expires_in': 0}
except linuxmanager_users.Failure as failure:
if failure.status == 5:
return 403, {'error': 'the passkey is not accepted' if 'passkey' in body else 'the password or the code is wrong'}
return USER_STATUS.get(failure.status, 500), {'error': str(failure)}
except (ldap.LDAPError, OSError) as exc:
return 500, {'error': f'the user directory failed: {exc}'}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
ticket = secrets.token_urlsafe(32)
now = time.monotonic()
with _shell_lock:
for unused in [key for key, (_, expires) in _shell_tickets.items() if expires <= now]:
del _shell_tickets[unused]
_shell_tickets[ticket] = (name, now + SHELL_TICKET_SECONDS)
_shell_used[name] = now
return 200, {'ticket': ticket, 'expires_in': SHELL_TICKET_SECONDS}
def shell_passkey(name):
"""Answer POST /shell/passkey: begin the proof of the signed-in user with one of its own passkeys for POST /shell,
{"id", "options"}, options those of navigator.credentials.get; the HTTP status and the JSON document. 403 while
the user may not use a shell (console_account), 400 for a user without a passkey."""
if not isinstance(name, str):
return 403, {'error': 'the shell needs a token of a signed-in user'}
try:
if not linuxmanager.load_config()['web']['shell']:
return 403, {'error': 'the shell is disabled; linux-manager-config-edit set web.shell true enables it'}
refusal = console_account(name)[1]
if refusal:
return 403, {'error': refusal}
with linuxmanager_users.session() as connection:
return 200, linuxmanager_passkeys.shell_begin(connection, name)
except linuxmanager_users.Failure as failure:
return USER_STATUS.get(failure.status, 500), {'error': str(failure)}
except (ldap.LDAPError, OSError) as exc:
return 500, {'error': f'the user directory failed: {exc}'}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
def console_account(name):
"""The system account the console of the user runs as and None, or None and the reason the user may not use a
console: its console login is off, or it has no TOTP token, which the PAM login session of the shell and sudo need
also when a passkey opens the shell."""
with linuxmanager_users.session() as connection:
user = linuxmanager_users.summary(linuxmanager_users.entry(connection, name))
if not user['console'] or not user['console']['enabled']:
return None, f'console login is not enabled for {name}; linux-manager-user console sets it'
if not user['otp']:
return None, f'{name} has no TOTP token; the shell needs one, enrolled on the Account page'
return user['console']['account'], None
def take_ticket(name, ticket):
"""Whether the ticket of POST /shell is unexpired and was given to the user; it is used up either way."""
with _shell_lock:
holder, expires = _shell_tickets.pop(ticket, (None, 0.0))
return holder == name and expires > time.monotonic()
def shell_command(account, unit, session):
"""The command that runs the login shell of the system user account in the transient unit, on its terminal, with
the shell's session in LINUX_MANAGER_SHELL for linux-manager-editor.
setsid makes the terminal the controlling one of systemd-run, which passes the window size on; systemd-run starts
the shell outside the web service's sandbox, in a PAM login session of the user.
"""
return ['setsid', '--ctty', 'systemd-run', '--quiet', '--collect', '--pty', '--send-sighup', f'--unit={unit}',
f'--uid={account.pw_name}', '--property=PAMName=login', f'--working-directory={account.pw_dir}',
'--setenv=TERM=xterm-256color', f'--setenv=LINUX_MANAGER_SHELL={session}', account.pw_shell or '/bin/sh', '-l']
def editor_file(account, action, path, content=b'', version=''):
"""Run editor-file as the system user account outside the sandbox: read the file, or write content to it when it
still has version. The exit status of editor-file and its output, or its message on failure."""
command = ['systemd-run', '--quiet', '--collect', '--pipe', f'--uid={account}', EDITOR_FILE, action, path]
try:
result = subprocess.run(command + ([version] if action == 'write' else []), input=content, capture_output=True,
timeout=EDITOR_TIMEOUT)
except (OSError, subprocess.TimeoutExpired) as exc:
return 1, f'editor-file cannot be run: {exc}'.encode()
return result.returncode, result.stdout if result.returncode == 0 else result.stderr.strip() or \
f'editor-file ended with status {result.returncode}'.encode()
def shell_editor(name, method, parts, body):
"""Answer a request of a file open in the editor of one of the user's shells: the HTTP status and the JSON document.
GET /shell/editor/<session>/<id> {"path", "content", "version"}, version the SHA-256 of the content
PUT /shell/editor/<session>/<id> {"content", "version"} writes the content when the file still has version,
else 409; answers {"path", "version"}
DELETE /shell/editor/<session>/<id> closes the file
The file is read and written as the shell's console account (editor_file). 404 for a session that is not a shell of
the user or a file it does not have open.
"""
if len(parts) != 4:
return 404, {'error': f'no route {method} /{"/".join(parts)}'}
with _shell_lock:
session = _shell_sessions.get(parts[2])
path = session['files'].get(parts[3]) if session and session['user'] == name else None
account = session['account'] if path else None
if path is None:
return 404, {'error': 'no such file is open in a shell of yours'}
if method == 'DELETE':
with _shell_lock:
session['files'].pop(parts[3], None)
return 200, {'path': path, 'closed': True}
if method == 'GET':
status, output = editor_file(account, 'read', path)
if status != 0:
return (400 if status == 2 else 500), {'error': output.decode(errors='replace')}
return 200, {'path': path, 'content': output.decode(), 'version': hashlib.sha256(output).hexdigest()}
if method != 'PUT':
return 405, {'error': f'{method} is not allowed on a file of the editor'}
if not isinstance(body, dict) or not isinstance(body.get('content'), str) or not isinstance(body.get('version'), str):
return 400, {'error': 'the body must be {"content", "version"}'}
content = body['content'].encode()
if len(content) > EDITOR_MAX_BYTES:
return 413, {'error': f'the content is larger than {EDITOR_MAX_BYTES // 1024} KiB'}
status, output = editor_file(account, 'write', path, content, body['version'])
if status != 0:
return {2: 400, 3: 409}.get(status, 500), {'error': output.decode(errors='replace')}
log(f'editor of {name} saved {path} as {account}')
return 200, {'path': path, 'version': hashlib.sha256(content).hexdigest()}
def editor_request(connection):
"""Answer one request of linux-manager-editor on the editor socket: {"session", "path"} on a line opens the file in the
editor of the shell of the session, whose client is sent {"editor": "open", "session", "id", "path"} as a text
message.
The peer must run as the shell's console account and the path must be absolute. The answer is {"ok": true} or
{"error"} on a line.
"""
try:
connection.settimeout(EDITOR_TIMEOUT)
_, uid, _ = struct.unpack('iII', connection.getsockopt(socket.SOL_SOCKET, socket.SO_PEERCRED,
struct.calcsize('iII')))
request = json.loads(connection.makefile('rb').readline(4096) or b'null')
if not isinstance(request, dict) or not isinstance(request.get('session'), str) or \
not isinstance(request.get('path'), str):
answer = {'error': 'the request must be {"session", "path"}'}
elif not os.path.isabs(request['path']):
answer = {'error': 'the path must be absolute'}
else:
with _shell_lock:
session = _shell_sessions.get(request['session'])
if session is None or session['uid'] != uid:
answer = {'error': 'no shell of the Manager has this session; lme works only in its shell'}
elif len(session['files']) >= EDITOR_FILES_MAX:
answer = {'error': f'the editor has {EDITOR_FILES_MAX} files open; close one first'}
else:
file_id = secrets.token_urlsafe(8)
session['files'][file_id] = request['path']
answer = None
if answer is None:
session['send'](TEXT, json.dumps({'editor': 'open', 'session': request['session'], 'id': file_id,
'path': request['path']}).encode())
log(f'editor of {session["user"]} opened {request["path"]}')
answer = {'ok': True}
connection.sendall(json.dumps(answer).encode() + b'\n')
except (OSError, ValueError, struct.error):
pass
finally:
connection.close()
def editor_listen():
"""Listen on EDITOR_SOCKET, which any local user may connect to, answering each connection in a thread with
editor_request; the socket of an earlier run is replaced. Does not return; a failure to listen is logged and the
editor stays closed."""
try:
if os.path.lexists(EDITOR_SOCKET):
os.unlink(EDITOR_SOCKET)
listener = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
listener.bind(EDITOR_SOCKET)
os.chmod(EDITOR_SOCKET, 0o666)
listener.listen(16)
except OSError as exc:
log(f'the editor socket {EDITOR_SOCKET} cannot be opened: {exc}')
return
while True:
try:
connection, _ = listener.accept()
except OSError:
continue
threading.Thread(target=editor_request, args=(connection,), name='editor request', daemon=True).start()
def frame(opcode, payload):
"""A final, unmasked WebSocket frame of a server."""
size = len(payload)
if size < 126:
length = bytes([size])
elif size < 65536:
length = bytes([126]) + size.to_bytes(2, 'big')
else:
length = bytes([127]) + size.to_bytes(8, 'big')
return bytes([0x80 | opcode]) + length + payload
def exact(handler, size):
"""The next size bytes from the client of the Handler handler; ConnectionError when it closes first."""
data = handler.rfile.read(size)
if len(data) != size:
raise ConnectionError('the client closed the connection')
return data
def message(handler, send):
"""The opcode and the payload of the next WebSocket data message of the client of the Handler handler, or CLOSE
and its payload.
Fragments are joined and unmasked; a ping is answered with send, a pong ignored. ValueError for an unmasked
frame or a message over MESSAGE_LIMIT.
"""
opcode, payload = None, b''
while True:
head = exact(handler, 2)
code, size = head[0] & 0x0F, head[1] & 0x7F
if not head[1] & 0x80:
raise ValueError('a frame of the client is not masked')
if size == 126:
size = int.from_bytes(exact(handler, 2), 'big')
elif size == 127:
size = int.from_bytes(exact(handler, 8), 'big')
if len(payload) + size > MESSAGE_LIMIT:
raise ValueError(f'a message is larger than {MESSAGE_LIMIT} bytes')
mask = exact(handler, 4) * (size // 4 + 1)
data = bytes(byte ^ key for byte, key in zip(exact(handler, size), mask))
if code == PING:
send(PONG, data)
continue
if code == PONG:
continue
if code == CLOSE:
return CLOSE, data
opcode = code if opcode is None else opcode
payload += data
if head[0] & 0x80:
return opcode, payload
def shell(handler, name):
"""Answer GET /shell?ticket=<ticket>, a WebSocket handshake the Handler handler received, with a login shell of the
user's console account (linuxmanager_users.console) on a terminal.
The ticket must come from POST /shell for the user named by the token. Binary messages of the client are its
keystrokes, a text message {"cols", "rows"} sets the window size, and binary messages of the server are the
terminal's output; a text message of the server {"editor": "open", "session", "id", "path"} opens a file the
shell named with linux-manager-editor (editor_request), which the client reads and saves with
/shell/editor/<session>/<id> (shell_editor) while the shell lives. The shell ends after web.shell_idle_minutes
without a keystroke, when it exits, when the client closes the connection, or when a check of the user (SHELL_CHECK_SECONDS) finds its console login off or
its TOTP token gone; the server then closes it with the reason. Each opening, end and refusal is logged
with its reason.
"""
def refuse(code, message):
log(f'shell of {name} from {handler.client()} refused: {message}')
body = json.dumps({'error': message}).encode() + b'\n'
try:
handler.send_response(code)
handler.send_header('Content-Type', 'application/json')
handler.send_header('Content-Length', str(len(body)))
handler.end_headers()
handler.wfile.write(body)
except ConnectionError:
handler.close_connection = True
key = handler.headers.get('Sec-WebSocket-Key', '').strip()
if handler.headers.get('Upgrade', '').lower() != 'websocket' or 'upgrade' not in handler.headers.get('Connection', '').lower() \
or handler.headers.get('Sec-WebSocket-Version') != '13' or not key:
return refuse(400, 'GET /shell must be a WebSocket handshake')
query = urllib.parse.parse_qs(urllib.parse.urlsplit(handler.path).query)
if not take_ticket(name, query.get('ticket', [''])[0]):
return refuse(403, 'the ticket is not valid; POST /shell gives one')
try:
options = linuxmanager.load_config()['web']
except SystemExit as exc:
return refuse(500, str(exc.code))
if not options['shell']:
return refuse(403, 'the shell is disabled; linux-manager-config-edit set web.shell true enables it')
try:
login, refusal = console_account(name)
except (linuxmanager_users.Failure, ldap.LDAPError, OSError) as exc:
return refuse(500, f'the user directory failed: {exc}')
if refusal:
return refuse(403, refusal)
try:
account = pwd.getpwnam(login)
except KeyError:
return refuse(500, f'the system does not know the account {login}; is nslcd running?')
unit = f'linux-manager-shell-{secrets.token_hex(4)}'
session = secrets.token_urlsafe(24)
master, terminal = os.openpty()
try:
process = subprocess.Popen(shell_command(account, unit, session), stdin=terminal, stdout=terminal, stderr=terminal,
env={'PATH': '/usr/sbin:/usr/bin:/sbin:/bin'})
except OSError as exc:
os.close(master)
return refuse(500, f'the shell cannot be started: {exc}')
finally:
os.close(terminal)
accept = base64.b64encode(hashlib.sha1((key + WEBSOCKET_GUID).encode()).digest()).decode()
handler.close_connection = True
try:
handler.send_response(101, 'Switching Protocols')
handler.send_header('Upgrade', 'websocket')
handler.send_header('Connection', 'Upgrade')
handler.send_header('Sec-WebSocket-Accept', accept)
handler.end_headers()
except ConnectionError:
# The client went away before the shell opened; the shell started for it is stopped.
run('systemctl', 'stop', unit)
os.close(master)
process.kill()
process.wait()
log(f'shell of {name} from {handler.client()} ended before it opened: the client closed the connection')
return
client, started, idle = handler.client(), time.monotonic(), options['shell_idle_minutes'] * 60
log(f'shell of {name} from {client} opened as {account.pw_name} in {unit}')
lock, ended, typed = threading.Lock(), threading.Event(), [started]
def send(opcode, payload):
with lock:
handler.wfile.write(frame(opcode, payload))
with _shell_lock:
_shell_sessions[session] = {'user': name, 'account': account.pw_name, 'uid': account.pw_uid, 'send': send,
'files': {}}
def receive():
"""Pass the client's keystrokes and window sizes to the terminal until it closes the connection."""
try:
while True:
opcode, payload = message(handler, send)
if opcode == CLOSE:
break
if opcode == BINARY:
os.write(master, payload)
typed[0] = time.monotonic()
with _shell_lock:
_shell_used[name] = typed[0]
elif opcode == TEXT:
size = json.loads(payload)
fcntl.ioctl(master, termios.TIOCSWINSZ, struct.pack('HHHH', int(size['rows']), int(size['cols']), 0, 0))
except (OSError, ValueError, KeyError, TypeError, struct.error):
pass
finally:
ended.set()
reader = threading.Thread(target=receive, name=f'{unit} input', daemon=True)
reader.start()
reason = 'the shell ended'
checked = started
try:
while not ended.is_set():
left = typed[0] + idle - time.monotonic()
if left <= 0:
reason = f'no keystroke for {options["shell_idle_minutes"]} minutes'
break
# The user's console login and TOTP token are checked again every SHELL_CHECK_SECONDS.
if time.monotonic() - checked >= SHELL_CHECK_SECONDS:
checked = time.monotonic()
try:
refusal = console_account(name)[1]
except (linuxmanager_users.Failure, ldap.LDAPError, OSError) as exc:
refusal = f'the user directory failed: {exc}'
if refusal:
reason = refusal
break
if select.select([master], [], [], min(left, 1))[0]:
try:
output = os.read(master, 65536)
except OSError:
output = b''
if not output:
break
send(BINARY, output)
else:
reason = 'the client closed the connection'
except OSError:
reason = 'the connection failed'
with _shell_lock:
_shell_sessions.pop(session, None)
run('systemctl', 'stop', unit)
try:
send(CLOSE, (1000).to_bytes(2, 'big') + reason.encode()[:120])
except OSError:
pass
try:
handler.connection.shutdown(socket.SHUT_RDWR)
except OSError:
pass
reader.join()
os.close(master)
try:
process.wait(timeout=10)
except subprocess.TimeoutExpired:
process.kill()
process.wait()
log(f'shell of {name} from {client} in {unit} ended after {int(time.monotonic() - started)} seconds: {reason}')
@@ -0,0 +1,153 @@
"""/sites: the settings of the web sites Caddy serves, the sites, and the files of each (site_files)."""
import mimetypes
import os
import re
import shutil
import urllib.parse
import ldap
import linuxmanager
import linuxmanager_sites
from linuxmanager_status import run
from .common import MIB, SITE_STATUS, section_change
#: A path of a site's files: the site and the path in its folder.
SITE_FILES = re.compile(r'^/sites/([^/]+)/files(?:/(.*))?$')
def sites(method, path, query, body):
"""Answer a /sites request but one of a site's files with linuxmanager_sites: the HTTP status and the JSON document.
GET /sites {"settings", "sites"}: the sites section of the configuration and the sites, each with its
settings but private_key, with private_key_set, modified and its folder
PUT /sites change any of the settings of the sites section
POST /sites add a site: {"name", and any of "enabled", "hosts", "port", "listen", "tls", "ca",
"certificate", "private_key", "index_file", "spa", "editors", "dns",
"dns_address"}; an existing name is 409
GET /sites/<name> the site
PUT /sites/<name> change any of the settings of POST but name; an empty or absent private_key keeps the key
DELETE /sites/<name> remove the site, with content=true also its folder
A change is applied by restarting linux-manager-setup without waiting for it.
"""
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
return 400, {'error': 'the body must be a JSON object'}
body = body or {}
parts = path.strip('/').split('/')
route = (method, len(parts))
try:
cfg = linuxmanager.load_config()
if route == ('GET', 1):
return 200, {'settings': cfg['sites'], 'sites': linuxmanager_sites.listing()}
if route == ('PUT', 1):
changed = section_change('sites', body)
return changed if isinstance(changed, tuple) else (200, {'settings': changed})
if route == ('POST', 1):
name = body.get('name')
document = linuxmanager_sites.save(cfg, name, {key: value for key, value in body.items() if key != 'name'},
True)
code = 201
elif route == ('GET', 2):
with linuxmanager_sites.database() as connection:
return 200, linuxmanager_sites.shown(linuxmanager_sites.get(connection, parts[1]))
elif route == ('PUT', 2):
document, code = linuxmanager_sites.save(cfg, parts[1], body, False), 200
elif route == ('DELETE', 2):
document = linuxmanager_sites.remove(parts[1], query.get('content', [''])[0] == 'true')
code = 200
else:
return 404, {'error': f'no route {method} {path}'}
except linuxmanager_sites.Failure as failure:
return SITE_STATUS.get(failure.status, 500), {'error': str(failure)}
except linuxmanager_sites.DatabaseError as exc:
return 500, {'error': str(exc)}
except (ldap.LDAPError, OSError) as exc:
return 500, {'error': f'the site cannot be changed: {exc}'}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
run('systemctl', 'restart', '--no-block', 'linux-manager-setup.service')
return code, document
def site_file_action(name, target, body):
"""Answer a POST to a site's file or folder: {"folder": true} creates the folder, {"rename": "<path>"} moves the file
or folder to the path in the site's folder; the HTTP status and the JSON document."""
if not isinstance(body, dict):
return 400, {'error': 'the body must be a JSON object'}
try:
if body.get('folder') is True:
linuxmanager_sites.make_folder(target)
return 201, {'path': linuxmanager_sites.relative_of(name, target), 'folder': True}
if isinstance(body.get('rename'), str):
moved = linuxmanager_sites.rename(name, target, body['rename'])
return 200, {'path': linuxmanager_sites.relative_of(name, moved)}
except linuxmanager_sites.Failure as failure:
return SITE_STATUS.get(failure.status, 500), {'error': str(failure)}
except OSError as exc:
return 500, {'error': f'{exc.filename or target.name}: {exc.strerror}'}
return 400, {'error': 'give {"folder": true} or {"rename": "<path>"}'}
def site_files(handler, name, relative, query, payload, send_body):
"""Answer a request of a site's files the Handler handler received: the HTTP status and the JSON document, or None
once a file was sent.
GET /sites/<name>/files/<path> a folder: {"path", "entries": [{name, folder, size, modified}]}; a file:
the file itself, as an attachment
PUT /sites/<name>/files/<path> write the body to the file, its folders created; with unzip=true the body
is a zip archive extracted into the folder of the path: {"path", "files"}
POST /sites/<name>/files/<path> {"folder": true} creates the folder, {"rename": "<path>"} moves it
DELETE /sites/<name>/files/<path> remove the file or folder
The body of a PUT needs a Content-Length of at most sites.upload_max_mb MiB.
"""
try:
target = linuxmanager_sites.resolve(name, urllib.parse.unquote(relative))
if handler.command in ('GET', 'HEAD'):
if target.is_dir():
return 200, linuxmanager_sites.entries(name, target)
if not target.is_file():
return 404, {'error': f'{relative} does not exist'}
with open(target, 'rb') as handle:
size = os.fstat(handle.fileno()).st_size
try:
handler.send_response(200)
handler.send_header('Content-Type', mimetypes.guess_type(target.name)[0] or 'application/octet-stream')
handler.send_header('Content-Length', str(size))
handler.send_header('Content-Disposition',
f"attachment; filename*=UTF-8''{urllib.parse.quote(target.name)}")
handler.send_header('Cache-Control', 'no-store')
handler.end_headers()
if send_body:
shutil.copyfileobj(handle, handler.wfile, linuxmanager_sites.CHUNK)
except ConnectionError:
# The client went away before the file was sent.
handler.close_connection = True
return None
if handler.command == 'PUT':
length = handler.headers.get('Content-Length')
if not length or not length.isdigit():
return 411, {'error': 'the body needs a Content-Length'}
limit = linuxmanager.load_config()['sites']['upload_max_mb'] * MIB
if query.get('unzip', [''])[0] == 'true':
count = linuxmanager_sites.extract(handler.rfile, int(length), target, limit)
return 200, {'path': linuxmanager_sites.relative_of(name, target), 'files': count}
linuxmanager_sites.receive(handler.rfile, int(length), target, limit)
return 200, {'path': linuxmanager_sites.relative_of(name, target), 'files': 1}
if handler.command == 'POST':
return site_file_action(name, target, payload)
if handler.command == 'DELETE':
if not target.exists() and not target.is_symlink():
return 404, {'error': f'{relative} does not exist'}
linuxmanager_sites.delete(name, target)
return 200, {'path': linuxmanager_sites.relative_of(name, target), 'removed': True}
except linuxmanager_sites.Failure as failure:
return SITE_STATUS.get(failure.status, 500), {'error': str(failure)}
except OSError as exc:
return 500, {'error': f'{exc.filename or relative}: {exc.strerror}'}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
return 405, {'error': f'{handler.command} is not allowed on a site\'s files'}
@@ -0,0 +1,344 @@
"""The status of the server: / for people, /status, /status/full, /config and /postgresql, with the cache of the
status that the requests of CACHE_SECONDS share (cached)."""
import json
import threading
import time
from html import escape
import linuxmanager
from linuxmanager_status import established, established_each, openvpn_clients, run, swanctl, wireguard_peers
from .common import current, unit_state
#: Seconds a collected status is answered again.
CACHE_SECONDS = 2
TAILWIND = 'https://cdn.jsdelivr.net/npm/@tailwindcss/browser@4'
#: Seconds after which the status page in a browser reloads itself.
REFRESH_SECONDS = 60
#: The PostgreSQL settings /config shows for each cluster.
POSTGRESQL_SETTINGS = ('server_version', 'listen_addresses', 'port', 'max_connections', 'shared_buffers',
'effective_cache_size', 'work_mem', 'password_encryption', 'ssl', 'timezone')
_cache = {'at': 0.0, 'data': None}
_lock = threading.Lock()
def package_version(package='linux-manager'):
"""The installed version of the package, or None when it is not installed."""
result = run('dpkg-query', '-W', '-f=${Version}', package)
return (result.stdout.strip() or None) if result and result.returncode == 0 else None
def docker_status():
"""Whether Docker is installed, and the state of its service and of the Docker rules when it is."""
result = run('systemctl', 'list-unit-files', '--no-legend', 'docker.service')
if result is None or result.returncode != 0 or not result.stdout.strip():
return {'installed': False}
return {
'installed': True,
'service': unit_state('docker.service'),
'rules': unit_state('linux-manager-docker.service'),
}
def postgresql_clusters():
"""The state of each PostgreSQL cluster unit (postgresql@<version>-<name>), or {'postgresql': 'no cluster'} without one.
postgresql.service, which only runs /bin/true, is not reported.
"""
lines = lines_of('systemctl', 'list-units', 'postgresql@*.service', '--all', '--plain', '--no-legend') or []
names = [line.split()[0].removesuffix('.service') for line in lines if line.strip()]
return {name: unit_state(f'{name}.service') for name in names} or {'postgresql': 'no cluster'}
def collect(state):
"""The status document. state says which features the server has."""
units = {'linux-manager-setup': unit_state('linux-manager-setup.service'), 'slapd': unit_state('slapd.service'),
'linux-manager-ui': unit_state('linux-manager-ui.service'), **postgresql_clusters()}
if state.get('site_to_site') or state.get('p2s'):
units['strongswan'] = unit_state('strongswan.service')
if state.get('wireguard'):
units['wg-quick@wg0'] = unit_state('wg-quick@wg0.service')
if state.get('openvpn'):
units['openvpn-server@linux-manager'] = unit_state('openvpn-server@linux-manager.service')
status = {
'status': 'ok' if all(value == 'active' for value in units.values()) else 'degraded',
'version': package_version(),
'time': time.strftime('%Y-%m-%dT%H:%M:%S%z'),
'units': units,
'docker': docker_status(),
}
names = state.get('site_to_site')
if isinstance(names, list) and names:
counts = established_each(names).values()
status['site_to_site'] = {'connections': len(names),
'established': None if None in counts else sum(1 for count in counts if count)}
if state.get('p2s'):
status['p2s'] = {'clients': established('road-warrior')}
if state.get('wireguard'):
status['wireguard'] = {'peers': wireguard_peers()}
if state.get('openvpn'):
status['openvpn'] = {'clients': openvpn_clients()}
return status
def shown(status, content, client):
"""The status document as web.status_content shows it to the client at its address.
Every content has status, time and client_ip, the client's address, and content, which one it is: basic nothing
more; standard adds version and services, the number of active units and of units in every
other state; full adds the whole status document of collect. Content that is not one of
linuxmanager.STATUS_CONTENTS, or not set, is basic.
"""
if content not in linuxmanager.STATUS_CONTENTS:
content = 'basic'
if content == 'full':
return {**status, 'content': content, 'client_ip': client}
document = {'status': status['status'], 'content': content, 'time': status['time'], 'client_ip': client}
if content == 'standard':
active = sum(1 for value in status['units'].values() if value == 'active')
document.update(version=status['version'],
services={'active': active, 'failed': len(status['units']) - active})
return document
def lines_of(*args):
"""The output lines of a command, or None when it fails."""
result = run(*args)
return result.stdout.splitlines() if result and result.returncode == 0 else None
def routes():
"""The IPv4 routes as ip -j prints them, or None when ip fails."""
result = run('ip', '-j', '-4', 'route', 'show')
if result is None or result.returncode != 0:
return None
try:
return json.loads(result.stdout)
except ValueError:
return None
def interfaces():
"""The network interfaces but loopback, each {name, state, mac, ipv4, ipv6} from ip -j addr show, with its IPv4
and its global IPv6 addresses as address/prefix, or None when ip fails."""
result = run('ip', '-j', 'addr', 'show')
if result is None or result.returncode != 0:
return None
try:
links = json.loads(result.stdout)
except ValueError:
return None
def addresses(link, family):
return [f'{entry["local"]}/{entry["prefixlen"]}' for entry in link.get('addr_info', [])
if entry.get('family') == family and 'local' in entry
and (family == 'inet' or entry.get('scope') == 'global')]
return [{
'name': link['ifname'],
'state': link.get('operstate'),
'mac': link.get('address'),
'ipv4': addresses(link, 'inet'),
'ipv6': addresses(link, 'inet6'),
} for link in links if link.get('link_type') != 'loopback' and 'ifname' in link]
def wireguard_details():
"""The interface and the peers of wg0, with endpoints and allowed IPs; no private or preshared key."""
result = run('wg', 'show', 'wg0', 'dump')
if result is None or result.returncode != 0:
return None
rows = [line.split('\t') for line in result.stdout.splitlines()]
now = int(time.time())
return {
'public_key': rows[0][1] if rows and len(rows[0]) > 2 else None,
'listen_port': int(rows[0][2]) if rows and len(rows[0]) > 2 else None,
'peers': [{
'public_key': row[0],
'endpoint': None if row[2] == '(none)' else row[2],
'allowed_ips': row[3].split(','),
'handshake_seconds_ago': now - int(row[4]) if int(row[4]) else None,
'received_bytes': int(row[5]),
'sent_bytes': int(row[6]),
} for row in rows[1:] if len(row) >= 7],
}
def postgresql_query(cluster, sql):
"""The JSON value an SQL statement answers on a cluster (version/name), run by psql as postgres, or None when it fails."""
result = run('runuser', '-u', 'postgres', '--', 'psql', '--cluster', cluster, '-X', '-q', '-t', '-A',
'-v', 'ON_ERROR_STOP=1', '-d', 'postgres', '-c', sql)
if result is None or result.returncode != 0:
return None
try:
return json.loads(result.stdout)
except ValueError:
return None
def postgresql_clusters_of():
"""The PostgreSQL clusters of pg_lsclusters, each with version, name, port, status and data_directory, or None when it fails."""
lines = lines_of('pg_lsclusters', '--no-header')
if lines is None:
return None
clusters = []
for line in lines:
fields = line.split()
if len(fields) >= 6:
clusters.append({'version': fields[0], 'name': fields[1], 'port': int(fields[2]) if fields[2].isdigit() else None,
'status': fields[3], 'data_directory': fields[5]})
return clusters
def postgresql_configuration():
"""The PostgreSQL clusters on the server with the settings of POSTGRESQL_SETTINGS of each one that is online (None
for one that is not), or None when pg_lsclusters fails."""
clusters = postgresql_clusters_of()
if clusters is None:
return None
names = ', '.join(f"'{name}'" for name in POSTGRESQL_SETTINGS)
for cluster in clusters:
cluster['settings'] = postgresql_query(
f"{cluster['version']}/{cluster['name']}",
f'SELECT json_object_agg(name, current_setting(name) ORDER BY name) FROM pg_settings WHERE name IN ({names})',
) if cluster['status'].startswith('online') else None
return {'clusters': clusters}
def postgresql_databases():
"""What /postgresql shows: the PostgreSQL clusters on the server, each with its databases (name, owner, encoding and
size in bytes) and their total size, None for a cluster that is not online; clusters is None when pg_lsclusters fails."""
clusters = postgresql_clusters_of()
if clusters is None:
return {'clusters': None}
for cluster in clusters:
databases = postgresql_query(
f"{cluster['version']}/{cluster['name']}",
"SELECT coalesce(json_agg(json_build_object('name', datname, 'owner', pg_get_userbyid(datdba), "
"'encoding', pg_encoding_to_char(encoding), 'size', pg_database_size(oid)) ORDER BY datname), '[]') "
'FROM pg_database',
) if cluster['status'].startswith('online') else None
cluster['databases'] = databases
cluster['size'] = sum(database['size'] for database in databases) if isinstance(databases, list) else None
return {'clusters': clusters}
def details(state):
"""What /config shows: the configuration without its secrets, and what the server resolved and runs; caddy is the
installed Caddy, {version, source, service}, service the state of caddy.service, null when it is not installed; oidc
is {enabled, issuer} of the OpenID Provider, issuer null while it is disabled."""
configuration = state.get('configuration', {})
caddy = package_version('caddy')
oidc = configuration.get('oidc', {}).get('enabled') is True
return {
'configuration': configuration,
'network': dict(state.get('network', {}), interfaces=interfaces(), routes=routes()),
'ipsec': {
'connections': swanctl('--list-conns'),
'sas': swanctl('--list-sas'),
},
'wireguard': wireguard_details(),
'docker': {
'exposed_ports': configuration.get('docker', {}).get('exposed_ports', []),
'rules': lines_of('iptables', '-S', 'LINUX-MANAGER-DOCKER'),
},
'postgresql': postgresql_configuration(),
# Caddy is upstream's once its repository is configured (linux-manager-caddy).
'caddy': {'version': caddy, 'source': 'upstream' if linuxmanager.CADDY_APT_SOURCE.is_file() else 'ubuntu',
'service': unit_state('caddy.service')} if caddy else None,
'oidc': {'enabled': oidc, 'issuer': linuxmanager.oidc_issuer(configuration) if oidc else None},
}
def text_of(status):
"""The status document of shown for people, as plain text: a header, a blank line, then name: value lines."""
lines = ['Linux Manager', f'Status: {status["status"]}']
if 'version' in status:
lines.append(f'Version: {status["version"] or "unknown"}')
lines += ['', f'Client IP: {status["client_ip"]}']
if 'services' in status:
lines.append(f'Services: {status["services"]["active"]} active, {status["services"]["failed"]} failed')
lines += [f'{name}: {value}' for name, value in status.get('units', {}).items()]
if 'docker' in status:
docker = status['docker']
lines.append('Docker: not installed' if not docker['installed']
else f'Docker: installed, service {docker["service"]}, rules {docker["rules"]}')
if 'site_to_site' in status:
established, connections = status['site_to_site']['established'], status['site_to_site']['connections']
lines.append(f'Site to site: {"unknown" if established is None else established} of {connections} established')
if 'p2s' in status:
clients = status['p2s']['clients']
lines.append(f'IKEv2: {"unknown" if clients is None else f"{clients} connected"}')
if 'openvpn' in status:
clients = status['openvpn']['clients']
lines.append(f'OpenVPN: {"unknown" if clients is None else f"{clients} connected"}')
if 'wireguard' in status:
peers = status['wireguard']['peers']
if peers is None:
lines.append('WireGuard: down')
else:
lines.append(f'WireGuard: {len(peers)} peers')
for number, peer in enumerate(peers, 1):
age = peer['handshake_seconds_ago']
lines.append(f' peer {number}: {"no handshake yet" if age is None else f"handshake {age} s ago"}, '
f'received {peer["received_bytes"]} B, sent {peer["sent_bytes"]} B')
return '\n'.join(lines) + '\n'
def html_of(status):
"""The status document of shown for a browser, as a page styled with Tailwind CSS."""
colour = 'bg-emerald-600 dark:bg-emerald-700' if status['status'] == 'ok' else 'bg-amber-600 dark:bg-amber-700'
version = f', version {status["version"] or "unknown"}' if 'version' in status else ''
# Each "name: value" line after the header of the text is a row; indented lines, such as WireGuard peers, are indented here too.
rows = []
for line in text_of(status).split('\n\n', 1)[1].splitlines():
if not line:
continue
name, _, value = line.partition(': ')
indent = ' pl-8' if line.startswith(' ') else ''
rows.append(f'<tr class="border-t border-slate-200 first:border-t-0 dark:border-slate-700">'
f'<th class="py-2 pr-4{indent} text-left font-medium text-slate-600 dark:text-slate-300">{escape(name.strip())}</th>'
f'<td class="py-2">{escape(value)}</td></tr>')
body = ''.join(rows)
return f"""<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="color-scheme" content="light dark">
<meta http-equiv="refresh" content="{REFRESH_SECONDS}">
<title>Linux Manager</title>
<script src="{TAILWIND}"></script>
</head>
<body class="min-h-screen bg-slate-100 text-slate-900 dark:bg-slate-900 dark:text-slate-100">
<div class="mx-auto max-w-2xl px-4 py-10">
<div class="overflow-hidden rounded-2xl border border-slate-200 bg-white shadow-lg dark:border-slate-700 dark:bg-slate-800">
<div class="{colour} px-6 py-4 text-white">
<h1 class="text-xl font-semibold">Linux Manager</h1>
<p class="mt-1 text-sm">Status: {escape(status['status'] + version)}</p>
</div>
<div class="px-6 py-4">
<table class="w-full text-sm"><tbody>{body}</tbody></table>
</div>
<p class="px-6 pb-6 text-xs text-slate-500 dark:text-slate-400">{escape(status['time'])} - <a class="underline" href="/status">JSON</a></p>
</div>
</div>
</body>
</html>
"""
def cached():
"""The status of collect, collected again when it is older than CACHE_SECONDS."""
with _lock:
if _cache['data'] is None or time.monotonic() - _cache['at'] > CACHE_SECONDS:
_cache['data'] = collect(current())
_cache['at'] = time.monotonic()
return _cache['data']
+903
View File
@@ -0,0 +1,903 @@
#!/usr/bin/python3
"""Serve the Manager: the web application and its API, which passes calls to the server.
Run by linux-manager-ui.service. It uses python3-requests to call the server and
python3-psycopg to keep the sessions; its settings and its listening loop are in
linuxmanager_ui.
Pages:
/api/info the version of this package and the server it manages, as JSON
/api/session POST signs in through the server's /auth with {"name", "password", "code"?},
GET answers the signed-in user, DELETE signs out
/api/session/passkey
POST begins a sign-in with a passkey at the server's /auth/passkey;
/api/session/passkey/complete POST signs in through /auth/passkey/complete
/api/updates GET the installed and the candidate version of linux-manager
from apt (see updates()), POST updates apt's package lists (see start_check());
needs a session, POST a member of admins
/api/upgrade GET the state and the output of the upgrade of that package (see upgrade_state()),
POST starts it (see start_upgrade()); needs a session, POST a member of admins
/api/system-updates
GET the packages of the system apt-get upgrade would upgrade, with their installed
and candidate versions (see system_updates()); needs a session
/api/system-upgrade
GET the state and the output of the upgrade of the system's packages, POST starts
it (see start_upgrade()); needs a session, POST a member of admins
/api/reboot POST reboots the system (see start_reboot()); needs a member of admins
/api/shell a WebSocket handshake goes to the server's /shell, and the WebSocket is relayed
(see Handler.relay()); POST passes as every other path does
/api/<path> the server's web service at /<path>, with the access token of the session;
the method, the body and the answer pass unchanged; the files of a web site,
/api/sites/<name>/files/..., pass streamed for GET, HEAD and PUT (see Handler.transfer())
/docs/<file> a file of the help in DOCS_DIR, Markdown, SVG or PNG (see doc_file()); needs a session
/<file> the web application from WEB_DIR; a path that names no file gives index.html
A session is a row of the table sessions in PostgreSQL, found by the SHA-256 of a random id in
the HttpOnly cookie COOKIE; it holds the user's access and refresh tokens and ends with the
refresh token. An access token that expires within REFRESH_MARGIN seconds is renewed through
the server's /auth/refresh before a call; a refused renewal ends the session. /api/<path>
without a session answers 401. Sign-in calls carry the browser's address in X-Forwarded-For,
which the server reads for its rate limit when the manager is one of its trusted proxies.
Without router_url the API answers 503.
The manager first initializes, without listening: on the server, unless the settings name a
database, it creates its role, password and database when they are missing; then it connects to
the sessions database and creates its table, retrying every INIT_RETRY_SECONDS until it succeeds
and logging each failure. Then it listens. Its unit starts it only once wait-for-libpq has found
libpq in the linker cache, where psycopg looks it up.
The service needs nothing else to start and does not end on its own. Every DISCOVERY_SECONDS it
works out where to listen, bind or the interfaces listen names, and runs a server at each address
(linuxmanager_ui.serve). Every request
reads the settings when it is made.
"""
import argparse
import hashlib
import http.cookies
import json
import mimetypes
import os
import re
import secrets
import signal
import socket
import ssl
import subprocess
import sys
import threading
import time
import urllib.parse
from http.server import BaseHTTPRequestHandler
sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
import psycopg
import requests
import urllib3
import linuxmanager_ui
from linuxmanager_ui import DATABASE, DATABASE_SECRET, ROUTER_CONFIG_FILE
#: The address a proxy on this host calls from.
LOOPBACK = '127.0.0.1'
CONFIG_FILE = linuxmanager_ui.CONFIG_FILE
WEB_DIR = '/usr/share/linux-manager/manager/web'
DOCS_DIR = '/usr/share/linux-manager/manager/docs'
#: The files of DOCS_DIR /docs serves, by extension, and their content types.
DOC_TYPES = {'.md': 'text/markdown; charset=utf-8', '.svg': 'image/svg+xml', '.png': 'image/png'}
#: The headers of every /docs answer: no type guessing, and an SVG opened on its own runs no script.
DOC_HEADERS = (('X-Content-Type-Options', 'nosniff'), ('Content-Security-Policy', "default-src 'none'; style-src 'unsafe-inline'"))
#: The cookie that holds the session id.
COOKIE = 'linux_manager_ui_session'
#: Seconds before its expiry an access token is renewed.
REFRESH_MARGIN = 30
#: Seconds between two attempts of the initialization.
INIT_RETRY_SECONDS = 15
#: The packages /api/updates reports.
PACKAGES = ('linux-manager',)
#: Touched by apt at every successful update of its package lists.
APT_UPDATED = '/var/lib/apt/periodic/update-success-stamp'
#: The transient units of the upgrade, of the reboot and of the update of the package lists, outside the manager's unit.
UPGRADE_UNIT = 'linux-manager-ui-upgrade'
SYSTEM_UPGRADE_UNIT = 'linux-manager-ui-system-upgrade'
REBOOT_UNIT = 'linux-manager-ui-reboot'
CHECK_UNIT = 'linux-manager-ui-check'
#: Seconds the update of the package lists waits for apt's lock, which Ubuntu's apt-daily or unattended-upgrades may hold.
CHECK_LOCK_SECONDS = 300
#: The upgrade helper, and the state and the output it records of the server's package and of the system's.
UPGRADE = '/usr/lib/linux-manager/upgrade'
UPGRADE_STATE = '/var/lib/linux-manager/upgrade.json'
UPGRADE_LOG = '/var/lib/linux-manager/upgrade.log'
SYSTEM_UPGRADE_STATE = '/var/lib/linux-manager/system-upgrade.json'
SYSTEM_UPGRADE_LOG = '/var/lib/linux-manager/system-upgrade.log'
#: A package apt-get -s upgrade would upgrade: Inst <name> [<installed>] (<candidate> <release> [<architecture>]).
SIMULATED = re.compile(r'^Inst (\S+) (?:\[([^\]]*)\] )?\((\S+)')
#: The last lines of the upgrade's output /api/upgrade answers.
LOG_LINES = 200
#: Written by packages that need a reboot, with the packages in REBOOT_REQUIRED.pkgs.
REBOOT_REQUIRED = '/var/run/reboot-required'
#: Seconds between the answer of /api/reboot and the reboot.
REBOOT_DELAY = 5
SCHEMA = ('CREATE TABLE IF NOT EXISTS sessions (id_hash text PRIMARY KEY, name text NOT NULL, access_token text NOT NULL, '
'access_expires timestamptz NOT NULL, refresh_token text NOT NULL, expires timestamptz NOT NULL)')
#: Seconds between two discoveries of where the service should listen.
DISCOVERY_SECONDS = 30
#: Seconds a call to the server may take; adding a user creates an RSA key.
ROUTER_TIMEOUT = 130
#: The largest request body passed to the server, in bytes; a CSR or a certificate chain in PEM is a few kilobytes.
BODY_LIMIT = 65536
#: The largest body of a save of the Shell page's editor, /api/shell/editor/...: a file of up to 1 MiB, as the server
#: takes, escaped in JSON.
EDITOR_BODY_LIMIT = 2 * 1024 * 1024 + BODY_LIMIT
#: The paths of a web site's files, whose GET, HEAD and PUT pass streamed (see Handler.transfer()).
SITE_FILES = re.compile(r'^/api/sites/[^/]+/files(/.*)?$')
def settings():
return linuxmanager_ui.settings(CONFIG_FILE)
def package_version():
try:
result = subprocess.run(['dpkg-query', '-W', '-f=${Version}', 'linux-manager'],
capture_output=True, text=True, timeout=5)
except (OSError, subprocess.TimeoutExpired):
return None
return result.stdout.strip() if result.returncode == 0 else None
def updates():
"""The installed and the candidate version of each of PACKAGES, from apt-cache policy, and whether an update is available.
The candidate is the version apt would install from the sources it is configured with, as of
its last update of the package lists, whose time checked gives (APT_UPDATED, None when it is
not known); a version apt does not know is None. checking is whether start_check's update of
the package lists runs. Where apt cannot be asked, available is False and the packages are empty.
"""
checking = unit_active(CHECK_UNIT)
try:
result = subprocess.run(['apt-cache', 'policy', *PACKAGES], capture_output=True, text=True, timeout=30)
except (OSError, subprocess.TimeoutExpired):
return {'available': False, 'checked': None, 'checking': checking, 'packages': []}
try:
checked = time.strftime('%Y-%m-%dT%H:%M:%S%z', time.localtime(os.path.getmtime(APT_UPDATED)))
except OSError:
checked = None
found = {}
package = None
for line in result.stdout.splitlines():
if line and not line[0].isspace() and line.endswith(':'):
package = found.setdefault(line[:-1], {'name': line[:-1], 'installed': None, 'candidate': None})
elif package is not None and line.strip().startswith(('Installed:', 'Candidate:')):
key, _, value = line.strip().partition(': ')
package[key.lower()] = None if value == '(none)' else value
packages = [dict(found.get(name, {'name': name, 'installed': None, 'candidate': None})) for name in PACKAGES]
for package in packages:
package['update'] = package['installed'] is not None and package['candidate'] not in (None, package['installed'])
return {'available': True, 'checked': checked, 'checking': checking, 'packages': packages}
def system_updates():
"""The packages of the system apt-get upgrade would upgrade, from its simulation (apt-get -s), as updates() answers
its packages, each with update true, sorted by name; the other keys are those of updates().
The simulation takes no lock and changes nothing; it answers what the package lists of apt's last update give.
"""
checking = unit_active(CHECK_UNIT)
try:
result = subprocess.run(['apt-get', '-s', '-q', 'upgrade'], capture_output=True, text=True, timeout=60,
env=dict(os.environ, LC_ALL='C'))
except (OSError, subprocess.TimeoutExpired):
return {'available': False, 'checked': None, 'checking': checking, 'packages': []}
if result.returncode != 0:
return {'available': False, 'checked': None, 'checking': checking, 'packages': []}
try:
checked = time.strftime('%Y-%m-%dT%H:%M:%S%z', time.localtime(os.path.getmtime(APT_UPDATED)))
except OSError:
checked = None
found = {}
for line in result.stdout.splitlines():
match = SIMULATED.match(line)
if match:
found[match[1]] = {'name': match[1], 'installed': match[2], 'candidate': match[3], 'update': True}
return {'available': True, 'checked': checked, 'checking': checking, 'packages': [found[name] for name in sorted(found)]}
def unit_active(unit):
"""Whether a unit of systemd is active or activating; False where systemctl cannot be asked."""
try:
result = subprocess.run(['systemctl', 'is-active', f'{unit}.service', f'{unit}.timer'],
capture_output=True, text=True, timeout=5)
except (OSError, subprocess.TimeoutExpired):
return False
return any(line.strip() in ('active', 'activating') for line in result.stdout.splitlines())
def upgrade_state(system=False):
"""The state of the upgrade of the server's package, as GET /api/upgrade answers it, or with system of the system's
packages, as GET /api/system-upgrade answers it.
state is idle before the first upgrade, running, succeeded or failed as the helper recorded it, and interrupted when
it recorded running but its unit is not active, as after a reboot during the upgrade; the other keys of the helper's
state follow, and log holds the last LOG_LINES lines of its output. reboot_required and reboot_packages tell whether
a package asks for a reboot, and rebooting whether one is about to happen.
"""
state_file, log_file = (SYSTEM_UPGRADE_STATE, SYSTEM_UPGRADE_LOG) if system else (UPGRADE_STATE, UPGRADE_LOG)
try:
with open(state_file, encoding='utf-8') as handle:
recorded = json.load(handle)
except (OSError, ValueError):
recorded = {}
if not isinstance(recorded, dict):
recorded = {}
running = unit_active(SYSTEM_UPGRADE_UNIT if system else UPGRADE_UNIT)
state = recorded.get('state', 'idle')
if running:
state = 'running'
elif state == 'running':
state = 'interrupted'
try:
with open(log_file, encoding='utf-8', errors='replace') as handle:
log = handle.read().splitlines()[-LOG_LINES:]
except OSError:
log = []
try:
with open(f'{REBOOT_REQUIRED}.pkgs', encoding='utf-8') as handle:
reboot_packages = sorted({line.strip() for line in handle if line.strip()})
except OSError:
reboot_packages = []
return {
'state': state,
'user': recorded.get('user'),
'started': recorded.get('started'),
'finished': recorded.get('finished'),
'exit_status': recorded.get('exit_status'),
'before': recorded.get('before'),
'after': recorded.get('after'),
'log': log,
'reboot_required': os.path.exists(REBOOT_REQUIRED),
'reboot_packages': reboot_packages,
'rebooting': unit_active(REBOOT_UNIT),
}
def transient(*args):
"""Start a transient unit with systemd-run and the arguments; the HTTP status and the message of a refusal, or None."""
try:
result = subprocess.run(['systemd-run', '--quiet', '--collect', *args], capture_output=True, text=True, timeout=15)
except (OSError, subprocess.TimeoutExpired) as exc:
return 503, f'systemd-run cannot be run here: {exc}'
if result.returncode != 0:
message = result.stderr.strip() or f'systemd-run ended with status {result.returncode}'
return (409 if 'already' in message.lower() else 500), message
return None
def busy():
"""Whether an upgrade, of the server's package or of the system's, or a reboot runs."""
return unit_active(UPGRADE_UNIT) or unit_active(SYSTEM_UPGRADE_UNIT) or unit_active(REBOOT_UNIT)
def start_upgrade(name, system=False):
"""Start the upgrade of the server's package in the transient unit UPGRADE_UNIT, or with system of the system's
packages in SYSTEM_UPGRADE_UNIT, outside this service, so the restarts it makes do not end it.
The HTTP status and the JSON document: 202 when it started, 409 while an upgrade or a reboot runs, 503 where apt
cannot be used.
"""
if busy():
return 409, {'error': 'an upgrade or a reboot is already running'}
if not os.path.exists('/usr/bin/apt-get'):
return 503, {'error': 'apt is not available here'}
if system:
refused = transient(f'--unit={SYSTEM_UPGRADE_UNIT}', '--description=Linux Manager system upgrade',
'--property=Type=exec', '--no-block', '/usr/bin/python3', UPGRADE, '--system', name)
else:
refused = transient(f'--unit={UPGRADE_UNIT}', '--description=Linux Manager package upgrade', '--property=Type=exec',
'--no-block', '/usr/bin/python3', UPGRADE, name)
if refused:
return refused[0], {'error': refused[1]}
return 202, {'started': True}
def start_check():
"""Update apt's package lists in the transient unit CHECK_UNIT, as root outside this service's sandbox, and touch
APT_UPDATED when it succeeds, so updates() answers the new candidates and the time of the check.
The HTTP status and the JSON document: 202 when it started, 409 while an update of the lists, an upgrade or a
reboot runs, 503 where apt cannot be used.
"""
if unit_active(CHECK_UNIT) or busy():
return 409, {'error': 'a check for updates, an upgrade or a reboot is already running'}
if not os.path.exists('/usr/bin/apt-get'):
return 503, {'error': 'apt is not available here'}
refused = transient(f'--unit={CHECK_UNIT}', '--description=Linux Manager package lists update', '--property=Type=oneshot',
f'--property=ExecStartPost=/usr/bin/touch {APT_UPDATED}', '--no-block',
'/usr/bin/apt-get', '-q', '-o', f'DPkg::Lock::Timeout={CHECK_LOCK_SECONDS}', 'update')
if refused:
return refused[0], {'error': refused[1]}
return 202, {'checking': True}
def start_reboot():
"""Reboot the system REBOOT_DELAY seconds from now, so the answer reaches the browser first.
The HTTP status and the JSON document: 202 when the reboot is scheduled, 409 while an upgrade or a reboot runs.
"""
if busy():
return 409, {'error': 'an upgrade or a reboot is already running'}
refused = transient(f'--unit={REBOOT_UNIT}', '--description=Linux Manager reboot', f'--on-active={REBOOT_DELAY}',
'/usr/bin/systemctl', 'reboot')
if refused:
return refused[0], {'error': refused[1]}
return 202, {'rebooting': True}
def admin(current, access, client):
"""Whether the signed-in user is a member of the server's admins, as the server's /account answers."""
response = router_call(current, 'GET', '/account', None, {'Authorization': f'Bearer {access}', 'X-Forwarded-For': client})
try:
return response.ok and response.json().get('admin') is True
except ValueError:
return False
def error(code, message):
return code, 'application/json', json.dumps({'error': message}).encode() + b'\n'
class Unavailable(Exception):
"""A setting the API needs is missing, or the server or the database cannot be reached; the HTTP status and message."""
def __init__(self, code, message):
super().__init__(message)
self.code = code
def router_call(current, method, path, body=None, headers=None):
"""The server's answer to a call; Unavailable when it cannot be reached."""
if not current['router_url']:
raise Unavailable(503, f'no router is configured; set router_url in {CONFIG_FILE}')
headers = dict(headers or {}, Accept='application/json')
if body is not None:
headers['Content-Type'] = 'application/json'
try:
return requests.request(method, current['router_url'] + path, data=body, headers=headers,
timeout=ROUTER_TIMEOUT, allow_redirects=False)
except requests.RequestException as failure:
raise Unavailable(502, f'the server at {current["router_url"]} cannot be reached: {failure}')
def answer_of(response):
"""The HTTP status, the content type and the body to pass on for an answer of the server."""
kind = response.headers.get('Content-Type', '')
if response.ok or kind.startswith('application/json'):
return response.status_code, kind or 'application/json', response.content
return error(response.status_code, f'the server answered {response.status_code}: {response.text.strip()}')
def database(current):
"""A connection to the sessions database, with its table; Unavailable when it is not configured or fails."""
if not current['database']:
raise Unavailable(503, f'no session database is configured; set database in {CONFIG_FILE}')
try:
connection = psycopg.connect(current['database'], autocommit=True)
connection.execute(SCHEMA)
except psycopg.Error as failure:
raise Unavailable(503, f'the session database cannot be used: {failure}')
return connection
def psql(sql):
"""The output of an SQL statement, read from standard input, run by psql as postgres with DATABASE as the variable v1."""
result = subprocess.run(['runuser', '-u', 'postgres', '--', 'psql', '-X', '-q', '-t', '-A', '-v', 'ON_ERROR_STOP=1',
'-v', f'v1={DATABASE}'], input=sql, capture_output=True, text=True)
if result.returncode != 0:
raise OSError(f'psql failed: {result.stderr.strip()}')
return result.stdout.strip()
def provision():
"""Create the role, its password file and the database on the server, each when missing.
The password, of URL-safe characters, reaches psql on standard input, not on its command line.
"""
if not psql("SELECT 1 FROM pg_roles WHERE rolname = :'v1'"):
if os.path.exists(DATABASE_SECRET):
os.unlink(DATABASE_SECRET)
psql('CREATE ROLE :"v1" LOGIN')
if not os.path.exists(DATABASE_SECRET):
password = secrets.token_urlsafe(32)
psql(f"ALTER ROLE :\"v1\" PASSWORD '{password}'")
os.makedirs(os.path.dirname(DATABASE_SECRET), exist_ok=True)
descriptor = os.open(DATABASE_SECRET, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)
with os.fdopen(descriptor, 'w', encoding='utf-8') as handle:
handle.write(password + '\n')
if not psql("SELECT 1 FROM pg_database WHERE datname = :'v1'"):
psql('CREATE DATABASE :"v1" OWNER :"v1"')
def initialize():
"""Create the sessions database on the server when it is missing and connect to it, retrying every INIT_RETRY_SECONDS."""
reported = None
while True:
phase = 'reading the settings'
try:
configured = linuxmanager_ui.text_setting(linuxmanager_ui.read_json(CONFIG_FILE), 'database')
if not configured and os.path.exists(ROUTER_CONFIG_FILE):
phase = 'creating the sessions database'
provision()
phase = 'connecting to the sessions database'
database(settings()).close()
except (OSError, Unavailable) as exc:
failure = f'{phase} failed: {exc}'
if failure != reported:
log(f'{failure}; retrying every {INIT_RETRY_SECONDS} seconds')
reported = failure
time.sleep(INIT_RETRY_SECONDS)
continue
log('ready')
return
def digest(session_id):
return hashlib.sha256(session_id.encode()).hexdigest()
def tokens_of(response):
"""The tokens of a 200 answer of /auth or /auth/refresh, or None when it holds none."""
try:
document = response.json()
values = (document['access_token'], int(document['expires_in']), document['refresh_token'],
int(document['refresh_expires_in']))
except (ValueError, KeyError, TypeError):
return None
return values if response.status_code == 200 and all(values) else None
def sign_in(current, path, body, client):
"""Sign in through the server's path, /auth or /auth/passkey/complete: the HTTP status, the content type, the body
and the session id, or None."""
response = router_call(current, 'POST', path, body, {'X-Forwarded-For': client})
tokens = tokens_of(response)
if tokens is None:
return (*answer_of(response), None)
try:
name = response.json()['name']
except (ValueError, KeyError, TypeError):
return (*error(502, 'the server named no user'), None)
session_id = secrets.token_urlsafe(32)
access, access_seconds, refresh, refresh_seconds = tokens
with database(current) as connection:
connection.execute('DELETE FROM sessions WHERE expires < now()')
connection.execute('INSERT INTO sessions VALUES (%s, %s, %s, now() + make_interval(secs => %s), %s, '
'now() + make_interval(secs => %s))',
(digest(session_id), name, access, access_seconds, refresh, refresh_seconds))
return 200, 'application/json', json.dumps({'name': name}).encode() + b'\n', session_id
def session(current, session_id, client):
"""The user name and a current access token of the session, renewed when it expires soon, or None."""
if not session_id:
return None
with database(current) as connection:
with connection.transaction():
row = connection.execute('SELECT name, access_token, access_expires < now() + make_interval(secs => %s), '
'refresh_token FROM sessions WHERE id_hash = %s AND expires > now() FOR UPDATE',
(REFRESH_MARGIN, digest(session_id))).fetchone()
if row is None:
return None
name, access, stale, refresh = row
if not stale:
return name, access
response = router_call(current, 'POST', '/auth/refresh', json.dumps({'refresh_token': refresh}).encode(),
{'X-Forwarded-For': client})
tokens = tokens_of(response)
if tokens is None:
connection.execute('DELETE FROM sessions WHERE id_hash = %s', (digest(session_id),))
return None
access, access_seconds, refresh, refresh_seconds = tokens
connection.execute('UPDATE sessions SET access_token = %s, access_expires = now() + make_interval(secs => %s), '
'refresh_token = %s, expires = now() + make_interval(secs => %s) WHERE id_hash = %s',
(access, access_seconds, refresh, refresh_seconds, digest(session_id)))
return name, access
def sign_out(current, session_id):
"""End the session and revoke its refresh token at the server."""
if not session_id:
return
with database(current) as connection:
row = connection.execute('DELETE FROM sessions WHERE id_hash = %s RETURNING refresh_token',
(digest(session_id),)).fetchone()
if row:
router_call(current, 'POST', '/auth/logout', json.dumps({'refresh_token': row[0]}).encode())
def static(path):
"""A file of the web application; a path that names no file gives index.html, so the application routes it."""
root = os.path.realpath(WEB_DIR)
name = os.path.realpath(os.path.join(root, path.lstrip('/')))
if not name.startswith(root + os.sep) or not os.path.isfile(name):
name = os.path.join(root, 'index.html')
try:
with open(name, 'rb') as handle:
data = handle.read()
except OSError:
return 404, 'text/plain; charset=utf-8', b'Not found\n'
return 200, mimetypes.guess_type(name)[0] or 'application/octet-stream', data
def doc_file(path, root=DOCS_DIR):
"""The file of the help that a /docs/<file> path names and its content type, or None: a file inside root whose
extension DOC_TYPES lists."""
root = os.path.realpath(root)
name = os.path.realpath(os.path.join(root, urllib.parse.unquote(path).lstrip('/')))
kind = DOC_TYPES.get(os.path.splitext(name)[1].lower())
if kind is None or not name.startswith(root + os.sep) or not os.path.isfile(name):
return None
return name, kind
class Stream:
"""A request body of length bytes read from a stream as requests sends it, which takes its length from len()."""
def __init__(self, stream, length):
self.stream = stream
self.length = length
self.left = length
def __len__(self):
return self.length
def read(self, size=-1):
size = self.left if size is None or size < 0 else min(size, self.left)
data = self.stream.read(size) if size else b''
self.left -= len(data)
return data
class Handler(BaseHTTPRequestHandler):
protocol_version = 'HTTP/1.1'
def session_id(self):
"""The session id of the request's cookie, or ''."""
cookies = http.cookies.SimpleCookie()
try:
cookies.load(self.headers.get('Cookie', ''))
except http.cookies.CookieError:
return ''
return cookies[COOKIE].value if COOKIE in cookies else ''
def proxied(self):
"""True for a call from a proxy on this host, such as the server's HTTPS proxy, which names the client."""
return self.client_address[0] == LOOPBACK and 'X-Forwarded-For' in self.headers
def client(self):
"""The browser's address: the last X-Forwarded-For entry of a proxied call, else the peer."""
if self.proxied():
return self.headers['X-Forwarded-For'].split(',')[-1].strip()
return self.client_address[0]
def api(self, path, route, payload):
"""Answer an /api/ request: the HTTP status, the content type, the body and the Set-Cookie value or None."""
current = settings()
client = self.client()
session_id = self.session_id()
if route == '/api/session/passkey' and self.command == 'POST':
return (*answer_of(router_call(current, 'POST', '/auth/passkey', b'{}', {'X-Forwarded-For': client})), None)
if route in ('/api/session', '/api/session/passkey/complete') and self.command == 'POST':
code, kind, body, created = sign_in(current, '/auth' if route == '/api/session' else '/auth/passkey/complete',
payload, client)
if created is None:
return code, kind, body, None
return code, kind, body, f'{COOKIE}={created}; HttpOnly; SameSite=Strict; Path=/'
if route == '/api/session' and self.command == 'DELETE':
sign_out(current, session_id)
return 200, 'application/json', b'{"signed_out": true}\n', f'{COOKIE}=; HttpOnly; SameSite=Strict; Path=/; Max-Age=0'
found = session(current, session_id, client)
if found is None:
return (*error(401, 'not signed in'), None)
name, access = found
if route == '/api/session':
return 200, 'application/json', json.dumps({'name': name}).encode() + b'\n', None
if route == '/api/updates' and self.command in ('GET', 'HEAD'):
return 200, 'application/json', json.dumps(updates()).encode() + b'\n', None
if route == '/api/updates' and self.command == 'POST':
if not admin(current, access, client):
return (*error(403, f'user {name} is not a member of admins'), None)
code, document = start_check()
return code, 'application/json', json.dumps(document).encode() + b'\n', None
if route == '/api/system-updates' and self.command in ('GET', 'HEAD'):
return 200, 'application/json', json.dumps(system_updates()).encode() + b'\n', None
if route in ('/api/upgrade', '/api/system-upgrade') and self.command in ('GET', 'HEAD'):
return 200, 'application/json', json.dumps(upgrade_state(route == '/api/system-upgrade')).encode() + b'\n', None
if route in ('/api/upgrade', '/api/system-upgrade', '/api/reboot') and self.command == 'POST':
if not admin(current, access, client):
return (*error(403, f'user {name} is not a member of admins'), None)
code, document = start_reboot() if route == '/api/reboot' else start_upgrade(name, route == '/api/system-upgrade')
return code, 'application/json', json.dumps(document).encode() + b'\n', None
return (*answer_of(router_call(current, self.command, path[4:], payload,
{'Authorization': f'Bearer {access}', 'X-Forwarded-For': client})), None)
def relay(self, path):
"""Pass the WebSocket handshake of /api/shell to the server's /shell with the session's access token, then
relay the bytes both ways until either side closes; the HTTP status, the content type and the body of a
refusal, or None after a relayed WebSocket.
The handshake must come from a page of this host (Origin), and asks the server for no extension, so the
frames pass unchanged.
"""
current = settings()
client = self.client()
if urllib.parse.urlsplit(self.headers.get('Origin', '')).netloc != self.headers.get('Host', ''):
return error(403, 'the shell must be opened from a page of the Manager')
found = session(current, self.session_id(), client)
if found is None:
return error(401, 'not signed in')
if not current['router_url']:
raise Unavailable(503, f'no router is configured; set router_url in {CONFIG_FILE}')
url = urllib.parse.urlsplit(current['router_url'] + path[4:])
secure = url.scheme == 'https'
request = '\r\n'.join([
f'GET {url.path}?{url.query} HTTP/1.1', f'Host: {url.netloc}', 'Upgrade: websocket', 'Connection: Upgrade',
f'Sec-WebSocket-Key: {self.headers.get("Sec-WebSocket-Key", "")}',
f'Sec-WebSocket-Version: {self.headers.get("Sec-WebSocket-Version", "")}',
f'Authorization: Bearer {found[1]}', f'X-Forwarded-For: {client}', '', ''])
try:
upstream = socket.create_connection((url.hostname, url.port or (443 if secure else 80)), timeout=ROUTER_TIMEOUT)
if secure:
upstream = ssl.create_default_context().wrap_socket(upstream, server_hostname=url.hostname)
upstream.sendall(request.encode())
reader = upstream.makefile('rb')
code = int(reader.readline().split(b' ', 2)[1])
headers = {}
while (line := reader.readline().strip()):
name, _, value = line.decode('latin-1').partition(':')
headers[name.strip().lower()] = value.strip()
if code != 101:
body = reader.read(int(headers.get('content-length', '0')))
upstream.close()
return code, headers.get('content-type', 'application/json'), body
upstream.settimeout(None)
except (OSError, ValueError, IndexError) as failure:
raise Unavailable(502, f'the server at {current["router_url"]} cannot open a shell: {failure}')
self.close_connection = True
try:
self.send_response(101, 'Switching Protocols')
self.send_header('Upgrade', 'websocket')
self.send_header('Connection', 'Upgrade')
self.send_header('Sec-WebSocket-Accept', headers.get('sec-websocket-accept', ''))
self.end_headers()
except ConnectionError:
# The client went away before the shell was relayed; the server ends the shell when upstream closes.
upstream.close()
return None
def pump(source, target):
try:
while data := source.read1(65536):
target.sendall(data)
except OSError:
pass
finally:
for end in (upstream, self.connection):
try:
end.shutdown(socket.SHUT_RDWR)
except OSError:
pass
thread = threading.Thread(target=pump, args=(reader, self.connection), name='shell output', daemon=True)
thread.start()
pump(self.rfile, upstream)
thread.join()
upstream.close()
return None
def transfer(self, path, send_body):
"""Pass a GET, HEAD or PUT of a web site's files to the server with the session's access token, streaming the
request body and the answer; the HTTP status, the content type and the body of a refusal, or None once the
server's answer was passed on.
The body of a PUT goes to the server as it arrives, with its Content-Type and Content-Length, so it is not held
to BODY_LIMIT; the answer passes with its Content-Type, Content-Length and Content-Disposition.
"""
current = settings()
client = self.client()
found = session(current, self.session_id(), client)
if found is None:
return error(401, 'not signed in')
if not current['router_url']:
raise Unavailable(503, f'no router is configured; set router_url in {CONFIG_FILE}')
headers = {'Authorization': f'Bearer {found[1]}', 'X-Forwarded-For': client}
body = None
if self.command == 'PUT':
length = self.headers.get('Content-Length', '')
if not length.isdigit():
return error(411, 'the body needs a Content-Length')
headers['Content-Type'] = self.headers.get('Content-Type') or 'application/octet-stream'
body = Stream(self.rfile, int(length))
try:
response = requests.request(self.command, current['router_url'] + path[4:], data=body, headers=headers,
timeout=ROUTER_TIMEOUT, allow_redirects=False, stream=True)
except requests.RequestException as failure:
raise Unavailable(502, f'the server at {current["router_url"]} cannot be reached: {failure}')
with response:
try:
self.send_response(response.status_code)
for name in ('Content-Type', 'Content-Length', 'Content-Disposition'):
if name in response.headers:
self.send_header(name, response.headers[name])
self.send_header('Cache-Control', 'no-store')
self.end_headers()
if send_body:
for chunk in response.raw.stream(65536, decode_content=False):
self.wfile.write(chunk)
except ConnectionError:
# The client went away before the answer was passed on.
self.close_connection = True
except urllib3.exceptions.HTTPError as failure:
# The server's answer broke off after its headers were passed on.
self.close_connection = True
log(f'the server\'s answer to {self.command} {path.split("?", 1)[0]} broke off: {failure}')
return None
def doc(self, path):
"""Answer GET /docs/<path> with a file of the help (doc_file): the HTTP status, the content type and the body; 401
without a session, 404 for a path that names no such file."""
try:
if session(settings(), self.session_id(), self.client()) is None:
return error(401, 'not signed in')
except Unavailable as failure:
return error(failure.code, str(failure))
except psycopg.Error as failure:
return error(503, f'the session database failed: {failure}')
found = doc_file(path)
if found is None:
return 404, 'text/plain; charset=utf-8', b'Not found\n'
try:
with open(found[0], 'rb') as handle:
return 200, found[1], handle.read()
except OSError:
return 404, 'text/plain; charset=utf-8', b'Not found\n'
def answer(self, send_body):
path = self.path.split('#', 1)[0]
route = path.split('?', 1)[0]
cookie = None
# A body that is not read must not be taken for the next request.
if self.command not in ('GET', 'HEAD'):
self.close_connection = True
if route == '/api/info':
https = self.proxied() and self.headers.get('X-Forwarded-Proto') == 'https'
status = linuxmanager_ui.status_url(CONFIG_FILE, self.headers.get('Host', ''), self.server.server_address[0], https)
body = json.dumps({'version': package_version(), 'status_url': status or None}).encode()
code, kind = 200, 'application/json'
elif route == '/api/shell' and self.command == 'GET' and self.headers.get('Upgrade', '').lower() == 'websocket':
try:
refused = self.relay(path)
except Unavailable as failure:
refused = error(failure.code, str(failure))
except psycopg.Error as failure:
refused = error(503, f'the session database failed: {failure}')
if refused is None:
return
code, kind, body = refused
elif SITE_FILES.match(route) and self.command in ('GET', 'HEAD', 'PUT'):
try:
refused = self.transfer(path, send_body)
except Unavailable as failure:
refused = error(failure.code, str(failure))
except psycopg.Error as failure:
refused = error(503, f'the session database failed: {failure}')
if refused is None:
return
code, kind, body = refused
elif route.startswith('/api/'):
limit = EDITOR_BODY_LIMIT if route.startswith('/api/shell/editor/') else BODY_LIMIT
payload = self.request_body(limit)
if payload is False:
code, kind, body = error(413, f'the body is larger than {limit} bytes')
else:
try:
code, kind, body, cookie = self.api(path, route, payload)
except Unavailable as failure:
code, kind, body = error(failure.code, str(failure))
except psycopg.Error as failure:
code, kind, body = error(503, f'the session database failed: {failure}')
elif route.startswith('/docs/') and self.command in ('GET', 'HEAD'):
code, kind, body = self.doc(route[len('/docs/'):])
elif self.command in ('GET', 'HEAD'):
code, kind, body = static(route)
else:
code, kind, body = 405, 'text/plain; charset=utf-8', b'Method not allowed\n'
try:
self.send_response(code)
self.send_header('Content-Type', kind)
self.send_header('Content-Length', str(len(body)))
self.send_header('Cache-Control', 'no-store' if route.startswith('/api/') or not route.startswith('/assets/') else 'max-age=31536000, immutable')
if cookie:
self.send_header('Set-Cookie', cookie)
for name, value in DOC_HEADERS if route.startswith('/docs/') else ():
self.send_header(name, value)
self.end_headers()
if send_body:
self.wfile.write(body)
except ConnectionError:
# The client went away before the answer was written.
self.close_connection = True
def request_body(self, limit=BODY_LIMIT):
"""The request body, None when there is none, or False when it is larger than limit bytes."""
length = self.headers.get('Content-Length')
if not length:
return None
try:
size = int(length)
except ValueError:
return False
if not 0 <= size <= limit:
return False
return self.rfile.read(size)
def do_GET(self):
self.answer(True)
def do_HEAD(self):
self.answer(False)
def do_POST(self):
self.answer(True)
def do_PUT(self):
self.answer(True)
def do_DELETE(self):
self.answer(True)
def log_message(self, format, *args):
sys.stderr.write('%s - - [%s] %s\n' % (self.client(), self.log_date_time_string(), format % args))
def log(message):
print(f'linux-manager-ui: {message}', file=sys.stderr, flush=True)
def main():
global CONFIG_FILE
parser = argparse.ArgumentParser(description='Serve the Ubuntu Linux Manager.')
parser.add_argument('--config', default=CONFIG_FILE, help=f'the settings file (default: {CONFIG_FILE})')
CONFIG_FILE = parser.parse_args().config
signal.signal(signal.SIGTERM, lambda *_: sys.exit(0))
initialize()
try:
linuxmanager_ui.serve(lambda: linuxmanager_ui.listen(CONFIG_FILE), Handler, log, DISCOVERY_SECONDS)
except KeyboardInterrupt:
pass
return 0
if __name__ == '__main__':
sys.exit(main())
@@ -1,19 +1,19 @@
#!/usr/bin/python3
"""Collect the metrics of the router, keep them in PostgreSQL, serve them for scraping and push them with OTLP.
"""Collect the metrics of the server, keep them in PostgreSQL, serve them for scraping and push them with OTLP.
Run by vpn-router-metrics.service while metrics.enabled is true; vpn-router-setup enables and
Run by linux-manager-metrics.service while metrics.enabled is true; linux-manager-setup enables and
disables the unit.
Every SAMPLE_SECONDS, on the minute, a sample is taken with psutil: the CPU utilization since the
previous sample, the memory used and in total, the free and the total space of /, the bytes
received and sent per second on the external and the internal interface, and the load averages;
and the VPN clients of each enabled VPN system (vpnrouter_status): IKEv2 (p2s) and
and the VPN clients of each enabled VPN system (linuxmanager_status): IKEv2 (p2s) and
OpenVPN clients and WireGuard peers with a recent handshake; and of each site-to-site connection
that is set up, whether it is up, the bytes and packets per second through its child SAs since
the previous sample, and how long its IKE SA has been up. The interfaces, the enabled systems and
the connections are those vpn-router-setup resolved, read from /run/vpn-router/web.json.
the connections are those linux-manager-setup resolved, read from /run/linux-manager/web.json.
The sample is written to the detail tables of vpnrouter_metrics. When the clock has passed into
The sample is written to the detail tables of linuxmanager_metrics. When the clock has passed into
a new hour, day, month or year, the complete periods are summarized, and once a day, after the
daily summary, the rows older than their retention are deleted; both also run on start. A
failure of the database is logged and tried again with the next sample.
@@ -21,8 +21,8 @@ failure of the database is logged and tried again with the next sample.
GET /metrics answers the last sample in the Prometheus text format, written by
prometheus_client, with OpenTelemetry semantic convention names in their Prometheus form, for an
OpenTelemetry Collector's prometheus receiver. It needs HTTP Basic authentication as the service
account metrics.service_account names, which vpn-router-setup keeps with the password in
/etc/vpn-router/metrics.secret, checked with a bind to the router's directory, so failed attempts
account metrics.service_account names, which linux-manager-setup keeps with the password in
/etc/linux-manager/metrics.secret, checked with a bind to the server's directory, so failed attempts
lock the account as for any user. The service listens on the interfaces metrics.listen
names and on the loopback address, where Caddy serves it over HTTPS at metrics.https_port; the
address of a call through Caddy is the last entry of its X-Forwarded-For header.
@@ -51,15 +51,15 @@ import psutil
from prometheus_client import CONTENT_TYPE_LATEST, generate_latest
from prometheus_client.core import CollectorRegistry, CounterMetricFamily, GaugeMetricFamily
import vpnrouter
import vpnrouter_metrics
import vpnrouter_net
import vpnrouter_serve
import vpnrouter_status
import vpnrouter_users
import linuxmanager
import linuxmanager_metrics
import linuxmanager_net
import linuxmanager_serve
import linuxmanager_status
import linuxmanager_users
STATE_FILE = '/run/vpn-router/web.json'
CONFIG_FILE = '/etc/vpn-router/vpn-router.json'
STATE_FILE = '/run/linux-manager/web.json'
CONFIG_FILE = '/etc/linux-manager/linux-manager.json'
SAMPLE_SECONDS = 60
@@ -78,7 +78,7 @@ _latest_lock = threading.Lock()
def log(message):
print(f'vpn-router-metrics: {message}', file=sys.stderr, flush=True)
print(f'linux-manager-metrics: {message}', file=sys.stderr, flush=True)
def read_json(path):
@@ -101,16 +101,16 @@ def clients(state):
"""The VPN clients of each enabled VPN system, None where it cannot be asked."""
counts = {}
if state.get('p2s'):
counts['p2s'] = vpnrouter_status.established('road-warrior')
counts['p2s'] = linuxmanager_status.established('road-warrior')
if state.get('openvpn'):
counts['openvpn'] = vpnrouter_status.openvpn_clients()
counts['openvpn'] = linuxmanager_status.openvpn_clients()
if state.get('wireguard'):
counts['wireguard'] = vpnrouter_status.wireguard_connected()
counts['wireguard'] = linuxmanager_status.wireguard_connected()
return counts
def sample(previous):
"""A sample of the router, with the rates since the previous sample, None for the first."""
"""A sample of the server, with the rates since the previous sample, None for the first."""
state = read_json(STATE_FILE)
network = section(state, 'network')
names = [name for name in dict.fromkeys((network.get('external_interface'), network.get('internal_interface')))
@@ -120,7 +120,7 @@ def sample(previous):
disk = psutil.disk_usage(MOUNTPOINT)
load1, load5, load15 = os.getloadavg()
connections = state.get('site_to_site')
tunnels = vpnrouter_status.site_to_site_tunnels(connections) if isinstance(connections, list) and connections else {}
tunnels = linuxmanager_status.site_to_site_tunnels(connections) if isinstance(connections, list) and connections else {}
now = time.time()
current = {
'time': now,
@@ -162,7 +162,7 @@ def sample(previous):
def periods(moment):
"""The hour, the day, the month and the year of a UTC moment, keyed by the levels of vpnrouter_metrics.LEVELS."""
"""The hour, the day, the month and the year of a UTC moment, keyed by the levels of linuxmanager_metrics.LEVELS."""
return {'hourly': moment.strftime('%Y-%m-%dT%H'), 'daily': moment.strftime('%Y-%m-%d'),
'monthly': moment.strftime('%Y-%m'), 'yearly': moment.strftime('%Y')}
@@ -174,14 +174,14 @@ def store(cfg, current, done):
it is updated only for what succeeded.
"""
now = periods(datetime.datetime.now(datetime.timezone.utc))
with vpnrouter_metrics.database(cfg) as connection:
vpnrouter_metrics.write(connection, current['system'], current['rates'], current['clients'], current['site_to_site'])
for level, _, _ in vpnrouter_metrics.LEVELS:
with linuxmanager_metrics.database(cfg) as connection:
linuxmanager_metrics.write(connection, current['system'], current['rates'], current['clients'], current['site_to_site'])
for level, _, _ in linuxmanager_metrics.LEVELS:
if done.get(level) != now[level]:
vpnrouter_metrics.summarize(connection, level)
linuxmanager_metrics.summarize(connection, level)
done[level] = now[level]
if done.get('garbage') != now['daily']:
vpnrouter_metrics.collect_garbage(connection, cfg)
linuxmanager_metrics.collect_garbage(connection, cfg)
done['garbage'] = now['daily']
@@ -212,15 +212,15 @@ def otlp_request(current, started):
gauge('system.cpu.load_average.1m', '{thread}', [(system['load1'], {})]),
gauge('system.cpu.load_average.5m', '{thread}', [(system['load5'], {})]),
gauge('system.cpu.load_average.15m', '{thread}', [(system['load15'], {})]),
gauge('vpn_router.clients', '{client}', [(count, {'kind': kind}) for kind, count in current['clients'].items()]),
gauge('vpn_router.site_to_site.up', '1', [(values['up'], {'connection': name})
gauge('linux_manager.clients', '{client}', [(count, {'kind': kind}) for kind, count in current['clients'].items()]),
gauge('linux_manager.site_to_site.up', '1', [(values['up'], {'connection': name})
for name, values in current['site_to_site'].items()]),
gauge('vpn_router.site_to_site.uptime', 's', [(values['uptime'], {'connection': name})
gauge('linux_manager.site_to_site.uptime', 's', [(values['uptime'], {'connection': name})
for name, values in current['site_to_site'].items()]),
gauge('vpn_router.site_to_site.io', 'By/s', [(values[value], {'connection': name, 'direction': direction})
gauge('linux_manager.site_to_site.io', 'By/s', [(values[value], {'connection': name, 'direction': direction})
for name, values in current['site_to_site'].items()
for value, direction in (('rx', 'receive'), ('tx', 'transmit'))]),
gauge('vpn_router.site_to_site.packets', '{packet}/s',
gauge('linux_manager.site_to_site.packets', '{packet}/s',
[(values[value], {'connection': name, 'direction': direction})
for name, values in current['site_to_site'].items()
for value, direction in (('packets_rx', 'receive'), ('packets_tx', 'transmit'))]),
@@ -232,8 +232,8 @@ def otlp_request(current, started):
for value, direction in zip(counters, ('receive', 'transmit'))]}},
]
return {'resourceMetrics': [{
'resource': {'attributes': attributes(**{'service.name': 'vpn-router', 'host.name': socket.gethostname()})},
'scopeMetrics': [{'scope': {'name': 'vpn-router-metrics'}, 'metrics': metrics}],
'resource': {'attributes': attributes(**{'service.name': 'linux-manager', 'host.name': socket.gethostname()})},
'scopeMetrics': [{'scope': {'name': 'linux-manager-metrics'}, 'metrics': metrics}],
}]}
@@ -265,7 +265,7 @@ def collector_loop():
while True:
time.sleep(SAMPLE_SECONDS - time.time() % SAMPLE_SECONDS)
try:
cfg = vpnrouter.load_config()
cfg = linuxmanager.load_config()
except SystemExit as exc:
report('configuration', str(exc.code))
continue
@@ -277,7 +277,7 @@ def collector_loop():
try:
store(cfg, current, done)
report('database', '')
except vpnrouter_metrics.DatabaseError as exc:
except linuxmanager_metrics.DatabaseError as exc:
report('database', str(exc))
metrics = cfg['metrics']
if metrics['otlp_endpoint']:
@@ -319,19 +319,19 @@ class Sample:
network.add_metric([name, 'receive'], received)
network.add_metric([name, 'transmit'], sent)
yield network
vpn = GaugeMetricFamily('vpn_router_clients', 'Connected VPN clients', labels=['kind'])
vpn = GaugeMetricFamily('linux_manager_clients', 'Connected VPN clients', labels=['kind'])
for kind, count in self.current['clients'].items():
if count is not None:
vpn.add_metric([kind], count)
yield vpn
tunnels = self.current['site_to_site']
up = GaugeMetricFamily('vpn_router_site_to_site_up', 'Whether the site-to-site connection is established, 1 or 0',
up = GaugeMetricFamily('linux_manager_site_to_site_up', 'Whether the site-to-site connection is established, 1 or 0',
labels=['connection'])
uptime = GaugeMetricFamily('vpn_router_site_to_site_uptime_seconds', 'Seconds the IKE SA of the site-to-site connection has been up',
uptime = GaugeMetricFamily('linux_manager_site_to_site_uptime_seconds', 'Seconds the IKE SA of the site-to-site connection has been up',
labels=['connection'])
io = GaugeMetricFamily('vpn_router_site_to_site_io_bytes_per_second',
io = GaugeMetricFamily('linux_manager_site_to_site_io_bytes_per_second',
'Bytes per second received and sent through the site-to-site connection', labels=['connection', 'direction'])
packets = GaugeMetricFamily('vpn_router_site_to_site_packets_per_second',
packets = GaugeMetricFamily('linux_manager_site_to_site_packets_per_second',
'Packets per second received and sent through the site-to-site connection',
labels=['connection', 'direction'])
for name, values in tunnels.items():
@@ -356,11 +356,11 @@ def authorized(header, account):
if name != account:
return False
try:
with vpnrouter_users.session() as connection:
if not vpnrouter_users.service(vpnrouter_users.entry(connection, name)):
with linuxmanager_users.session() as connection:
if not linuxmanager_users.service(linuxmanager_users.entry(connection, name)):
return False
vpnrouter_users.login(name, password, None)
except vpnrouter_users.Failure:
linuxmanager_users.login(name, password, None)
except linuxmanager_users.Failure:
return False
return True
@@ -372,7 +372,7 @@ class Handler(BaseHTTPRequestHandler):
"""The client address: the last X-Forwarded-For entry of a call through Caddy on the loopback address, else the peer."""
peer = self.client_address[0]
forwarded = self.headers.get('X-Forwarded-For', '')
if peer == vpnrouter_net.LOOPBACK and forwarded.strip():
if peer == linuxmanager_net.LOOPBACK and forwarded.strip():
return forwarded.split(',')[-1].strip()
return peer
@@ -381,7 +381,7 @@ class Handler(BaseHTTPRequestHandler):
extra = []
path = self.path.split('?', 1)[0]
account = str(section(read_json(CONFIG_FILE), 'metrics').get('service_account',
vpnrouter.SCHEMA['metrics']['service_account'])).strip()
linuxmanager.SCHEMA['metrics']['service_account'])).strip()
with _latest_lock:
current = _latest['sample']
if path != '/metrics':
@@ -396,7 +396,7 @@ class Handler(BaseHTTPRequestHandler):
body, kind, code = f'the user directory failed: {exc}\n'.encode(), 'text/plain; charset=utf-8', 500
if allowed is False:
body, kind, code = b'Unauthorized\n', 'text/plain; charset=utf-8', 401
extra.append(('WWW-Authenticate', 'Basic realm="vpn-router metrics", charset="UTF-8"'))
extra.append(('WWW-Authenticate', 'Basic realm="linux-manager metrics", charset="UTF-8"'))
elif allowed:
self.principal = account
if current is None:
@@ -405,15 +405,19 @@ class Handler(BaseHTTPRequestHandler):
registry = CollectorRegistry()
registry.register(Sample(current))
body, kind, code = generate_latest(registry), CONTENT_TYPE_LATEST, 200
self.send_response(code)
self.send_header('Content-Type', kind)
self.send_header('Content-Length', str(len(body)))
self.send_header('Cache-Control', 'no-store')
for name, value in extra:
self.send_header(name, value)
self.end_headers()
if send_body:
self.wfile.write(body)
try:
self.send_response(code)
self.send_header('Content-Type', kind)
self.send_header('Content-Length', str(len(body)))
self.send_header('Cache-Control', 'no-store')
for name, value in extra:
self.send_header(name, value)
self.end_headers()
if send_body:
self.wfile.write(body)
except ConnectionError:
# The client went away before the answer was written.
self.close_connection = True
def do_GET(self):
self.answer(True)
@@ -430,23 +434,23 @@ def wanted():
"""Where the service listens: the addresses of the interfaces metrics.listen names and LOOPBACK, at metrics.port."""
configuration = read_json(CONFIG_FILE)
metrics = section(configuration, 'metrics')
defaults = vpnrouter.SCHEMA['metrics']
defaults = linuxmanager.SCHEMA['metrics']
if metrics.get('enabled', defaults['enabled']) is False:
return [], 'metrics are disabled in the configuration'
port = metrics.get('port', defaults['port'])
if isinstance(port, bool) or not isinstance(port, int) or not 1 <= port <= 65535:
port = defaults['port']
addresses = vpnrouter_net.listen_addresses(configuration, metrics.get('listen', defaults['listen']))
addresses = linuxmanager_net.listen_addresses(configuration, metrics.get('listen', defaults['listen']))
if not addresses:
return [], 'the addresses to listen on are not known yet'
return [(address, port) for address in [*addresses, vpnrouter_net.LOOPBACK]], ''
return [(address, port) for address in [*addresses, linuxmanager_net.LOOPBACK]], ''
def main():
signal.signal(signal.SIGTERM, lambda *_: sys.exit(0))
threading.Thread(target=collector_loop, name='collector', daemon=True).start()
try:
vpnrouter_serve.serve(wanted, Handler, log, DISCOVERY_SECONDS)
linuxmanager_serve.serve(wanted, Handler, log, DISCOVERY_SECONDS)
except KeyboardInterrupt:
pass
return 0
@@ -6,10 +6,10 @@ the live system: routes, DNS-derived identity, WireGuard keys, firewall rules
and service reloads. Idempotent - a second run changes nothing.
Until a run has succeeded (SETUP_DONE), every run starts the bootstrap
services, BOOTSTRAP_UNITS, which show the progress and the failure recorded in
SETUP_STATE on vpnrouter.BOOTSTRAP_PORTS, and the firewall opens those ports on the
service, BOOTSTRAP_UNITS, which shows the progress and the failure recorded in
SETUP_STATE on linuxmanager.BOOTSTRAP_PORTS, and the firewall opens those ports on the
internal interface. The first successful run writes SETUP_DONE, closes the
ports and stops the bootstrap services; a failed run leaves them running.
ports and stops the bootstrap service; a failed run leaves it running.
Interface names come from the configuration and are never guessed. The
firewall and NAT rules are written against them, so choosing the wrong one
@@ -32,76 +32,80 @@ sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
import ldap
import vpnrouter
import vpnrouter_auth
import vpnrouter_dns
import vpnrouter_ldap
import vpnrouter_pki
import vpnrouter_sites
import vpnrouter_users
from vpnrouter import fail, warn
import linuxmanager
import linuxmanager_auth
import linuxmanager_dns
import linuxmanager_ldap
import linuxmanager_pki
import linuxmanager_sites
import linuxmanager_users
from linuxmanager import fail, warn
BEFORE_RULES = pathlib.Path('/etc/ufw/before.rules')
UFW_DEFAULTS = pathlib.Path('/etc/default/ufw')
WG_DIR = pathlib.Path('/etc/wireguard')
WEB_STATE = vpnrouter.WEB_STATE
WEB_STATE = linuxmanager.WEB_STATE
#: The state, the current step and the failure of the last run, which the bootstrap services show.
SETUP_STATE = pathlib.Path('/run/vpn-router/setup.json')
#: The state, the current step and the failure of the last run, which the bootstrap service shows.
SETUP_STATE = pathlib.Path('/run/linux-manager/setup.json')
#: Written once a run has succeeded; until then setup runs the bootstrap services and opens their ports.
SETUP_DONE = pathlib.Path('/var/lib/vpn-router/setup.done')
#: Written once a run has succeeded; until then setup runs the bootstrap service and opens its ports.
SETUP_DONE = pathlib.Path('/var/lib/linux-manager/setup.done')
#: The units of the bootstrap services, whose ports are vpnrouter.BOOTSTRAP_PORTS.
BOOTSTRAP_UNITS = ('vpn-router-bootstrap.service', 'vpn-router-manager-bootstrap.service')
#: The unit of the bootstrap service, whose ports are linuxmanager.BOOTSTRAP_PORTS.
BOOTSTRAP_UNITS = ('linux-manager-bootstrap.service',)
#: The default admin of the Router Manager, {"username", "password"}, which its package writes from
#: The default admin of the Manager, {"username", "password"}, which the package's postinst writes from
#: its debconf answers; setup creates the account and removes the file.
MANAGER_ADMIN = pathlib.Path('/etc/vpn-router-manager/admin.json')
MANAGER_ADMIN = pathlib.Path('/etc/linux-manager/manager-admin.json')
#: The password of the metrics service account, which setup makes and keeps (root only).
METRICS_SECRET = pathlib.Path('/etc/vpn-router/metrics.secret')
METRICS_SECRET = pathlib.Path('/etc/linux-manager/metrics.secret')
#: The schedule of the backup timer, written from backup.schedule.
BACKUP_SCHEDULE = pathlib.Path('/etc/systemd/system/vpn-router-backup.timer.d/schedule.conf')
BACKUP_SCHEDULE = pathlib.Path('/etc/systemd/system/linux-manager-backup.timer.d/schedule.conf')
#: The configuration of Caddy, the HTTPS proxy, which the package's drop-in for caddy.service names;
#: Caddy runs as the user and group CADDY_GROUP, which the caddy package creates.
CADDYFILE = pathlib.Path('/etc/caddy/vpn-router.Caddyfile')
CADDYFILE = pathlib.Path('/etc/caddy/linux-manager.Caddyfile')
CADDY_UNIT = 'caddy.service'
CADDY_GROUP = 'caddy'
#: The unit that installs Caddy from https.caddy_source with vpn-router-caddy, outside setup.
CADDY_INSTALL_UNIT = 'vpn-router-caddy.service'
#: The unit that installs Caddy from https.caddy_source with linux-manager-caddy, outside setup.
CADDY_INSTALL_UNIT = 'linux-manager-caddy.service'
#: The certificates and keys of the issuing CAs behind Caddy's ACME server and the sites of tls ca, <name>/cert.pem and
#: key.pem.
CADDY_PKI_DIR = pathlib.Path('/etc/caddy/vpn-router-pki')
CADDY_PKI_DIR = pathlib.Path('/etc/caddy/linux-manager-pki')
#: The certificates and keys of the sites of tls file, <site>/cert.pem and key.pem.
CADDY_SITES_DIR = pathlib.Path('/etc/caddy/vpn-router-sites')
CADDY_SITES_DIR = pathlib.Path('/etc/caddy/linux-manager-sites')
#: The unit of dnsmasq, the resolver of the remote access clients.
RESOLVER_UNIT = 'vpn-router-resolver.service'
RESOLVER_UNIT = 'linux-manager-resolver.service'
#: The unit of rclone's WebDAV server of the sites' folders.
WEBDAV_UNIT = 'vpn-router-webdav.service'
WEBDAV_UNIT = 'linux-manager-webdav.service'
#: The unit that installs the packages of a feature with install-packages, outside setup; {} is the feature.
INSTALL_UNIT = 'vpn-router-install@{}.service'
INSTALL_UNIT = 'linux-manager-install@{}.service'
#: The timer of vpn-router-ca maintain, which keeps the CRLs fresh.
PKI_TIMER = 'vpn-router-pki.timer'
#: The timer of linux-manager-ca maintain, which keeps the CRLs fresh.
PKI_TIMER = 'linux-manager-pki.timer'
#: The system's side of console login (vpnrouter_users.console): nslcd's configuration, which serves the posixAccounts
#: The system's side of console login (linuxmanager_users.console): nslcd's configuration, which serves the posixAccounts
#: of the directory with console login on, the name service switch, the sudo rule of their group, and the drop-in of
#: sshd that lets them sign in with their password while ssh.ldap_login is true.
NSLCD_CONF = pathlib.Path('/etc/nslcd.conf')
NSSWITCH = pathlib.Path('/etc/nsswitch.conf')
SUDOERS = pathlib.Path('/etc/sudoers.d/vpn-router')
SSHD_DROPIN = pathlib.Path('/etc/ssh/sshd_config.d/vpn-router.conf')
SUDOERS = pathlib.Path('/etc/sudoers.d/linux-manager')
SSHD_DROPIN = pathlib.Path('/etc/ssh/sshd_config.d/linux-manager.conf')
PAM_SESSION = pathlib.Path('/etc/pam.d/common-session')
#: The kernel settings setup writes while the router or the VPN is on: forwarding for both, and a loose rp_filter for
#: the router's asymmetric forwarding between its interfaces.
SYSCTL_CONF = pathlib.Path('/etc/sysctl.d/99-linux-manager.conf')
FILTER_ANCHOR = '# End required lines'
#: Configuration options left out wherever the configuration is shown.
@@ -195,32 +199,36 @@ def strip_blocks(text):
def is_configured(ctx):
"""True once any feature is actually configured.
"""True once any feature is actually configured: routing, a VPN system, or a listener of the core.
While nothing is configured the package leaves the firewall alone: an
install that has not been filled in yet must not start blocking traffic on
a host that was working before.
"""
return any((vpnrouter.s2s_ready(ctx), vpnrouter.routing_ready(ctx),
vpnrouter.p2s_ready(ctx), vpnrouter.wg_ready(ctx),
vpnrouter.openvpn_ready(ctx)))
return any((linuxmanager.s2s_ready(ctx), linuxmanager.routing_ready(ctx),
linuxmanager.p2s_ready(ctx), linuxmanager.wg_ready(ctx),
linuxmanager.openvpn_ready(ctx), ctx['web_enabled'], linuxmanager.https_ready(ctx),
linuxmanager.metrics_ready(ctx), linuxmanager.dns_ready(ctx), bool(ctx['sites'])))
def filter_block(ctx):
lines = [
'',
'# IPSEC RULES START',
'-A ufw-before-input -p udp --dport 500 -j ACCEPT',
'-A ufw-before-input -p udp --dport 4500 -j ACCEPT',
'-A ufw-before-input -p esp -j ACCEPT',
'-A ufw-before-input -m policy --dir in --pol ipsec -j ACCEPT',
'-A ufw-before-output -m policy --dir out --pol ipsec -j ACCEPT',
'-A ufw-before-forward -m policy --dir in --pol ipsec -j ACCEPT',
'-A ufw-before-forward -m policy --dir out --pol ipsec -j ACCEPT',
'# IPSEC RULES END',
]
lines = []
if vpnrouter.wg_ready(ctx):
if linuxmanager.s2s_ready(ctx) or linuxmanager.p2s_ready(ctx):
lines += [
'',
'# IPSEC RULES START',
'-A ufw-before-input -p udp --dport 500 -j ACCEPT',
'-A ufw-before-input -p udp --dport 4500 -j ACCEPT',
'-A ufw-before-input -p esp -j ACCEPT',
'-A ufw-before-input -m policy --dir in --pol ipsec -j ACCEPT',
'-A ufw-before-output -m policy --dir out --pol ipsec -j ACCEPT',
'-A ufw-before-forward -m policy --dir in --pol ipsec -j ACCEPT',
'-A ufw-before-forward -m policy --dir out --pol ipsec -j ACCEPT',
'# IPSEC RULES END',
]
if linuxmanager.wg_ready(ctx):
lines += [
'',
'# WIREGUARD RULES START',
@@ -228,7 +236,7 @@ def filter_block(ctx):
'# WIREGUARD RULES END',
]
if vpnrouter.openvpn_ready(ctx):
if linuxmanager.openvpn_ready(ctx):
device = ctx['openvpn_device']
lines += [
'',
@@ -242,9 +250,7 @@ def filter_block(ctx):
]
lines += ['# OPENVPN RULES END']
pools = [pool for ready, pool in ((vpnrouter.p2s_ready(ctx), ctx['p2s_address_pool']),
(vpnrouter.openvpn_ready(ctx), ctx['openvpn_pool']),
(vpnrouter.wg_ready(ctx), ctx['wg_network'])) if ready]
pools = linuxmanager.vpn_pools(ctx)
# The remote access clients' resolver answers on the addresses of p2s.dns_listen.
if pools and ctx['p2s_resolver_addrs']:
lines += ['', '# P2S DNS RULES START']
@@ -256,8 +262,9 @@ def filter_block(ctx):
]
lines += ['# P2S DNS RULES END']
# The web service, the Router Manager and the metrics service choose their interfaces themselves, so their ports are open on both.
interfaces = [name for name in (ctx['int_iface'], ctx['wan_iface']) if name]
# The web service, the Manager and the metrics service choose their interfaces themselves, so their ports are open on
# both; without the router both are the one interface.
interfaces = list(dict.fromkeys(name for name in (ctx['int_iface'], ctx['wan_iface']) if name))
if ctx['web_enabled'] and interfaces:
lines += [
'',
@@ -271,12 +278,12 @@ def filter_block(ctx):
'',
'# BOOTSTRAP RULES START',
*(f'-A ufw-before-input -i {ctx["int_iface"]} -p tcp --dport {port} -j ACCEPT'
for port in vpnrouter.BOOTSTRAP_PORTS),
for port in linuxmanager.BOOTSTRAP_PORTS),
'# BOOTSTRAP RULES END',
]
if vpnrouter.MANAGER.exists() and interfaces:
port = vpnrouter.manager_port()
if interfaces:
port = linuxmanager.manager_port()
lines += [
'',
'# MANAGER RULES START',
@@ -284,7 +291,7 @@ def filter_block(ctx):
'# MANAGER RULES END',
]
if vpnrouter.metrics_ready(ctx) and interfaces:
if linuxmanager.metrics_ready(ctx) and interfaces:
lines += [
'',
'# METRICS RULES START',
@@ -294,19 +301,19 @@ def filter_block(ctx):
# Let's Encrypt reaches the HTTP challenge on the external interface, and the CA files published on port 80 are
# read on both; the HTTPS ports are open on both.
if vpnrouter.https_ready(ctx) and interfaces:
if linuxmanager.https_ready(ctx) and interfaces:
publish = ctx['pki_publish']
ports = [port for wanted, port in ((ctx['web_enabled'] or 'https' in publish or ctx['pki_acme'],
ctx['https_web_port']),
(vpnrouter.MANAGER.exists(), ctx['https_manager_port']),
(vpnrouter.metrics_ready(ctx), ctx['metrics_https_port']),
(vpnrouter.webdav_ready(ctx), ctx['webdav_port']),
('http-80' in publish, vpnrouter.ACME_HTTP_PORT),
('https-default' in publish, vpnrouter.HTTPS_DEFAULT_PORT)) if wanted]
(True, ctx['https_manager_port']),
(linuxmanager.metrics_ready(ctx), ctx['metrics_https_port']),
(linuxmanager.webdav_ready(ctx), ctx['webdav_port']),
('http-80' in publish, linuxmanager.ACME_HTTP_PORT),
('https-default' in publish, linuxmanager.HTTPS_DEFAULT_PORT)) if wanted]
lines += [
'',
'# HTTPS RULES START',
*([f'-A ufw-before-input -i {ctx["wan_iface"]} -p tcp --dport {vpnrouter.ACME_HTTP_PORT} -j ACCEPT']
*([f'-A ufw-before-input -i {ctx["wan_iface"]} -p tcp --dport {linuxmanager.ACME_HTTP_PORT} -j ACCEPT']
if ctx['wan_iface'] and 'http-80' not in publish else []),
*(f'-A ufw-before-input -i {name} -p tcp --dport {port} -j ACCEPT' for name in interfaces for port in ports),
'# HTTPS RULES END',
@@ -319,7 +326,7 @@ def filter_block(ctx):
sites = ctx['sites'] or []
rules = sorted({(name, site['port']) for site in sites for name in by_listen[site['listen']] if name})
if ctx['wan_iface'] and any(site['tls'] == 'acme' for site in sites):
rules = sorted(set(rules) | {(ctx['wan_iface'], vpnrouter.ACME_HTTP_PORT)})
rules = sorted(set(rules) | {(ctx['wan_iface'], linuxmanager.ACME_HTTP_PORT)})
if rules:
lines += [
'',
@@ -328,7 +335,8 @@ def filter_block(ctx):
'# SITES RULES END',
]
dns_interfaces = [name for name in by_listen[ctx['dns_listen']] if name] if vpnrouter.dns_ready(ctx) else []
dns_interfaces = (list(dict.fromkeys(name for name in by_listen[ctx['dns_listen']] if name))
if linuxmanager.dns_ready(ctx) else [])
if dns_interfaces:
lines += [
'',
@@ -338,7 +346,7 @@ def filter_block(ctx):
'# DNS RULES END',
]
if vpnrouter.routing_ready(ctx):
if linuxmanager.routing_ready(ctx):
lines += ['', '# ROUTER FORWARD RULES START']
for subnet in ctx['local_cidr_list']:
lines += [
@@ -353,7 +361,7 @@ def filter_block(ctx):
def nat_and_mangle_block(ctx):
lines = []
if vpnrouter.routing_ready(ctx):
if linuxmanager.routing_ready(ctx):
subnets = ctx['local_cidr_list']
lines += [
'',
@@ -374,16 +382,29 @@ def nat_and_mangle_block(ctx):
lines.append(
f'-A POSTROUTING -s {subnet} -o {ctx["wan_iface"]} -j MASQUERADE')
lines += ['COMMIT', '# ROUTER NAT RULES END']
elif not ctx['router_enabled'] and ctx['wan_iface'] and linuxmanager.vpn_pools(ctx):
# Without the router the remote access clients reach the server's networks from its own address.
lines += [
'',
'# ROUTER NAT RULES START',
'*nat',
':POSTROUTING ACCEPT [0:0]',
'-F POSTROUTING',
*(f'-A POSTROUTING -s {pool} -o {ctx["wan_iface"]} -j MASQUERADE' for pool in linuxmanager.vpn_pools(ctx)),
'COMMIT',
'# ROUTER NAT RULES END',
]
lines += [
'',
'# ROUTER MSS RULES START',
'*mangle',
':FORWARD ACCEPT [0:0]',
'-A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu',
'COMMIT',
'# ROUTER MSS RULES END',
]
if ctx['router_enabled'] or ctx['vpn_enabled']:
lines += [
'',
'# ROUTER MSS RULES START',
'*mangle',
':FORWARD ACCEPT [0:0]',
'-A FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu',
'COMMIT',
'# ROUTER MSS RULES END',
]
return ''.join(line + '\n' for line in lines)
@@ -397,6 +418,24 @@ def _insert_after_line(text, marker, block):
return None
def nat_rules(text):
"""The POSTROUTING rules of the package's NAT block in the text of before.rules."""
lines = [line.strip() for line in text.splitlines()]
start, end = '# ROUTER NAT RULES START', '# ROUTER NAT RULES END'
if start not in lines or end not in lines:
return []
return [line for line in lines[lines.index(start):lines.index(end)] if line.startswith('-A POSTROUTING ')]
def remove_stale_nat(original, text):
"""Delete from the running nat table the rules of the old NAT block that the new one no longer has; ufw reload
leaves the nat table alone."""
wanted = set(nat_rules(text))
for rule in nat_rules(original):
if rule not in wanted:
run(['iptables', '-t', 'nat', '-D', *rule.split()[1:]], check=False)
def setup_ufw(ctx):
"""Rewrite the package's blocks in before.rules. Returns True on change."""
if not BEFORE_RULES.is_file():
@@ -410,6 +449,7 @@ def setup_ufw(ctx):
if text == original:
return False
BEFORE_RULES.write_text(text, encoding='utf-8')
remove_stale_nat(original, text)
return True
if UFW_DEFAULTS.is_file():
@@ -438,6 +478,7 @@ def setup_ufw(ctx):
if text == original:
return False
BEFORE_RULES.write_text(text, encoding='utf-8')
remove_stale_nat(original, text)
return True
@@ -453,8 +494,10 @@ def manage_strongswan(ctx, changed):
(ExecStartPost) and re-reads it on reload, so starting it is enough on a
cold boot and a reload is the right response to a configuration change.
"""
wanted = vpnrouter.s2s_ready(ctx) or vpnrouter.p2s_ready(ctx)
wanted = linuxmanager.s2s_ready(ctx) or linuxmanager.p2s_ready(ctx)
if not wanted:
if unit_is_active('strongswan'):
run(['systemctl', 'disable', '--now', 'strongswan'], check=False)
return
if not unit_is_active('strongswan'):
@@ -467,7 +510,7 @@ def manage_wireguard(ctx, changed):
"""Bring wg0 up when WireGuard is configured, and down when it is not."""
unit = 'wg-quick@wg0'
if not vpnrouter.wg_ready(ctx):
if not linuxmanager.wg_ready(ctx):
if unit_is_active(unit):
run(['systemctl', 'disable', '--now', unit], check=False)
return
@@ -480,19 +523,19 @@ def manage_wireguard(ctx, changed):
def manage_directory():
"""Provision the user directory in slapd, whether or not a P2S system is configured; the problem, or ''."""
if vpnrouter_ldap.ensure_services() or not unit_is_active('slapd'):
if linuxmanager_ldap.ensure_services() or not unit_is_active('slapd'):
result = run(['systemctl', 'restart', 'slapd'], check=False)
if result.returncode != 0:
return f'slapd cannot be restarted: {result.stderr.strip()}'
try:
vpnrouter_ldap.provision()
linuxmanager_ldap.provision()
except Exception as exc:
return f'the user directory cannot be provisioned: {exc!r}'
return ''
def manage_admin():
"""Create the Router Manager's default admin from MANAGER_ADMIN, then remove the file; the problem, or ''.
"""Create the Manager's default admin from MANAGER_ADMIN, then remove the file; the problem, or ''.
The admin is a Manager-only account and a member of the admins group, which is created with it
when it does not exist.
@@ -502,17 +545,17 @@ def manage_admin():
try:
answers = json.loads(MANAGER_ADMIN.read_text(encoding='utf-8'))
name, password = answers['username'], answers['password']
with vpnrouter_users.session() as connection:
vpnrouter_users.add(connection, name, password, {})
with linuxmanager_users.session() as connection:
linuxmanager_users.add(connection, name, password, {})
try:
vpnrouter_users.group_member_add(connection, vpnrouter_users.ADMIN_GROUP, name)
except vpnrouter_users.Failure as failure:
linuxmanager_users.group_member_add(connection, linuxmanager_users.ADMIN_GROUP, name)
except linuxmanager_users.Failure as failure:
if failure.status != 3:
raise
vpnrouter_users.group_add(connection, vpnrouter_users.ADMIN_GROUP, 'Router administrators', [name])
linuxmanager_users.group_add(connection, linuxmanager_users.ADMIN_GROUP, 'Router administrators', [name])
MANAGER_ADMIN.unlink()
except (OSError, ValueError, KeyError, TypeError, ldap.LDAPError, vpnrouter_users.Failure) as exc:
return f'the Router Manager admin cannot be created from {MANAGER_ADMIN}: {exc}'
except (OSError, ValueError, KeyError, TypeError, ldap.LDAPError, linuxmanager_users.Failure) as exc:
return f'the Manager admin cannot be created from {MANAGER_ADMIN}: {exc}'
return ''
@@ -520,41 +563,41 @@ def manage_metrics_account(ctx):
"""Keep the service account metrics.service_account names, whose password is in METRICS_SECRET; the problem, or ''.
While metrics are enabled, setup owns the account, an internal service account: it adds it as a service account
with a random password and the labels of vpnrouter_users.internal_accounts when it is missing, sets those labels
with a random password and the labels of linuxmanager_users.internal_accounts when it is missing, sets those labels
when they differ, and gives it a new random password, ending any lockout, when METRICS_SECRET is missing or does
not hold its password. The password of a disabled account is left as it is. An account of that name that is not a
service account is left alone and is a problem.
"""
if not vpnrouter.metrics_ready(ctx):
if not linuxmanager.metrics_ready(ctx):
return ''
name = ctx['metrics_service_account']
wanted = vpnrouter_users.internal_accounts().get(name, {})
wanted = linuxmanager_users.internal_accounts().get(name, {})
try:
vpnrouter_users.check_name(name)
linuxmanager_users.check_name(name)
try:
password = METRICS_SECRET.read_text(encoding='utf-8').strip()
except OSError:
password = ''
with vpnrouter_users.session() as connection:
with linuxmanager_users.session() as connection:
try:
current = vpnrouter_users.entry(connection, name)
except vpnrouter_users.Failure as failure:
current = linuxmanager_users.entry(connection, name)
except linuxmanager_users.Failure as failure:
if failure.status != 3:
raise
current = None
if current is not None and not vpnrouter_users.service(current):
if current is not None and not linuxmanager_users.service(current):
return f'metrics.service_account: {name} is a user, not a service account'
if current is not None and any(vpnrouter_users.summary(current)[key] != value for key, value in wanted.items()):
vpnrouter_users.update_service(connection, name, wanted)
if current is not None and any(linuxmanager_users.summary(current)[key] != value for key, value in wanted.items()):
linuxmanager_users.update_service(connection, name, wanted)
# An administrator who disabled the account meant it; its password is left as it is.
if current is not None and not vpnrouter_users.summary(current)['enabled']:
if current is not None and not linuxmanager_users.summary(current)['enabled']:
return ''
valid = False
if current is not None and password:
try:
vpnrouter_users.login(name, password, None)
linuxmanager_users.login(name, password, None)
valid = True
except vpnrouter_users.Failure:
except linuxmanager_users.Failure:
pass
if not valid:
password = secrets.token_urlsafe(32)
@@ -562,11 +605,11 @@ def manage_metrics_account(ctx):
METRICS_SECRET.chmod(0o600)
METRICS_SECRET.write_text(password + '\n', encoding='utf-8')
if current is None:
vpnrouter_users.add(connection, name, password, wanted, service=True)
linuxmanager_users.add(connection, name, password, wanted, service=True)
else:
vpnrouter_users.passwd(connection, name, password)
vpnrouter_users.unlock(connection, name)
except (OSError, ldap.LDAPError, vpnrouter_users.Failure) as exc:
linuxmanager_users.passwd(connection, name, password)
linuxmanager_users.unlock(connection, name)
except (OSError, ldap.LDAPError, linuxmanager_users.Failure) as exc:
return f'the metrics service account cannot be kept: {exc}'
return ''
@@ -583,16 +626,31 @@ def write_if_changed(path, text, mode=0o644):
return True
def manage_sysctl(ctx):
"""Write SYSCTL_CONF while the router or the VPN is on, remove it while both are off, and apply the settings when
the file changed. Removing the file leaves the running values alone, as Docker also needs forwarding."""
if ctx['router_enabled'] or ctx['vpn_enabled']:
lines = ['net.ipv4.ip_forward = 1']
if ctx['router_enabled']:
lines += ['net.ipv4.conf.all.rp_filter = 0', 'net.ipv4.conf.default.rp_filter = 0']
changed = write_if_changed(SYSCTL_CONF, ''.join(line + '\n' for line in lines))
else:
changed = SYSCTL_CONF.exists()
SYSCTL_CONF.unlink(missing_ok=True)
if changed:
run(['sysctl', '--system'], check=False)
def manage_console(cfg):
"""Make the system serve console login; the problem, or ''.
The local group vpnrouter_ldap.CONSOLE_GROUP is created when missing, and sudo lets its members run any command
The local group linuxmanager_ldap.CONSOLE_GROUP is created when missing, and sudo lets its members run any command
with their password. nslcd serves the directory's posixAccounts with console login on through the local socket,
and checks their passwords by binding as the user; the passwd line of the name service switch gains ldap, which
restarts the web service, and PAM creates a home at the first login. While ssh.ldap_login is true, sshd accepts the password of the group's members;
a drop-in sshd -t refuses is removed again.
"""
group = vpnrouter_ldap.CONSOLE_GROUP
group = linuxmanager_ldap.CONSOLE_GROUP
try:
grp.getgrnam(group)
except KeyError:
@@ -603,7 +661,7 @@ def manage_console(cfg):
rule = f'%{group} ALL=(ALL:ALL) ALL\n'
try:
if not SUDOERS.is_file() or SUDOERS.read_text(encoding='utf-8') != rule:
staged = SUDOERS.with_name('.vpn-router.new')
staged = SUDOERS.with_name('.linux-manager.new')
staged.write_text(rule, encoding='utf-8')
staged.chmod(0o440)
checked = run(['visudo', '-cf', str(staged)], check=False)
@@ -612,16 +670,16 @@ def manage_console(cfg):
return f'the sudo rule of {group} is refused: {checked.stdout.strip()} {checked.stderr.strip()}'
staged.replace(SUDOERS)
uri = 'ldapi://' + vpnrouter_ldap.SLAPD_SOCKET.replace('/', '%2f') + '/'
uri = 'ldapi://' + linuxmanager_ldap.SLAPD_SOCKET.replace('/', '%2f') + '/'
nslcd = '\n'.join([
'uid nslcd',
'gid nslcd',
f'uri {uri}',
f'base {vpnrouter_ldap.BASE}',
'filter passwd (&(objectClass=posixAccount)(vpnRouterConsoleLogin=TRUE))',
f'nss_min_uid {vpnrouter_ldap.CONSOLE_UID_MIN}',
'pam_authz_search (&(objectClass=posixAccount)(objectClass=oathTOTPUser)(uid=$username)(vpnRouterConsoleLogin=TRUE))',
'pam_password_prohibit_message The password is changed in the Router Manager or with vpn-router-user.',
f'base {linuxmanager_ldap.BASE}',
'filter passwd (&(objectClass=posixAccount)(linuxManagerConsoleLogin=TRUE))',
f'nss_min_uid {linuxmanager_ldap.CONSOLE_UID_MIN}',
'pam_authz_search (&(objectClass=posixAccount)(objectClass=oathTOTPUser)(uid=$username)(linuxManagerConsoleLogin=TRUE))',
'pam_password_prohibit_message The password is changed in the Manager or with linux-manager-user.',
'',
])
if write_if_changed(NSLCD_CONF, nslcd, 0o640) or not unit_is_active('nslcd'):
@@ -636,7 +694,7 @@ def manage_console(cfg):
NSSWITCH.write_text(''.join(switched), encoding='utf-8')
# A running process keeps the name service switch it read first, so the web service's shell would not find
# the console accounts of the directory until it restarts.
run(['systemctl', 'try-restart', 'vpn-router-web.service'], check=False)
run(['systemctl', 'try-restart', 'linux-manager-web.service'], check=False)
if 'pam_mkhomedir' not in PAM_SESSION.read_text(encoding='utf-8'):
run(['pam-auth-update', '--enable', 'mkhomedir'], check=False)
@@ -660,9 +718,9 @@ def manage_console(cfg):
def manage_openvpn(ctx, changed):
"""Run the OpenVPN server when it is configured, and stop it when it is not."""
unit = 'openvpn-server@vpn-router'
unit = 'openvpn-server@linux-manager'
if not vpnrouter.openvpn_ready(ctx):
if not linuxmanager.openvpn_ready(ctx):
if unit_is_active(unit):
run(['systemctl', 'disable', '--now', unit], check=False)
return
@@ -683,7 +741,7 @@ def caddy_address(host, port, tls):
def manage_resolver(changed):
"""Run dnsmasq, the resolver of the remote access clients, while configure wrote its configuration, restarting it when
that changed, and stop it when not. It starts after systemd-resolved has left the addresses it takes."""
if not vpnrouter.RESOLVER_CONF.exists():
if not linuxmanager.RESOLVER_CONF.exists():
if unit_is_active(RESOLVER_UNIT):
run(['systemctl', 'disable', '--now', RESOLVER_UNIT], check=False)
return
@@ -694,13 +752,13 @@ def manage_resolver(changed):
def manage_caddy(cfg, ctx):
"""Run Caddy, the HTTPS proxy of the router's services and the server of the web sites, while either is wanted, and
"""Run Caddy, the HTTPS proxy of the server's services and the server of the web sites, while either is wanted, and
stop it when not.
While HTTPS is ready, CADDYFILE serves the web service on https.web_port, while the manager is installed the
manager on https.manager_port, while metrics are enabled the metrics service on metrics.https_port, and while
While HTTPS is ready, CADDYFILE serves the web service on https.web_port, the manager on
https.manager_port, while metrics are enabled the metrics service on metrics.https_port, and while
WebDAV is ready WebDAV on sites.webdav_port, all for wan.local_fqdn with a certificate from Let's Encrypt, and all
calling the service on its loopback address. It also serves the CA files of vpnrouter_pki.PUBLISH_DIR at /pki/ on
calling the service on its loopback address. It also serves the CA files of linuxmanager_pki.PUBLISH_DIR at /pki/ on
the endpoints of pki.publish that are Caddy's, and an ACME server at /acme/<name>/ on https.web_port for each
issuing CA with acme and a certificate.
@@ -714,20 +772,20 @@ def manage_caddy(cfg, ctx):
setup, whose next run serves with it.
"""
sites = ctx['sites']
if not vpnrouter.https_ready(ctx) and not sites:
if not linuxmanager.https_ready(ctx) and not sites:
if sites is None:
return
vpnrouter.remove(CADDYFILE)
linuxmanager.remove(CADDYFILE)
if unit_is_active(CADDY_UNIT):
run(['systemctl', 'disable', '--now', CADDY_UNIT], check=False)
return
source = cfg['https']['caddy_source']
if shutil.which('caddy') is None or (source == 'upstream' and not vpnrouter.CADDY_APT_SOURCE.is_file()):
if shutil.which('caddy') is None or (source == 'upstream' and not linuxmanager.CADDY_APT_SOURCE.is_file()):
run(['systemctl', 'start', '--no-block', CADDY_INSTALL_UNIT], check=False)
warn(f'Caddy is installed from the {source} source by {CADDY_INSTALL_UNIT}, which runs setup again')
return
if source == 'ubuntu' and vpnrouter.CADDY_APT_SOURCE.is_file():
if source == 'ubuntu' and linuxmanager.CADDY_APT_SOURCE.is_file():
warn("https.caddy_source ubuntu is ignored: Caddy's repository is configured, and Caddy stays upstream's")
if sites is None:
@@ -739,11 +797,11 @@ def manage_caddy(cfg, ctx):
group = grp.getgrnam(CADDY_GROUP).gr_gid
except KeyError:
fail(f'the group {CADDY_GROUP} does not exist; is the caddy package installed?')
https = vpnrouter.https_ready(ctx)
https = linuxmanager.https_ready(ctx)
wanted = list(dict.fromkeys([*(ctx['pki_acme'] if https else []),
*(site['ca'] for site in sites if site['tls'] == 'ca')]))
try:
credentials = {name: vpnrouter_pki.authority_files(name) for name in wanted}
credentials = {name: linuxmanager_pki.authority_files(name) for name in wanted}
except ldap.LDAPError as exc:
warn(f'the ACME servers and the sites of issuing CAs are left out: the PKI cannot be read: {exc}')
credentials = {}
@@ -762,7 +820,7 @@ def manage_caddy(cfg, ctx):
path.mkdir(mode=0o750, exist_ok=True)
os.chown(path, 0, group)
for file, data in zip(('cert.pem', 'key.pem'), credentials[name]):
copied |= vpnrouter_pki.write(folder / file, data, 0o640)
copied |= linuxmanager_pki.write(folder / file, data, 0o640)
os.chown(folder / file, 0, group)
files = {site['name']: site for site in sites if site['tls'] == 'file'}
for folder in (CADDY_SITES_DIR.iterdir() if CADDY_SITES_DIR.is_dir() else []):
@@ -774,24 +832,24 @@ def manage_caddy(cfg, ctx):
path.mkdir(mode=0o750, exist_ok=True)
os.chown(path, 0, group)
for file, data in (('cert.pem', site['certificate']), ('key.pem', site['private_key'])):
copied |= vpnrouter_pki.write(folder / file, data.strip().encode() + b'\n', 0o640)
copied |= linuxmanager_pki.write(folder / file, data.strip().encode() + b'\n', 0o640)
os.chown(folder / file, 0, group)
vpnrouter_sites.ensure_folders(site['name'] for site in sites)
names = vpnrouter.issuing_cas(cfg)
caddy_ctx = dict(ctx, https=https, manager_port=vpnrouter.manager_port() if vpnrouter.MANAGER.exists() else 0,
pki_publish_dir=str(vpnrouter_pki.PUBLISH_DIR),
linuxmanager_sites.ensure_folders(site['name'] for site in sites)
names = linuxmanager.issuing_cas(cfg)
caddy_ctx = dict(ctx, https=https, manager_port=linuxmanager.manager_port(),
pki_publish_dir=str(linuxmanager_pki.PUBLISH_DIR),
pki_cas=[{'id': name, 'name': names[name]['name'].strip(),
'cert': str(CADDY_PKI_DIR / name / 'cert.pem'),
'key': str(CADDY_PKI_DIR / name / 'key.pem')} for name in authorities],
acme_cas=[name for name in authorities if https and name in ctx['pki_acme']],
webdav=vpnrouter.webdav_ready(ctx), webdav_loopback_port=vpnrouter.WEBDAV_LOOPBACK_PORT,
webdav=linuxmanager.webdav_ready(ctx), webdav_loopback_port=linuxmanager.WEBDAV_LOOPBACK_PORT,
sites=[{'name': site['name'], 'tls': site['tls'], 'ca': site['ca'], 'spa': site['spa'],
'index': site['index_file'], 'root': str(vpnrouter_sites.SITES_DIR / site['name']),
'index': site['index_file'], 'root': str(linuxmanager_sites.SITES_DIR / site['name']),
'addresses': [caddy_address(host, site['port'], site['tls']) for host in site['hosts']]
or [caddy_address('', site['port'], site['tls'])],
'cert': str(CADDY_SITES_DIR / site['name'] / 'cert.pem'),
'key': str(CADDY_SITES_DIR / site['name'] / 'key.pem')} for site in sites])
changed = vpnrouter.render(vpnrouter._jinja_env(), caddy_ctx, 'Caddyfile.j2', CADDYFILE, 0o640)
changed = linuxmanager.render(linuxmanager._jinja_env(), caddy_ctx, 'Caddyfile.j2', CADDYFILE, 0o640)
os.chown(CADDYFILE, 0, group)
if not unit_is_active(CADDY_UNIT):
run(['systemctl', 'enable', '--now', CADDY_UNIT], check=False)
@@ -809,33 +867,53 @@ def install_packages(feature):
def manage_webdav(ctx):
"""Run rclone's WebDAV server of the sites' folders while WebDAV is ready, and stop it when not; rclone is installed
when it is missing."""
if not vpnrouter.webdav_ready(ctx):
if not linuxmanager.webdav_ready(ctx):
if unit_is_active(WEBDAV_UNIT):
run(['systemctl', 'disable', '--now', WEBDAV_UNIT], check=False)
return
if shutil.which('rclone') is None:
install_packages('webdav')
return
vpnrouter_sites.ensure_folders([])
linuxmanager_sites.ensure_folders([])
run(['systemctl', 'enable', '--now', WEBDAV_UNIT], check=False)
def site_addresses(cfg, ctx):
"""The pairs of a host name and the server's address of dns_address of each enabled site with dns router: public,
wan.public_address; internal, the internal interface's; external, the external interface's first. A site whose
address is not known gets none, with a warning."""
addresses = {'public': cfg['wan']['public_address'].strip(), 'internal': ctx['int_addr'],
'external': next(iter(ctx['local_addr_list']), '')}
pairs = []
for site in ctx['sites']:
if site['dns'] != 'router':
continue
address = addresses[site['dns_address']]
if not address:
warn(f'site {site["name"]}: the server has no {site["dns_address"]} address, so its host names get no '
'records')
continue
pairs += [(host, address) for host in site['hosts']]
return pairs
def manage_dns(cfg, ctx):
"""Run PowerDNS, the primary name server of the domains of vpnrouter_dns, on the addresses of dns.listen while DNS
"""Run PowerDNS, the primary name server of the domains of linuxmanager_dns, on the addresses of dns.listen while DNS
is enabled, and stop it when not; its packages are installed when they are missing.
The settings, vpnrouter_dns.PDNS_CONF, are written before the packages are installed, so the server their
installation starts answers on those addresses only, beside systemd-resolved's stub on 127.0.0.53. The domains
with site records get the host names of the enabled sites, and keep those they have while the sites cannot be read.
The settings, linuxmanager_dns.PDNS_CONF, are written before the packages are installed, so the server their
installation starts answers on those addresses only, beside systemd-resolved's stub on 127.0.0.53. The host names
of the enabled sites with dns router get records of the server's address they choose (site_addresses), and the
domains keep those they have while the sites cannot be read.
"""
if not vpnrouter.dns_ready(ctx):
if unit_is_active(vpnrouter_dns.UNIT):
run(['systemctl', 'disable', '--now', vpnrouter_dns.UNIT], check=False)
vpnrouter.remove(vpnrouter_dns.PDNS_CONF)
if vpnrouter_dns.DATABASE_SECRET.is_file():
if not linuxmanager.dns_ready(ctx):
if unit_is_active(linuxmanager_dns.UNIT):
run(['systemctl', 'disable', '--now', linuxmanager_dns.UNIT], check=False)
linuxmanager.remove(linuxmanager_dns.PDNS_CONF)
if linuxmanager_dns.DATABASE_SECRET.is_file():
try:
vpnrouter_dns.apply(cfg, [])
except vpnrouter_dns.DatabaseError as exc:
linuxmanager_dns.apply(cfg, [])
except linuxmanager_dns.DatabaseError as exc:
warn(str(exc))
return
addresses = {'internal': [ctx['int_addr']], 'external': ctx['local_addr_list'],
@@ -845,23 +923,23 @@ def manage_dns(cfg, ctx):
warn(f'the name server does not run: the interfaces of dns.listen {ctx["dns_listen"]} have no address')
return
try:
vpnrouter_auth.provision_database(vpnrouter_dns.DATABASE, vpnrouter_dns.DATABASE_SECRET)
vpnrouter_dns.PDNS_CONF.parent.mkdir(mode=0o755, parents=True, exist_ok=True)
changed = write_if_changed(vpnrouter_dns.PDNS_CONF, vpnrouter_dns.config_text(addresses), 0o640)
if not vpnrouter_dns.PDNS_SERVER.exists() or not vpnrouter_dns.PDNS_SCHEMA.exists():
linuxmanager_auth.provision_database(linuxmanager_dns.DATABASE, linuxmanager_dns.DATABASE_SECRET)
linuxmanager_dns.PDNS_CONF.parent.mkdir(mode=0o755, parents=True, exist_ok=True)
changed = write_if_changed(linuxmanager_dns.PDNS_CONF, linuxmanager_dns.config_text(addresses), 0o640)
if not linuxmanager_dns.PDNS_SERVER.exists() or not linuxmanager_dns.PDNS_SCHEMA.exists():
install_packages('dns')
return
os.chown(vpnrouter_dns.PDNS_CONF, 0, grp.getgrnam(vpnrouter_dns.GROUP).gr_gid)
vpnrouter_dns.provision()
os.chown(linuxmanager_dns.PDNS_CONF, 0, grp.getgrnam(linuxmanager_dns.GROUP).gr_gid)
linuxmanager_dns.provision()
if ctx['sites'] is not None:
vpnrouter_dns.apply(cfg, [host for site in ctx['sites'] for host in site['hosts']])
except (OSError, KeyError, vpnrouter_dns.DatabaseError) as exc:
linuxmanager_dns.apply(cfg, site_addresses(cfg, ctx))
except (OSError, KeyError, linuxmanager_dns.DatabaseError) as exc:
warn(f'the name server cannot be set up: {exc}')
return
if not unit_is_active(vpnrouter_dns.UNIT):
run(['systemctl', 'enable', '--now', vpnrouter_dns.UNIT], check=False)
if not unit_is_active(linuxmanager_dns.UNIT):
run(['systemctl', 'enable', '--now', linuxmanager_dns.UNIT], check=False)
elif changed:
run(['systemctl', 'restart', vpnrouter_dns.UNIT], check=False)
run(['systemctl', 'restart', linuxmanager_dns.UNIT], check=False)
def manage_metrics(ctx):
@@ -870,8 +948,8 @@ def manage_metrics(ctx):
The service reads its settings itself, so it is restarted only to run the code of a new package version, which
every run of setup after an upgrade does.
"""
unit = 'vpn-router-metrics.service'
if not vpnrouter.metrics_ready(ctx):
unit = 'linux-manager-metrics.service'
if not linuxmanager.metrics_ready(ctx):
run(['systemctl', 'disable', '--now', unit], check=False)
return
run(['systemctl', 'enable', unit], check=False)
@@ -883,7 +961,7 @@ def manage_backup(cfg):
The schedule is a drop-in of the timer, BACKUP_SCHEDULE, which replaces the OnCalendar of the shipped unit.
"""
unit = 'vpn-router-backup.timer'
unit = 'linux-manager-backup.timer'
if not cfg['backup']['enabled']:
run(['systemctl', 'disable', '--now', unit], check=False)
if BACKUP_SCHEDULE.exists():
@@ -901,7 +979,7 @@ def manage_backup(cfg):
def run_platform_setup(cfg, ctx):
"""Let the platform module apply its own state, if it defines any."""
module = vpnrouter.platform_module(cfg)
module = linuxmanager.platform_module(cfg)
apply = getattr(module, 'apply', None)
if apply is None:
return
@@ -914,19 +992,19 @@ def run_platform_setup(cfg, ctx):
def refresh_docker_rules():
"""Apply the new settings to the Docker rules while Docker is running.
When Docker is not running, vpn-router-docker applies them after
When Docker is not running, linux-manager-docker applies them after
docker.service starts. The restart is waited for, so the rules are in place
when this setup ends; vpn-router-docker is therefore not ordered after
vpn-router-setup.
when this setup ends; linux-manager-docker is therefore not ordered after
linux-manager-setup.
"""
if unit_is_active('docker.service'):
run(['systemctl', 'restart', 'vpn-router-docker.service'], check=False)
run(['systemctl', 'restart', 'linux-manager-docker.service'], check=False)
def refresh_web(cfg, ctx):
"""Hand the web service the features and facts it shows.
The service reads the file at each request: the features of the router,
The service reads the file at each request: the features of the server,
the configuration without its secret options and the facts resolved from
the system, which it cannot work out with the standard library alone. The
file is readable by root only. The service follows a changed address or
@@ -942,10 +1020,12 @@ def refresh_web(cfg, ctx):
WEB_STATE.touch(mode=0o600)
WEB_STATE.chmod(0o600)
WEB_STATE.write_text(json.dumps({
'router': bool(ctx['router_enabled']),
'vpn': bool(ctx['vpn_enabled']),
'site_to_site': [connection['name'] for connection in ctx['connections'] if connection['ready']],
'p2s': bool(vpnrouter.p2s_ready(ctx)),
'wireguard': bool(vpnrouter.wg_ready(ctx)),
'openvpn': bool(vpnrouter.openvpn_ready(ctx)),
'p2s': bool(linuxmanager.p2s_ready(ctx)),
'wireguard': bool(linuxmanager.wg_ready(ctx)),
'openvpn': bool(linuxmanager.openvpn_ready(ctx)),
'configuration': public(cfg),
'network': {
'external_interface': ctx['wan_iface'],
@@ -954,7 +1034,7 @@ def refresh_web(cfg, ctx):
'internal_address': ctx['int_addr'],
'internal_gateway': ctx['int_gateway_ip'],
'local_cidrs': ctx['local_cidr_list'],
'client_dns': ctx['p2s_resolver_addrs'] if vpnrouter.RESOLVER_CONF.exists() else [],
'client_dns': ctx['p2s_resolver_addrs'] if linuxmanager.RESOLVER_CONF.exists() else [],
'site_to_site': [{
'name': connection['name'],
'local_identity': connection['local_id'],
@@ -978,14 +1058,14 @@ def progress(state, step, error=None):
def start_bootstrap():
"""Start the bootstrap services while no run has succeeded; a unit that is not installed is skipped."""
"""Start the bootstrap service while no run has succeeded."""
if not SETUP_DONE.exists():
for unit in BOOTSTRAP_UNITS:
run(['systemctl', 'start', '--no-block', unit], check=False)
def end_bootstrap(ctx):
"""After the first successful run: write SETUP_DONE, close the bootstrap ports and stop the bootstrap services."""
"""After the first successful run: write SETUP_DONE, close the bootstrap ports and stop the bootstrap service."""
if SETUP_DONE.exists():
return
SETUP_DONE.parent.mkdir(parents=True, exist_ok=True)
@@ -999,11 +1079,11 @@ def end_bootstrap(ctx):
def main():
start_bootstrap()
progress('running', 'configuration')
cfg = vpnrouter.load_config()
if vpnrouter.deferred(cfg):
print('vpn-router: platform is none, configuration deferred; '
f'set general.platform in {vpnrouter.CONFIG_FILE}')
progress('deferred', 'configuration', f'general.platform is none; set it in {vpnrouter.CONFIG_FILE}')
cfg = linuxmanager.load_config()
if linuxmanager.deferred(cfg):
print('linux-manager: platform is none, configuration deferred; '
f'set general.platform in {linuxmanager.CONFIG_FILE}')
progress('deferred', 'configuration', f'general.platform is none; set it in {linuxmanager.CONFIG_FILE}')
return 0
# The directory holds the PKI, which the configuration needs; its problem ends setup only after every other step.
@@ -1011,16 +1091,19 @@ def main():
progress('running', failed_step)
problem = manage_directory()
progress('running', 'configuration')
ctx, changed = vpnrouter.configure(cfg)
ctx, changed = linuxmanager.configure(cfg)
progress('running', 'kernel settings')
manage_sysctl(ctx)
progress('running', 'routes')
if vpnrouter.routing_ready(ctx):
if linuxmanager.routing_ready(ctx):
apply_routes(ctx)
if ctx['local_fqdn'] and any(connection['local_id_mode'] == 'public_ip' for connection in ctx['connections']):
progress('running', 'public address')
address = resolve_public_ip(ctx['local_fqdn'])
ctx, more = vpnrouter.configure(cfg, public_ip=address)
ctx, more = linuxmanager.configure(cfg, public_ip=address)
changed |= more
# The enabled sites while sites are on, None while they cannot be read.
@@ -1028,8 +1111,8 @@ def main():
if cfg['sites']['enabled']:
progress('running', 'web sites')
try:
ctx['sites'] = vpnrouter_sites.served()
except vpnrouter_sites.DatabaseError as exc:
ctx['sites'] = linuxmanager_sites.served()
except linuxmanager_sites.DatabaseError as exc:
warn(str(exc))
ctx['sites'] = None
@@ -1055,7 +1138,7 @@ def main():
manage_wireguard(ctx, changed)
if not problem:
failed_step = 'Router Manager admin'
failed_step = 'Ubuntu Linux Manager admin'
progress('running', failed_step)
problem = manage_admin()
if not problem:
@@ -1095,7 +1178,7 @@ def main():
manage_backup(cfg)
if problem:
progress('failed', failed_step, problem)
print(f'vpn-router: error: {problem}', file=sys.stderr)
print(f'linux-manager: error: {problem}', file=sys.stderr)
return 1
progress('done', 'done')
end_bootstrap(ctx)
@@ -1112,7 +1195,7 @@ def record_failure(error):
if __name__ == '__main__':
# A failing step ends with SystemExit carrying its message (vpnrouter.fail).
# A failing step ends with SystemExit carrying its message (linuxmanager.fail).
try:
code = main()
except SystemExit as exc:
+108
View File
@@ -0,0 +1,108 @@
#!/usr/bin/python3
"""Upgrade the server's package, linux-manager, or with --system every package of the system, recording the state and
the output.
Started by the Manager as the transient unit linux-manager-ui-upgrade, or with --system
linux-manager-ui-system-upgrade (systemd-run), so it runs outside the manager's unit: the
restarts of the manager and of the server's web service that the package's scripts make do not
stop it. Usage: upgrade [--system] <user>, the user who started it.
The state file, STATE_FILE or with --system SYSTEM_STATE_FILE, holds the state, running while it
runs, then succeeded or failed, with the user, the start and the end, the exit status of apt-get
and the installed versions of PACKAGES before and after; it is replaced by a rename, so a reader
never sees half of it. The log file, LOG_FILE or SYSTEM_LOG_FILE, holds the output of dpkg and
apt-get, from the start of the run. A dpkg run that was interrupted is finished first
(dpkg --configure -a). Then apt-get install --only-upgrade PACKAGES, or with --system apt-get
upgrade, which upgrades the installed packages without installing or removing any. apt-get waits
up to LOCK_SECONDS for the dpkg lock, which Ubuntu's apt-daily or unattended-upgrades may hold,
takes the installed version of a changed configuration file and asks nothing.
"""
import json
import os
import subprocess
import sys
import time
STATE_DIR = '/var/lib/linux-manager'
STATE_FILE = os.path.join(STATE_DIR, 'upgrade.json')
LOG_FILE = os.path.join(STATE_DIR, 'upgrade.log')
SYSTEM_STATE_FILE = os.path.join(STATE_DIR, 'system-upgrade.json')
SYSTEM_LOG_FILE = os.path.join(STATE_DIR, 'system-upgrade.log')
PACKAGES = ('linux-manager',)
#: Seconds apt-get waits for the dpkg lock.
LOCK_SECONDS = 300
#: The environment of dpkg and apt-get: no questions, the installed configuration files kept, and needrestart left out,
#: as the package's scripts restart their own services.
ENVIRONMENT = {
'DEBIAN_FRONTEND': 'noninteractive',
'UCF_FORCE_CONFFOLD': '1',
'APT_LISTCHANGES_FRONTEND': 'none',
'NEEDRESTART_SUSPEND': '1',
}
def now():
return time.strftime('%Y-%m-%dT%H:%M:%S%z')
def versions():
"""The installed version of each of PACKAGES, None for one that is not installed."""
found = {}
for package in PACKAGES:
result = subprocess.run(['dpkg-query', '-W', '-f=${Version}', package], capture_output=True, text=True)
found[package] = (result.stdout.strip() or None) if result.returncode == 0 else None
return found
def save(state, path):
"""Write the state to the file of path by a rename."""
temporary = path + '.new'
with open(temporary, 'w', encoding='utf-8') as handle:
json.dump(state, handle)
handle.write('\n')
os.chmod(temporary, 0o644)
os.replace(temporary, path)
def main(argv):
system = argv[:1] == ['--system']
if system:
argv = argv[1:]
if len(argv) != 1:
print('usage: upgrade [--system] <user>', file=sys.stderr)
return 2
state_file, log_file = (SYSTEM_STATE_FILE, SYSTEM_LOG_FILE) if system else (STATE_FILE, LOG_FILE)
os.makedirs(STATE_DIR, exist_ok=True)
state = {'state': 'running', 'user': argv[0], 'started': now(), 'finished': None, 'exit_status': None,
'before': versions(), 'after': None}
save(state, state_file)
environment = dict(os.environ, **ENVIRONMENT)
with open(log_file, 'w', encoding='utf-8') as log:
os.chmod(log_file, 0o644)
status = 1
for command in (
['dpkg', '--configure', '-a'],
['apt-get', '-y', '-q', '-o', f'DPkg::Lock::Timeout={LOCK_SECONDS}', '-o', 'Dpkg::Options::=--force-confdef',
'-o', 'Dpkg::Options::=--force-confold', *(['upgrade'] if system else ['install', '--only-upgrade', *PACKAGES])],
):
log.write(f'$ {" ".join(command)}\n')
log.flush()
try:
status = subprocess.run(command, stdin=subprocess.DEVNULL, stdout=log, stderr=subprocess.STDOUT, env=environment).returncode
except OSError as exc:
log.write(f'{exc}\n')
status = 1
if status != 0:
break
log.write(f'exit status {status}\n')
state.update(state='succeeded' if status == 0 else 'failed', finished=now(), exit_status=status, after=versions())
save(state, state_file)
return status
if __name__ == '__main__':
sys.exit(main(sys.argv[1:]))
@@ -1,7 +1,7 @@
#!/bin/sh
# Wait until libpq is in the dynamic linker cache, where psycopg looks it up.
# ExecStartPre= of vpn-router-setup.service, vpn-router-web.service and vpn-router-metrics.service;
# TimeoutStartSec= of the unit bounds the wait.
# ExecStartPre= of linux-manager-setup.service, linux-manager-web.service, linux-manager-metrics.service and
# linux-manager-ui.service; TimeoutStartSec= of the unit bounds the wait.
reported=
until ldconfig -p | grep -q '^[[:space:]]*libpq\.so\.5 '; do
+99
View File
@@ -0,0 +1,99 @@
#!/usr/bin/python3
"""Serve the status of the server on its internal address, its external address or both, as web.listen sets.
Run by linux-manager-web.service. Users are managed with linuxmanager_users, the module that
linux-manager-user uses too.
The paths it serves and the modules that answer them are described in the package linuxmanager_web.
The service first initializes, without listening: it makes the signing key, the OpenID Provider's key
(linuxmanager_oidc), the refresh tokens' database (linuxmanager_auth) and the web sites' database (linuxmanager_sites), retrying every INIT_RETRY_SECONDS until all are ready and
logging each failure. Then it listens. Its unit starts it only once wait-for-libpq has found libpq
in the linker cache, where psycopg looks it up.
The service needs nothing else to start and does not end on its own. Every DISCOVERY_SECONDS it
works out from /etc/linux-manager/linux-manager.json the addresses of the interfaces web.listen names,
internal, external or both, as general.mode directs, with web.port and web.enabled
(linuxmanager_net.web_listen), and runs a server on each (linuxmanager_serve.serve): it starts them,
moves them when an address or the port changes, and stops them while the service is disabled or
no address is known. Every request reads what it needs when it is made: the signing
key, the configuration, and the features of the server and the facts shown by /config from
/run/linux-manager/web.json, which linux-manager-setup writes. Before setup has written that file
the server has no features, and /config shows no configuration or interfaces.
"""
import os
import signal
import sys
import threading
import time
sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
import linuxmanager
import linuxmanager_auth
import linuxmanager_net
import linuxmanager_oidc
import linuxmanager_serve
import linuxmanager_sites
from linuxmanager_web.common import CONFIG_FILE, log, read_json
from linuxmanager_web.handler import Handler
from linuxmanager_web.shell import editor_listen
#: Seconds between two attempts of the initialization.
INIT_RETRY_SECONDS = 15
#: The port when web.port is not set.
DEFAULT_PORT = 7081
#: Seconds between two discoveries of where the service should listen.
DISCOVERY_SECONDS = 30
def initialize():
"""Make the signing key, the OpenID Provider's key, the refresh tokens' database and the web sites' database,
retrying every INIT_RETRY_SECONDS until all are ready."""
reported = None
while True:
phase = 'reading the configuration'
try:
cfg = linuxmanager.load_config()
phase = 'making the signing key'
linuxmanager_auth.ensure_key()
phase = 'making the OpenID Provider key'
linuxmanager_oidc.ensure_key()
phase = 'creating the token database'
linuxmanager_auth.provision(cfg)
phase = 'connecting to the token database'
with linuxmanager_auth.database(cfg):
pass
phase = 'creating the database of the web sites'
linuxmanager_sites.provision()
phase = 'connecting to the database of the web sites'
with linuxmanager_sites.database():
pass
except (OSError, SystemExit, linuxmanager_auth.DatabaseError, linuxmanager_sites.DatabaseError) as exc:
error = f'{phase} failed: {exc.code if isinstance(exc, SystemExit) else exc}'
if error != reported:
log(f'{error}; retrying every {INIT_RETRY_SECONDS} seconds')
reported = error
time.sleep(INIT_RETRY_SECONDS)
continue
log('ready')
return
def main():
signal.signal(signal.SIGTERM, lambda *_: sys.exit(0))
initialize()
threading.Thread(target=editor_listen, name='editor socket', daemon=True).start()
try:
linuxmanager_serve.serve(lambda: linuxmanager_net.web_listen(read_json(CONFIG_FILE), DEFAULT_PORT), Handler, log,
DISCOVERY_SECONDS)
except KeyboardInterrupt:
pass
return 0
if __name__ == '__main__':
sys.exit(main())
@@ -1,10 +1,10 @@
#!/usr/bin/python3
"""The auth proxy of rclone's WebDAV server of the web sites' folders (vpn-router-webdav.service).
"""The auth proxy of rclone's WebDAV server of the web sites' folders (linux-manager-webdav.service).
rclone runs it for each new user and password with {"user", "pass"} on standard input. The user signs in with the
password of the router's directory, followed by the TOTP code for an enrolled user. A member of
vpnrouter_users.ADMIN_GROUP sees the folders of all sites, any other user those of the sites whose editors group
the user is a member of (vpnrouter_sites.editable), each as a folder named after its site. The answer on standard output
password of the server's directory, followed by the TOTP code for an enrolled user. A member of
linuxmanager_users.ADMIN_GROUP sees the folders of all sites, any other user those of the sites whose editors group
the user is a member of (linuxmanager_sites.editable), each as a folder named after its site. The answer on standard output
is rclone's combine backend of those folders; a failed sign-in, a user with no site or any failure is a message on
standard error and the exit status 1, which rclone refuses the user with.
"""
@@ -17,32 +17,32 @@ sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
import ldap
import vpnrouter_ldap
import vpnrouter_sites
import vpnrouter_users
import linuxmanager_ldap
import linuxmanager_sites
import linuxmanager_users
def main():
try:
request = json.load(sys.stdin)
name, password = request['user'], request['pass']
vpnrouter_users.login(name, password, None)
connection = vpnrouter_ldap.manager()
linuxmanager_users.login(name, password, None)
connection = linuxmanager_ldap.manager()
try:
groups = vpnrouter_users.groups_of(connection, name)
groups = linuxmanager_users.groups_of(connection, name)
finally:
connection.unbind_s()
if vpnrouter_users.ADMIN_GROUP in groups:
sites = [site['name'] for site in vpnrouter_sites.listing()]
if linuxmanager_users.ADMIN_GROUP in groups:
sites = [site['name'] for site in linuxmanager_sites.listing()]
else:
sites = vpnrouter_sites.editable(groups)
except (ValueError, KeyError, TypeError, OSError, ldap.LDAPError, vpnrouter_users.Failure,
vpnrouter_sites.DatabaseError) as exc:
sites = linuxmanager_sites.editable(groups)
except (ValueError, KeyError, TypeError, OSError, ldap.LDAPError, linuxmanager_users.Failure,
linuxmanager_sites.DatabaseError) as exc:
sys.exit(f'webdav-auth: refused: {exc}')
if not sites:
sys.exit(f'webdav-auth: refused: {name} may change no site')
vpnrouter_sites.ensure_folders(sites)
upstreams = ' '.join(f'{site}={vpnrouter_sites.SITES_DIR / site}' for site in sites)
linuxmanager_sites.ensure_folders(sites)
upstreams = ' '.join(f'{site}={linuxmanager_sites.SITES_DIR / site}' for site in sites)
json.dump({'type': 'combine', '_root': '', 'upstreams': upstreams}, sys.stdout)
return 0
@@ -0,0 +1,10 @@
# Caddy is the HTTPS proxy of linux-manager: it reads the configuration linux-manager-setup writes, and
# does not start until that exists.
[Unit]
ConditionPathExists=/etc/caddy/linux-manager.Caddyfile
[Service]
ExecStart=
ExecStart=/usr/bin/caddy run --environ --adapter caddyfile --config /etc/caddy/linux-manager.Caddyfile
ExecReload=
ExecReload=/usr/bin/caddy reload --adapter caddyfile --config /etc/caddy/linux-manager.Caddyfile --force
@@ -1,10 +0,0 @@
# Caddy is the HTTPS proxy of vpn-router: it reads the configuration vpn-router-setup writes, and
# does not start until that exists.
[Unit]
ConditionPathExists=/etc/caddy/vpn-router.Caddyfile
[Service]
ExecStart=
ExecStart=/usr/bin/caddy run --environ --adapter caddyfile --config /etc/caddy/vpn-router.Caddyfile
ExecReload=
ExecReload=/usr/bin/caddy reload --adapter caddyfile --config /etc/caddy/vpn-router.Caddyfile --force
@@ -1,55 +0,0 @@
#!/usr/bin/python3
"""Serve diagnostics of the router's first setup on PORT, with the Python standard library alone.
Run by vpn-router-bootstrap.service, which vpn-router-setup starts until a setup has succeeded and
then stops. Every request on any address answers, as plain text, the output of COMMANDS run at that
moment: the state of the last setup run, the state of the services, the addresses, and the end of
the logs.
"""
import subprocess
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
PORT = 17081
#: The title of each section and its command.
COMMANDS = (
('Setup state', ['cat', '/run/vpn-router/setup.json']),
('Services', ['systemctl', 'is-active', 'vpn-router-setup', 'vpn-router-web', 'slapd', 'postgresql',
'strongswan', 'openvpn-server@vpn-router', 'wg-quick@wg0']),
('Failed units', ['systemctl', '--failed', '--no-legend', '--plain']),
('Addresses', ['ip', '-brief', 'address']),
('Routes', ['ip', 'route']),
('cloud-init', ['cloud-init', 'status', '--long']),
('vpn-router-setup log', ['journalctl', '--unit', 'vpn-router-setup', '--lines', '60', '--no-pager', '--output', 'short-iso']),
('vpn-router-web log', ['journalctl', '--unit', 'vpn-router-web', '--lines', '30', '--no-pager', '--output', 'short-iso']),
)
def report():
"""The output of every command of COMMANDS under its title."""
sections = []
for title, command in COMMANDS:
try:
result = subprocess.run(command, capture_output=True, text=True, timeout=10)
text = (result.stdout + result.stderr).rstrip()
except (OSError, subprocess.TimeoutExpired) as exc:
text = f'cannot run {command[0]}: {exc}'
sections.append(f'== {title} ==\n{text}\n')
return '\n'.join(sections)
class Handler(BaseHTTPRequestHandler):
def do_GET(self):
body = report().encode()
self.send_response(200)
self.send_header('Content-Type', 'text/plain; charset=utf-8')
self.send_header('Content-Length', str(len(body)))
self.send_header('Cache-Control', 'no-store')
self.send_header('Refresh', '10')
self.end_headers()
self.wfile.write(body)
if __name__ == '__main__':
ThreadingHTTPServer(('', PORT), Handler).serve_forever()
@@ -1,584 +0,0 @@
"""The router's OpenID Provider, for the web service, with python3-oauthlib and python3-jwt.
While oidc.enabled is true, the web service answers OpenID Connect at the issuer vpnrouter.oidc_issuer, its HTTPS site
that Caddy serves; the web service answers it only on the loopback address, where Caddy passes the requests. The users
of the router's directory sign in, with the password and the TOTP code or with a passkey (vpnrouter_passkeys), to its
clients, the service accounts with redirect URIs (vpnrouter_users.client_settings): a client's ID is its name and its
secret its password, sent with HTTP Basic authentication or in the body of the token request. A client with allowed
groups admits only their members; a service account never signs in as a user.
GET /.well-known/openid-configuration the discovery document
GET /oidc/jwks the public key the ID tokens are signed with
GET, POST /oidc/authorize the authorization code flow, PKCE with S256 required; the sign-in page while
the browser has no session, or with prompt=login or an older one than max_age
POST /oidc/login the sign-in page's form: the name, the password and the code, or a passkey
POST /oidc/passkey begin a sign-in with a passkey: {"id", "options"}
POST /oidc/token exchange a code or a refresh token for tokens
GET, POST /oidc/userinfo the claims of the user an access token was issued for
GET /oidc/logout end the session, then go to a post-logout redirect URI of the client
ID tokens are RS256 JWTs signed with KEY, an RSA key the web service makes on its first run on the router, named by
the RFC 7638 thumbprint of its public key. sub is the entryUUID of the user's entry, so a user added again under the
same name is another subject. The scopes: openid, required; profile, preferred_username and the names; email; groups,
the names of the user's groups; their claims are in the ID token and the userinfo answer. A code is valid
CODE_SECONDS, an access token and an ID token ACCESS_SECONDS, a refresh token REFRESH_SECONDS, used once. A session,
the cookie COOKIE of the issuer's /oidc/ path, lasts SESSION_SECONDS. Codes, tokens and sessions are kept in the
database of vpnrouter_auth as their SHA-256 (vpnrouter_auth.OIDC_SQL); vpnrouter_auth.revoke removes a user's. A
client is registered by an administrator, so a user is not asked for consent.
"""
import base64
import collections
import contextlib
import datetime
import hashlib
import http.cookies
import json
import pathlib
import secrets
import time
import types
import urllib.parse
import jinja2
import jwt
import ldap
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric import rsa
from oauthlib.common import add_params_to_uri
from oauthlib.oauth2 import AccessDeniedError, FatalClientError, InvalidRequestError, OAuth2Error
from oauthlib.openid import RequestValidator, Server
from oauthlib.openid.connect.core.exceptions import LoginRequired
import vpnrouter
import vpnrouter_auth
import vpnrouter_ldap
import vpnrouter_passkeys
import vpnrouter_users
KEY = pathlib.Path('/etc/vpn-router/oidc.key')
KEY_BITS = 3072
CODE_SECONDS = 60
ACCESS_SECONDS = 300
REFRESH_SECONDS = 8 * 3600
SESSION_SECONDS = 8 * 3600
COOKIE = 'vpn_router_oidc'
#: The claims of each scope but openid, which gives sub.
SCOPE_CLAIMS = {
'profile': ('preferred_username', 'name', 'given_name', 'family_name'),
'email': ('email',),
'groups': ('groups',),
}
SCOPES = ('openid', *SCOPE_CLAIMS)
#: The authentication methods (RFC 8176) of a sign-in with a password, with a password and a TOTP code, and with a
#: passkey, which verifies the user.
PASSWORD_AMR = ['pwd']
OTP_AMR = ['pwd', 'otp']
PASSKEY_AMR = ['hwk', 'user']
TEMPLATE = 'oidc-page.html.j2'
#: The headers of the pages: no framing, and nothing loaded but from the page itself and the router.
PAGE_HEADERS = [('Content-Security-Policy', "default-src 'none'; script-src 'unsafe-inline'; style-src 'unsafe-inline'; "
"connect-src 'self'; frame-ancestors 'none'")]
#: The headers of oauthlib's answers that the web service sets itself.
OWN_HEADERS = ('content-type', 'cache-control', 'content-length')
#: A user's sign-in at the OpenID Provider: the user name, when, a datetime, and how (amr).
Session = collections.namedtuple('Session', 'name auth_time amr')
_pages = jinja2.Environment(loader=jinja2.FileSystemLoader(str(vpnrouter.TEMPLATE_DIR)), autoescape=True,
undefined=jinja2.StrictUndefined)
def ensure_key():
"""Make the signing key when it is missing."""
if KEY.is_file() and KEY.stat().st_size:
return
key = rsa.generate_private_key(public_exponent=65537, key_size=KEY_BITS)
KEY.parent.mkdir(parents=True, exist_ok=True)
KEY.touch(mode=0o600)
KEY.chmod(0o600)
KEY.write_bytes(key.private_bytes(serialization.Encoding.PEM, serialization.PrivateFormat.PKCS8,
serialization.NoEncryption()))
def private_key():
return serialization.load_pem_private_key(KEY.read_bytes(), None)
def public_jwk(key):
"""The JWK of the key's public key, its kid the RFC 7638 thumbprint."""
numbers = key.public_key().public_numbers()
jwk = {'e': numbers.e, 'kty': 'RSA', 'n': numbers.n}
jwk = {name: vpnrouter_auth.b64encode(value.to_bytes((value.bit_length() + 7) // 8, 'big')) if isinstance(value, int)
else value for name, value in jwk.items()}
thumbprint = hashlib.sha256(json.dumps(jwk, sort_keys=True, separators=(',', ':')).encode()).digest()
return {**jwk, 'kid': vpnrouter_auth.b64encode(thumbprint), 'use': 'sig', 'alg': 'RS256'}
def sign(claims):
key = private_key()
return jwt.encode(claims, key, algorithm='RS256', headers={'kid': public_jwk(key)['kid']})
def discovery(issuer):
"""The discovery document of OpenID Connect Discovery 1.0."""
return {
'issuer': issuer,
'authorization_endpoint': f'{issuer}/oidc/authorize',
'token_endpoint': f'{issuer}/oidc/token',
'userinfo_endpoint': f'{issuer}/oidc/userinfo',
'jwks_uri': f'{issuer}/oidc/jwks',
'end_session_endpoint': f'{issuer}/oidc/logout',
'scopes_supported': list(SCOPES),
'response_types_supported': ['code'],
'response_modes_supported': ['query'],
'grant_types_supported': ['authorization_code', 'refresh_token'],
'subject_types_supported': ['public'],
'id_token_signing_alg_values_supported': ['RS256'],
'token_endpoint_auth_methods_supported': ['client_secret_basic', 'client_secret_post'],
'code_challenge_methods_supported': ['S256'],
'claims_supported': ['iss', 'sub', 'aud', 'exp', 'iat', 'auth_time', 'nonce', 'amr', 'at_hash',
*(claim for claims in SCOPE_CLAIMS.values() for claim in claims)],
}
@contextlib.contextmanager
def directory():
"""A connection to the router's directory as the user manager account, for reading."""
connection = vpnrouter_ldap.manager()
try:
yield connection
finally:
connection.unbind_s()
def client_of(client_id):
"""The client of the ID, a namespace of its vpnrouter_users.summary with client_id, when it is an enabled service
account with redirect URIs; else None."""
if not isinstance(client_id, str):
return None
with directory() as connection:
try:
user = vpnrouter_users.summary(vpnrouter_users.entry(connection, client_id))
except vpnrouter_users.Failure:
return None
if not user['service'] or not user['enabled'] or not user['redirect_uris']:
return None
return types.SimpleNamespace(client_id=client_id, **user)
def subject(name, client):
"""The claims of every scope of the user, with sub, when the user may sign in to the client: an enabled, unlocked
user who is not a service account, a member of one of the client's allowed groups when it has any; else None."""
with directory() as connection:
try:
attributes = vpnrouter_users.entry(connection, name)
except vpnrouter_users.Failure:
return None
user = vpnrouter_users.summary(attributes)
groups = sorted(vpnrouter_users.groups_of(connection, name))
uuid = connection.search_s(vpnrouter_users.dn_of(name), ldap.SCOPE_BASE, attrlist=['entryUUID'])[0][1]['entryUUID']
if user['service'] or not user['enabled'] or user['locked']:
return None
if client.allowed_groups and not set(client.allowed_groups) & set(groups):
return None
full = ' '.join(part for part in (user['first_name'], user['last_name']) if part)
claims = {'sub': uuid[0].decode(), 'preferred_username': name, 'name': full or None, 'given_name': user['first_name'],
'family_name': user['last_name'], 'email': user['email'], 'groups': groups}
return {claim: value for claim, value in claims.items() if value is not None}
def claims_of(person, scopes):
"""The claims of the scopes of a subject's claims (subject)."""
return {'sub': person['sub'], **{claim: person[claim] for scope in scopes for claim in SCOPE_CLAIMS.get(scope, ())
if claim in person}}
def cookie_value(cookies):
"""The value of the session cookie in a Cookie header, or ''."""
try:
morsel = http.cookies.SimpleCookie(cookies or '').get(COOKIE)
except http.cookies.CookieError:
return ''
return morsel.value if morsel is not None else ''
def session_of(cfg, cookies):
"""The unexpired Session the session cookie in a Cookie header names, or None."""
value = cookie_value(cookies)
if not value:
return None
with vpnrouter_auth.database(cfg) as connection:
row = connection.execute('SELECT * FROM oidc_session_get(%s)', (vpnrouter_auth.digest(value),)).fetchone()
return Session(*row) if row else None
class Validator(RequestValidator):
"""The checks and the storage of oauthlib's server for the router's clients, users and database.
Only the authorization code grant with PKCE S256 and the refresh token grant are allowed. The values of a code, a
token or a session found are kept on the request as oidc_code, oidc_scopes and oidc_person.
"""
def __init__(self, cfg):
super().__init__()
self.cfg = cfg
def fetch(self, sql, values):
with vpnrouter_auth.database(self.cfg) as connection:
return connection.execute(sql, values).fetchone()
def call(self, *statements):
with vpnrouter_auth.database(self.cfg) as connection:
for sql, values in statements:
connection.execute(sql, values)
def session(self, request):
return session_of(self.cfg, request.headers.get('Cookie'))
# Clients.
def validate_client_id(self, client_id, request, *args, **kwargs):
request.client = client_of(client_id)
return request.client is not None
def validate_redirect_uri(self, client_id, redirect_uri, request, *args, **kwargs):
return redirect_uri in request.client.redirect_uris
def get_default_redirect_uri(self, client_id, request, *args, **kwargs):
return None
def validate_response_type(self, client_id, response_type, client, request, *args, **kwargs):
return response_type == 'code'
def validate_scopes(self, client_id, scopes, client, request, *args, **kwargs):
return 'openid' in scopes and set(scopes) <= set(SCOPES)
def get_default_scopes(self, client_id, request, *args, **kwargs):
return ['openid']
def is_pkce_required(self, client_id, request):
return True
def authenticate_client(self, request, *args, **kwargs):
"""Check the client's secret, the password of its service account, given with HTTP Basic authentication, its
ID and secret form-encoded (RFC 6749 2.3.1), or in the body."""
name, secret = request.client_id, request.client_secret
header = request.headers.get('Authorization', '')
if header[:6].lower() == 'basic ':
try:
name, _, secret = base64.b64decode(header[6:].strip(), validate=True).decode().partition(':')
except (ValueError, UnicodeDecodeError):
return False
name, secret = urllib.parse.unquote_plus(name), urllib.parse.unquote_plus(secret)
client = client_of(name)
if client is None:
return False
try:
vpnrouter_users.login(name, secret, None)
except vpnrouter_users.Failure:
return False
request.client = client
return True
def authenticate_client_id(self, client_id, request, *args, **kwargs):
return False
def validate_grant_type(self, client_id, grant_type, client, request, *args, **kwargs):
return grant_type in ('authorization_code', 'refresh_token')
def validate_user(self, username, password, client, request, *args, **kwargs):
return False
# Codes.
def save_authorization_code(self, client_id, code, request, *args, **kwargs):
self.call(('CALL oidc_code_save(%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)',
(vpnrouter_auth.digest(code['code']), client_id, request.user, request.redirect_uri, request.scopes,
request.nonce, request.code_challenge, request.code_challenge_method, request.auth_time, request.amr,
CODE_SECONDS)))
def code(self, code):
row = self.fetch('SELECT * FROM oidc_code_get(%s)', (vpnrouter_auth.digest(code),)) if isinstance(code, str) else None
return dict(zip(('client_id', 'name', 'redirect_uri', 'scopes', 'nonce', 'code_challenge', 'code_challenge_method',
'auth_time', 'amr'), row)) if row else None
def get_authorization_code_scopes(self, client_id, code, redirect_uri, request):
found = self.code(code)
return found['scopes'] if found else []
def validate_code(self, client_id, code, client, request, *args, **kwargs):
found = self.code(code)
if found is None or found['client_id'] != client_id:
return False
person = subject(found['name'], client)
if person is None:
return False
request.user, request.scopes, request.auth_time, request.amr = found['name'], found['scopes'], found['auth_time'], found['amr']
request.oidc_code, request.oidc_person = found, person
return True
def get_code_challenge(self, code, request):
return request.oidc_code['code_challenge']
def get_code_challenge_method(self, code, request):
return request.oidc_code['code_challenge_method']
def confirm_redirect_uri(self, client_id, code, redirect_uri, client, request, *args, **kwargs):
return redirect_uri == request.oidc_code['redirect_uri']
def get_authorization_code_nonce(self, client_id, code, redirect_uri, request):
return request.oidc_code['nonce']
def invalidate_authorization_code(self, client_id, code, request, *args, **kwargs):
self.call(('CALL oidc_code_delete(%s)', (vpnrouter_auth.digest(code),)))
# Tokens.
def token(self, token, kind):
row = self.fetch('SELECT * FROM oidc_token_get(%s, %s)', (vpnrouter_auth.digest(token), kind)) \
if isinstance(token, str) and token else None
return dict(zip(('client_id', 'name', 'scopes', 'auth_time', 'amr'), row)) if row else None
def save_bearer_token(self, token, request, *args, **kwargs):
"""Keep the access token and the refresh token; a refresh token that was exchanged is used up."""
values = (request.client.client_id, request.user, request.scopes, request.auth_time, request.amr)
statements = [('CALL oidc_token_save(%s, %s, %s, %s, %s, %s, %s, %s)',
(vpnrouter_auth.digest(token['access_token']), 'access', *values, ACCESS_SECONDS))]
if token.get('refresh_token'):
statements.append(('CALL oidc_token_save(%s, %s, %s, %s, %s, %s, %s, %s)',
(vpnrouter_auth.digest(token['refresh_token']), 'refresh', *values, REFRESH_SECONDS)))
if request.grant_type == 'refresh_token':
statements.append(('CALL oidc_token_delete(%s)', (vpnrouter_auth.digest(request.refresh_token),)))
self.call(*statements)
def validate_refresh_token(self, refresh_token, client, request, *args, **kwargs):
found = self.token(refresh_token, 'refresh')
if found is None or found['client_id'] != client.client_id:
return False
person = subject(found['name'], client)
if person is None:
return False
request.user, request.auth_time, request.amr = found['name'], found['auth_time'], found['amr']
request.oidc_scopes, request.oidc_person = found['scopes'], person
return True
def get_original_scopes(self, refresh_token, request, *args, **kwargs):
return request.oidc_scopes
def validate_bearer_token(self, token, scopes, request):
found = self.token(token, 'access')
if found is None or not set(scopes) <= set(found['scopes']):
return False
client = client_of(found['client_id'])
person = subject(found['name'], client) if client else None
if person is None:
return False
request.client, request.user, request.oidc_scopes, request.oidc_person = client, found['name'], found['scopes'], person
return True
def get_userinfo_claims(self, request):
return claims_of(request.oidc_person, request.oidc_scopes)
# OpenID Connect.
def finalize_id_token(self, id_token, token, token_handler, request):
now = int(time.time())
id_token.update(claims_of(request.oidc_person, request.scopes))
id_token.update({'iss': vpnrouter.oidc_issuer(self.cfg), 'exp': now + ACCESS_SECONDS,
'auth_time': int(request.auth_time.timestamp()), 'amr': request.amr})
return sign(id_token)
def validate_silent_login(self, request):
session = self.session(request)
return session is not None and subject(session.name, request.client) is not None
def validate_silent_authorization(self, request):
return True
def validate_user_match(self, id_token_hint, scopes, claims, request):
"""Whether the user of the session, when there is one, is the subject of an ID token the router signed."""
session = self.session(request) if id_token_hint else None
if session is None:
return True
try:
hint = jwt.decode(id_token_hint, private_key().public_key(), algorithms=['RS256'],
options={'verify_exp': False, 'verify_aud': False})
except jwt.InvalidTokenError:
return False
person = subject(session.name, request.client)
return person is not None and hint.get('sub') == person['sub']
def page(status, cfg, title, message='', error='', form=False, request='', name='', headers=()):
"""An HTML page of TEMPLATE: a message, an error, and with form the sign-in form for the authorization request."""
body = _pages.get_template(TEMPLATE).render(title=title, message=message, error=error, form=form, request=request,
name=name, issuer=vpnrouter.oidc_issuer(cfg))
return status, 'text/html; charset=utf-8', body.encode(), [*PAGE_HEADERS, *headers]
def document(status, value, headers=()):
return status, 'application/json', json.dumps(value, indent=2).encode() + b'\n', list(headers)
def relayed(headers, body, status):
"""An answer of oauthlib's server, without the headers the web service sets."""
kind = next((value for name, value in headers.items() if name.lower() == 'content-type'), 'application/json')
return status, kind, (body or '').encode(), [(name, value) for name, value in headers.items()
if name.lower() not in OWN_HEADERS]
def redirect(location, headers=()):
return 302, 'text/plain; charset=utf-8', b'', [('Location', location), *headers]
def cookie(value, seconds):
return 'Set-Cookie', f'{COOKIE}={value}; Path=/oidc/; Max-Age={seconds}; Secure; HttpOnly; SameSite=Lax'
def authorize(server, cfg, params, headers, fresh=None, name='', error=''):
"""Answer an authorization request, the query or the form params: the sign-in page while the browser has no
session, or with prompt=login or max_age older than the session, else a redirect to the client with a code.
fresh is the session of a sign-in just made, which counts whatever prompt and max_age say; the redirect then sets
its cookie, fresh.cookie. error is why a sign-in failed: the sign-in page shows it, filled with the name.
"""
uri = f'{vpnrouter.oidc_issuer(cfg)}/oidc/authorize?{params}'
try:
scopes, info = server.validate_authorization_request(uri, 'GET', None, headers)
except FatalClientError as error:
return page(error.status_code, cfg, 'The sign-in request is refused', error=error.description or error.error)
except OAuth2Error as error:
if not error.redirect_uri:
return page(error.status_code, cfg, 'The sign-in request is refused', error=error.description or error.error)
return redirect(error.in_uri(error.redirect_uri))
request = info['request']
title = f'Sign in to {request.client.display_name or request.client.client_id}'
if info.get('code_challenge_method') != 'S256':
return redirect(InvalidRequestError(description='PKCE with code_challenge_method S256 is required',
request=request).in_uri(request.redirect_uri))
if error:
return page(401, cfg, title, error=error, form=True, request=params, name=name)
session = fresh.session if fresh else session_of(cfg, headers.get('Cookie'))
max_age = getattr(request, 'max_age', None)
stale = session is not None and isinstance(max_age, str) and max_age.isdigit() and \
time.time() - session.auth_time.timestamp() > int(max_age)
prompt = info.get('prompt') or set()
if session is None or (not fresh and ('login' in prompt or stale)):
if 'none' in prompt:
return redirect(LoginRequired(request=request).in_uri(request.redirect_uri))
return page(200, cfg, title, form=True, request=params, name=name)
if subject(session.name, request.client) is None:
if fresh:
return redirect(AccessDeniedError(description=f'{session.name} may not sign in to this client',
request=request).in_uri(request.redirect_uri), [fresh.cookie])
return page(403, cfg, title, error=f'{session.name} may not sign in here; sign in as another user.', form=True,
request=params)
answer, _, _ = server.create_authorization_response(uri, 'GET', None, headers, scopes, credentials={
'user': session.name, 'auth_time': session.auth_time, 'amr': session.amr})
return redirect(answer['Location'], [fresh.cookie] if fresh else [])
def login(server, cfg, form, headers):
"""Sign a user in with the sign-in page's form, start the session and go on with its authorization request."""
params = form.get('request', '')
name = form.get('name', '').strip()
try:
if form.get('passkey_id'):
with vpnrouter_users.session() as connection:
name = vpnrouter_passkeys.login_complete(connection, form['passkey_id'], json.loads(form.get('credential', '')))
amr = PASSKEY_AMR
else:
code = form.get('code', '').strip()
vpnrouter_users.login(name, form.get('password', ''), code or None)
with directory() as connection:
if vpnrouter_users.service(vpnrouter_users.entry(connection, name)):
raise vpnrouter_users.Failure('the name, the password or the code is wrong', 5)
amr = OTP_AMR if code else PASSWORD_AMR
except vpnrouter_users.Failure as failure:
return authorize(server, cfg, params, headers, name=name, error=str(failure))
except ValueError:
return authorize(server, cfg, params, headers, name=name, error='the passkey is not accepted')
value = secrets.token_urlsafe(32)
with vpnrouter_auth.database(cfg) as connection:
connection.execute('CALL oidc_session_save(%s, %s, %s, %s)', (vpnrouter_auth.digest(value), name, amr, SESSION_SECONDS))
fresh = types.SimpleNamespace(session=Session(name, datetime.datetime.now(datetime.timezone.utc), amr),
cookie=cookie(value, SESSION_SECONDS))
return authorize(server, cfg, params, headers, fresh=fresh)
def logout(cfg, query, headers):
"""End the session the cookie names, then go to post_logout_redirect_uri, with state, when it is a post-logout
redirect URI of the client that client_id or the audience of id_token_hint names; else show that the user is
signed out."""
values = dict(urllib.parse.parse_qsl(query))
value = cookie_value(headers.get('Cookie'))
if value:
with vpnrouter_auth.database(cfg) as connection:
connection.execute('CALL oidc_session_delete(%s)', (vpnrouter_auth.digest(value),))
client_id = values.get('client_id')
if not client_id and values.get('id_token_hint'):
try:
audience = jwt.decode(values['id_token_hint'], private_key().public_key(), algorithms=['RS256'],
options={'verify_exp': False, 'verify_aud': False}).get('aud')
except jwt.InvalidTokenError:
audience = None
client_id = audience[0] if isinstance(audience, list) and audience else audience
client = client_of(client_id)
target = values.get('post_logout_redirect_uri')
expired = cookie('', 0)
if client is not None and target in client.post_logout_redirect_uris:
return redirect(add_params_to_uri(target, [('state', values['state'])]) if values.get('state') else target, [expired])
return page(200, cfg, 'Signed out', message='You are signed out of the router.', headers=[expired])
def answer(method, target, headers, body):
"""Answer a request of the OpenID Provider: the HTTP status, the content type, the body and [(header, value)]; 404
while oidc.enabled is false. target is the path with the query, headers a mapping of the request's headers and
body its bytes; HEAD is answered as GET."""
path, _, query = target.partition('?')
method = 'GET' if method == 'HEAD' else method
try:
cfg = vpnrouter.load_config()
if not cfg['oidc']['enabled']:
return 404, 'text/plain; charset=utf-8', b'Not found\n', []
issuer = vpnrouter.oidc_issuer(cfg)
headers = dict(headers.items())
text = body.decode('utf-8', 'replace') if body else ''
server = Server(Validator(cfg), token_expires_in=ACCESS_SECONDS)
if (method, path) == ('GET', '/.well-known/openid-configuration'):
return document(200, discovery(issuer))
if (method, path) == ('GET', '/oidc/jwks'):
return document(200, {'keys': [public_jwk(private_key())]})
if path == '/oidc/authorize' and method in ('GET', 'POST'):
return authorize(server, cfg, query if method == 'GET' else text, headers)
if (method, path) == ('POST', '/oidc/login'):
return login(server, cfg, dict(urllib.parse.parse_qsl(text)), headers)
if (method, path) == ('POST', '/oidc/passkey'):
return document(200, vpnrouter_passkeys.login_begin())
if (method, path) == ('POST', '/oidc/token'):
return relayed(*server.create_token_response(f'{issuer}{path}', 'POST', text, headers))
if path == '/oidc/userinfo' and method in ('GET', 'POST'):
return relayed(*server.create_userinfo_response(f'{issuer}{target}', method, text or None, headers))
if (method, path) == ('GET', '/oidc/logout'):
return logout(cfg, query, headers)
return 404, 'text/plain; charset=utf-8', b'Not found\n', []
except OAuth2Error as error:
return error.status_code, 'application/json', error.json.encode(), list(error.headers.items())
except vpnrouter_users.Failure as failure:
return document(400 if failure.status == 2 else 500, {'error': str(failure)})
except (ldap.LDAPError, OSError) as exc:
return document(500, {'error': f'the OpenID Provider failed: {exc}'})
except vpnrouter_auth.DatabaseError as exc:
return document(500, {'error': str(exc)})
except SystemExit as exc:
return document(500, {'error': str(exc.code)})
except ValueError as exc:
return document(400, {'error': 'invalid_request', 'error_description': str(exc)})
Loaded 100 of 272 files, more files were not shown because too many files have changed in this diff. Show more