slawek/linux-cloud-router
3
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
5303779559211c422370f7b8123242ab58ee8995
linux-cloud-router/debian-package/src/usr/lib/vpn-router/web
T

2075 lines
110 KiB
Python
Raw Blame History

#!/usr/bin/python3
"""Serve the status of the router on its internal address, its external address or both, as web.listen sets.
Run by vpn-router-web.service. Users are managed with vpnrouter_users, the module that
vpn-router-user uses too.
Pages:
/ the status for people: plain text for curl and wget, a styled page
(Tailwind CSS from a CDN) for a browser
/status the status as JSON
/status/full
the whole status as JSON, whatever web.status_content says, for the Router Manager
/config the configuration without its secrets, and what the router resolved and
runs, with the installed Caddy and where it comes from, as JSON
/users the users in the router's directory, whether or not a P2S system is enabled: list,
add, change, set the password, disable, enable, remove, the OpenVPN client
profile, and TOTP enrolment (see users()); service accounts are not among them
/service-accounts
the service accounts: list, add, change the e-mail address, set the password,
disable, enable, unlock and remove; the internal ones are only listed (see
service_accounts())
/groups the groups of the router's directory: list, add, rename or describe, remove, and
add and take out members (see groups())
/account the account of the signed-in user: its details, its password, its TOTP
enrolment and its passkeys (see account())
/site-to-site
the site-to-site connections of the configuration: list, add, change and remove,
applied by vpn-router-setup (see site_to_site())
/backup the backup settings, the last run and the local archives: show, change and run
now (see backup())
/remote-access
the settings of IKEv2, OpenVPN and WireGuard access and whether each
runs: show and change, applied by vpn-router-setup (see remote_access())
/ca the PKI: its settings, the root and issuing CAs, the certificates issued; change the
settings, issue and revoke certificates, and take the certificate of an issuing CA
and a root CRL signed offline (see ca())
/pki/<name>.crt, /pki/<name>.crl
the DER certificate and CRL of the root (root) and of each issuing CA, which the
certificates name in their AIA and CRL distribution points; no authentication
/shell while web.shell is true, a login shell of the user's console account on a terminal, for
a member of admins whose console login is on and who has a TOTP token: POST signs the user in again, unless
the user used a shell within web.shell_auth_timeout minutes, for a one-time ticket
(see shell_ticket()), GET with it is a WebSocket
that carries the terminal (see Handler.shell())
/auth sign-in with a user's password and TOTP code or with a passkey, refresh and sign-out
(see auth())
/metrics/history
the metrics the metrics service keeps, for a view: hour, day, month or year (see
metrics_history())
/metrics/settings
the settings of the metrics service: show and change, applied by vpn-router-setup (see
metrics_settings())
/logs the last lines of the journal of the router's units and of the files web.log_files lists
(see logs())
/sites the web sites Caddy serves: the settings of sites, list, add, change and remove (see sites()),
and the files of each: list, download, upload, extract a zip archive, create a folder, rename and
remove (see Handler.site_files()); a member of a site's editors group may use the site's files
/dns the primary name server of the router's domains: its settings, and the domains: add, change and
remove, each with its secondaries, the records it serves, and its manual records (see dns())
/postgresql
the databases of each PostgreSQL cluster on the router with their sizes, as JSON (see
postgresql_databases()); /config has the clusters' settings
/.well-known/openid-configuration, /oidc/...
while oidc.enabled is true, the OpenID Provider (vpnrouter_oidc), answered only on the
loopback address, where Caddy passes the requests of the HTTPS site; POST /oidc/login counts
towards the limit of /auth calls
/status/full, /config, /metrics/history, /metrics/settings, /postgresql, /users, /service-accounts, /groups, /site-to-site, /backup, /remote-access, /ca, /logs, /sites, /dns, /shell and /account need an HS256 JWT in an Authorization: Bearer header signed
with vpnrouter_auth.WEB_KEY (see vpnrouter_auth.verify). A token of /auth names a user in sub;
/account serves that user, and the other paths need a member of vpnrouter_users.ADMIN_GROUP. A
token of vpn-router-token has the admin claim and names no user; it may call every path but
/account, and only such a token may change ADMIN_GROUP: create, rename to it, change, remove it,
add or take out its members, and remove one of its members, and switch a user's console login.
These paths and /auth are not
served (404) while the key does not exist. /auth but its sign-out and the beginning of a passkey
sign-in is limited to AUTH_LIMIT calls per client
address in AUTH_WINDOW seconds; the address of a call from one of web.trusted_proxies, by
default the address the call came in on, is the last entry of its X-Forwarded-For header.
/ and /status show only the status, as much as web.status_content says (see shown): basic, the
default, the overall status, the time and the client's address (its address as for the rate limit
below); standard also the package version, the installed version of vpn-router-manager
(manager_version, null when it is not installed) and the number of active and other services; full
whether each service runs, whether Docker is installed, how many of the site-to-site connections are established
(site_to_site: connections and established, null when swanctl cannot be asked), and IKEv2, OpenVPN
and WireGuard. No configuration data, key or connection name is shown, and no address but the client's.
The service first initializes, without listening: it makes the signing key, the OpenID Provider's key
(vpnrouter_oidc), the refresh tokens' database (vpnrouter_auth) and the web sites' database (vpnrouter_sites), retrying every INIT_RETRY_SECONDS until all are ready and
logging each failure. Then it listens. Its unit starts it only once wait-for-libpq has found libpq
in the linker cache, where psycopg looks it up.
The service needs nothing else to start and does not end on its own. Every DISCOVERY_SECONDS it
works out from /etc/vpn-router/vpn-router.json the addresses of the interfaces web.listen names,
internal, external or both, as general.mode directs, with web.port and web.enabled
(vpnrouter_net.web_listen), and runs a server on each (vpnrouter_serve.serve): it starts them,
moves them when an address or the port changes, and stops them while the service is disabled or
no address is known. Every request reads what it needs when it is made: the signing
key, the configuration, and the features of the router and the facts shown by /config from
/run/vpn-router/web.json, which vpn-router-setup writes. Before setup has written that file
the router has no features, and /config shows no configuration or interfaces.
"""
import base64
import collections
import configparser
import fcntl
import hashlib
import ipaddress
import json
import mimetypes
import os
import pwd
import re
import secrets
import select
import shutil
import signal
import socket
import struct
import subprocess
import sys
import termios
import threading
import time
import urllib.parse
from html import escape
from http.server import BaseHTTPRequestHandler
sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
import ldap
import vpnrouter
import vpnrouter_auth
import vpnrouter_backup
import vpnrouter_dns
import vpnrouter_metrics
import vpnrouter_net
import vpnrouter_oidc
import vpnrouter_passkeys
import vpnrouter_pki
import vpnrouter_serve
import vpnrouter_sites
import vpnrouter_users
from vpnrouter_status import established, established_each, openvpn_clients, run, swanctl, wireguard_peers
STATE_FILE = '/run/vpn-router/web.json'
CONFIG_FILE = '/etc/vpn-router/vpn-router.json'
#: Seconds between two attempts of the initialization.
INIT_RETRY_SECONDS = 15
#: The port when web.port is not set.
DEFAULT_PORT = 7081
#: Seconds between two discoveries of where the service should listen.
DISCOVERY_SECONDS = 30
CACHE_SECONDS = 2
#: The calls of /auth allowed per client address in AUTH_WINDOW seconds.
AUTH_LIMIT = 10
AUTH_WINDOW = 60
TAILWIND = 'https://cdn.jsdelivr.net/npm/@tailwindcss/browser@4'
#: Seconds after which the status page in a browser reloads itself.
REFRESH_SECONDS = 60
#: The PostgreSQL settings /config shows for each cluster.
POSTGRESQL_SETTINGS = ('server_version', 'listen_addresses', 'port', 'max_connections', 'shared_buffers',
'effective_cache_size', 'work_mem', 'password_encryption', 'ssl', 'timezone')
#: The units whose journal /logs/journal shows, besides the whole journal: the package's own and the services it runs.
JOURNAL_UNITS = ('vpn-router-setup', 'vpn-router-web', 'vpn-router-metrics', 'vpn-router-pki', 'vpn-router-backup',
'vpn-router-caddy', 'vpn-router-manager', 'caddy', 'slapd', 'nslcd', 'postgresql', 'strongswan',
'wg-quick@wg0', 'openvpn-server@vpn-router', 'ssh', 'vpn-router-resolver', 'vpn-router-webdav', 'pdns')
#: The most lines /logs answers, the number it answers without lines, and the most bytes it reads from the end of a file.
LOG_LINES_MAX = 1000
LOG_LINES_DEFAULT = 200
LOG_BYTES_MAX = 1024 * 1024
#: The largest request body accepted, in bytes; a CSR or a certificate chain in PEM is a few kilobytes.
BODY_LIMIT = 65536
#: The settings of a site_to_site connection that /site-to-site adds and changes.
CONNECTION_SETTINGS = ('addrs', 'id', 'cidrs', 'local_cidrs', 'local_id_mode', 'auth', 'psk', 'remote_ca')
#: The sections of the remote access systems /remote-access changes: IKEv2, OpenVPN and WireGuard.
REMOTE_ACCESS = ('p2s', 'openvpn', 'wireguard')
#: The HTTP status for each status of a vpnrouter_users.Failure; any other is 500.
USER_STATUS = {2: 400, 3: 404, 4: 409, 5: 401}
#: The HTTP status for each status of a vpnrouter_pki.Failure; any other is 500.
PKI_STATUS = {2: 400, 3: 404, 4: 409}
#: The HTTP status for each status of a vpnrouter_sites.Failure; any other is 500.
SITE_STATUS = {2: 400, 3: 404, 4: 409}
#: A path of a site's files: the site and the path in its folder.
SITE_FILES = re.compile(r'^/sites/([^/]+)/files(?:/(.*))?$')
#: The bytes in a MiB, the unit of sites.upload_max_mb.
MIB = 1024 * 1024
#: A file /pki/ serves: the certificate or the CRL of a CA, and its content type.
PUBLISHED = {'.crt': 'application/pkix-cert', '.crl': 'application/pkix-crl'}
#: Seconds a ticket of POST /shell may open a shell.
SHELL_TICKET_SECONDS = 30
#: Seconds between the checks of an open shell's user, which end the shell once its console is refused.
SHELL_CHECK_SECONDS = 60
#: The largest WebSocket message accepted from a client, in bytes.
MESSAGE_LIMIT = 65536
#: The GUID of RFC 6455 that the Sec-WebSocket-Accept of a handshake is made with.
WEBSOCKET_GUID = '258EAFA5-E914-47DA-95CA-C5AB0DC85B11'
#: WebSocket opcodes.
TEXT, BINARY, CLOSE, PING, PONG = 1, 2, 8, 9, 10
_cache = {'at': 0.0, 'data': None}
_lock = threading.Lock()
#: The times of the recent /auth calls of each client address.
_auth_calls = collections.defaultdict(collections.deque)
_auth_lock = threading.Lock()
#: The unused tickets of POST /shell: ticket -> (user, monotonic time of expiry).
_shell_tickets = {}
_shell_lock = threading.Lock()
#: The last use of a shell of each user, a sign-in of POST /shell or a keystroke: user -> monotonic time.
_shell_used = {}
def read_json(path):
"""The object in a JSON file, or {} when the file is missing or holds no object."""
try:
with open(path, encoding='utf-8') as handle:
data = json.load(handle)
except (OSError, ValueError):
return {}
return data if isinstance(data, dict) else {}
def section(configuration, name):
"""A section of the configuration, or {} when it is missing or not an object."""
value = configuration.get(name)
return value if isinstance(value, dict) else {}
def current():
"""What a request needs, read now: the features and facts setup resolved, and the signing key.
Before setup has run the router has no features, and without a key the protected paths and
/auth are not served.
"""
return dict(read_json(STATE_FILE), key=vpnrouter_auth.key())
def unit_state(unit):
result = run('systemctl', 'is-active', unit)
return result.stdout.strip() if result and result.stdout.strip() else 'unknown'
def package_version(package='vpn-router'):
"""The installed version of the package, or None when it is not installed."""
result = run('dpkg-query', '-W', '-f=${Version}', package)
return (result.stdout.strip() or None) if result and result.returncode == 0 else None
def docker_status():
"""Whether Docker is installed, and the state of its service and of the Docker rules when it is."""
result = run('systemctl', 'list-unit-files', '--no-legend', 'docker.service')
if result is None or result.returncode != 0 or not result.stdout.strip():
return {'installed': False}
return {
'installed': True,
'service': unit_state('docker.service'),
'rules': unit_state('vpn-router-docker.service'),
}
def postgresql_clusters():
"""The state of each PostgreSQL cluster unit (postgresql@<version>-<name>), or {'postgresql': 'no cluster'} without one.
postgresql.service, which only runs /bin/true, is not reported.
"""
lines = lines_of('systemctl', 'list-units', 'postgresql@*.service', '--all', '--plain', '--no-legend') or []
names = [line.split()[0].removesuffix('.service') for line in lines if line.strip()]
return {name: unit_state(f'{name}.service') for name in names} or {'postgresql': 'no cluster'}
def collect(state):
"""The status document. state says which features the router has."""
units = {'vpn-router-setup': unit_state('vpn-router-setup.service'), 'slapd': unit_state('slapd.service'),
**postgresql_clusters()}
manager_version = package_version('vpn-router-manager')
if manager_version:
units['vpn-router-manager'] = unit_state('vpn-router-manager.service')
if state.get('site_to_site') or state.get('p2s'):
units['strongswan'] = unit_state('strongswan.service')
if state.get('wireguard'):
units['wg-quick@wg0'] = unit_state('wg-quick@wg0.service')
if state.get('openvpn'):
units['openvpn-server@vpn-router'] = unit_state('openvpn-server@vpn-router.service')
status = {
'status': 'ok' if all(value == 'active' for value in units.values()) else 'degraded',
'version': package_version(),
'manager_version': manager_version,
'time': time.strftime('%Y-%m-%dT%H:%M:%S%z'),
'units': units,
'docker': docker_status(),
}
names = state.get('site_to_site')
if isinstance(names, list) and names:
counts = established_each(names).values()
status['site_to_site'] = {'connections': len(names),
'established': None if None in counts else sum(1 for count in counts if count)}
if state.get('p2s'):
status['p2s'] = {'clients': established('road-warrior')}
if state.get('wireguard'):
status['wireguard'] = {'peers': wireguard_peers()}
if state.get('openvpn'):
status['openvpn'] = {'clients': openvpn_clients()}
return status
def shown(status, content, client):
"""The status document as web.status_content shows it to the client at its address.
Every content has status, time and client_ip, the client's address, and content, which one it is: basic nothing
more; standard adds version, manager_version and services, the number of active units and of units in every
other state; full adds the whole status document of collect. Content that is not one of
vpnrouter.STATUS_CONTENTS, or not set, is basic.
"""
if content not in vpnrouter.STATUS_CONTENTS:
content = 'basic'
if content == 'full':
return {**status, 'content': content, 'client_ip': client}
document = {'status': status['status'], 'content': content, 'time': status['time'], 'client_ip': client}
if content == 'standard':
active = sum(1 for value in status['units'].values() if value == 'active')
document.update(version=status['version'], manager_version=status['manager_version'],
services={'active': active, 'failed': len(status['units']) - active})
return document
def lines_of(*args):
"""The output lines of a command, or None when it fails."""
result = run(*args)
return result.stdout.splitlines() if result and result.returncode == 0 else None
def routes():
"""The IPv4 routes as ip -j prints them, or None when ip fails."""
result = run('ip', '-j', '-4', 'route', 'show')
if result is None or result.returncode != 0:
return None
try:
return json.loads(result.stdout)
except ValueError:
return None
def wireguard_details():
"""The interface and the peers of wg0, with endpoints and allowed IPs; no private or preshared key."""
result = run('wg', 'show', 'wg0', 'dump')
if result is None or result.returncode != 0:
return None
rows = [line.split('\t') for line in result.stdout.splitlines()]
now = int(time.time())
return {
'public_key': rows[0][1] if rows and len(rows[0]) > 2 else None,
'listen_port': int(rows[0][2]) if rows and len(rows[0]) > 2 else None,
'peers': [{
'public_key': row[0],
'endpoint': None if row[2] == '(none)' else row[2],
'allowed_ips': row[3].split(','),
'handshake_seconds_ago': now - int(row[4]) if int(row[4]) else None,
'received_bytes': int(row[5]),
'sent_bytes': int(row[6]),
} for row in rows[1:] if len(row) >= 7],
}
def postgresql_query(cluster, sql):
"""The JSON value an SQL statement answers on a cluster (version/name), run by psql as postgres, or None when it fails."""
result = run('runuser', '-u', 'postgres', '--', 'psql', '--cluster', cluster, '-X', '-q', '-t', '-A',
'-v', 'ON_ERROR_STOP=1', '-d', 'postgres', '-c', sql)
if result is None or result.returncode != 0:
return None
try:
return json.loads(result.stdout)
except ValueError:
return None
def postgresql_clusters_of():
"""The PostgreSQL clusters of pg_lsclusters, each with version, name, port, status and data_directory, or None when it fails."""
lines = lines_of('pg_lsclusters', '--no-header')
if lines is None:
return None
clusters = []
for line in lines:
fields = line.split()
if len(fields) >= 6:
clusters.append({'version': fields[0], 'name': fields[1], 'port': int(fields[2]) if fields[2].isdigit() else None,
'status': fields[3], 'data_directory': fields[5]})
return clusters
def postgresql_configuration():
"""The PostgreSQL clusters on the router with the settings of POSTGRESQL_SETTINGS of each one that is online (None
for one that is not), or None when pg_lsclusters fails."""
clusters = postgresql_clusters_of()
if clusters is None:
return None
names = ', '.join(f"'{name}'" for name in POSTGRESQL_SETTINGS)
for cluster in clusters:
cluster['settings'] = postgresql_query(
f"{cluster['version']}/{cluster['name']}",
f'SELECT json_object_agg(name, current_setting(name) ORDER BY name) FROM pg_settings WHERE name IN ({names})',
) if cluster['status'].startswith('online') else None
return {'clusters': clusters}
def postgresql_databases():
"""What /postgresql shows: the PostgreSQL clusters on the router, each with its databases (name, owner, encoding and
size in bytes) and their total size, None for a cluster that is not online; clusters is None when pg_lsclusters fails."""
clusters = postgresql_clusters_of()
if clusters is None:
return {'clusters': None}
for cluster in clusters:
databases = postgresql_query(
f"{cluster['version']}/{cluster['name']}",
"SELECT coalesce(json_agg(json_build_object('name', datname, 'owner', pg_get_userbyid(datdba), "
"'encoding', pg_encoding_to_char(encoding), 'size', pg_database_size(oid)) ORDER BY datname), '[]') "
'FROM pg_database',
) if cluster['status'].startswith('online') else None
cluster['databases'] = databases
cluster['size'] = sum(database['size'] for database in databases) if isinstance(databases, list) else None
return {'clusters': clusters}
def details(state):
"""What /config shows: the configuration without its secrets, and what the router resolved and runs; caddy is the
installed Caddy, {version, source, service}, service the state of caddy.service, null when it is not installed."""
configuration = state.get('configuration', {})
caddy = package_version('caddy')
return {
'configuration': configuration,
'network': dict(state.get('network', {}), routes=routes()),
'ipsec': {
'connections': swanctl('--list-conns'),
'sas': swanctl('--list-sas'),
},
'wireguard': wireguard_details(),
'docker': {
'exposed_ports': configuration.get('docker', {}).get('exposed_ports', []),
'rules': lines_of('iptables', '-S', 'VPN-ROUTER-DOCKER'),
},
'postgresql': postgresql_configuration(),
# Caddy is upstream's once its repository is configured (vpn-router-caddy).
'caddy': {'version': caddy, 'source': 'upstream' if vpnrouter.CADDY_APT_SOURCE.is_file() else 'ubuntu',
'service': unit_state('caddy.service')} if caddy else None,
}
def text_of(status):
"""The status document of shown for people, as plain text: a header, a blank line, then name: value lines."""
lines = ['VPN Router', f'Status: {status["status"]}']
if 'version' in status:
lines.append(f'Version: {status["version"] or "unknown"}')
lines += ['', f'Client IP: {status["client_ip"]}']
if 'manager_version' in status:
lines.append(f'Router Manager: {status["manager_version"] or "not installed"}')
if 'services' in status:
lines.append(f'Services: {status["services"]["active"]} active, {status["services"]["failed"]} failed')
lines += [f'{name}: {value}' for name, value in status.get('units', {}).items()]
if 'docker' in status:
docker = status['docker']
lines.append('Docker: not installed' if not docker['installed']
else f'Docker: installed, service {docker["service"]}, rules {docker["rules"]}')
if 'site_to_site' in status:
established, connections = status['site_to_site']['established'], status['site_to_site']['connections']
lines.append(f'Site to site: {"unknown" if established is None else established} of {connections} established')
if 'p2s' in status:
clients = status['p2s']['clients']
lines.append(f'IKEv2: {"unknown" if clients is None else f"{clients} connected"}')
if 'openvpn' in status:
clients = status['openvpn']['clients']
lines.append(f'OpenVPN: {"unknown" if clients is None else f"{clients} connected"}')
if 'wireguard' in status:
peers = status['wireguard']['peers']
if peers is None:
lines.append('WireGuard: down')
else:
lines.append(f'WireGuard: {len(peers)} peers')
for number, peer in enumerate(peers, 1):
age = peer['handshake_seconds_ago']
lines.append(f' peer {number}: {"no handshake yet" if age is None else f"handshake {age} s ago"}, '
f'received {peer["received_bytes"]} B, sent {peer["sent_bytes"]} B')
return '\n'.join(lines) + '\n'
def html_of(status):
"""The status document of shown for a browser, as a page styled with Tailwind CSS."""
colour = 'bg-emerald-600 dark:bg-emerald-700' if status['status'] == 'ok' else 'bg-amber-600 dark:bg-amber-700'
version = f', version {status["version"] or "unknown"}' if 'version' in status else ''
# Each "name: value" line after the header of the text is a row; indented lines, such as WireGuard peers, are indented here too.
rows = []
for line in text_of(status).split('\n\n', 1)[1].splitlines():
if not line:
continue
name, _, value = line.partition(': ')
indent = ' pl-8' if line.startswith(' ') else ''
rows.append(f'<tr class="border-t border-slate-200 first:border-t-0 dark:border-slate-700">'
f'<th class="py-2 pr-4{indent} text-left font-medium text-slate-600 dark:text-slate-300">{escape(name.strip())}</th>'
f'<td class="py-2">{escape(value)}</td></tr>')
body = ''.join(rows)
return f"""<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="color-scheme" content="light dark">
<meta http-equiv="refresh" content="{REFRESH_SECONDS}">
<title>VPN Router</title>
<script src="{TAILWIND}"></script>
</head>
<body class="min-h-screen bg-slate-100 text-slate-900 dark:bg-slate-900 dark:text-slate-100">
<div class="mx-auto max-w-2xl px-4 py-10">
<div class="overflow-hidden rounded-2xl border border-slate-200 bg-white shadow-lg dark:border-slate-700 dark:bg-slate-800">
<div class="{colour} px-6 py-4 text-white">
<h1 class="text-xl font-semibold">VPN Router</h1>
<p class="mt-1 text-sm">Status: {escape(status['status'] + version)}</p>
</div>
<div class="px-6 py-4">
<table class="w-full text-sm"><tbody>{body}</tbody></table>
</div>
<p class="px-6 pb-6 text-xs text-slate-500 dark:text-slate-400">{escape(status['time'])} - <a class="underline" href="/status">JSON</a></p>
</div>
</div>
</body>
</html>
"""
def users(state, method, path, body):
"""Answer a /users request with vpnrouter_users: the HTTP status and the JSON document.
GET /users the users, and {"enabled", "address"} of wireguard, whose network their
WireGuard peers take their addresses from
POST /users add a user: {"name", "password", "first_name", "last_name", "email"};
without the names a Manager-only account
PUT /users/<name> change any of {"first_name", "last_name", "email", "wireguard_peers"};
wireguard_peers, [{"public_key", "ips"}], replaces the user's peers
PUT /users/<name>/password set the password: {"password": ...}
POST /users/<name>/disable lock the user out of OpenVPN
POST /users/<name>/enable let the user in again
POST /users/<name>/unlock end a lockout after failed logins
DELETE /users/<name> remove the user and revoke the certificate
GET /users/<name>/profile the OpenVPN client profile
POST /users/<name>/apple-profile
the Apple configuration profile of IKEv2 access: {"password"}, which
encrypts the user's key in it; {"name", "profile"}
POST /users/<name>/otp a new TOTP secret and its otpauth URI; nothing is stored
PUT /users/<name>/otp store a TOTP secret: {"secret", "code"}, code a code of the secret
DELETE /users/<name>/otp remove the TOTP token
DELETE /users/<name>/passkeys remove all the user's passkeys
PUT /users/<name>/console switch console login on or off: {"enabled", "account"?}, account a local account,
without which the user's own posixAccount; only a token of vpn-router-token
Service accounts are served by /service-accounts: they are not listed, and a request naming one is 404.
"""
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
return 400, {'error': 'the body must be a JSON object'}
body = body or {}
parts = path.strip('/').split('/')
route = (method, len(parts), parts[2] if len(parts) == 3 else None)
name = parts[1] if len(parts) > 1 else None
actions = {
('GET', 1, None): lambda connection: {
'users': [user for user in vpnrouter_users.list_users(connection)['users'] if not user['service']],
'wireguard': {option: vpnrouter.load_config()['wireguard'][option] for option in ('enabled', 'address')}},
('POST', 1, None): lambda connection: vpnrouter_users.add(connection, body.get('name'), body.get('password'), body),
('PUT', 2, None): lambda connection: vpnrouter_users.update(connection, name, body),
('PUT', 3, 'password'): lambda connection: vpnrouter_users.passwd(connection, name, body.get('password')),
('POST', 3, 'disable'): lambda connection: vpnrouter_users.disable(connection, name),
('POST', 3, 'enable'): lambda connection: vpnrouter_users.enable(connection, name),
('POST', 3, 'unlock'): lambda connection: vpnrouter_users.unlock(connection, name),
('DELETE', 2, None): lambda connection: vpnrouter_users.remove(connection, name),
('GET', 3, 'profile'): lambda connection: vpnrouter_users.profile(connection, name),
('POST', 3, 'apple-profile'): lambda connection: vpnrouter_users.apple_profile(connection, name,
body.get('password')),
('POST', 3, 'otp'): lambda connection: vpnrouter_users.otp_new(connection, name),
('PUT', 3, 'otp'): lambda connection: vpnrouter_users.otp_enrol(connection, name, body.get('secret'), body.get('code')),
('DELETE', 3, 'otp'): lambda connection: vpnrouter_users.otp_remove(connection, name),
('DELETE', 3, 'passkeys'): lambda connection: vpnrouter_passkeys.remove(connection, name),
('PUT', 3, 'console'): lambda connection: vpnrouter_users.console(connection, name, body.get('enabled'), body.get('account')),
}
# The vpnrouter_users operation of each route that may revoke refresh tokens.
operations = {('PUT', 3, 'password'): 'passwd', ('POST', 3, 'disable'): 'disable', ('POST', 3, 'unlock'): 'unlock',
('DELETE', 2, None): 'remove'}
if route not in actions:
return 404, {'error': f'no route {method} {path}'}
try:
with vpnrouter_users.session() as connection:
if name is not None and vpnrouter_users.service(vpnrouter_users.entry(connection, name)):
raise vpnrouter_users.Failure(f'{name} is a service account; see /service-accounts', 3)
document = actions[route](connection)
if operations.get(route) in vpnrouter_auth.REVOKING:
vpnrouter_auth.revoke(vpnrouter.load_config(), name)
except vpnrouter_users.Failure as failure:
return USER_STATUS.get(failure.status, 500), {'error': str(failure)}
except (ldap.LDAPError, OSError) as exc:
return 500, {'error': f'the user directory failed: {exc}'}
except vpnrouter_auth.DatabaseError as exc:
return 500, {'error': str(exc)}
except SystemExit as exc:
# vpnrouter ends with SystemExit, carrying the reason, on a configuration it cannot read.
return 500, {'error': str(exc.code)}
return (201 if route == ('POST', 1, None) else 200), document
def oidc_settings():
"""{"enabled", "issuer"} of the OpenID Provider, issuer null while it is disabled."""
cfg = vpnrouter.load_config()
enabled = cfg['oidc']['enabled']
return {'enabled': enabled, 'issuer': vpnrouter.oidc_issuer(cfg) if enabled else None}
def service_accounts(method, path, body):
"""Answer a /service-accounts request with vpnrouter_users: the HTTP status and the JSON document.
GET /service-accounts the service accounts: {"service_accounts": [...], "oidc": {"enabled",
"issuer"}}, issuer that of the OpenID Provider, null while it is disabled
POST /service-accounts add one: {"name", "password", "display_name"?, "description"?, "email"?,
"redirect_uris"?, "post_logout_redirect_uris"?, "allowed_groups"?}
PUT /service-accounts/<name> change any of {"display_name", "description", "email", "redirect_uris",
"post_logout_redirect_uris", "allowed_groups"}; "" removes one of the
first three, [] the values of the others
PUT /service-accounts/<name>/password set the password: {"password": ...}
POST /service-accounts/<name>/disable lock it out
POST /service-accounts/<name>/enable let it in again
POST /service-accounts/<name>/unlock end a lockout after failed logins
DELETE /service-accounts/<name> remove it
A service account with redirect URIs is a client of the OpenID Provider (vpnrouter_users.client_settings). A name
that is not a service account is 404. The internal service accounts (vpnrouter_users.internal_accounts), which
the router manages, are only listed: every other request on one is refused with 403.
"""
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
return 400, {'error': 'the body must be a JSON object'}
body = body or {}
parts = path.strip('/').split('/')
route = (method, len(parts), parts[2] if len(parts) == 3 else None)
name = parts[1] if len(parts) > 1 else None
settings = {key: body.get(key) for key in ('email', 'display_name', 'description', 'redirect_uris',
'post_logout_redirect_uris', 'allowed_groups')}
actions = {
('GET', 1, None): lambda connection: {
'service_accounts': [user for user in vpnrouter_users.list_users(connection)['users'] if user['service']],
'oidc': oidc_settings()},
('POST', 1, None): lambda connection: vpnrouter_users.add(connection, body.get('name'), body.get('password'), settings,
service=True),
('PUT', 2, None): lambda connection: vpnrouter_users.update_service(connection, name, settings),
('PUT', 3, 'password'): lambda connection: vpnrouter_users.passwd(connection, name, body.get('password')),
('POST', 3, 'disable'): lambda connection: vpnrouter_users.disable(connection, name),
('POST', 3, 'enable'): lambda connection: vpnrouter_users.enable(connection, name),
('POST', 3, 'unlock'): lambda connection: vpnrouter_users.unlock(connection, name),
('DELETE', 2, None): lambda connection: vpnrouter_users.remove(connection, name),
}
if route not in actions:
return 404, {'error': f'no route {method} {path}'}
if name is not None and name in vpnrouter_users.internal_accounts():
return 403, {'error': f'{name} is an internal service account, which the router manages'}
try:
with vpnrouter_users.session() as connection:
if name is not None and not vpnrouter_users.service(vpnrouter_users.entry(connection, name)):
raise vpnrouter_users.Failure(f'{name} is not a service account', 3)
document = actions[route](connection)
except vpnrouter_users.Failure as failure:
return USER_STATUS.get(failure.status, 500), {'error': str(failure)}
except (ldap.LDAPError, OSError) as exc:
return 500, {'error': f'the user directory failed: {exc}'}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
return (201 if route == ('POST', 1, None) else 200), document
def groups(state, method, path, body):
"""Answer a /groups request with vpnrouter_users: the HTTP status and the JSON document.
GET /groups the groups with their members
POST /groups add a group: {"name", "members": [...], "description"?}
PUT /groups/<name> rename it or change the description: {"name"?, "description"?}
DELETE /groups/<name> remove the group; its members are kept
POST /groups/<name>/members add a member: {"user": ...}
DELETE /groups/<name>/members/<user> take a member out; the last member cannot leave
An internal service account (vpnrouter_users.internal_accounts) is not put in or taken out of a group: such a
request is refused with 403.
"""
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
return 400, {'error': 'the body must be a JSON object'}
body = body or {}
parts = path.strip('/').split('/')
name = parts[1] if len(parts) > 1 else None
route = (method, len(parts), parts[2] if len(parts) > 2 else None)
members = body.get('members') if route == ('POST', 1, None) else [body.get('user')] if route == ('POST', 3, 'members') \
else [parts[3]] if route == ('DELETE', 4, 'members') else []
internal = sorted(set(vpnrouter_users.internal_accounts()).intersection(
member for member in (members if isinstance(members, list) else []) if isinstance(member, str)))
if internal:
return 403, {'error': f'{", ".join(internal)}: internal service accounts, which the router manages, are not group members'}
actions = {
('GET', 1, None): lambda connection: vpnrouter_users.list_groups(connection),
('POST', 1, None): lambda connection: vpnrouter_users.group_add(
connection, body.get('name'), body.get('description'), body.get('members')),
('PUT', 2, None): lambda connection: vpnrouter_users.group_update(
connection, name, body.get('name'), body.get('description')),
('DELETE', 2, None): lambda connection: vpnrouter_users.group_remove(connection, name),
('POST', 3, 'members'): lambda connection: vpnrouter_users.group_member_add(connection, name, body.get('user')),
('DELETE', 4, 'members'): lambda connection: vpnrouter_users.group_member_remove(connection, name, parts[3]),
}
if route not in actions:
return 404, {'error': f'no route {method} {path}'}
try:
with vpnrouter_users.session() as connection:
document = actions[route](connection)
except vpnrouter_users.Failure as failure:
return USER_STATUS.get(failure.status, 500), {'error': str(failure)}
except (ldap.LDAPError, OSError) as exc:
return 500, {'error': f'the user directory failed: {exc}'}
return (201 if route == ('POST', 1, None) else 200), document
def connection_of(name, connection, count):
"""A site_to_site connection as /site-to-site shows it: its settings but psk, psk_set and established.
count is the number of its established IKE SAs, None when it is not known.
"""
return {'name': name, **{option: value for option, value in connection.items() if option != 'psk'},
'psk_set': bool(connection['psk']), 'established': None if count is None else count > 0}
def site_to_site(method, path, body):
"""Answer a /site-to-site request: the HTTP status and the JSON document.
GET /site-to-site the connections, {"connections": [...]}: each with its settings but psk, with
psk_set, whether psk holds a key, and established, whether its tunnel is up,
null when swanctl cannot be asked
POST /site-to-site add a connection: {"name", "addrs", "id", "cidrs", "local_cidrs", "local_id_mode",
"auth", "psk", "remote_ca"}; psk is required with auth psk, the default, and an
existing name is 409
PUT /site-to-site/<name> change any of the settings of POST but name; an empty or absent psk keeps the
current key, and a psk given replaces psk_file
DELETE /site-to-site/<name> remove the connection
A change is made holding vpnrouter.config_lock, checked as vpn-router-setup checks the configuration (400 with
the reason), written with vpnrouter.save_config, and applied by restarting vpn-router-setup without waiting for it.
"""
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
return 400, {'error': 'the body must be a JSON object'}
body = body or {}
parts = path.strip('/').split('/')
name = parts[1] if len(parts) == 2 else None
if (method, len(parts)) not in (('GET', 1), ('POST', 1), ('PUT', 2), ('DELETE', 2)):
return 404, {'error': f'no route {method} {path}'}
try:
if method == 'GET':
connections = vpnrouter.load_config()['site_to_site']
counts = established_each(connections)
return 200, {'connections': [connection_of(key, value, counts[key]) for key, value in connections.items()]}
settings = {option: body[option] for option in CONNECTION_SETTINGS if option in body}
with vpnrouter.config_lock():
data = vpnrouter.read_settings()
connections = data.setdefault('site_to_site', {})
if method == 'POST':
name = body.get('name')
if not isinstance(name, str) or not name:
return 400, {'error': 'name is required'}
if name in connections:
return 409, {'error': f'connection {name} exists'}
if settings.get('auth', 'psk') == 'psk' and (not isinstance(settings.get('psk'), str) or not settings['psk']):
return 400, {'error': 'psk is required with auth psk'}
connections[name] = settings
elif name not in connections:
return 404, {'error': f'no connection {name}'}
elif method == 'PUT':
connection = dict(connections[name]) if isinstance(connections[name], dict) else {}
if settings.get('psk'):
connection.pop('psk_file', None)
else:
settings.pop('psk', None)
connections[name] = {**connection, **settings}
else:
del connections[name]
cfg = vpnrouter.with_defaults(data)
vpnrouter.validate(cfg)
vpnrouter.save_config(data)
except vpnrouter.ConfigError as exc:
return 400, {'error': str(exc)}
except OSError as exc:
return 500, {'error': f'the configuration cannot be written: {exc}'}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
run('systemctl', 'restart', '--no-block', 'vpn-router-setup.service')
if method == 'DELETE':
return 200, {'name': name, 'removed': True}
return (201 if method == 'POST' else 200), connection_of(name, cfg['site_to_site'][name], None)
def backup_settings(settings):
"""The backup section as /backup shows it: its settings but pbs_password, with pbs_password_set."""
return {**{option: value for option, value in settings.items() if option != 'pbs_password'},
'pbs_password_set': bool(settings['pbs_password'])}
def rclone_remotes():
"""The names of the remotes of vpnrouter_backup.RCLONE_CONFIG; [] when it cannot be read, as when it is encrypted."""
parser = configparser.ConfigParser(interpolation=None)
try:
parser.read(vpnrouter_backup.RCLONE_CONFIG, encoding='utf-8')
except (OSError, configparser.Error):
return []
return parser.sections()
def backup(method, path, body):
"""Answer a /backup request: the HTTP status and the JSON document.
GET /backup {"settings", "state", "archives", "next", "client", "rclone"}: the backup section without
pbs_password, with pbs_password_set; the last run (vpnrouter_backup.state), null before the
first; the local archives of this host, newest first, [{name, time, size}], [] with the pbs
target; the next run of the timer as systemd shows it, null while backups are disabled;
whether proxmox-backup-client is installed; and {"installed", "config", "remotes"} of rclone
PUT /backup change any of the settings of the backup section; an empty or absent pbs_password keeps the
current secret
POST /backup/run start a backup now, without waiting for it; 409 while one runs
A change is made as /site-to-site makes one, and applied by restarting vpn-router-setup without waiting for it.
"""
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
return 400, {'error': 'the body must be a JSON object'}
body = body or {}
if (method, path) == ('POST', '/backup/run'):
if vpnrouter_backup.running():
return 409, {'error': 'a backup is running'}
result = run('systemctl', 'start', '--no-block', vpnrouter_backup.UNIT)
if result is None or result.returncode != 0:
return 500, {'error': f'the backup cannot be started: {result.stderr.strip() if result else "no systemctl"}'}
return 202, {'started': True}
if (method, path) not in (('GET', '/backup'), ('PUT', '/backup')):
return 404, {'error': f'no route {method} {path}'}
try:
if method == 'GET':
settings = vpnrouter.load_config()['backup']
timer = run('systemctl', 'show', '-p', 'NextElapseUSecRealtime', '--value', vpnrouter_backup.TIMER)
next_run = timer.stdout.strip() if timer and timer.returncode == 0 else ''
return 200, {
'settings': backup_settings(settings),
'state': vpnrouter_backup.state() or None,
'archives': vpnrouter_backup.archives(settings['path']) if settings['target'] == 'local' else [],
'next': next_run if settings['enabled'] and next_run not in ('', 'n/a') else None,
'client': vpnrouter_backup.PBS_CLIENT.exists(),
'rclone': {'installed': shutil.which('rclone') is not None,
'config': vpnrouter_backup.RCLONE_CONFIG.is_file(), 'remotes': rclone_remotes()},
}
settings = {option: body[option] for option in vpnrouter.SCHEMA['backup'] if option in body}
if not settings.get('pbs_password'):
settings.pop('pbs_password', None)
with vpnrouter.config_lock():
data = vpnrouter.read_settings()
current = data.get('backup')
data['backup'] = {**(current if isinstance(current, dict) else {}), **settings}
cfg = vpnrouter.with_defaults(data)
vpnrouter.validate(cfg)
vpnrouter.save_config(data)
except vpnrouter.ConfigError as exc:
return 400, {'error': str(exc)}
except OSError as exc:
return 500, {'error': f'the configuration cannot be written: {exc}'}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
run('systemctl', 'restart', '--no-block', 'vpn-router-setup.service')
return 200, {'settings': backup_settings(cfg['backup'])}
def wireguard_server(cfg, state):
"""What a WireGuard peer's configuration needs of the router: {public_key, endpoint, allowed_ips, dns}; public_key is
null before setup made the key pair, endpoint, <wan.local_fqdn>:<wireguard.listen_port>, null without a name,
allowed_ips the networks reached through the tunnel, the router's local CIDRs and the WireGuard network, and dns the
addresses of the remote access clients' resolver, [] while it does not run."""
try:
public_key = vpnrouter.WG_PUBLIC_KEY.read_text(encoding='utf-8').strip() or None
except OSError:
public_key = None
fqdn = cfg['wan']['local_fqdn'].strip()
address = cfg['wireguard']['address'].strip()
networks = section(state, 'network').get('local_cidrs')
allowed = [cidr for cidr in networks if isinstance(cidr, str)] if isinstance(networks, list) else []
if address:
allowed.append(str(ipaddress.ip_interface(address).network))
dns = section(state, 'network').get('client_dns')
return {'public_key': public_key, 'endpoint': f'{fqdn}:{cfg["wireguard"]["listen_port"]}' if fqdn else None,
'allowed_ips': list(dict.fromkeys(allowed)),
'dns': [address for address in dns if isinstance(address, str)] if isinstance(dns, list) else []}
def remote_access(state, method, path, body):
"""Answer a /remote-access request: the HTTP status and the JSON document.
GET /remote-access {"p2s", "openvpn", "wireguard", "running", "local_fqdn", "wireguard_server"}: the
settings of the three remote access systems (REMOTE_ACCESS), whether setup runs
each (running: {p2s, openvpn, wireguard}), wan.local_fqdn, which IKEv2 and OpenVPN
need with the router's VPN server certificate, and what a WireGuard peer's
configuration needs of the router (wireguard_server)
PUT /remote-access/<system> change any of the settings of the system's section; answers the section
A change is made as /site-to-site makes one, and applied by restarting vpn-router-setup without waiting for it.
"""
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
return 400, {'error': 'the body must be a JSON object'}
body = body or {}
parts = path.strip('/').split('/')
try:
if (method, len(parts)) == ('GET', 1):
cfg = vpnrouter.load_config()
return 200, {**{system: cfg[system] for system in REMOTE_ACCESS},
'running': {system: bool(state.get(system)) for system in REMOTE_ACCESS},
'local_fqdn': cfg['wan']['local_fqdn'],
'wireguard_server': wireguard_server(cfg, state)}
if method != 'PUT' or len(parts) != 2 or parts[1] not in REMOTE_ACCESS:
return 404, {'error': f'no route {method} {path}'}
system = parts[1]
settings = {option: body[option] for option in vpnrouter.SCHEMA[system] if option in body}
with vpnrouter.config_lock():
data = vpnrouter.read_settings()
current = data.get(system)
data[system] = {**(current if isinstance(current, dict) else {}), **settings}
cfg = vpnrouter.with_defaults(data)
vpnrouter.validate(cfg)
vpnrouter.save_config(data)
except vpnrouter.ConfigError as exc:
return 400, {'error': str(exc)}
except OSError as exc:
return 500, {'error': f'the configuration cannot be written: {exc}'}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
run('systemctl', 'restart', '--no-block', 'vpn-router-setup.service')
return 200, cfg[system]
def ca(method, path, body):
"""Answer a /ca request with vpnrouter_pki: the HTTP status and the JSON document.
GET /ca {"settings", "templates", "extended_key_usages", "status",
"certificates"}: the pki section, the templates with the built-in
ones, the extended key usages, each {oid, builtin}, the root, the
issuing CAs, the retired CAs and the warnings (vpnrouter_pki.status),
and the certificates issued, newest first
PUT /ca change any of the settings of the pki section
POST /ca/certificates issue a certificate: {"ca", "template", "csr"} answers {"serial",
"certificate", "chain"}; {"ca", "template", "subject", "sans",
"password"} answers {"serial", "pfx"}, a PKCS#12 file in base64
POST /ca/certificates/<serial>/revoke revoke a certificate: {"reason"?}
GET /ca/bundle/<name> the PEM certificates a client trusts the CA with: {"name", "pem"}; for
root the root alone, for an issuing CA the issuing CA and the root
GET /ca/issuing/<name>/csr the CSR of an issuing CA that waits for the offline root, with the
openssl extensions to sign it with: {"name", "csr", "extensions"}
PUT /ca/issuing/<name>/certificate take its certificate signed by the root: {"certificate"}
PUT /ca/root/crl take a root CRL signed offline: {"crl"}
A change of the settings is made as /site-to-site makes one; it, a revoked router certificate and an imported
issuing CA certificate are applied by restarting vpn-router-setup without waiting for it.
"""
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
return 400, {'error': 'the body must be a JSON object'}
body = body or {}
parts = path.strip('/').split('/')
route = (method, *parts[1:])
try:
cfg = vpnrouter.load_config()
if len(route) == 3 and route[:2] == ('GET', 'bundle'):
return 200, vpnrouter_pki.bundle(route[2])
if route == ('GET',):
return 200, {'settings': cfg['pki'], 'templates': vpnrouter.pki_templates(cfg),
'extended_key_usages': {name: {'oid': oid, 'builtin': name in vpnrouter.EXTENDED_KEY_USAGES}
for name, oid in vpnrouter.extended_key_usages(cfg).items()},
'status': vpnrouter_pki.status(cfg), 'certificates': vpnrouter_pki.certificates()}
if route == ('PUT',):
settings = {option: body[option] for option in vpnrouter.SCHEMA['pki'] if option in body}
with vpnrouter.config_lock():
data = vpnrouter.read_settings()
current = data.get('pki')
data['pki'] = {**(current if isinstance(current, dict) else {}), **settings}
cfg = vpnrouter.with_defaults(data)
vpnrouter.validate(cfg)
vpnrouter.save_config(data)
run('systemctl', 'restart', '--no-block', 'vpn-router-setup.service')
return 200, {'settings': cfg['pki'], 'templates': vpnrouter.pki_templates(cfg)}
with vpnrouter_pki.locked():
if route == ('POST', 'certificates'):
return 201, vpnrouter_pki.issue(cfg, body.get('ca'), body.get('template'), csr=body.get('csr'),
subject=body.get('subject'), sans=body.get('sans'),
password=body.get('password'))
if len(route) == 4 and route[:2] == ('POST', 'certificates') and route[3] == 'revoke':
document = vpnrouter_pki.revoke(cfg, route[2], body.get('reason'))
elif len(route) == 4 and route[:2] == ('GET', 'issuing') and route[3] == 'csr':
return 200, vpnrouter_pki.csr(cfg, route[2])
elif len(route) == 4 and route[:2] == ('PUT', 'issuing') and route[3] == 'certificate':
document = vpnrouter_pki.import_ca(cfg, route[2], body.get('certificate'))
elif route == ('PUT', 'root', 'crl'):
return 200, vpnrouter_pki.import_root_crl(body.get('crl'))
else:
return 404, {'error': f'no route {method} {path}'}
except vpnrouter.ConfigError as exc:
return 400, {'error': str(exc)}
except vpnrouter_pki.Failure as failure:
return PKI_STATUS.get(failure.status, 500), {'error': str(failure)}
except ldap.LDAPError as exc:
return 500, {'error': f'the directory, which holds the PKI, failed: {exc}'}
except OSError as exc:
return 500, {'error': f'the PKI cannot be changed: {exc}'}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
if route[1] == 'issuing' or document.get('reissue'):
run('systemctl', 'restart', '--no-block', 'vpn-router-setup.service')
return 200, document
def account(name, method, path, body):
"""Answer an /account request of the user named by the token with vpnrouter_users: the HTTP status and the JSON document.
GET /account the user, with admin, whether the user is a member of ADMIN_GROUP
PUT /account/password set the password: {"current_password", "current_code"?, "password"}
POST /account/otp a new TOTP secret and its otpauth URI; nothing is stored
PUT /account/otp store a TOTP secret: {"current_password", "current_code"?, "secret", "code"},
code a code of the secret
POST /account/passkeys begin the registration of a passkey: {"current_password", "current_code"?};
{"id", "options"} for navigator.credentials.create
PUT /account/passkeys store the passkey: {"id", "credential", "label"?}, credential the browser's answer
DELETE /account/passkeys/<id>
remove a passkey
POST /account/profile the user's OpenVPN client profile: {"current_password", "current_code"?}; {"name",
"profile"}
POST /account/apple-profile
the user's Apple configuration profile of IKEv2 access:
{"current_password", "current_code"?, "password"}, password encrypting the user's key in
it; {"name", "profile"}
A change but the removal of a passkey, and the profiles, need the current password and, for an enrolled user, a
current code; a wrong
one is 403 and counts as a failed login. A password change revokes the user's refresh tokens.
"""
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
return 400, {'error': 'the body must be a JSON object'}
body = body or {}
route = (method, path.rstrip('/'))
removing = method == 'DELETE' and path.startswith('/account/passkeys/') and len(path.strip('/').split('/')) == 3
if not removing and route not in (('GET', '/account'), ('PUT', '/account/password'), ('POST', '/account/otp'),
('PUT', '/account/otp'), ('POST', '/account/passkeys'), ('PUT', '/account/passkeys'),
('POST', '/account/profile'), ('POST', '/account/apple-profile')):
return 404, {'error': f'no route {method} {path}'}
try:
if route == ('PUT', '/account/password'):
vpnrouter_users.check_password(body.get('password'))
if route in (('PUT', '/account/password'), ('PUT', '/account/otp'), ('POST', '/account/passkeys'),
('POST', '/account/profile'), ('POST', '/account/apple-profile')):
vpnrouter_users.login(name, body.get('current_password'), body.get('current_code'))
with vpnrouter_users.session() as connection:
if route == ('GET', '/account'):
document = {**vpnrouter_users.summary(vpnrouter_users.entry(connection, name)),
'admin': vpnrouter_users.is_admin(connection, name)}
elif route == ('PUT', '/account/password'):
document = vpnrouter_users.passwd(connection, name, body['password'])
elif route == ('POST', '/account/otp'):
document = vpnrouter_users.otp_new(connection, name)
elif route == ('POST', '/account/profile'):
document = vpnrouter_users.profile(connection, name)
elif route == ('POST', '/account/apple-profile'):
document = vpnrouter_users.apple_profile(connection, name, body.get('password'))
elif route == ('POST', '/account/passkeys'):
document = vpnrouter_passkeys.register_begin(connection, name)
elif route == ('PUT', '/account/passkeys'):
document = vpnrouter_passkeys.register_complete(connection, name, body.get('id'), body.get('credential'),
body.get('label'))
elif removing:
document = vpnrouter_passkeys.remove(connection, name, urllib.parse.unquote(path.strip('/').split('/')[2]))
else:
document = vpnrouter_users.otp_enrol(connection, name, body.get('secret'), body.get('code'))
if route == ('PUT', '/account/password'):
vpnrouter_auth.revoke(vpnrouter.load_config(), name)
except vpnrouter_users.Failure as failure:
if failure.status == 5:
return 403, {'error': 'the current password or code is wrong'}
return USER_STATUS.get(failure.status, 500), {'error': str(failure)}
except (ldap.LDAPError, OSError) as exc:
return 500, {'error': f'the user directory failed: {exc}'}
except vpnrouter_auth.DatabaseError as exc:
return 500, {'error': str(exc)}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
return 200, document
def tail(path, count):
"""The last count lines of a file, read from at most LOG_BYTES_MAX bytes at its end, invalid UTF-8 replaced."""
with open(path, 'rb') as handle:
size = handle.seek(0, os.SEEK_END)
start = max(0, size - LOG_BYTES_MAX)
handle.seek(start)
lines = handle.read().decode('utf-8', 'replace').splitlines()
# A read that starts inside the file starts inside a line, which is left out.
return (lines[1:] if start and lines else lines)[-count:]
def logs(method, path, query):
"""Answer a /logs request: the HTTP status and the JSON document.
GET /logs {"units": JOURNAL_UNITS, "files": web.log_files, "max_lines": LOG_LINES_MAX}
GET /logs/journal {"lines": [...]}: the last lines of the journal, of the unit unit= names, one of
JOURNAL_UNITS, or of the whole journal without it
GET /logs/file {"lines": [...]}: the last lines of the file path= names, one of web.log_files
lines= is the number of lines, 1 to LOG_LINES_MAX, LOG_LINES_DEFAULT without it.
"""
if method not in ('GET', 'HEAD'):
return 405, {'error': f'{method} is not allowed on /logs'}
try:
files = [item.strip() for item in vpnrouter.load_config()['web']['log_files'] if item.strip()]
except SystemExit as exc:
return 500, {'error': str(exc.code)}
if path == '/logs':
return 200, {'units': list(JOURNAL_UNITS), 'files': files, 'max_lines': LOG_LINES_MAX}
text = query.get('lines', [str(LOG_LINES_DEFAULT)])[0]
if not text.isdigit() or not 1 <= int(text) <= LOG_LINES_MAX:
return 400, {'error': f'lines must be a whole number from 1 to {LOG_LINES_MAX}'}
count = int(text)
if path == '/logs/journal':
unit = query.get('unit', [''])[0]
if unit and unit not in JOURNAL_UNITS:
return 404, {'error': f'{unit} is not one of the units whose journal /logs shows'}
try:
result = subprocess.run(['journalctl', '--no-pager', '--quiet', '--output=short-iso', f'--lines={count}',
*([f'--unit={unit}.service'] if unit else [])], capture_output=True, text=True,
errors='replace', timeout=30)
except (OSError, subprocess.TimeoutExpired) as exc:
return 500, {'error': f'journalctl cannot be run: {exc}'}
if result.returncode != 0:
return 500, {'error': result.stderr.strip() or f'journalctl ended with status {result.returncode}'}
return 200, {'lines': result.stdout.splitlines()[-count:]}
if path == '/logs/file':
name = query.get('path', [''])[0]
if name not in files:
return 404, {'error': f'{name!r} is not one of web.log_files'}
try:
return 200, {'lines': tail(name, count)}
except OSError as exc:
return (404 if isinstance(exc, FileNotFoundError) else 500), {'error': f'{name} cannot be read: {exc.strerror}'}
return 404, {'error': f'no route {method} {path}'}
def section_change(name, body):
"""Change any of the settings of the configuration's section as /backup changes its own; the section, or the HTTP
status and the error document as a tuple."""
settings = {option: body[option] for option in vpnrouter.SCHEMA[name] if option in body}
try:
with vpnrouter.config_lock():
data = vpnrouter.read_settings()
current = data.get(name)
data[name] = {**(current if isinstance(current, dict) else {}), **settings}
cfg = vpnrouter.with_defaults(data)
vpnrouter.validate(cfg)
vpnrouter.save_config(data)
except vpnrouter.ConfigError as exc:
return 400, {'error': str(exc)}
except OSError as exc:
return 500, {'error': f'the configuration cannot be written: {exc}'}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
run('systemctl', 'restart', '--no-block', 'vpn-router-setup.service')
return cfg[name]
def sites(method, path, query, body):
"""Answer a /sites request but one of a site's files with vpnrouter_sites: the HTTP status and the JSON document.
GET /sites {"settings", "sites"}: the sites section of the configuration and the sites, each with its
settings but private_key, with private_key_set, modified and its folder
PUT /sites change any of the settings of the sites section
POST /sites add a site: {"name", and any of "enabled", "hosts", "port", "listen", "tls", "ca",
"certificate", "private_key", "index_file", "spa", "editors"}; an existing name is 409
GET /sites/<name> the site
PUT /sites/<name> change any of the settings of POST but name; an empty or absent private_key keeps the key
DELETE /sites/<name> remove the site, with content=true also its folder
A change is applied by restarting vpn-router-setup without waiting for it.
"""
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
return 400, {'error': 'the body must be a JSON object'}
body = body or {}
parts = path.strip('/').split('/')
route = (method, len(parts))
try:
cfg = vpnrouter.load_config()
if route == ('GET', 1):
return 200, {'settings': cfg['sites'], 'sites': vpnrouter_sites.listing()}
if route == ('PUT', 1):
changed = section_change('sites', body)
return changed if isinstance(changed, tuple) else (200, {'settings': changed})
if route == ('POST', 1):
name = body.get('name')
document = vpnrouter_sites.save(cfg, name, {key: value for key, value in body.items() if key != 'name'},
True)
code = 201
elif route == ('GET', 2):
with vpnrouter_sites.database() as connection:
return 200, vpnrouter_sites.shown(vpnrouter_sites.get(connection, parts[1]))
elif route == ('PUT', 2):
document, code = vpnrouter_sites.save(cfg, parts[1], body, False), 200
elif route == ('DELETE', 2):
document = vpnrouter_sites.remove(parts[1], query.get('content', [''])[0] == 'true')
code = 200
else:
return 404, {'error': f'no route {method} {path}'}
except vpnrouter_sites.Failure as failure:
return SITE_STATUS.get(failure.status, 500), {'error': str(failure)}
except vpnrouter_sites.DatabaseError as exc:
return 500, {'error': str(exc)}
except (ldap.LDAPError, OSError) as exc:
return 500, {'error': f'the site cannot be changed: {exc}'}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
run('systemctl', 'restart', '--no-block', 'vpn-router-setup.service')
return code, document
def site_file_action(name, target, body):
"""Answer a POST to a site's file or folder: {"folder": true} creates the folder, {"rename": "<path>"} moves the file
or folder to the path in the site's folder; the HTTP status and the JSON document."""
if not isinstance(body, dict):
return 400, {'error': 'the body must be a JSON object'}
try:
if body.get('folder') is True:
vpnrouter_sites.make_folder(target)
return 201, {'path': vpnrouter_sites.relative_of(name, target), 'folder': True}
if isinstance(body.get('rename'), str):
moved = vpnrouter_sites.rename(name, target, body['rename'])
return 200, {'path': vpnrouter_sites.relative_of(name, moved)}
except vpnrouter_sites.Failure as failure:
return SITE_STATUS.get(failure.status, 500), {'error': str(failure)}
except OSError as exc:
return 500, {'error': f'{exc.filename or target.name}: {exc.strerror}'}
return 400, {'error': 'give {"folder": true} or {"rename": "<path>"}'}
def dns(method, path, query, body):
"""Answer a /dns request with vpnrouter_dns: the HTTP status and the JSON document.
GET /dns {"settings", "installed", "running", "domains", "problem"}: the dns section of the
configuration, whether PowerDNS is installed and runs, the domains, each {"name", "sites",
"ttl", "secondaries", "manual", "records"} with its secondaries {"nameserver", "address"},
its manual records and the records the name server serves, and why the domains cannot be
read, null when they can
PUT /dns change any of the settings of the dns section
POST /dns/domains add a domain: {"name", "sites"?, "ttl"?}; sites gives the host names of the enabled sites
in it records; ttl is dns.ttl when null or absent; an existing name is 409
PUT /dns/domains/<domain>
change "sites" or "ttl" of the domain
DELETE /dns/domains/<domain>
remove the domain with its secondaries and manual records
POST /dns/domains/<domain>/secondaries
add a secondary name server of the domain, or change its address: {"nameserver", "address"}
DELETE /dns/domains/<domain>/secondaries/<nameserver>
remove the secondary name server
POST /dns/domains/<domain>/records
add a manual record: {"name", "type", "content", "ttl"?}; name is relative to the domain,
@ for the domain itself, or absolute with a final dot; ttl is the domain's without it
DELETE /dns/domains/<domain>/records
remove the manual records of name= and type=, with content= only that one
A change is applied by restarting vpn-router-setup without waiting for it.
"""
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
return 400, {'error': 'the body must be a JSON object'}
body = body or {}
parts = path.strip('/').split('/')
route = (method, *parts[:2], *parts[3:4])
try:
cfg = vpnrouter.load_config()
if route == ('GET', 'dns'):
domains, problem = [], None
try:
domains = vpnrouter_dns.listing()
except vpnrouter_dns.DatabaseError as exc:
problem = str(exc)
return 200, {'settings': cfg['dns'], 'installed': vpnrouter_dns.PDNS_SERVER.exists(),
'running': unit_state(vpnrouter_dns.UNIT) == 'active', 'domains': domains,
'problem': problem}
if route == ('PUT', 'dns'):
changed = section_change('dns', body)
return changed if isinstance(changed, tuple) else (200, {'settings': changed})
if route == ('POST', 'dns', 'domains') and len(parts) == 2:
document, code = vpnrouter_dns.domain_add(body), 201
elif route == ('PUT', 'dns', 'domains') and len(parts) == 3:
document, code = vpnrouter_dns.domain_change(parts[2], body), 200
elif route == ('DELETE', 'dns', 'domains') and len(parts) == 3:
document, code = vpnrouter_dns.domain_remove(parts[2]), 200
elif route == ('POST', 'dns', 'domains', 'secondaries') and len(parts) == 4:
document, code = vpnrouter_dns.secondary_add(parts[2], body), 201
elif route == ('DELETE', 'dns', 'domains', 'secondaries') and len(parts) == 5:
document, code = vpnrouter_dns.secondary_remove(parts[2], parts[4]), 200
elif route == ('POST', 'dns', 'domains', 'records') and len(parts) == 4:
document, code = vpnrouter_dns.add(cfg, parts[2], body), 201
elif route == ('DELETE', 'dns', 'domains', 'records') and len(parts) == 4:
content = query.get('content', [None])[0]
document = vpnrouter_dns.remove(parts[2], query.get('name', [''])[0], query.get('type', [''])[0], content)
code = 200
else:
return 404, {'error': f'no route {method} {path}'}
except vpnrouter_dns.Failure as failure:
return {3: 404, 4: 409}.get(failure.status, 400), {'error': str(failure)}
except vpnrouter_dns.DatabaseError as exc:
return 500, {'error': str(exc)}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
run('systemctl', 'restart', '--no-block', 'vpn-router-setup.service')
return code, document
def metrics_history(view):
"""Answer GET /metrics/history?range=<view> with vpnrouter_metrics.history: the HTTP status and the JSON document.
view is hour, the default, day, month or year (vpnrouter_metrics.RANGES).
"""
if view not in vpnrouter_metrics.RANGES:
return 400, {'error': f'range must be one of {", ".join(vpnrouter_metrics.RANGES)}'}
try:
with vpnrouter_metrics.database(vpnrouter.load_config()) as connection:
return 200, vpnrouter_metrics.history(connection, view)
except vpnrouter_metrics.DatabaseError as exc:
return 500, {'error': str(exc)}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
def metrics_shown(settings):
"""The metrics section as /metrics/settings shows it: its settings but otlp_password, with otlp_password_set."""
return {**{option: value for option, value in settings.items() if option != 'otlp_password'},
'otlp_password_set': bool(settings['otlp_password'])}
def metrics_settings(method, body):
"""Answer a /metrics/settings request: the HTTP status and the JSON document.
GET /metrics/settings {"settings"}: the metrics section without otlp_password, with otlp_password_set
PUT /metrics/settings change any of the settings of the metrics section; an empty or absent otlp_password keeps
the current secret, and an empty otlp_username removes it
A change is applied by restarting vpn-router-setup without waiting for it.
"""
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
return 400, {'error': 'the body must be a JSON object'}
if method in ('GET', 'HEAD'):
try:
return 200, {'settings': metrics_shown(vpnrouter.load_config()['metrics'])}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
if method != 'PUT':
return 404, {'error': f'no route {method} /metrics/settings'}
body = dict(body or {})
if not body.get('otlp_password'):
body.pop('otlp_password', None)
if body.get('otlp_username') == '':
body['otlp_password'] = ''
changed = section_change('metrics', body)
return changed if isinstance(changed, tuple) else (200, {'settings': metrics_shown(changed)})
def changes_admins(connection, method, parts, body):
"""Whether a /users, /service-accounts or /groups request creates, renames to, changes or removes ADMIN_GROUP, or
removes a member of it."""
admins = vpnrouter_users.ADMIN_GROUP
if parts[0] == 'groups':
if method == 'POST' and len(parts) == 1:
return body.get('name') == admins
if method != 'GET' and len(parts) > 1 and parts[1] == admins:
return True
return method == 'PUT' and len(parts) == 2 and body.get('name') == admins
if parts[0] in ('users', 'service-accounts') and method == 'DELETE' and len(parts) == 2:
return admins in vpnrouter_users.groups_of(connection, parts[1])
return False
def refusal(claims, method, path, body):
"""Why the holder of the claims may not make the request to a protected path, or '' when it may.
The files of a site may also be read and changed by a member of the site's editors group.
"""
name = claims.get('sub')
if path == '/account' or path.startswith('/account/'):
return '' if isinstance(name, str) else 'the token names no user'
if claims.get('admin') is True:
return ''
if not isinstance(name, str):
return 'the token names no user'
admins = vpnrouter_users.ADMIN_GROUP
with vpnrouter_users.session() as connection:
if not vpnrouter_users.is_admin(connection, name):
files = SITE_FILES.match(path)
if files and files.group(1) in vpnrouter_sites.editable(vpnrouter_users.groups_of(connection, name)):
return ''
return f'user {name} is not a member of {admins}'
parts = path.strip('/').split('/')
if changes_admins(connection, method, parts, body if isinstance(body, dict) else {}):
return f'only a token of vpn-router-token may change the group {admins} or remove its members'
if parts[0] == 'users' and len(parts) == 3 and parts[2] == 'console' and method != 'GET':
return 'only a token of vpn-router-token may change console login'
return ''
def auth(path, body):
"""Answer a POST to /auth with vpnrouter_users and vpnrouter_auth: the HTTP status and the JSON document.
POST /auth sign in: {"name", "password", "code"?}; the tokens and the user's name
POST /auth/passkey begin a sign-in with a passkey: {"id", "options"} for navigator.credentials.get
POST /auth/passkey/complete
{"id", "credential"}, credential the browser's answer; the tokens and the user's name
POST /auth/refresh {"refresh_token"}; new tokens, the refresh token used up
POST /auth/logout {"refresh_token"}; the refresh token revoked
"""
if not isinstance(body, dict):
return 400, {'error': 'the body must be a JSON object'}
try:
cfg = vpnrouter.load_config()
if path == '/auth':
vpnrouter_users.login(body.get('name'), body.get('password'), body.get('code'))
with vpnrouter_users.session() as connection:
if vpnrouter_users.service(vpnrouter_users.entry(connection, body['name'])):
raise vpnrouter_users.Failure('the name, the password or the code is wrong', 5)
return 200, {**vpnrouter_auth.issue(cfg, body['name']), 'name': body['name']}
if path == '/auth/passkey':
return 200, vpnrouter_passkeys.login_begin()
if path == '/auth/passkey/complete':
with vpnrouter_users.session() as connection:
name = vpnrouter_passkeys.login_complete(connection, body.get('id'), body.get('credential'))
return 200, {**vpnrouter_auth.issue(cfg, name), 'name': name}
name = vpnrouter_auth.take(cfg, body.get('refresh_token'))
if path == '/auth/logout':
return 200, {'revoked': name is not None}
if name is None:
return 401, {'error': 'the refresh token is not valid'}
with vpnrouter_users.session() as connection:
user = vpnrouter_users.summary(vpnrouter_users.entry(connection, name))
if not user['enabled'] or user['locked']:
return 401, {'error': f'user {name} is disabled or locked'}
return 200, vpnrouter_auth.issue(cfg, name)
except vpnrouter_users.Failure as failure:
return (401 if failure.status == 3 else USER_STATUS.get(failure.status, 500)), {'error': str(failure)}
except (ldap.LDAPError, OSError) as exc:
return 500, {'error': f'the user directory failed: {exc}'}
except vpnrouter_auth.DatabaseError as exc:
return 500, {'error': str(exc)}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
def shell_ticket(name, body):
"""Answer POST /shell: the HTTP status and the JSON document.
The user named by the token signs in again with {"password", "code"?}; a wrong password or code is 403 and counts
as a failed login. Without a password, the user is not asked to sign in while its last use of a shell (see
_shell_used) is less than web.shell_auth_timeout minutes ago. The answer is {"ticket", "expires_in"}: a ticket that
opens one shell (GET /shell) for the same user within SHELL_TICKET_SECONDS, or a null ticket when the user must
sign in. 403 while web.shell is false, the user's console login is not on or the user has no TOTP token.
"""
if not isinstance(name, str):
return 403, {'error': 'the shell needs a token of a signed-in user'}
if not isinstance(body, dict):
return 400, {'error': 'the body must be a JSON object'}
try:
options = vpnrouter.load_config()['web']
if not options['shell']:
return 403, {'error': 'the shell is disabled; vpn-config-edit set web.shell true enables it'}
refusal = console_account(name)[1]
if refusal:
return 403, {'error': refusal}
if 'password' not in body:
with _shell_lock:
used = _shell_used.get(name)
if used is None or time.monotonic() - used >= options['shell_auth_timeout'] * 60:
return 200, {'ticket': None, 'expires_in': 0}
else:
vpnrouter_users.login(name, body.get('password'), body.get('code'))
except vpnrouter_users.Failure as failure:
if failure.status == 5:
return 403, {'error': 'the password or the code is wrong'}
return USER_STATUS.get(failure.status, 500), {'error': str(failure)}
except (ldap.LDAPError, OSError) as exc:
return 500, {'error': f'the user directory failed: {exc}'}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
ticket = secrets.token_urlsafe(32)
now = time.monotonic()
with _shell_lock:
for unused in [key for key, (_, expires) in _shell_tickets.items() if expires <= now]:
del _shell_tickets[unused]
_shell_tickets[ticket] = (name, now + SHELL_TICKET_SECONDS)
_shell_used[name] = now
return 200, {'ticket': ticket, 'expires_in': SHELL_TICKET_SECONDS}
def console_account(name):
"""The system account the console of the user runs as and None, or None and the reason the user may not use a
console: its console login is off, or it has no TOTP token."""
with vpnrouter_users.session() as connection:
user = vpnrouter_users.summary(vpnrouter_users.entry(connection, name))
if not user['console'] or not user['console']['enabled']:
return None, f'console login is not enabled for {name}; vpn-router-user console sets it'
if not user['otp']:
return None, f'{name} has no TOTP token; the shell needs one, enrolled on the Account page'
return user['console']['account'], None
def take_ticket(name, ticket):
"""Whether the ticket of POST /shell is unexpired and was given to the user; it is used up either way."""
with _shell_lock:
holder, expires = _shell_tickets.pop(ticket, (None, 0.0))
return holder == name and expires > time.monotonic()
def shell_command(account, unit):
"""The command that runs the login shell of the system user account in the transient unit, on its terminal.
setsid makes the terminal the controlling one of systemd-run, which passes the window size on; systemd-run starts
the shell outside the web service's sandbox, in a PAM login session of the user.
"""
return ['setsid', '--ctty', 'systemd-run', '--quiet', '--collect', '--pty', '--send-sighup', f'--unit={unit}',
f'--uid={account.pw_name}', '--property=PAMName=login', f'--working-directory={account.pw_dir}',
'--setenv=TERM=xterm-256color', account.pw_shell or '/bin/sh', '-l']
def frame(opcode, payload):
"""A final, unmasked WebSocket frame of a server."""
size = len(payload)
if size < 126:
length = bytes([size])
elif size < 65536:
length = bytes([126]) + size.to_bytes(2, 'big')
else:
length = bytes([127]) + size.to_bytes(8, 'big')
return bytes([0x80 | opcode]) + length + payload
def auth_wait(address):
"""Seconds until the client address may call /auth again, 0 when it may now; a call that may is counted."""
now = time.monotonic()
with _auth_lock:
for idle in [key for key, times in _auth_calls.items() if times[-1] <= now - AUTH_WINDOW]:
del _auth_calls[idle]
calls = _auth_calls[address]
while calls and calls[0] <= now - AUTH_WINDOW:
calls.popleft()
if len(calls) >= AUTH_LIMIT:
return int(calls[0] + AUTH_WINDOW - now) + 1
calls.append(now)
return 0
class Handler(BaseHTTPRequestHandler):
protocol_version = 'HTTP/1.1'
def status(self):
with _lock:
if _cache['data'] is None or time.monotonic() - _cache['at'] > CACHE_SECONDS:
_cache['data'] = collect(current())
_cache['at'] = time.monotonic()
return _cache['data']
def claims(self, secret):
"""The claims of the request's token signed with the secret, or None without an acceptable one."""
header = self.headers.get('Authorization', '')
return vpnrouter_auth.verify(header[7:].strip(), secret) if header.startswith('Bearer ') else None
def client(self):
"""The client address: the last X-Forwarded-For entry when the peer is a trusted proxy, else the peer."""
peer = self.client_address[0]
trusted = section(read_json(CONFIG_FILE), 'web').get('trusted_proxies') or [self.server.server_address[0]]
forwarded = self.headers.get('X-Forwarded-For', '')
if isinstance(trusted, list) and peer in trusted and forwarded.strip():
return forwarded.split(',')[-1].strip()
return peer
def refusal(self, claims, path, payload):
"""The HTTP status and the message that refuse a request to a protected path, or None when it may be made."""
try:
reason = refusal(claims, self.command, path, payload)
except (ldap.LDAPError, OSError) as exc:
return 500, f'the user directory failed: {exc}'
except vpnrouter_sites.DatabaseError as exc:
return 500, str(exc)
return (403, reason) if reason else None
def site_files(self, name, relative, query, payload, send_body):
"""Answer a request of a site's files: the HTTP status and the JSON document, or None once a file was sent.
GET /sites/<name>/files/<path> a folder: {"path", "entries": [{name, folder, size, modified}]}; a file:
the file itself, as an attachment
PUT /sites/<name>/files/<path> write the body to the file, its folders created; with unzip=true the body
is a zip archive extracted into the folder of the path: {"path", "files"}
POST /sites/<name>/files/<path> {"folder": true} creates the folder, {"rename": "<path>"} moves it
DELETE /sites/<name>/files/<path> remove the file or folder
The body of a PUT needs a Content-Length of at most sites.upload_max_mb MiB.
"""
try:
target = vpnrouter_sites.resolve(name, urllib.parse.unquote(relative))
if self.command in ('GET', 'HEAD'):
if target.is_dir():
return 200, vpnrouter_sites.entries(name, target)
if not target.is_file():
return 404, {'error': f'{relative} does not exist'}
with open(target, 'rb') as handle:
size = os.fstat(handle.fileno()).st_size
self.send_response(200)
self.send_header('Content-Type', mimetypes.guess_type(target.name)[0] or 'application/octet-stream')
self.send_header('Content-Length', str(size))
self.send_header('Content-Disposition',
f"attachment; filename*=UTF-8''{urllib.parse.quote(target.name)}")
self.send_header('Cache-Control', 'no-store')
self.end_headers()
if send_body:
shutil.copyfileobj(handle, self.wfile, vpnrouter_sites.CHUNK)
return None
if self.command == 'PUT':
length = self.headers.get('Content-Length')
if not length or not length.isdigit():
return 411, {'error': 'the body needs a Content-Length'}
limit = vpnrouter.load_config()['sites']['upload_max_mb'] * MIB
if query.get('unzip', [''])[0] == 'true':
count = vpnrouter_sites.extract(self.rfile, int(length), target, limit)
return 200, {'path': vpnrouter_sites.relative_of(name, target), 'files': count}
vpnrouter_sites.receive(self.rfile, int(length), target, limit)
return 200, {'path': vpnrouter_sites.relative_of(name, target), 'files': 1}
if self.command == 'POST':
return site_file_action(name, target, payload)
if self.command == 'DELETE':
if not target.exists() and not target.is_symlink():
return 404, {'error': f'{relative} does not exist'}
vpnrouter_sites.delete(name, target)
return 200, {'path': vpnrouter_sites.relative_of(name, target), 'removed': True}
except vpnrouter_sites.Failure as failure:
return SITE_STATUS.get(failure.status, 500), {'error': str(failure)}
except OSError as exc:
return 500, {'error': f'{exc.filename or relative}: {exc.strerror}'}
except SystemExit as exc:
return 500, {'error': str(exc.code)}
return 405, {'error': f'{self.command} is not allowed on a site\'s files'}
def exact(self, size):
"""The next size bytes from the client; ConnectionError when it closes first."""
data = self.rfile.read(size)
if len(data) != size:
raise ConnectionError('the client closed the connection')
return data
def message(self, send):
"""The opcode and the payload of the client's next WebSocket data message, or CLOSE and its payload.
Fragments are joined and unmasked; a ping is answered with send, a pong ignored. ValueError for an unmasked
frame or a message over MESSAGE_LIMIT.
"""
opcode, payload = None, b''
while True:
head = self.exact(2)
code, size = head[0] & 0x0F, head[1] & 0x7F
if not head[1] & 0x80:
raise ValueError('a frame of the client is not masked')
if size == 126:
size = int.from_bytes(self.exact(2), 'big')
elif size == 127:
size = int.from_bytes(self.exact(8), 'big')
if len(payload) + size > MESSAGE_LIMIT:
raise ValueError(f'a message is larger than {MESSAGE_LIMIT} bytes')
mask = self.exact(4) * (size // 4 + 1)
data = bytes(byte ^ key for byte, key in zip(self.exact(size), mask))
if code == PING:
send(PONG, data)
continue
if code == PONG:
continue
if code == CLOSE:
return CLOSE, data
opcode = code if opcode is None else opcode
payload += data
if head[0] & 0x80:
return opcode, payload
def shell(self, name):
"""Answer GET /shell?ticket=<ticket>, a WebSocket handshake, with a login shell of the user's console account
(vpnrouter_users.console) on a terminal.
The ticket must come from POST /shell for the user named by the token. Binary messages of the client are its
keystrokes, a text message {"cols", "rows"} sets the window size, and binary messages of the server are the
terminal's output. The shell ends after web.shell_idle_minutes without a keystroke, when it exits, when the
client closes the connection, or when a check of the user (SHELL_CHECK_SECONDS) finds its console login off or
its TOTP token gone; the server then closes it with the reason. Each opening, end and refusal is logged
with its reason.
"""
def refuse(code, message):
log(f'shell of {name} from {self.client()} refused: {message}')
body = json.dumps({'error': message}).encode() + b'\n'
self.send_response(code)
self.send_header('Content-Type', 'application/json')
self.send_header('Content-Length', str(len(body)))
self.end_headers()
self.wfile.write(body)
key = self.headers.get('Sec-WebSocket-Key', '').strip()
if self.headers.get('Upgrade', '').lower() != 'websocket' or 'upgrade' not in self.headers.get('Connection', '').lower() \
or self.headers.get('Sec-WebSocket-Version') != '13' or not key:
return refuse(400, 'GET /shell must be a WebSocket handshake')
query = urllib.parse.parse_qs(urllib.parse.urlsplit(self.path).query)
if not take_ticket(name, query.get('ticket', [''])[0]):
return refuse(403, 'the ticket is not valid; POST /shell gives one')
try:
options = vpnrouter.load_config()['web']
except SystemExit as exc:
return refuse(500, str(exc.code))
if not options['shell']:
return refuse(403, 'the shell is disabled; vpn-config-edit set web.shell true enables it')
try:
login, refusal = console_account(name)
except (vpnrouter_users.Failure, ldap.LDAPError, OSError) as exc:
return refuse(500, f'the user directory failed: {exc}')
if refusal:
return refuse(403, refusal)
try:
account = pwd.getpwnam(login)
except KeyError:
return refuse(500, f'the system does not know the account {login}; is nslcd running?')
unit = f'vpn-router-shell-{secrets.token_hex(4)}'
master, terminal = os.openpty()
try:
process = subprocess.Popen(shell_command(account, unit), stdin=terminal, stdout=terminal, stderr=terminal,
env={'PATH': '/usr/sbin:/usr/bin:/sbin:/bin'})
except OSError as exc:
os.close(master)
return refuse(500, f'the shell cannot be started: {exc}')
finally:
os.close(terminal)
accept = base64.b64encode(hashlib.sha1((key + WEBSOCKET_GUID).encode()).digest()).decode()
self.send_response(101, 'Switching Protocols')
self.send_header('Upgrade', 'websocket')
self.send_header('Connection', 'Upgrade')
self.send_header('Sec-WebSocket-Accept', accept)
self.end_headers()
self.close_connection = True
client, started, idle = self.client(), time.monotonic(), options['shell_idle_minutes'] * 60
log(f'shell of {name} from {client} opened as {account.pw_name} in {unit}')
lock, ended, typed = threading.Lock(), threading.Event(), [started]
def send(opcode, payload):
with lock:
self.wfile.write(frame(opcode, payload))
def receive():
"""Pass the client's keystrokes and window sizes to the terminal until it closes the connection."""
try:
while True:
opcode, payload = self.message(send)
if opcode == CLOSE:
break
if opcode == BINARY:
os.write(master, payload)
typed[0] = time.monotonic()
with _shell_lock:
_shell_used[name] = typed[0]
elif opcode == TEXT:
size = json.loads(payload)
fcntl.ioctl(master, termios.TIOCSWINSZ, struct.pack('HHHH', int(size['rows']), int(size['cols']), 0, 0))
except (OSError, ValueError, KeyError, TypeError, struct.error):
pass
finally:
ended.set()
reader = threading.Thread(target=receive, name=f'{unit} input', daemon=True)
reader.start()
reason = 'the shell ended'
checked = started
try:
while not ended.is_set():
left = typed[0] + idle - time.monotonic()
if left <= 0:
reason = f'no keystroke for {options["shell_idle_minutes"]} minutes'
break
# The user's console login and TOTP token are checked again every SHELL_CHECK_SECONDS.
if time.monotonic() - checked >= SHELL_CHECK_SECONDS:
checked = time.monotonic()
try:
refusal = console_account(name)[1]
except (vpnrouter_users.Failure, ldap.LDAPError, OSError) as exc:
refusal = f'the user directory failed: {exc}'
if refusal:
reason = refusal
break
if select.select([master], [], [], min(left, 1))[0]:
try:
output = os.read(master, 65536)
except OSError:
output = b''
if not output:
break
send(BINARY, output)
else:
reason = 'the client closed the connection'
except OSError:
reason = 'the connection failed'
run('systemctl', 'stop', unit)
try:
send(CLOSE, (1000).to_bytes(2, 'big') + reason.encode()[:120])
except OSError:
pass
try:
self.connection.shutdown(socket.SHUT_RDWR)
except OSError:
pass
reader.join()
os.close(master)
try:
process.wait(timeout=10)
except subprocess.TimeoutExpired:
process.kill()
process.wait()
log(f'shell of {name} from {client} in {unit} ended after {int(time.monotonic() - started)} seconds: {reason}')
def answer(self, send_body):
path = self.path.split('?', 1)[0]
extra = []
self.principal = None
# A body that is not read must not be taken for the next request.
if self.command not in ('GET', 'HEAD'):
self.close_connection = True
directory = next((name for name in ('users', 'service-accounts', 'groups', 'account', 'site-to-site', 'backup',
'ca', 'shell', 'remote-access', 'logs', 'sites', 'dns')
if path == f'/{name}' or path.startswith(f'/{name}/')), None)
protected = path in ('/status/full', '/config', '/metrics/history', '/metrics/settings', '/postgresql') or \
directory is not None
signing = path in ('/auth', '/auth/passkey', '/auth/passkey/complete', '/auth/refresh', '/auth/logout')
oidc = (path == '/.well-known/openid-configuration' or path.startswith('/oidc/')) and \
self.server.server_address[0] == vpnrouter_net.LOOPBACK
files = SITE_FILES.match(path) if directory == 'sites' else None
query = urllib.parse.parse_qs(urllib.parse.urlsplit(self.path).query)
state = current() if protected or signing else {}
claims = self.claims(state['key']) if protected and state['key'] else None
# The body of a file upload is read as a stream, after the request is allowed.
payload = self.request_body() if (protected or signing) and self.command not in ('GET', 'HEAD') and \
not (files and self.command == 'PUT') else None
if claims is not None:
self.principal = 'admin' if claims.get('admin') is True else claims.get('sub')
if (protected or signing) and not state['key']:
body, kind, code = b'Not found\n', 'text/plain; charset=utf-8', 404
elif signing and self.command != 'POST':
body, kind, code = b'Method not allowed\n', 'text/plain; charset=utf-8', 405
elif signing:
wait = auth_wait(self.client()) if path not in ('/auth/logout', '/auth/passkey') else 0
if wait:
code, document = 429, {'error': f'too many sign-in calls; retry in {wait} seconds'}
extra.append(('Retry-After', str(wait)))
else:
code, document = auth(path, payload)
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif protected and claims is None:
body, kind, code = b'Unauthorized\n', 'text/plain; charset=utf-8', 401
extra.append(('WWW-Authenticate', 'Bearer'))
elif protected and (refused := self.refusal(claims, path, payload)):
code, kind = refused[0], 'application/json'
body = json.dumps({'error': refused[1]}, indent=2).encode() + b'\n'
elif path == '/status/full' and self.command in ('GET', 'HEAD'):
document = shown(self.status(), 'full', self.client())
body, kind, code = json.dumps(document, indent=2).encode() + b'\n', 'application/json', 200
elif path == '/config' and self.command in ('GET', 'HEAD'):
body, kind, code = json.dumps(details(state), indent=2).encode() + b'\n', 'application/json', 200
elif path == '/postgresql' and self.command in ('GET', 'HEAD'):
body, kind, code = json.dumps(postgresql_databases(), indent=2).encode() + b'\n', 'application/json', 200
elif path == '/metrics/history' and self.command in ('GET', 'HEAD'):
query = urllib.parse.parse_qs(urllib.parse.urlsplit(self.path).query)
code, document = metrics_history(query.get('range', ['hour'])[0])
body, kind = json.dumps(document).encode() + b'\n', 'application/json'
elif path == '/metrics/settings':
code, document = metrics_settings(self.command, payload)
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif directory == 'shell' and (path, self.command) == ('/shell', 'GET'):
return self.shell(claims.get('sub'))
elif directory == 'shell':
code, document = shell_ticket(claims.get('sub'), payload) if (path, self.command) == ('/shell', 'POST') \
else (404, {'error': f'no route {self.command} {path}'})
if code != 200:
log(f'shell ticket of {claims.get("sub")} from {self.client()} refused: {document["error"]}')
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif directory == 'account':
code, document = account(claims['sub'], self.command, path, payload)
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif directory == 'site-to-site':
code, document = site_to_site(self.command, path, payload)
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif directory == 'remote-access':
code, document = remote_access(state, self.command, path, payload)
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif directory == 'backup':
code, document = backup(self.command, path, payload)
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif directory == 'ca':
code, document = ca(self.command, path, payload)
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif directory == 'logs':
code, document = logs(self.command, path, query)
body, kind = json.dumps(document).encode() + b'\n', 'application/json'
elif files:
answered = self.site_files(files.group(1), files.group(2) or '', query, payload, send_body)
if answered is None:
return
code, document = answered
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif directory == 'sites':
code, document = sites(self.command, path, query, payload)
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif directory == 'dns':
code, document = dns(self.command, path, query, payload)
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif directory == 'service-accounts':
code, document = service_accounts(self.command, path, payload)
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif directory is not None:
code, document = (users if directory == 'users' else groups)(state, self.command, path, payload)
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif oidc:
wait = auth_wait(self.client()) if (self.command, path) == ('POST', '/oidc/login') else 0
if wait:
code, kind = 429, 'application/json'
body = json.dumps({'error': f'too many sign-in calls; retry in {wait} seconds'}).encode() + b'\n'
extra.append(('Retry-After', str(wait)))
else:
code, kind, body, headers = vpnrouter_oidc.answer(self.command, self.path, self.headers, self.raw_body())
extra += headers
elif self.command not in ('GET', 'HEAD'):
body, kind, code = b'Method not allowed\n', 'text/plain; charset=utf-8', 405
elif path.startswith('/pki/') and vpnrouter.PKI_NAME.match(path[5:-4]) and path[-4:] in PUBLISHED and \
(vpnrouter_pki.PUBLISH_DIR / path[5:]).is_file():
body, kind, code = (vpnrouter_pki.PUBLISH_DIR / path[5:]).read_bytes(), PUBLISHED[path[-4:]], 200
elif path in ('/status', '/'):
document = shown(self.status(), section(read_json(CONFIG_FILE), 'web').get('status_content'), self.client())
agent = self.headers.get('User-Agent', '').lower()
if path == '/status':
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
elif 'curl' in agent or 'wget' in agent:
body, kind = text_of(document).encode(), 'text/plain; charset=utf-8'
else:
body, kind = html_of(document).encode(), 'text/html; charset=utf-8'
code = 200
else:
body, kind, code = b'Not found\n', 'text/plain; charset=utf-8', 404
self.send_response(code)
self.send_header('Content-Type', kind)
self.send_header('Content-Length', str(len(body)))
self.send_header('Cache-Control', 'no-store')
for name, value in extra:
self.send_header(name, value)
self.end_headers()
if send_body:
self.wfile.write(body)
def raw_body(self):
"""The request body's bytes, b'' without one or over BODY_LIMIT."""
try:
size = int(self.headers.get('Content-Length') or 0)
except ValueError:
return b''
return self.rfile.read(size) if 0 < size <= BODY_LIMIT else b''
def request_body(self):
"""The request body as JSON, None when there is none, or the string 'invalid'."""
length = self.headers.get('Content-Length')
if not length:
return None
try:
size = int(length)
if not 0 <= size <= BODY_LIMIT:
return 'invalid'
return json.loads(self.rfile.read(size) or b'null')
except ValueError:
return 'invalid'
def do_GET(self):
self.answer(True)
def do_HEAD(self):
self.answer(False)
def do_POST(self):
self.answer(True)
def do_PUT(self):
self.answer(True)
def do_DELETE(self):
self.answer(True)
def log_message(self, format, *args):
principal = getattr(self, 'principal', None) or '-'
sys.stderr.write('%s - %s [%s] %s\n' % (self.client_address[0], principal, self.log_date_time_string(), format % args))
def log(message):
print(f'vpn-router-web: {message}', file=sys.stderr, flush=True)
def initialize():
"""Make the signing key, the OpenID Provider's key, the refresh tokens' database and the web sites' database,
retrying every INIT_RETRY_SECONDS until all are ready."""
reported = None
while True:
phase = 'reading the configuration'
try:
cfg = vpnrouter.load_config()
phase = 'making the signing key'
vpnrouter_auth.ensure_key()
phase = 'making the OpenID Provider key'
vpnrouter_oidc.ensure_key()
phase = 'creating the token database'
vpnrouter_auth.provision(cfg)
phase = 'connecting to the token database'
with vpnrouter_auth.database(cfg):
pass
phase = 'creating the database of the web sites'
vpnrouter_sites.provision()
phase = 'connecting to the database of the web sites'
with vpnrouter_sites.database():
pass
except (OSError, SystemExit, vpnrouter_auth.DatabaseError, vpnrouter_sites.DatabaseError) as exc:
error = f'{phase} failed: {exc.code if isinstance(exc, SystemExit) else exc}'
if error != reported:
log(f'{error}; retrying every {INIT_RETRY_SECONDS} seconds')
reported = error
time.sleep(INIT_RETRY_SECONDS)
continue
log('ready')
return
def main():
signal.signal(signal.SIGTERM, lambda *_: sys.exit(0))
initialize()
try:
vpnrouter_serve.serve(lambda: vpnrouter_net.web_listen(read_json(CONFIG_FILE), DEFAULT_PORT), Handler, log,
DISCOVERY_SECONDS)
except KeyboardInterrupt:
pass
return 0
if __name__ == '__main__':
sys.exit(main())
Reference in new issue View Git Blame Copy Permalink