feature: Caddy serves static web sites from folders on the router, defined in their own database, with files uploaded through the web service or WebDAV, and an optional PowerDNS name server answers for a zone of the sites.
Release / release (push) Successful in 21s
Release / release (push) Successful in 21s
This commit is contained in:
1 parent
77bdbcb058
commit
ea8ca97417
27 files changed
+2234
-134
No files matched your search
@@ -1,3 +1,39 @@
|
||||
vpn-router (0.31.0-1) unstable; urgency=medium
|
||||
|
||||
feature: Caddy serves static web sites from folders on the router, defined in their own database, with files uploaded through the web service or WebDAV, and an optional PowerDNS name server answers for a zone of the sites.
|
||||
|
||||
* The new database vpn-router-sites holds the web sites: host names,
|
||||
port, interfaces, TLS (none, Let's Encrypt, an issuing CA or a
|
||||
certificate and key), index file, single-page fallback and the group
|
||||
whose members change the files. sites.enabled serves them from
|
||||
/var/lib/vpn-router/sites/<name>; setup writes their blocks into the
|
||||
Caddyfile and opens their ports, and Caddy also runs without HTTPS of
|
||||
the router while a site is served.
|
||||
* vpn-router-site lists, shows, adds, changes and removes the sites, and
|
||||
bash completes it.
|
||||
* /sites manages the sites and the settings of the sites section, and
|
||||
under /sites/<name>/files/ lists, downloads, uploads, extracts zip
|
||||
archives into, creates, renames and removes the files of a site, up to
|
||||
sites.upload_max_mb; members of the site's editors group may use its
|
||||
files.
|
||||
* sites.webdav runs rclone's WebDAV server of the sites' folders behind
|
||||
Caddy on sites.webdav_port, signing users in with the directory and
|
||||
showing each the sites it may change.
|
||||
* dns.enabled runs PowerDNS with its PostgreSQL backend in the new
|
||||
database vpn-router-dns for dns.zone on the addresses of dns.listen:
|
||||
SOA, NS, the A and AAAA records of dns.addresses for the zone, the
|
||||
name server and the sites' host names in the zone, and manual records
|
||||
added with /dns.
|
||||
* rclone and PowerDNS are installed from the Ubuntu archive by
|
||||
vpn-router-install@<feature>.service when their feature is on.
|
||||
* Backups hold the sites' folders and the dumps of both databases; a
|
||||
restore puts them back. Purge removes the databases, their roles and
|
||||
password files, and the settings of PowerDNS.
|
||||
* Renaming a group renames it in the sites; a group a site has cannot be
|
||||
removed.
|
||||
|
||||
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Mon, 26 Oct 2026 21:00:00 +0100
|
||||
|
||||
vpn-router (0.30.0-1) unstable; urgency=medium
|
||||
|
||||
feature: the web service serves the last lines of the journal of the router's units and of the log files web.log_files lists, at /logs.
|
||||
|
||||
@@ -42,6 +42,8 @@ Depends: ${misc:Depends},
|
||||
Recommends: bash-completion
|
||||
Suggests: caddy,
|
||||
rclone,
|
||||
pdns-server,
|
||||
pdns-backend-pgsql,
|
||||
proxmox-backup-client-static
|
||||
Description: Linux VPN router with IPSec, road-warrior, OpenVPN and WireGuard support
|
||||
Configures a Linux host or virtual machine as a VPN router: site-to-site
|
||||
@@ -52,7 +54,9 @@ Description: Linux VPN router with IPSec, road-warrior, OpenVPN and WireGuard su
|
||||
needs. Its two-level CA, whose root may be kept offline, has issuing CAs for
|
||||
the VPN clients, for site-to-site and for general use, and publishes their
|
||||
certificates and CRLs. Caddy, installed by the package from the Ubuntu
|
||||
archive or from Caddy's repository, serves HTTPS, and an optional OpenID
|
||||
archive or from Caddy's repository, serves HTTPS and static web sites, whose
|
||||
files are uploaded through the web service or WebDAV, an optional PowerDNS
|
||||
name server answers for a zone of the sites, and an optional OpenID
|
||||
Provider signs the directory's users in to other systems. It keeps metrics of the
|
||||
router in PostgreSQL and serves them for an OpenTelemetry Collector, and
|
||||
makes scheduled backups to a local folder, copied to the cloud with rclone,
|
||||
|
||||
@@ -2,3 +2,4 @@ etc/vpn-router
|
||||
usr/lib/vpn-router
|
||||
usr/lib/vpn-router/vpnrouter_platforms
|
||||
usr/share/vpn-router/templates
|
||||
var/lib/vpn-router/sites
|
||||
@@ -11,6 +11,10 @@ src/usr/lib/vpn-router/vpnrouter_metrics.py usr/lib/vpn-router/
|
||||
src/usr/lib/vpn-router/vpnrouter_status.py usr/lib/vpn-router/
|
||||
src/usr/lib/vpn-router/vpnrouter_backup.py usr/lib/vpn-router/
|
||||
src/usr/lib/vpn-router/vpnrouter_pki.py usr/lib/vpn-router/
|
||||
src/usr/lib/vpn-router/vpnrouter_sites.py usr/lib/vpn-router/
|
||||
src/usr/lib/vpn-router/vpnrouter_dns.py usr/lib/vpn-router/
|
||||
src/usr/lib/vpn-router/install-packages usr/lib/vpn-router/
|
||||
src/usr/lib/vpn-router/webdav-auth usr/lib/vpn-router/
|
||||
src/usr/lib/vpn-router/openvpn-same-user usr/lib/vpn-router/
|
||||
src/usr/lib/vpn-router/openvpn-issuer usr/lib/vpn-router/
|
||||
src/usr/sbin/vpn-router-user usr/sbin/
|
||||
@@ -20,6 +24,7 @@ src/usr/sbin/vpn-config-edit usr/sbin/
|
||||
src/usr/sbin/vpn-router-backup usr/sbin/
|
||||
src/usr/sbin/vpn-router-ca usr/sbin/
|
||||
src/usr/sbin/vpn-router-caddy usr/sbin/
|
||||
src/usr/sbin/vpn-router-site usr/sbin/
|
||||
src/usr/lib/vpn-router/configure usr/lib/vpn-router/
|
||||
src/usr/lib/vpn-router/docker-rules usr/lib/vpn-router/
|
||||
src/usr/lib/vpn-router/setup usr/lib/vpn-router/
|
||||
|
||||
@@ -11,10 +11,10 @@ strip_ufw_blocks() {
|
||||
|
||||
tmp="$(mktemp)"
|
||||
awk '
|
||||
/^# (IPSEC|WIREGUARD|OPENVPN|P2S DNS|WEB|BOOTSTRAP|MANAGER|HTTPS|METRICS|ROUTER FORWARD|ROUTER NAT|ROUTER MSS) RULES START$/ {
|
||||
/^# (IPSEC|WIREGUARD|OPENVPN|P2S DNS|WEB|BOOTSTRAP|MANAGER|HTTPS|METRICS|SITES|DNS|ROUTER FORWARD|ROUTER NAT|ROUTER MSS) RULES START$/ {
|
||||
skip = 1; pending = 0; next
|
||||
}
|
||||
/^# (IPSEC|WIREGUARD|OPENVPN|P2S DNS|WEB|BOOTSTRAP|MANAGER|HTTPS|METRICS|ROUTER FORWARD|ROUTER NAT|ROUTER MSS) RULES END$/ {
|
||||
/^# (IPSEC|WIREGUARD|OPENVPN|P2S DNS|WEB|BOOTSTRAP|MANAGER|HTTPS|METRICS|SITES|DNS|ROUTER FORWARD|ROUTER NAT|ROUTER MSS) RULES END$/ {
|
||||
skip = 0; next
|
||||
}
|
||||
skip { next }
|
||||
@@ -50,6 +50,7 @@ case "$1" in
|
||||
/etc/openvpn/server/vpn-router.conf \
|
||||
/etc/openvpn/server/vpn-router-ldap.conf \
|
||||
/etc/caddy/vpn-router.Caddyfile \
|
||||
/etc/powerdns/pdns.d/vpn-router.conf \
|
||||
/etc/vpn-router/vpn-router.json \
|
||||
/etc/vpn-router/web.key \
|
||||
/etc/vpn-router/oidc.key \
|
||||
@@ -63,7 +64,7 @@ case "$1" in
|
||||
# The name service no longer looks up the console accounts of the router's directory.
|
||||
sed -i '/^passwd:/s/ ldap\b//' /etc/nsswitch.conf
|
||||
rm -rf /var/lib/vpn-router/backup/staging /var/lib/vpn-router/backup/restore \
|
||||
/var/lib/vpn-router/pki-publish /etc/caddy/vpn-router-pki \
|
||||
/var/lib/vpn-router/pki-publish /etc/caddy/vpn-router-pki /etc/caddy/vpn-router-sites \
|
||||
/etc/openvpn/server/vpn-router-ca.pem /etc/openvpn/server/vpn-router-crl.pem
|
||||
rmdir --ignore-fail-on-non-empty /var/lib/vpn-router/backup /var/lib/vpn-router 2>/dev/null || true
|
||||
|
||||
@@ -74,12 +75,23 @@ case "$1" in
|
||||
fi
|
||||
rm -f /etc/vpn-router/database.secret
|
||||
|
||||
# Key material and users are not package state: /etc/vpn-router/pki,
|
||||
# The databases and roles of the web sites and of the name server, when the package created them.
|
||||
for database in vpn-router-sites vpn-router-dns; do
|
||||
secret="/etc/vpn-router/${database#vpn-router-}-database.secret"
|
||||
if [ -f "$secret" ] && command -v psql >/dev/null 2>&1; then
|
||||
runuser -u postgres -- psql -X -q -c "DROP DATABASE IF EXISTS \"$database\"" || true
|
||||
runuser -u postgres -- psql -X -q -c "DROP ROLE IF EXISTS \"$database\"" || true
|
||||
fi
|
||||
rm -f "$secret"
|
||||
done
|
||||
|
||||
# Key material, users and content are not package state: /etc/vpn-router/pki,
|
||||
# /etc/vpn-router/ldap, /etc/vpn-router/rclone.conf, the user directory
|
||||
# in slapd, the WireGuard key pair, everything under /etc/swanctl, the
|
||||
# backups and what a restore replaced are left in place. Caddy stays
|
||||
# installed, so its repository, which vpn-router-caddy configures, stays
|
||||
# too.
|
||||
# files of the web sites in /var/lib/vpn-router/sites, the backups and
|
||||
# what a restore replaced are left in place. Caddy, rclone and PowerDNS
|
||||
# stay installed, so Caddy's repository, which vpn-router-caddy
|
||||
# configures, stays too.
|
||||
rmdir --ignore-fail-on-non-empty /etc/vpn-router 2>/dev/null || true
|
||||
;;
|
||||
esac
|
||||
|
||||
@@ -11,3 +11,5 @@ override_dh_installsystemd:
|
||||
dh_installsystemd --name=vpn-router-backup --no-enable --no-start
|
||||
dh_installsystemd --name=vpn-router-pki --no-enable --no-start
|
||||
dh_installsystemd --name=vpn-router-caddy --no-enable --no-start
|
||||
dh_installsystemd --name=vpn-router-webdav --no-enable --no-start
|
||||
dh_installsystemd --name=vpn-router-install@ --no-enable --no-start
|
||||
@@ -0,0 +1,10 @@
|
||||
[Unit]
|
||||
Description=VPN Router installation of the packages of %i
|
||||
Documentation=file:/usr/share/doc/vpn-router/README.md
|
||||
After=network-online.target
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
ExecStart=/usr/lib/vpn-router/install-packages %i
|
||||
TimeoutStartSec=20min
|
||||
@@ -15,7 +15,7 @@ NoNewPrivileges=yes
|
||||
ProtectSystem=strict
|
||||
RuntimeDirectory=vpn-router
|
||||
RuntimeDirectoryPreserve=yes
|
||||
ReadWritePaths=-/etc/vpn-router -/etc/swanctl/x509crl -/etc/openvpn/server -/var/lib/vpn-router/pki-publish
|
||||
ReadWritePaths=-/etc/vpn-router -/etc/swanctl/x509crl -/etc/openvpn/server -/var/lib/vpn-router/pki-publish -/var/lib/vpn-router/sites
|
||||
ProtectHome=yes
|
||||
PrivateTmp=yes
|
||||
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
[Unit]
|
||||
Description=VPN Router WebDAV of the web sites' folders
|
||||
Documentation=file:/usr/share/doc/vpn-router/README.md
|
||||
After=network-online.target slapd.service postgresql.service
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
ExecStart=/usr/bin/rclone serve webdav --addr 127.0.0.1:7446 --auth-proxy /usr/lib/vpn-router/webdav-auth --config /dev/null --cache-dir /var/cache/vpn-router-webdav --vfs-cache-mode writes --umask 022
|
||||
Restart=on-failure
|
||||
RestartSec=15
|
||||
NoNewPrivileges=yes
|
||||
ProtectSystem=strict
|
||||
ReadWritePaths=/var/lib/vpn-router/sites
|
||||
CacheDirectory=vpn-router-webdav
|
||||
RuntimeDirectory=vpn-router-webdav
|
||||
ProtectHome=yes
|
||||
PrivateTmp=yes
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
+36
@@ -0,0 +1,36 @@
|
||||
#!/usr/bin/python3
|
||||
"""Install the packages of a feature from the Ubuntu archive, then restart vpn-router-setup without waiting.
|
||||
|
||||
Run by vpn-router-install@<feature>.service, which vpn-router-setup starts when the feature is on and its packages are
|
||||
missing: webdav (rclone) and dns (PowerDNS with its gpgsql backend). Recommended packages are left out, so PowerDNS
|
||||
comes without its bind backend. A failure is a message on standard error and the exit status 1.
|
||||
|
||||
install-packages FEATURE
|
||||
"""
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
#: The packages of each feature.
|
||||
FEATURES = {'webdav': ['rclone'], 'dns': ['pdns-server', 'pdns-backend-pgsql']}
|
||||
|
||||
#: apt-get, waiting up to ten minutes for dpkg's lock, which a package installation that runs setup holds.
|
||||
APT = ['apt-get', '-o', 'DPkg::Lock::Timeout=600']
|
||||
|
||||
|
||||
def main():
|
||||
if len(sys.argv) != 2 or sys.argv[1] not in FEATURES:
|
||||
sys.exit(f'usage: install-packages {"|".join(FEATURES)}')
|
||||
environment = dict(os.environ, DEBIAN_FRONTEND='noninteractive')
|
||||
for args in ([*APT, 'update'], [*APT, '-y', '--no-install-recommends', '-o', 'Dpkg::Options::=--force-confdef',
|
||||
'-o', 'Dpkg::Options::=--force-confold', 'install', *FEATURES[sys.argv[1]]]):
|
||||
print(f'$ {" ".join(args)}', flush=True)
|
||||
if subprocess.run(args, env=environment).returncode != 0:
|
||||
sys.exit(f'install-packages: {args[0]} failed')
|
||||
subprocess.run(['systemctl', 'restart', '--no-block', 'vpn-router-setup.service'])
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
@@ -7,7 +7,7 @@ and service reloads. Idempotent - a second run changes nothing.
|
||||
|
||||
Until a run has succeeded (SETUP_DONE), every run starts the bootstrap
|
||||
services, BOOTSTRAP_UNITS, which show the progress and the failure recorded in
|
||||
SETUP_STATE on BOOTSTRAP_PORTS, and the firewall opens those ports on the
|
||||
SETUP_STATE on vpnrouter.BOOTSTRAP_PORTS, and the firewall opens those ports on the
|
||||
internal interface. The first successful run writes SETUP_DONE, closes the
|
||||
ports and stops the bootstrap services; a failed run leaves them running.
|
||||
|
||||
@@ -33,8 +33,11 @@ sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
|
||||
import ldap
|
||||
|
||||
import vpnrouter
|
||||
import vpnrouter_auth
|
||||
import vpnrouter_dns
|
||||
import vpnrouter_ldap
|
||||
import vpnrouter_pki
|
||||
import vpnrouter_sites
|
||||
import vpnrouter_users
|
||||
from vpnrouter import fail, warn
|
||||
|
||||
@@ -49,18 +52,12 @@ SETUP_STATE = pathlib.Path('/run/vpn-router/setup.json')
|
||||
#: Written once a run has succeeded; until then setup runs the bootstrap services and opens their ports.
|
||||
SETUP_DONE = pathlib.Path('/var/lib/vpn-router/setup.done')
|
||||
|
||||
#: The units of the bootstrap services, and their ports: the Router Manager's and the router's.
|
||||
#: The units of the bootstrap services, whose ports are vpnrouter.BOOTSTRAP_PORTS.
|
||||
BOOTSTRAP_UNITS = ('vpn-router-bootstrap.service', 'vpn-router-manager-bootstrap.service')
|
||||
BOOTSTRAP_PORTS = (17080, 17081)
|
||||
|
||||
#: The Router Manager, vpn-router-manager, when it is installed, and its settings.
|
||||
MANAGER = pathlib.Path('/usr/lib/vpn-router-manager/manager')
|
||||
MANAGER_CONFIG = pathlib.Path('/etc/vpn-router-manager/manager.json')
|
||||
|
||||
#: The default admin of the Router Manager, {"username", "password"}, which its package writes from
|
||||
#: its debconf answers; setup creates the account and removes the file.
|
||||
MANAGER_ADMIN = pathlib.Path('/etc/vpn-router-manager/admin.json')
|
||||
MANAGER_PORT = 7080
|
||||
|
||||
#: The password of the metrics service account, which setup makes and keeps (root only).
|
||||
METRICS_SECRET = pathlib.Path('/etc/vpn-router/metrics.secret')
|
||||
@@ -77,9 +74,19 @@ CADDY_GROUP = 'caddy'
|
||||
#: The unit that installs Caddy from https.caddy_source with vpn-router-caddy, outside setup.
|
||||
CADDY_INSTALL_UNIT = 'vpn-router-caddy.service'
|
||||
|
||||
#: The certificates and keys of the issuing CAs behind Caddy's ACME server, <name>/cert.pem and key.pem.
|
||||
#: The certificates and keys of the issuing CAs behind Caddy's ACME server and the sites of tls ca, <name>/cert.pem and
|
||||
#: key.pem.
|
||||
CADDY_PKI_DIR = pathlib.Path('/etc/caddy/vpn-router-pki')
|
||||
|
||||
#: The certificates and keys of the sites of tls file, <site>/cert.pem and key.pem.
|
||||
CADDY_SITES_DIR = pathlib.Path('/etc/caddy/vpn-router-sites')
|
||||
|
||||
#: The unit of rclone's WebDAV server of the sites' folders.
|
||||
WEBDAV_UNIT = 'vpn-router-webdav.service'
|
||||
|
||||
#: The unit that installs the packages of a feature with install-packages, outside setup; {} is the feature.
|
||||
INSTALL_UNIT = 'vpn-router-install@{}.service'
|
||||
|
||||
#: The timer of vpn-router-ca maintain, which keeps the CRLs fresh.
|
||||
PKI_TIMER = 'vpn-router-pki.timer'
|
||||
|
||||
@@ -107,6 +114,8 @@ BLOCKS = (
|
||||
('# MANAGER RULES START', '# MANAGER RULES END'),
|
||||
('# HTTPS RULES START', '# HTTPS RULES END'),
|
||||
('# METRICS RULES START', '# METRICS RULES END'),
|
||||
('# SITES RULES START', '# SITES RULES END'),
|
||||
('# DNS RULES START', '# DNS RULES END'),
|
||||
('# ROUTER FORWARD RULES START', '# ROUTER FORWARD RULES END'),
|
||||
('# ROUTER NAT RULES START', '# ROUTER NAT RULES END'),
|
||||
('# ROUTER MSS RULES START', '# ROUTER MSS RULES END'),
|
||||
@@ -194,15 +203,6 @@ def is_configured(ctx):
|
||||
vpnrouter.openvpn_ready(ctx)))
|
||||
|
||||
|
||||
def manager_port():
|
||||
"""The port of the Router Manager: port in its settings, or MANAGER_PORT when it is not set or not a port."""
|
||||
try:
|
||||
port = json.loads(MANAGER_CONFIG.read_text(encoding='utf-8')).get('port')
|
||||
except (OSError, ValueError, AttributeError):
|
||||
return MANAGER_PORT
|
||||
return port if isinstance(port, int) and not isinstance(port, bool) and 1 <= port <= 65535 else MANAGER_PORT
|
||||
|
||||
|
||||
def filter_block(ctx):
|
||||
lines = [
|
||||
'',
|
||||
@@ -265,12 +265,13 @@ def filter_block(ctx):
|
||||
lines += [
|
||||
'',
|
||||
'# BOOTSTRAP RULES START',
|
||||
*(f'-A ufw-before-input -i {ctx["int_iface"]} -p tcp --dport {port} -j ACCEPT' for port in BOOTSTRAP_PORTS),
|
||||
*(f'-A ufw-before-input -i {ctx["int_iface"]} -p tcp --dport {port} -j ACCEPT'
|
||||
for port in vpnrouter.BOOTSTRAP_PORTS),
|
||||
'# BOOTSTRAP RULES END',
|
||||
]
|
||||
|
||||
if MANAGER.exists() and interfaces:
|
||||
port = manager_port()
|
||||
if vpnrouter.MANAGER.exists() and interfaces:
|
||||
port = vpnrouter.manager_port()
|
||||
lines += [
|
||||
'',
|
||||
'# MANAGER RULES START',
|
||||
@@ -292,8 +293,9 @@ def filter_block(ctx):
|
||||
publish = ctx['pki_publish']
|
||||
ports = [port for wanted, port in ((ctx['web_enabled'] or 'https' in publish or ctx['pki_acme'],
|
||||
ctx['https_web_port']),
|
||||
(MANAGER.exists(), ctx['https_manager_port']),
|
||||
(vpnrouter.MANAGER.exists(), ctx['https_manager_port']),
|
||||
(vpnrouter.metrics_ready(ctx), ctx['metrics_https_port']),
|
||||
(vpnrouter.webdav_ready(ctx), ctx['webdav_port']),
|
||||
('http-80' in publish, vpnrouter.ACME_HTTP_PORT),
|
||||
('https-default' in publish, vpnrouter.HTTPS_DEFAULT_PORT)) if wanted]
|
||||
lines += [
|
||||
@@ -305,6 +307,32 @@ def filter_block(ctx):
|
||||
'# HTTPS RULES END',
|
||||
]
|
||||
|
||||
# Each site's port is open on the interfaces of its listen; a site with a certificate of Let's Encrypt needs the HTTP
|
||||
# challenge on the external interface.
|
||||
by_listen = {'internal': [ctx['int_iface']], 'external': [ctx['wan_iface']],
|
||||
'both': [ctx['int_iface'], ctx['wan_iface']]}
|
||||
sites = ctx['sites'] or []
|
||||
rules = sorted({(name, site['port']) for site in sites for name in by_listen[site['listen']] if name})
|
||||
if ctx['wan_iface'] and any(site['tls'] == 'acme' for site in sites):
|
||||
rules = sorted(set(rules) | {(ctx['wan_iface'], vpnrouter.ACME_HTTP_PORT)})
|
||||
if rules:
|
||||
lines += [
|
||||
'',
|
||||
'# SITES RULES START',
|
||||
*(f'-A ufw-before-input -i {name} -p tcp --dport {port} -j ACCEPT' for name, port in rules),
|
||||
'# SITES RULES END',
|
||||
]
|
||||
|
||||
dns_interfaces = [name for name in by_listen[ctx['dns_listen']] if name] if vpnrouter.dns_ready(ctx) else []
|
||||
if dns_interfaces:
|
||||
lines += [
|
||||
'',
|
||||
'# DNS RULES START',
|
||||
*(f'-A ufw-before-input -i {name} -p {protocol} --dport 53 -j ACCEPT'
|
||||
for name in dns_interfaces for protocol in ('udp', 'tcp')),
|
||||
'# DNS RULES END',
|
||||
]
|
||||
|
||||
if vpnrouter.routing_ready(ctx):
|
||||
lines += ['', '# ROUTER FORWARD RULES START']
|
||||
for subnet in ctx['local_cidr_list']:
|
||||
@@ -640,22 +668,37 @@ def manage_openvpn(ctx, changed):
|
||||
run(['systemctl', 'restart', unit], check=False)
|
||||
|
||||
|
||||
def manage_caddy(cfg, ctx):
|
||||
"""Run Caddy, the HTTPS proxy of the web service, the Router Manager and the metrics service, while HTTPS is ready, and stop it when not.
|
||||
def caddy_address(host, port, tls):
|
||||
"""The address of a site block of the Caddyfile: the host, an IPv6 address in brackets, and the port, with http://
|
||||
for a plain HTTP site; no host answers any name."""
|
||||
host = f'[{host}]' if ':' in host else host
|
||||
return f'{"http://" if tls == "none" else ""}{host}:{port}'
|
||||
|
||||
CADDYFILE serves the web service on https.web_port, while the manager is installed the
|
||||
manager on https.manager_port, and while metrics are enabled the metrics service on
|
||||
metrics.https_port, all for wan.local_fqdn with a certificate from Let's Encrypt, and all
|
||||
calling the service on its loopback address. It also serves the CA files of
|
||||
vpnrouter_pki.PUBLISH_DIR at /pki/ on the endpoints of pki.publish that are Caddy's, and an
|
||||
ACME server at /acme/<name>/ on https.web_port for each issuing CA with acme and a
|
||||
certificate, whose certificate and key are copied to CADDY_PKI_DIR.
|
||||
|
||||
def manage_caddy(cfg, ctx):
|
||||
"""Run Caddy, the HTTPS proxy of the router's services and the server of the web sites, while either is wanted, and
|
||||
stop it when not.
|
||||
|
||||
While HTTPS is ready, CADDYFILE serves the web service on https.web_port, while the manager is installed the
|
||||
manager on https.manager_port, while metrics are enabled the metrics service on metrics.https_port, and while
|
||||
WebDAV is ready WebDAV on sites.webdav_port, all for wan.local_fqdn with a certificate from Let's Encrypt, and all
|
||||
calling the service on its loopback address. It also serves the CA files of vpnrouter_pki.PUBLISH_DIR at /pki/ on
|
||||
the endpoints of pki.publish that are Caddy's, and an ACME server at /acme/<name>/ on https.web_port for each
|
||||
issuing CA with acme and a certificate.
|
||||
|
||||
It serves each site of ctx's sites from its folder, with the TLS of the site: an issuing CA of tls ca issues its
|
||||
certificates, and the certificate and key of tls file are written to CADDY_SITES_DIR/<name>. The certificates and
|
||||
keys of the issuing CAs of both are copied to CADDY_PKI_DIR. While the sites cannot be read (ctx's sites is None),
|
||||
the Caddyfile is left as it is.
|
||||
|
||||
Caddy is installed by CADDY_INSTALL_UNIT, outside setup, when it is missing or when
|
||||
https.caddy_source is upstream and Caddy's repository is not configured; the unit restarts
|
||||
setup, whose next run serves with it.
|
||||
"""
|
||||
if not vpnrouter.https_ready(ctx):
|
||||
sites = ctx['sites']
|
||||
if not vpnrouter.https_ready(ctx) and not sites:
|
||||
if sites is None:
|
||||
return
|
||||
vpnrouter.remove(CADDYFILE)
|
||||
if unit_is_active(CADDY_UNIT):
|
||||
run(['systemctl', 'disable', '--now', CADDY_UNIT], check=False)
|
||||
@@ -669,21 +712,33 @@ def manage_caddy(cfg, ctx):
|
||||
if source == 'ubuntu' and vpnrouter.CADDY_APT_SOURCE.is_file():
|
||||
warn("https.caddy_source ubuntu is ignored: Caddy's repository is configured, and Caddy stays upstream's")
|
||||
|
||||
if sites is None:
|
||||
warn('the web sites cannot be read; the Caddyfile is left as it is')
|
||||
if not unit_is_active(CADDY_UNIT) and CADDYFILE.exists():
|
||||
run(['systemctl', 'enable', '--now', CADDY_UNIT], check=False)
|
||||
return
|
||||
try:
|
||||
group = grp.getgrnam(CADDY_GROUP).gr_gid
|
||||
except KeyError:
|
||||
fail(f'the group {CADDY_GROUP} does not exist; is the caddy package installed?')
|
||||
https = vpnrouter.https_ready(ctx)
|
||||
wanted = list(dict.fromkeys([*(ctx['pki_acme'] if https else []),
|
||||
*(site['ca'] for site in sites if site['tls'] == 'ca')]))
|
||||
try:
|
||||
credentials = {name: vpnrouter_pki.authority_files(name) for name in ctx['pki_acme']}
|
||||
credentials = {name: vpnrouter_pki.authority_files(name) for name in wanted}
|
||||
except ldap.LDAPError as exc:
|
||||
warn(f'the ACME servers are left out: the PKI cannot be read: {exc}')
|
||||
warn(f'the ACME servers and the sites of issuing CAs are left out: the PKI cannot be read: {exc}')
|
||||
credentials = {}
|
||||
acme = [name for name, files in credentials.items() if files is not None]
|
||||
authorities = [name for name, files in credentials.items() if files is not None]
|
||||
for site in sites:
|
||||
if site['tls'] == 'ca' and site['ca'] not in authorities:
|
||||
warn(f'site {site["name"]} is left out: issuing CA {site["ca"]} has no certificate')
|
||||
sites = [site for site in sites if site['tls'] != 'ca' or site['ca'] in authorities]
|
||||
copied = False
|
||||
for folder in (CADDY_PKI_DIR.iterdir() if CADDY_PKI_DIR.is_dir() else []):
|
||||
if folder.name not in acme:
|
||||
if folder.name not in authorities:
|
||||
shutil.rmtree(folder)
|
||||
for name in acme:
|
||||
for name in authorities:
|
||||
folder = CADDY_PKI_DIR / name
|
||||
for path in (CADDY_PKI_DIR, folder):
|
||||
path.mkdir(mode=0o750, exist_ok=True)
|
||||
@@ -691,12 +746,33 @@ def manage_caddy(cfg, ctx):
|
||||
for file, data in zip(('cert.pem', 'key.pem'), credentials[name]):
|
||||
copied |= vpnrouter_pki.write(folder / file, data, 0o640)
|
||||
os.chown(folder / file, 0, group)
|
||||
authorities = vpnrouter.issuing_cas(cfg)
|
||||
caddy_ctx = dict(ctx, manager_port=manager_port() if MANAGER.exists() else 0,
|
||||
files = {site['name']: site for site in sites if site['tls'] == 'file'}
|
||||
for folder in (CADDY_SITES_DIR.iterdir() if CADDY_SITES_DIR.is_dir() else []):
|
||||
if folder.name not in files:
|
||||
shutil.rmtree(folder)
|
||||
for name, site in files.items():
|
||||
folder = CADDY_SITES_DIR / name
|
||||
for path in (CADDY_SITES_DIR, folder):
|
||||
path.mkdir(mode=0o750, exist_ok=True)
|
||||
os.chown(path, 0, group)
|
||||
for file, data in (('cert.pem', site['certificate']), ('key.pem', site['private_key'])):
|
||||
copied |= vpnrouter_pki.write(folder / file, data.strip().encode() + b'\n', 0o640)
|
||||
os.chown(folder / file, 0, group)
|
||||
vpnrouter_sites.ensure_folders(site['name'] for site in sites)
|
||||
names = vpnrouter.issuing_cas(cfg)
|
||||
caddy_ctx = dict(ctx, https=https, manager_port=vpnrouter.manager_port() if vpnrouter.MANAGER.exists() else 0,
|
||||
pki_publish_dir=str(vpnrouter_pki.PUBLISH_DIR),
|
||||
acme_cas=[{'id': name, 'name': authorities[name]['name'].strip(),
|
||||
'cert': str(CADDY_PKI_DIR / name / 'cert.pem'),
|
||||
'key': str(CADDY_PKI_DIR / name / 'key.pem')} for name in acme])
|
||||
pki_cas=[{'id': name, 'name': names[name]['name'].strip(),
|
||||
'cert': str(CADDY_PKI_DIR / name / 'cert.pem'),
|
||||
'key': str(CADDY_PKI_DIR / name / 'key.pem')} for name in authorities],
|
||||
acme_cas=[name for name in authorities if https and name in ctx['pki_acme']],
|
||||
webdav=vpnrouter.webdav_ready(ctx), webdav_loopback_port=vpnrouter.WEBDAV_LOOPBACK_PORT,
|
||||
sites=[{'name': site['name'], 'tls': site['tls'], 'ca': site['ca'], 'spa': site['spa'],
|
||||
'index': site['index_file'], 'root': str(vpnrouter_sites.SITES_DIR / site['name']),
|
||||
'addresses': [caddy_address(host, site['port'], site['tls']) for host in site['hosts']]
|
||||
or [caddy_address('', site['port'], site['tls'])],
|
||||
'cert': str(CADDY_SITES_DIR / site['name'] / 'cert.pem'),
|
||||
'key': str(CADDY_SITES_DIR / site['name'] / 'key.pem')} for site in sites])
|
||||
changed = vpnrouter.render(vpnrouter._jinja_env(), caddy_ctx, 'Caddyfile.j2', CADDYFILE, 0o640)
|
||||
os.chown(CADDYFILE, 0, group)
|
||||
if not unit_is_active(CADDY_UNIT):
|
||||
@@ -705,6 +781,71 @@ def manage_caddy(cfg, ctx):
|
||||
run(['systemctl', 'reload', CADDY_UNIT], check=False)
|
||||
|
||||
|
||||
def install_packages(feature):
|
||||
"""Start the installation of the packages of a feature by INSTALL_UNIT, which runs setup again, and say so."""
|
||||
unit = INSTALL_UNIT.format(feature)
|
||||
run(['systemctl', 'start', '--no-block', unit], check=False)
|
||||
warn(f'the packages of {feature} are installed by {unit}, which runs setup again')
|
||||
|
||||
|
||||
def manage_webdav(ctx):
|
||||
"""Run rclone's WebDAV server of the sites' folders while WebDAV is ready, and stop it when not; rclone is installed
|
||||
when it is missing."""
|
||||
if not vpnrouter.webdav_ready(ctx):
|
||||
if unit_is_active(WEBDAV_UNIT):
|
||||
run(['systemctl', 'disable', '--now', WEBDAV_UNIT], check=False)
|
||||
return
|
||||
if shutil.which('rclone') is None:
|
||||
install_packages('webdav')
|
||||
return
|
||||
vpnrouter_sites.ensure_folders([])
|
||||
run(['systemctl', 'enable', '--now', WEBDAV_UNIT], check=False)
|
||||
|
||||
|
||||
def manage_dns(cfg, ctx):
|
||||
"""Run PowerDNS, the name server of dns.zone, on the addresses of dns.listen while DNS is enabled, and stop it when
|
||||
not; its packages are installed when they are missing.
|
||||
|
||||
The settings, vpnrouter_dns.PDNS_CONF, are written before the packages are installed, so the server their
|
||||
installation starts answers on those addresses only, beside systemd-resolved's stub on 127.0.0.53. The zone gets
|
||||
the host names of the enabled sites, and keeps those it has while the sites cannot be read.
|
||||
"""
|
||||
if not vpnrouter.dns_ready(ctx):
|
||||
if unit_is_active(vpnrouter_dns.UNIT):
|
||||
run(['systemctl', 'disable', '--now', vpnrouter_dns.UNIT], check=False)
|
||||
vpnrouter.remove(vpnrouter_dns.PDNS_CONF)
|
||||
if vpnrouter_dns.DATABASE_SECRET.is_file():
|
||||
try:
|
||||
vpnrouter_dns.apply(cfg, [])
|
||||
except vpnrouter_dns.DatabaseError as exc:
|
||||
warn(str(exc))
|
||||
return
|
||||
addresses = {'internal': [ctx['int_addr']], 'external': ctx['local_addr_list'],
|
||||
'both': [ctx['int_addr'], *ctx['local_addr_list']]}[ctx['dns_listen']]
|
||||
addresses = [address for address in addresses if address]
|
||||
if not addresses:
|
||||
warn(f'the name server does not run: the interfaces of dns.listen {ctx["dns_listen"]} have no address')
|
||||
return
|
||||
try:
|
||||
vpnrouter_auth.provision_database(vpnrouter_dns.DATABASE, vpnrouter_dns.DATABASE_SECRET)
|
||||
vpnrouter_dns.PDNS_CONF.parent.mkdir(mode=0o755, parents=True, exist_ok=True)
|
||||
changed = write_if_changed(vpnrouter_dns.PDNS_CONF, vpnrouter_dns.config_text(addresses), 0o640)
|
||||
if not vpnrouter_dns.PDNS_SERVER.exists() or not vpnrouter_dns.PDNS_SCHEMA.exists():
|
||||
install_packages('dns')
|
||||
return
|
||||
os.chown(vpnrouter_dns.PDNS_CONF, 0, grp.getgrnam(vpnrouter_dns.GROUP).gr_gid)
|
||||
vpnrouter_dns.provision()
|
||||
if ctx['sites'] is not None:
|
||||
vpnrouter_dns.apply(cfg, [host for site in ctx['sites'] for host in site['hosts']])
|
||||
except (OSError, KeyError, vpnrouter_dns.DatabaseError) as exc:
|
||||
warn(f'the name server cannot be set up: {exc}')
|
||||
return
|
||||
if not unit_is_active(vpnrouter_dns.UNIT):
|
||||
run(['systemctl', 'enable', '--now', vpnrouter_dns.UNIT], check=False)
|
||||
elif changed:
|
||||
run(['systemctl', 'restart', vpnrouter_dns.UNIT], check=False)
|
||||
|
||||
|
||||
def manage_metrics(ctx):
|
||||
"""Run the metrics service while metrics are enabled, and stop it when not.
|
||||
|
||||
@@ -863,6 +1004,16 @@ def main():
|
||||
ctx, more = vpnrouter.configure(cfg, public_ip=address)
|
||||
changed |= more
|
||||
|
||||
# The enabled sites while sites are on, None while they cannot be read.
|
||||
ctx['sites'] = []
|
||||
if cfg['sites']['enabled']:
|
||||
progress('running', 'web sites')
|
||||
try:
|
||||
ctx['sites'] = vpnrouter_sites.served()
|
||||
except vpnrouter_sites.DatabaseError as exc:
|
||||
warn(str(exc))
|
||||
ctx['sites'] = None
|
||||
|
||||
progress('running', 'WireGuard keys')
|
||||
setup_wireguard(ctx)
|
||||
|
||||
@@ -900,6 +1051,10 @@ def main():
|
||||
manage_openvpn(ctx, changed)
|
||||
progress('running', 'HTTPS proxy')
|
||||
manage_caddy(cfg, ctx)
|
||||
progress('running', 'WebDAV')
|
||||
manage_webdav(ctx)
|
||||
progress('running', 'name server')
|
||||
manage_dns(cfg, ctx)
|
||||
progress('running', 'PKI timer')
|
||||
run(['systemctl', 'enable', '--now', PKI_TIMER], check=False)
|
||||
|
||||
|
||||
@@ -63,6 +63,8 @@ SCHEMA = {
|
||||
'ssh': {'ldap_login': False},
|
||||
'https': {'enabled': True, 'email': '', 'web_port': 7444, 'manager_port': 7443, 'caddy_source': 'ubuntu'},
|
||||
'oidc': {'enabled': False},
|
||||
'sites': {'enabled': False, 'upload_max_mb': 512, 'webdav': False, 'webdav_port': 7445},
|
||||
'dns': {'enabled': False, 'zone': '', 'nameserver': '', 'addresses': [], 'listen': 'external', 'ttl': 300},
|
||||
'pki': {'root_name': 'VPN Router Root CA', 'root_days': 7300, 'ca_days': 3650, 'key': 'rsa-4096',
|
||||
'crl_days': 7, 'root_crl_days': 180, 'publish': ['http'], 'p2s_name': 'VPN Router P2S CA',
|
||||
's2s_name': 'VPN Router Site-to-Site CA', 'issuing_cas': {}, 'templates': {},
|
||||
@@ -130,6 +132,26 @@ HTTPS_DEFAULT_PORT = 443
|
||||
#: The most files web.log_files lists.
|
||||
LOG_FILES_MAX = 32
|
||||
|
||||
#: The port rclone serves WebDAV on, on the loopback address, behind Caddy.
|
||||
WEBDAV_LOOPBACK_PORT = 7446
|
||||
|
||||
#: The largest sites.upload_max_mb, 10 GiB.
|
||||
UPLOAD_MAX_MB = 10240
|
||||
|
||||
#: The shortest and the longest dns.ttl, in seconds.
|
||||
DNS_TTL_RANGE = (60, 86400)
|
||||
|
||||
#: A DNS name without a wildcard, in lowercase, of dns.zone and dns.nameserver.
|
||||
ZONE_NAME = re.compile(r'^[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?(\.[a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?)+$')
|
||||
|
||||
#: The ports of the bootstrap services: the Router Manager's and the router's.
|
||||
BOOTSTRAP_PORTS = (17080, 17081)
|
||||
|
||||
#: The Router Manager, vpn-router-manager, when it is installed, its settings and its port when they set none.
|
||||
MANAGER = pathlib.Path('/usr/lib/vpn-router-manager/manager')
|
||||
MANAGER_CONFIG = pathlib.Path('/etc/vpn-router-manager/manager.json')
|
||||
MANAGER_PORT = 7080
|
||||
|
||||
#: Where vpn-router-setup installs Caddy from, https.caddy_source: the Ubuntu archive or Caddy's own repository.
|
||||
CADDY_SOURCES = ('ubuntu', 'upstream')
|
||||
|
||||
@@ -539,6 +561,30 @@ def validate(cfg):
|
||||
'https://<wan.local_fqdn>:<https.web_port>, which needs web.enabled, https.enabled and '
|
||||
'wan.local_fqdn')
|
||||
|
||||
upload_max = _get(cfg, 'sites', 'upload_max_mb')
|
||||
if not 1 <= upload_max <= UPLOAD_MAX_MB:
|
||||
raise ConfigError(f'sites.upload_max_mb: {upload_max} is not between 1 and {UPLOAD_MAX_MB}')
|
||||
if _get(cfg, 'sites', 'webdav') and not (_get(cfg, 'https', 'enabled') and _get(cfg, 'wan', 'local_fqdn')):
|
||||
raise ConfigError('sites.webdav: WebDAV is served by Caddy at https://<wan.local_fqdn>:<sites.webdav_port>, '
|
||||
'which needs https.enabled and wan.local_fqdn')
|
||||
|
||||
dns = cfg['dns']
|
||||
if dns['enabled']:
|
||||
for option in ('zone', 'nameserver'):
|
||||
if not ZONE_NAME.match(dns[option].strip()):
|
||||
raise ConfigError(f'dns.{option}: {dns[option]!r} is not a DNS name in lowercase')
|
||||
if not _get_list(cfg, 'dns', 'addresses'):
|
||||
raise ConfigError('dns.enabled needs dns.addresses, the public addresses of the zone')
|
||||
for item in _get_list(cfg, 'dns', 'addresses'):
|
||||
_check_address(item, 'dns.addresses')
|
||||
if dns['listen'] not in LISTEN_CHOICES:
|
||||
raise ConfigError(f'dns.listen: {dns["listen"]!r} is not one of {", ".join(LISTEN_CHOICES)}')
|
||||
if not DNS_TTL_RANGE[0] <= dns['ttl'] <= DNS_TTL_RANGE[1]:
|
||||
raise ConfigError(f'dns.ttl: {dns["ttl"]} is not between {DNS_TTL_RANGE[0]} and {DNS_TTL_RANGE[1]} seconds')
|
||||
if dns['enabled'] and dns['listen'] != 'internal' and (_get(cfg, 'p2s', 'enabled') or _get(cfg, 'openvpn', 'enabled')):
|
||||
raise ConfigError('dns.listen: port 53 of the external addresses is the resolver of the remote access clients '
|
||||
'while p2s.enabled or openvpn.enabled is true; the name server can listen on internal only')
|
||||
|
||||
for option in ('root_name', 'p2s_name', 's2s_name'):
|
||||
value = _get(cfg, 'pki', option)
|
||||
if not value or len(value) > CA_NAME_MAX or not value.isprintable():
|
||||
@@ -629,13 +675,13 @@ def validate(cfg):
|
||||
if _get(cfg, 'openvpn', 'protocol') == 'tcp' and _get(cfg, 'openvpn', 'port') == HTTPS_DEFAULT_PORT:
|
||||
raise ConfigError(f'pki.publish: https-default takes port {HTTPS_DEFAULT_PORT}, the port of openvpn.port')
|
||||
taken[HTTPS_DEFAULT_PORT] = 'pki.publish https-default'
|
||||
for option in ('port', 'https_port'):
|
||||
port = _get(cfg, 'metrics', option)
|
||||
for section_name, option in (('metrics', 'port'), ('metrics', 'https_port'), ('sites', 'webdav_port')):
|
||||
port = _get(cfg, section_name, option)
|
||||
if isinstance(port, bool) or not isinstance(port, int) or not 1 <= port <= 65535:
|
||||
raise ConfigError(f'metrics.{option}: {port!r} is not a valid port')
|
||||
raise ConfigError(f'{section_name}.{option}: {port!r} is not a valid port')
|
||||
if port in taken:
|
||||
raise ConfigError(f'metrics.{option}: {port} is the port of {taken[port]}')
|
||||
taken[port] = f'metrics.{option}'
|
||||
raise ConfigError(f'{section_name}.{option}: {port} is the port of {taken[port]}')
|
||||
taken[port] = f'{section_name}.{option}'
|
||||
|
||||
metrics_listen = _get(cfg, 'metrics', 'listen')
|
||||
if metrics_listen not in LISTEN_CHOICES:
|
||||
@@ -764,6 +810,15 @@ def pki_base_urls(cfg):
|
||||
return [bases[item] for item in _get_list(cfg, 'pki', 'publish') if item in bases] if fqdn else []
|
||||
|
||||
|
||||
def manager_port():
|
||||
"""The port of the Router Manager: port in its settings, or MANAGER_PORT when it is not set or not a port."""
|
||||
try:
|
||||
port = json.loads(MANAGER_CONFIG.read_text(encoding='utf-8')).get('port')
|
||||
except (OSError, ValueError, AttributeError):
|
||||
return MANAGER_PORT
|
||||
return port if isinstance(port, int) and not isinstance(port, bool) and 1 <= port <= 65535 else MANAGER_PORT
|
||||
|
||||
|
||||
def oidc_issuer(cfg):
|
||||
"""The issuer of the OpenID Provider: the web service's HTTPS site, without the port when it is HTTPS_DEFAULT_PORT."""
|
||||
port = _get(cfg, 'https', 'web_port')
|
||||
@@ -959,6 +1014,11 @@ def build_context(cfg, public_ip=None):
|
||||
'metrics_port': _get(cfg, 'metrics', 'port'),
|
||||
'metrics_https_port': _get(cfg, 'metrics', 'https_port'),
|
||||
'metrics_service_account': _get(cfg, 'metrics', 'service_account'),
|
||||
'sites_enabled': _get(cfg, 'sites', 'enabled'),
|
||||
'webdav_enabled': _get(cfg, 'sites', 'webdav'),
|
||||
'webdav_port': _get(cfg, 'sites', 'webdav_port'),
|
||||
'dns_enabled': _get(cfg, 'dns', 'enabled'),
|
||||
'dns_listen': _get(cfg, 'dns', 'listen'),
|
||||
'loopback': LOOPBACK,
|
||||
}
|
||||
|
||||
@@ -998,6 +1058,16 @@ def metrics_ready(ctx):
|
||||
return bool(ctx['metrics_enabled'])
|
||||
|
||||
|
||||
def webdav_ready(ctx):
|
||||
"""WebDAV of the sites' folders is served by Caddy, on the router's own name, while it is on and HTTPS is ready."""
|
||||
return bool(ctx['webdav_enabled'] and https_ready(ctx))
|
||||
|
||||
|
||||
def dns_ready(ctx):
|
||||
"""The name server answers for dns.zone while dns.enabled is true."""
|
||||
return bool(ctx['dns_enabled'])
|
||||
|
||||
|
||||
def _jinja_env():
|
||||
return jinja2.Environment(
|
||||
loader=jinja2.FileSystemLoader(str(TEMPLATE_DIR)),
|
||||
|
||||
@@ -127,13 +127,19 @@ def admin_token(seconds):
|
||||
return {'token': sign({'admin': True, 'iat': now, 'exp': now + seconds}, secret), 'expires': now + seconds}
|
||||
|
||||
|
||||
def local_connection(name, secret):
|
||||
"""The connection string of a database provision_database() created; its password, of URL-safe characters, is
|
||||
written unquoted."""
|
||||
password = secret.read_text(encoding='utf-8').strip()
|
||||
return f'host=localhost dbname={name} user={name} password={password}'
|
||||
|
||||
|
||||
def connection_string(cfg):
|
||||
"""web.database, or the database provision() creates; its password, of URL-safe characters, is written unquoted."""
|
||||
"""web.database, or the database provision() creates."""
|
||||
configured = cfg['web']['database'].strip()
|
||||
if configured:
|
||||
return configured
|
||||
password = DATABASE_SECRET.read_text(encoding='utf-8').strip()
|
||||
return f'host=localhost dbname={DATABASE} user={DATABASE} password={password}'
|
||||
return local_connection(DATABASE, DATABASE_SECRET)
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
@@ -204,21 +210,33 @@ def psql(sql, name):
|
||||
return result.stdout.strip()
|
||||
|
||||
|
||||
def provision(cfg):
|
||||
"""Create the role, its password file and the database when web.database is not set and they are missing.
|
||||
def provision_database(name, secret):
|
||||
"""Create the role of the name, its password file secret and its database of the same name when they are missing.
|
||||
|
||||
The password, of URL-safe characters, reaches psql on standard input, not on its command line.
|
||||
"""
|
||||
if cfg['web']['database'].strip():
|
||||
return
|
||||
if not psql("SELECT 1 FROM pg_roles WHERE rolname = :'v1'", DATABASE):
|
||||
DATABASE_SECRET.unlink(missing_ok=True)
|
||||
psql("CREATE ROLE :\"v1\" LOGIN", DATABASE)
|
||||
if not DATABASE_SECRET.is_file():
|
||||
if not psql("SELECT 1 FROM pg_roles WHERE rolname = :'v1'", name):
|
||||
secret.unlink(missing_ok=True)
|
||||
psql("CREATE ROLE :\"v1\" LOGIN", name)
|
||||
if not secret.is_file():
|
||||
password = secrets.token_urlsafe(32)
|
||||
psql(f"ALTER ROLE :\"v1\" PASSWORD '{password}'", DATABASE)
|
||||
DATABASE_SECRET.touch(mode=0o600)
|
||||
DATABASE_SECRET.chmod(0o600)
|
||||
DATABASE_SECRET.write_text(password + '\n', encoding='utf-8')
|
||||
if not psql("SELECT 1 FROM pg_database WHERE datname = :'v1'", DATABASE):
|
||||
psql("CREATE DATABASE :\"v1\" OWNER :\"v1\"", DATABASE)
|
||||
psql(f"ALTER ROLE :\"v1\" PASSWORD '{password}'", name)
|
||||
secret.touch(mode=0o600)
|
||||
secret.chmod(0o600)
|
||||
secret.write_text(password + '\n', encoding='utf-8')
|
||||
if not psql("SELECT 1 FROM pg_database WHERE datname = :'v1'", name):
|
||||
psql("CREATE DATABASE :\"v1\" OWNER :\"v1\"", name)
|
||||
|
||||
|
||||
def reset_password(name, secret):
|
||||
"""Give the role of the name the password of its file secret, when both exist, as after a restore."""
|
||||
if secret.is_file() and psql("SELECT 1 FROM pg_roles WHERE rolname = :'v1'", name):
|
||||
password = secret.read_text(encoding='utf-8').strip()
|
||||
psql(f"ALTER ROLE :\"v1\" PASSWORD '{password}'", name)
|
||||
|
||||
|
||||
def provision(cfg):
|
||||
"""Create the role, its password file and the database of the refresh tokens when web.database is not set and they
|
||||
are missing."""
|
||||
if not cfg['web']['database'].strip():
|
||||
provision_database(DATABASE, DATABASE_SECRET)
|
||||
@@ -1,9 +1,10 @@
|
||||
"""The backups of the router, for vpn-router-backup and the web service.
|
||||
|
||||
A backup is a consistent copy of the state that setup cannot make again, staged in STAGING: the
|
||||
trees of TREES, the files of site_to_site psk_file outside them under files/, the directory as
|
||||
slapcat writes it (ldap/vpn-router.ldif), with backup.metrics the database as pg_dump writes it
|
||||
(postgresql/vpn-router.dump), and manifest.json. The trees and the directory are copied while
|
||||
trees of TREES, the folders of the web sites among them, the files of site_to_site psk_file outside them under files/,
|
||||
the directory as slapcat writes it (ldap/vpn-router.ldif), with backup.metrics the database as pg_dump writes it
|
||||
(postgresql/vpn-router.dump), the databases of the web sites and of the name server (DUMPS) once they exist, and
|
||||
manifest.json. The trees and the directory are copied while
|
||||
the configuration lock and the users lock are held, so the configuration, the PKI and the users
|
||||
match; pg_dump reads a snapshot of its own.
|
||||
|
||||
@@ -33,7 +34,9 @@ import time
|
||||
|
||||
import vpnrouter
|
||||
import vpnrouter_auth
|
||||
import vpnrouter_dns
|
||||
import vpnrouter_ldap
|
||||
import vpnrouter_sites
|
||||
import vpnrouter_users
|
||||
|
||||
STATE = pathlib.Path('/var/lib/vpn-router/backup.json')
|
||||
@@ -49,7 +52,12 @@ UNIT = 'vpn-router-backup.service'
|
||||
TIMER = 'vpn-router-backup.timer'
|
||||
|
||||
#: The state a backup copies: directories and files, each where it is on the system.
|
||||
TREES = ('/etc/vpn-router', '/etc/wireguard', '/etc/ufw', '/etc/default/ufw', '/etc/vpn-router-manager')
|
||||
TREES = ('/etc/vpn-router', '/etc/wireguard', '/etc/ufw', '/etc/default/ufw', '/etc/vpn-router-manager',
|
||||
str(vpnrouter_sites.SITES_DIR))
|
||||
|
||||
#: The databases a backup always dumps, each while it exists: the dump -> the database and its password file.
|
||||
DUMPS = {'postgresql/vpn-router-sites.dump': (vpnrouter_sites.DATABASE, vpnrouter_sites.DATABASE_SECRET),
|
||||
'postgresql/vpn-router-dns.dump': (vpnrouter_dns.DATABASE, vpnrouter_dns.DATABASE_SECRET)}
|
||||
|
||||
#: The name of a local archive: the host and the UTC time of the backup.
|
||||
ARCHIVE = re.compile(r'^vpn-router-(?P<host>[A-Za-z0-9][A-Za-z0-9.-]*)-(?P<stamp>\d{8}T\d{6}Z)\.tar\.zst$')
|
||||
@@ -64,7 +72,8 @@ PERIODS = {'keep_hourly': '%Y-%m-%d %H', 'keep_daily': '%Y-%m-%d', 'keep_weekly'
|
||||
'keep_yearly': '%Y'}
|
||||
|
||||
#: The units a restore stops while it replaces the directory and the database, and the ones it restarts after setup.
|
||||
RESTORE_STOPS = ('vpn-router-web.service', 'vpn-router-metrics.service', 'slapd.service')
|
||||
RESTORE_STOPS = ('vpn-router-web.service', 'vpn-router-metrics.service', 'vpn-router-webdav.service', 'pdns.service',
|
||||
'slapd.service')
|
||||
RESTORE_RESTARTS = ('wg-quick@wg0.service', 'openvpn-server@vpn-router.service', 'strongswan.service',
|
||||
'vpn-router-manager.service')
|
||||
|
||||
@@ -223,14 +232,16 @@ def stage(cfg):
|
||||
(STAGING / LDIF).parent.mkdir()
|
||||
command(['slapcat', '-b', vpnrouter_ldap.BASE, '-l', str(STAGING / LDIF)])
|
||||
items = trees + [f'files{name}' for name in files] + [LDIF]
|
||||
if cfg['backup']['metrics']:
|
||||
(STAGING / DUMP).parent.mkdir()
|
||||
prefix, database = database_args(cfg)
|
||||
with open(STAGING / DUMP, 'wb') as dump:
|
||||
dumps = {**({DUMP: database_args(cfg)} if cfg['backup']['metrics'] else {}),
|
||||
**{item: (['runuser', '-u', 'postgres', '--'], [f'--dbname={name}'])
|
||||
for item, (name, secret) in DUMPS.items() if secret.is_file()}}
|
||||
for item, (prefix, database) in dumps.items():
|
||||
(STAGING / item).parent.mkdir(exist_ok=True)
|
||||
with open(STAGING / item, 'wb') as dump:
|
||||
result = subprocess.run([*prefix, 'pg_dump', '-Fc', *database], stdout=dump, stderr=subprocess.PIPE)
|
||||
if result.returncode != 0:
|
||||
raise Failure(f'pg_dump failed: {result.stderr.decode(errors="replace").strip()}')
|
||||
items.append(DUMP)
|
||||
raise Failure(f'pg_dump for {item} failed: {result.stderr.decode(errors="replace").strip()}')
|
||||
items.append(item)
|
||||
version = subprocess.run(['dpkg-query', '-W', '-f=${Version}', 'vpn-router'], capture_output=True, text=True)
|
||||
manifest = {'host': host(), 'time': now(), 'version': version.stdout.strip() or None,
|
||||
'schedule': cfg['backup']['schedule'], 'items': items}
|
||||
@@ -389,19 +400,22 @@ def restore(source, pbs, confirm):
|
||||
subprocess.run(['systemctl', 'start', 'slapd.service'], capture_output=True)
|
||||
restored = vpnrouter.load_config()
|
||||
if not restored['web']['database'].strip():
|
||||
secret = vpnrouter_auth.DATABASE_SECRET
|
||||
if secret.is_file() and vpnrouter_auth.psql("SELECT 1 FROM pg_roles WHERE rolname = :'v1'",
|
||||
vpnrouter_auth.DATABASE):
|
||||
password = secret.read_text(encoding='utf-8').strip()
|
||||
vpnrouter_auth.psql(f"ALTER ROLE :\"v1\" PASSWORD '{password}'", vpnrouter_auth.DATABASE)
|
||||
vpnrouter_auth.reset_password(vpnrouter_auth.DATABASE, vpnrouter_auth.DATABASE_SECRET)
|
||||
vpnrouter_auth.provision(restored)
|
||||
if DUMP in manifest['items']:
|
||||
prefix, database = database_args(restored)
|
||||
with open(RESTORE_DIR / DUMP, 'rb') as dump:
|
||||
dumps = {DUMP: database_args(restored)}
|
||||
for item, (name, secret) in DUMPS.items():
|
||||
if item in manifest['items']:
|
||||
vpnrouter_auth.reset_password(name, secret)
|
||||
vpnrouter_auth.provision_database(name, secret)
|
||||
dumps[item] = (['runuser', '-u', 'postgres', '--'], [f'--dbname={name}'])
|
||||
for item, (prefix, database) in dumps.items():
|
||||
if item not in manifest['items']:
|
||||
continue
|
||||
with open(RESTORE_DIR / item, 'rb') as dump:
|
||||
result = subprocess.run([*prefix, 'pg_restore', '--clean', '--if-exists', *database], stdin=dump,
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
|
||||
if result.returncode != 0:
|
||||
raise Failure(f'pg_restore failed: {result.stderr.decode(errors="replace").strip()}')
|
||||
raise Failure(f'pg_restore of {item} failed: {result.stderr.decode(errors="replace").strip()}')
|
||||
subprocess.run(['systemctl', 'start', 'vpn-router-web.service'], capture_output=True)
|
||||
command(['systemctl', 'restart', 'vpn-router-setup.service'])
|
||||
for unit in RESTORE_RESTARTS:
|
||||
|
||||
@@ -0,0 +1,268 @@
|
||||
"""The router's authoritative name server, PowerDNS with its gpgsql backend, for vpn-router-setup and the web service.
|
||||
|
||||
The name server answers for one zone, dns.zone, on the addresses of the interfaces of dns.listen. Its records are in
|
||||
the vpn-router-dns database, which provision() creates with its role and the password in DATABASE_SECRET, and fills with
|
||||
the schema of pdns-backend-pgsql, PDNS_SCHEMA; DNS_SQL defines the router's own tables and procedures, and the first
|
||||
connection of a process loads it. apply() writes the zone: its SOA, its NS, dns.nameserver, A and AAAA records of
|
||||
dns.addresses for the zone's apex, for the name server when it is in the zone, and for every host name of an enabled
|
||||
site in the zone, and the records an administrator added (manual records), but those of a name and type the records
|
||||
before them have. The serial of the SOA grows only when the records change.
|
||||
|
||||
Every failure of the database raises DatabaseError; a record that cannot be added raises Failure.
|
||||
"""
|
||||
|
||||
import contextlib
|
||||
import ipaddress
|
||||
import pathlib
|
||||
import re
|
||||
|
||||
import psycopg
|
||||
from psycopg.rows import dict_row
|
||||
|
||||
import vpnrouter_auth
|
||||
|
||||
DATABASE = 'vpn-router-dns'
|
||||
DATABASE_SECRET = pathlib.Path('/etc/vpn-router/dns-database.secret')
|
||||
DNS_SQL = pathlib.Path('/usr/share/vpn-router/sql/dns.sql')
|
||||
|
||||
#: The schema of PowerDNS's gpgsql backend, which pdns-backend-pgsql installs.
|
||||
PDNS_SCHEMA = pathlib.Path('/usr/share/pdns-backend-pgsql/schema/schema.pgsql.sql')
|
||||
|
||||
#: The settings of PowerDNS that vpn-router-setup writes, read after /etc/powerdns/pdns.conf; the server, its unit and
|
||||
#: the group it runs as.
|
||||
PDNS_CONF = pathlib.Path('/etc/powerdns/pdns.d/vpn-router.conf')
|
||||
PDNS_SERVER = pathlib.Path('/usr/sbin/pdns_server')
|
||||
UNIT = 'pdns.service'
|
||||
GROUP = 'pdns'
|
||||
|
||||
#: The types of the manual records.
|
||||
TYPES = ('A', 'AAAA', 'CNAME', 'TXT', 'MX', 'SRV', 'CAA')
|
||||
|
||||
#: The most manual records.
|
||||
MANUAL_MAX = 1000
|
||||
|
||||
#: The refresh, retry and expire times of the SOA, in seconds.
|
||||
SOA_TIMES = '10800 3600 604800'
|
||||
|
||||
#: A label of a record's name: letters, digits, hyphens and underscores, or * as the first label.
|
||||
LABEL = re.compile(r'^[a-z0-9_]([a-z0-9_-]{0,61}[a-z0-9_])?$')
|
||||
|
||||
#: The content of an MX, SRV and CAA record, and a DNS name a record points to.
|
||||
MX = re.compile(r'^(\d{1,5}) (\S+)$')
|
||||
SRV = re.compile(r'^(\d{1,5}) (\d{1,5}) (\d{1,5}) (\S+)$')
|
||||
CAA = re.compile(r'^(0|128) (issue|issuewild|iodef) "[^"\\]*"$')
|
||||
TARGET = re.compile(r'^[a-z0-9_]([a-z0-9_-]{0,61}[a-z0-9_])?(\.[a-z0-9_]([a-z0-9_-]{0,61}[a-z0-9_])?)*$')
|
||||
|
||||
#: The longest content of a TXT record, in characters.
|
||||
TXT_MAX = 2048
|
||||
|
||||
#: Whether this process has loaded DNS_SQL.
|
||||
_loaded = False
|
||||
|
||||
|
||||
class DatabaseError(Exception):
|
||||
"""The database of the name server failed."""
|
||||
|
||||
|
||||
class Failure(Exception):
|
||||
"""A record that cannot be added or removed; status is 2 invalid, 3 no such record."""
|
||||
|
||||
def __init__(self, message, status=2):
|
||||
super().__init__(message)
|
||||
self.status = status
|
||||
|
||||
|
||||
def provision():
|
||||
"""Create the role, its password file and the database of the name server when they are missing, and PowerDNS's
|
||||
tables in it; DatabaseError while pdns-backend-pgsql is not installed."""
|
||||
vpnrouter_auth.provision_database(DATABASE, DATABASE_SECRET)
|
||||
try:
|
||||
with psycopg.connect(vpnrouter_auth.local_connection(DATABASE, DATABASE_SECRET), autocommit=True) as connection:
|
||||
if connection.execute("SELECT to_regclass('public.domains')").fetchone()[0] is None:
|
||||
connection.execute(PDNS_SCHEMA.read_text(encoding='utf-8'))
|
||||
except OSError as exc:
|
||||
raise DatabaseError(f'the schema of PowerDNS cannot be read: {exc}') from exc
|
||||
except psycopg.Error as exc:
|
||||
raise DatabaseError(f'the database of the name server failed: {exc}') from exc
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def database():
|
||||
"""A connection to the database of the name server, with the router's tables and procedures, closed at the end;
|
||||
DatabaseError on any failure, also before provision() made it."""
|
||||
global _loaded
|
||||
try:
|
||||
schema = None if _loaded else DNS_SQL.read_text(encoding='utf-8')
|
||||
target = vpnrouter_auth.local_connection(DATABASE, DATABASE_SECRET)
|
||||
except OSError as exc:
|
||||
raise DatabaseError(f'the database of the name server is not ready: {exc}') from exc
|
||||
try:
|
||||
with psycopg.connect(target, autocommit=True, row_factory=dict_row) as connection:
|
||||
if schema is not None:
|
||||
connection.execute(schema)
|
||||
_loaded = True
|
||||
yield connection
|
||||
except psycopg.Error as exc:
|
||||
raise DatabaseError(f'the database of the name server failed: {exc}') from exc
|
||||
|
||||
|
||||
def in_zone(zone, name):
|
||||
return name == zone or name.endswith(f'.{zone}')
|
||||
|
||||
|
||||
def absolute(zone, name):
|
||||
"""The record name in the zone: @ or '' is the zone itself, a name ending in a dot is taken as it is, any other is
|
||||
relative to the zone; Failure for a name outside the zone or not a DNS name."""
|
||||
if not isinstance(name, str):
|
||||
raise Failure('name must be a string')
|
||||
name = name.strip().lower()
|
||||
if name in ('', '@'):
|
||||
return zone
|
||||
full = name[:-1] if name.endswith('.') else f'{name}.{zone}'
|
||||
labels = full.split('.')
|
||||
if not in_zone(zone, full) or not all(LABEL.match(label) or (index == 0 and label == '*')
|
||||
for index, label in enumerate(labels)):
|
||||
raise Failure(f'{name!r} is not a name in {zone}')
|
||||
return full
|
||||
|
||||
|
||||
def check_record(zone, record):
|
||||
"""The manual record {name, type, content, ttl?} with its name made absolute and its content checked; Failure when
|
||||
it is not valid. A TXT content not in quotes is quoted."""
|
||||
if not isinstance(record, dict):
|
||||
raise Failure('a record is an object of name, type, content and ttl')
|
||||
name = absolute(zone, record.get('name'))
|
||||
kind = record.get('type')
|
||||
content = record.get('content')
|
||||
ttl = record.get('ttl')
|
||||
if kind not in TYPES:
|
||||
raise Failure(f'type must be one of {", ".join(TYPES)}')
|
||||
if not isinstance(content, str) or not content.strip():
|
||||
raise Failure('content must be a string')
|
||||
if ttl is not None and (isinstance(ttl, bool) or not isinstance(ttl, int) or not 60 <= ttl <= 86400):
|
||||
raise Failure('ttl must be 60 to 86400 seconds')
|
||||
content = content.strip()
|
||||
if kind in ('A', 'AAAA'):
|
||||
try:
|
||||
address = ipaddress.ip_address(content)
|
||||
except ValueError:
|
||||
address = None
|
||||
if address is None or address.version != (4 if kind == 'A' else 6):
|
||||
raise Failure(f'{content!r} is not an IPv{4 if kind == "A" else 6} address')
|
||||
content = str(address)
|
||||
elif kind == 'CNAME':
|
||||
if name == zone:
|
||||
raise Failure('the zone itself cannot have a CNAME record')
|
||||
content = content.lower().rstrip('.')
|
||||
if not TARGET.match(content):
|
||||
raise Failure(f'{content!r} is not a DNS name')
|
||||
elif kind == 'TXT':
|
||||
if not (content.startswith('"') and content.endswith('"') and len(content) > 1):
|
||||
content = '"' + content.replace('\\', '\\\\').replace('"', '\\"') + '"'
|
||||
if len(content) > TXT_MAX or not content.isprintable():
|
||||
raise Failure(f'the text must have at most {TXT_MAX} printable characters')
|
||||
elif kind in ('MX', 'SRV'):
|
||||
match = (MX if kind == 'MX' else SRV).match(content.lower())
|
||||
if not match or any(int(value) > 65535 for value in match.groups()[:-1]) or \
|
||||
not TARGET.match(match.groups()[-1].rstrip('.')):
|
||||
raise Failure(f'{kind} content is ' + ('"<preference> <host>"' if kind == 'MX'
|
||||
else '"<priority> <weight> <port> <host>"'))
|
||||
content = ' '.join([*match.groups()[:-1], match.groups()[-1].rstrip('.')])
|
||||
elif not CAA.match(content):
|
||||
raise Failure('CAA content is "<0 or 128> <issue, issuewild or iodef> \\"<value>\\""')
|
||||
return {'name': name, 'type': kind, 'content': content, 'ttl': ttl}
|
||||
|
||||
|
||||
def manual():
|
||||
"""The manual records."""
|
||||
with database() as connection:
|
||||
return connection.execute('SELECT * FROM dns_manual_list()').fetchall()
|
||||
|
||||
|
||||
def add(cfg, record):
|
||||
"""Add the manual record, or set the TTL of the same one; the record."""
|
||||
zone = cfg['dns']['zone'].strip()
|
||||
if not zone:
|
||||
raise Failure('dns.zone is not set')
|
||||
record = check_record(zone, record)
|
||||
record['ttl'] = record['ttl'] or cfg['dns']['ttl']
|
||||
with database() as connection:
|
||||
records = connection.execute('SELECT * FROM dns_manual_list()').fetchall()
|
||||
same = [row for row in records if (row['name'], row['type']) == (record['name'], record['type'])]
|
||||
others = [row for row in records if row['name'] == record['name'] and row['type'] != record['type']]
|
||||
if len(records) >= MANUAL_MAX and record not in records:
|
||||
raise Failure(f'the zone has {MANUAL_MAX} manual records')
|
||||
if record['type'] == 'CNAME' and (others or [row for row in same if row['content'] != record['content']]):
|
||||
raise Failure(f'{record["name"]} has other records; a CNAME record must be its only record')
|
||||
if any(row['type'] == 'CNAME' for row in others):
|
||||
raise Failure(f'{record["name"]} has a CNAME record, which must be its only record')
|
||||
connection.execute('CALL dns_manual_put(%s, %s, %s, %s)',
|
||||
(record['name'], record['type'], record['content'], record['ttl']))
|
||||
return record
|
||||
|
||||
|
||||
def remove(cfg, name, kind, content):
|
||||
"""Remove the manual records of the name and type, only the one of the content when it is given."""
|
||||
zone = cfg['dns']['zone'].strip()
|
||||
full = absolute(zone, name)
|
||||
with database() as connection:
|
||||
found = [row for row in connection.execute('SELECT * FROM dns_manual_list()').fetchall()
|
||||
if (row['name'], row['type']) == (full, kind) and content in (None, row['content'])]
|
||||
if not found:
|
||||
raise Failure(f'no {kind} record of {full}', 3)
|
||||
connection.execute('CALL dns_manual_remove(%s, %s, %s)', (full, kind, content))
|
||||
return {'name': full, 'type': kind, 'removed': len(found)}
|
||||
|
||||
|
||||
def served(zone):
|
||||
"""The records of the zone as the name server answers them, [] when it serves none."""
|
||||
with database() as connection:
|
||||
return connection.execute('SELECT * FROM dns_served(%s)', (zone,)).fetchall()
|
||||
|
||||
|
||||
def records(cfg, hosts, manual_records):
|
||||
"""The records of the zone besides its SOA, as (name, type, content, ttl): its NS, the A and AAAA records of
|
||||
dns.addresses for the apex, the name server and each host name in the zone, and the manual records."""
|
||||
dns = cfg['dns']
|
||||
zone, nameserver, ttl = dns['zone'].strip(), dns['nameserver'].strip(), dns['ttl']
|
||||
addresses = [ipaddress.ip_address(item.strip()) for item in dns['addresses'] if item.strip()]
|
||||
found = [(zone, 'NS', nameserver, ttl)]
|
||||
for name in dict.fromkeys([zone, *([nameserver] if in_zone(zone, nameserver) else []),
|
||||
*(host for host in hosts if in_zone(zone, host))]):
|
||||
found += [(name, 'A' if address.version == 4 else 'AAAA', str(address), ttl) for address in addresses]
|
||||
taken = {(name, kind) for name, kind, _, _ in found}
|
||||
found += [(row['name'], row['type'], row['content'], row['ttl']) for row in manual_records
|
||||
if (row['name'], row['type']) not in taken]
|
||||
return list(dict.fromkeys(found))
|
||||
|
||||
|
||||
def apply(cfg, hosts):
|
||||
"""Make the name server serve dns.zone with records(), or no zone while dns.enabled is false."""
|
||||
with database() as connection:
|
||||
if not cfg['dns']['enabled']:
|
||||
connection.execute('CALL dns_clear()')
|
||||
return
|
||||
dns = cfg['dns']
|
||||
zone = dns['zone'].strip()
|
||||
rows = records(cfg, hosts, connection.execute('SELECT * FROM dns_manual_list()').fetchall())
|
||||
soa = f'{dns["nameserver"].strip()} hostmaster.{zone} {{serial}} {SOA_TIMES} {dns["ttl"]}'
|
||||
connection.execute('CALL dns_apply(%s, %s, %s, %s, %s, %s, %s)',
|
||||
(zone, soa, dns['ttl'], [row[0] for row in rows], [row[1] for row in rows],
|
||||
[row[2] for row in rows], [row[3] for row in rows]))
|
||||
|
||||
|
||||
def config_text(addresses):
|
||||
"""The settings of PowerDNS: the gpgsql backend on the database of the name server, and the addresses it answers
|
||||
on."""
|
||||
password = DATABASE_SECRET.read_text(encoding='utf-8').strip()
|
||||
return '\n'.join([
|
||||
'# Written by vpn-router-setup from /etc/vpn-router/vpn-router.json; edits are overwritten.',
|
||||
'launch=gpgsql',
|
||||
'gpgsql-host=localhost',
|
||||
f'gpgsql-dbname={DATABASE}',
|
||||
f'gpgsql-user={DATABASE}',
|
||||
f'gpgsql-password={password}',
|
||||
f'local-address={", ".join(addresses)}',
|
||||
'local-port=53',
|
||||
'',
|
||||
])
|
||||
@@ -0,0 +1,439 @@
|
||||
"""The static web sites Caddy serves, for the web service, vpn-router-site, vpn-router-setup and webdav-auth.
|
||||
|
||||
A site is a row of the sites table in the vpn-router-sites database, which the web service creates on its first run
|
||||
(provision) with its role and the password in DATABASE_SECRET; SITES_SQL defines the table and its procedures, and the
|
||||
first connection of a process loads it. A site has a name, which also names its folder SITES_DIR/<name>, the host names
|
||||
Caddy answers on its port (hosts, the names TLS clients send as SNI; none answers any name), the interfaces whose
|
||||
firewall opens the port (listen), and TLS: none (plain HTTP), acme (a certificate of Let's Encrypt), ca (certificates
|
||||
Caddy issues with one of pki.issuing_cas) or file (a certificate and key given with the site). Caddy serves the files of
|
||||
the folder, index_file for a folder, and with spa index_file of the site for any path without a file. editors names
|
||||
the group whose members change the site's files besides the admins, through the web service and WebDAV.
|
||||
|
||||
The sites are served while sites.enabled is true; vpn-router-setup writes Caddy's configuration and opens the ports. A
|
||||
site is changed only through this module, which checks it (check), and its callers then restart vpn-router-setup.
|
||||
|
||||
The files of a site are read and written by path relative to its folder; resolve() refuses a path that leaves it.
|
||||
receive() writes a file from a stream, extract() a zip archive into a folder; both stop at sites.upload_max_mb.
|
||||
|
||||
Every failure of the database raises DatabaseError; a request that cannot be done raises Failure, whose status says
|
||||
why: 2 invalid, 3 no such site, 4 the site exists, 1 anything else.
|
||||
"""
|
||||
|
||||
import contextlib
|
||||
import datetime
|
||||
import os
|
||||
import pathlib
|
||||
import re
|
||||
import shutil
|
||||
import stat
|
||||
import tempfile
|
||||
import zipfile
|
||||
|
||||
import psycopg
|
||||
from psycopg.rows import dict_row
|
||||
from cryptography import x509
|
||||
from cryptography.hazmat.primitives import serialization
|
||||
|
||||
import vpnrouter
|
||||
import vpnrouter_auth
|
||||
import vpnrouter_pki
|
||||
import vpnrouter_users
|
||||
|
||||
DATABASE = 'vpn-router-sites'
|
||||
DATABASE_SECRET = pathlib.Path('/etc/vpn-router/sites-database.secret')
|
||||
SITES_SQL = pathlib.Path('/usr/share/vpn-router/sql/sites.sql')
|
||||
|
||||
#: The folders of the sites, SITES_DIR/<name>.
|
||||
SITES_DIR = pathlib.Path('/var/lib/vpn-router/sites')
|
||||
|
||||
#: The name of a site, which also names its folder.
|
||||
NAME = re.compile(r'^[a-z][a-z0-9-]{0,30}$')
|
||||
|
||||
#: A file name of index_file.
|
||||
INDEX = re.compile(r'^[A-Za-z0-9_-][A-Za-z0-9._-]{0,63}$')
|
||||
|
||||
LISTEN = vpnrouter.LISTEN_CHOICES
|
||||
TLS = ('none', 'acme', 'ca', 'file')
|
||||
|
||||
#: The most host names of one site.
|
||||
HOSTS_MAX = 16
|
||||
|
||||
#: The most members a zip archive extract() takes.
|
||||
ARCHIVE_MEMBERS_MAX = 10000
|
||||
|
||||
#: The bytes copied at once from a stream.
|
||||
CHUNK = 65536
|
||||
|
||||
#: The settings of a site and their defaults; the type of a default is the required type.
|
||||
SETTINGS = {'enabled': True, 'hosts': [], 'port': 443, 'listen': 'external', 'tls': 'acme', 'ca': '', 'certificate': '',
|
||||
'private_key': '', 'index_file': 'index.html', 'spa': False, 'editors': ''}
|
||||
|
||||
#: Whether this process has loaded SITES_SQL.
|
||||
_loaded = False
|
||||
|
||||
|
||||
class DatabaseError(Exception):
|
||||
"""The database of the sites failed."""
|
||||
|
||||
|
||||
class Failure(Exception):
|
||||
"""A request that cannot be done; status is 2 invalid, 3 no such site, 4 the site exists, 1 other."""
|
||||
|
||||
def __init__(self, message, status=1):
|
||||
super().__init__(message)
|
||||
self.status = status
|
||||
|
||||
|
||||
def provision():
|
||||
"""Create the role, its password file and the database of the sites when they are missing."""
|
||||
vpnrouter_auth.provision_database(DATABASE, DATABASE_SECRET)
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def database():
|
||||
"""A connection to the database of the sites, with its table and procedures, closed at the end; DatabaseError on any
|
||||
failure, also before the web service has created the database."""
|
||||
global _loaded
|
||||
try:
|
||||
schema = None if _loaded else SITES_SQL.read_text(encoding='utf-8')
|
||||
target = vpnrouter_auth.local_connection(DATABASE, DATABASE_SECRET)
|
||||
except OSError as exc:
|
||||
raise DatabaseError(f'the database of the sites is not ready: {exc}') from exc
|
||||
try:
|
||||
with psycopg.connect(target, autocommit=True, row_factory=dict_row) as connection:
|
||||
if schema is not None:
|
||||
connection.execute(schema)
|
||||
_loaded = True
|
||||
yield connection
|
||||
except psycopg.Error as exc:
|
||||
raise DatabaseError(f'the database of the sites failed: {exc}') from exc
|
||||
|
||||
|
||||
def shown(row):
|
||||
"""A site as the answers show it: its settings without the private key, with private_key_set and its folder."""
|
||||
site = {key: value for key, value in row.items() if key != 'private_key'}
|
||||
site['private_key_set'] = bool(row.get('private_key'))
|
||||
site['modified'] = row['modified'].isoformat(timespec='seconds')
|
||||
site['directory'] = str(SITES_DIR / row['name'])
|
||||
return site
|
||||
|
||||
|
||||
def listing():
|
||||
"""Every site, as shown()."""
|
||||
with database() as connection:
|
||||
return [shown(row) for row in connection.execute('SELECT * FROM sites_list()').fetchall()]
|
||||
|
||||
|
||||
def served():
|
||||
"""The enabled sites with their private keys."""
|
||||
with database() as connection:
|
||||
return connection.execute('SELECT * FROM sites_served()').fetchall()
|
||||
|
||||
|
||||
def get(connection, name):
|
||||
"""The site with its private key; Failure with status 3 when there is none."""
|
||||
if not isinstance(name, str) or not NAME.match(name):
|
||||
raise Failure(f'no site {name}', 3)
|
||||
row = connection.execute('SELECT * FROM site_get(%s)', (name,)).fetchone()
|
||||
if row is None:
|
||||
raise Failure(f'no site {name}', 3)
|
||||
return row
|
||||
|
||||
|
||||
def editable(groups):
|
||||
"""The names of the sites whose files the members of the groups may change."""
|
||||
with database() as connection:
|
||||
return [row['name'] for row in connection.execute('SELECT * FROM sites_editable(%s)', (list(groups),))]
|
||||
|
||||
|
||||
def sites_of_group(group):
|
||||
"""The names of the sites whose editors are the group; [] before the web service has created the database."""
|
||||
if not DATABASE_SECRET.is_file():
|
||||
return []
|
||||
with database() as connection:
|
||||
return [row['name'] for row in connection.execute('SELECT * FROM sites_of_group(%s)', (group,))]
|
||||
|
||||
|
||||
def rename_group(old, new):
|
||||
"""Give the sites whose editors are the group old its new name."""
|
||||
if DATABASE_SECRET.is_file():
|
||||
with database() as connection:
|
||||
connection.execute('CALL sites_rename_group(%s, %s)', (old, new))
|
||||
|
||||
|
||||
def taken_ports(cfg):
|
||||
"""The TCP ports of the router's own services, port -> what has it, which no site may take."""
|
||||
taken = {22: 'SSH', cfg['web']['port']: 'web.port', cfg['https']['web_port']: 'https.web_port',
|
||||
cfg['https']['manager_port']: 'https.manager_port', cfg['metrics']['port']: 'metrics.port',
|
||||
cfg['metrics']['https_port']: 'metrics.https_port', vpnrouter.manager_port(): 'the Router Manager',
|
||||
cfg['sites']['webdav_port']: 'sites.webdav_port', vpnrouter.WEBDAV_LOOPBACK_PORT: 'rclone\'s WebDAV',
|
||||
53: 'DNS'}
|
||||
for port in vpnrouter.BOOTSTRAP_PORTS:
|
||||
taken[port] = 'the bootstrap services'
|
||||
if cfg['openvpn']['protocol'] == 'tcp':
|
||||
taken[cfg['openvpn']['port']] = 'openvpn.port'
|
||||
for item in cfg['docker']['exposed_ports']:
|
||||
first, _, last = item.strip().partition(':')
|
||||
if first.isdigit() and (not last or last.isdigit()):
|
||||
for port in range(int(first), int(last or first) + 1):
|
||||
taken[port] = 'docker.exposed_ports'
|
||||
return taken
|
||||
|
||||
|
||||
def check_certificate(certificate, private_key, hosts):
|
||||
"""Refuse a PEM certificate chain whose first certificate is not the key's or names none of the hosts."""
|
||||
try:
|
||||
chain = x509.load_pem_x509_certificates(certificate.encode())
|
||||
except ValueError:
|
||||
raise Failure('certificate must be PEM certificates, the site\'s first', 2)
|
||||
try:
|
||||
key = serialization.load_pem_private_key(private_key.encode(), password=None)
|
||||
except (ValueError, TypeError):
|
||||
raise Failure('private_key must be an unencrypted PEM private key', 2)
|
||||
if not vpnrouter_pki.same_key(chain[0].public_key(), key.public_key()):
|
||||
raise Failure('private_key is not the key of the certificate', 2)
|
||||
names = {value.partition(':')[2] for value in vpnrouter_pki.san_values(chain[0]) if not value.startswith('email:')}
|
||||
missing = [host for host in hosts if host not in names and f'*.{host.partition(".")[2]}' not in names]
|
||||
if missing:
|
||||
raise Failure(f'the certificate does not name {", ".join(missing)}', 2)
|
||||
|
||||
|
||||
def check(cfg, sites, site):
|
||||
"""Refuse a site, a dict of SETTINGS with its name, that is invalid or collides with the other sites (sites, rows of
|
||||
the table) or with the router's own ports."""
|
||||
name = site['name']
|
||||
if not isinstance(name, str) or not NAME.match(name):
|
||||
raise Failure('a site name starts with a lowercase letter and has at most 31 lowercase letters, digits and '
|
||||
'hyphens', 2)
|
||||
for option, default in SETTINGS.items():
|
||||
value = site[option]
|
||||
if type(value) is not type(default) or (isinstance(value, list) and not all(isinstance(item, str) for item in value)):
|
||||
raise Failure(f'{option} must be {vpnrouter.TYPE_NAMES[type(default)]}', 2)
|
||||
hosts = site['hosts']
|
||||
if len(hosts) > HOSTS_MAX or len(set(hosts)) != len(hosts):
|
||||
raise Failure(f'hosts must be at most {HOSTS_MAX} different host names', 2)
|
||||
for host in hosts:
|
||||
if not (vpnrouter_pki.DNS_NAME.match(host) or vpnrouter_pki.is_address(host)):
|
||||
raise Failure(f'hosts: {host!r} is not a host name in lowercase or an IP address', 2)
|
||||
if host == cfg['wan']['local_fqdn'].strip():
|
||||
raise Failure(f'hosts: {host} is wan.local_fqdn, the name of the router\'s own services', 2)
|
||||
port = site['port']
|
||||
if not 1 <= port <= 65535:
|
||||
raise Failure(f'port: {port} is not a port', 2)
|
||||
taken = taken_ports(cfg)
|
||||
if port in taken:
|
||||
raise Failure(f'port: {port} is the port of {taken[port]}', 2)
|
||||
if site['listen'] not in LISTEN:
|
||||
raise Failure(f'listen: {site["listen"]!r} is not one of {", ".join(LISTEN)}', 2)
|
||||
tls = site['tls']
|
||||
if tls not in TLS:
|
||||
raise Failure(f'tls: {tls!r} is not one of {", ".join(TLS)}', 2)
|
||||
if tls != 'none' and not hosts:
|
||||
raise Failure(f'tls {tls} needs hosts, the names of the certificate', 2)
|
||||
if tls == 'acme' and any(host.startswith('*.') or vpnrouter_pki.is_address(host) for host in hosts):
|
||||
raise Failure('tls acme takes host names only, without wildcards', 2)
|
||||
if tls == 'ca':
|
||||
authorities = {key: value for key, value in vpnrouter.issuing_cas(cfg).items() if not value['builtin']}
|
||||
if site['ca'] not in authorities:
|
||||
raise Failure(f'ca: {site["ca"]!r} is not one of pki.issuing_cas', 2)
|
||||
if tls == 'file':
|
||||
check_certificate(site['certificate'], site['private_key'], hosts)
|
||||
if not INDEX.match(site['index_file']):
|
||||
raise Failure('index_file must be a file name', 2)
|
||||
if site['editors']:
|
||||
with vpnrouter_users.session() as connection:
|
||||
try:
|
||||
vpnrouter_users.group_entry(connection, site['editors'])
|
||||
except vpnrouter_users.Failure:
|
||||
raise Failure(f'editors: group {site["editors"]} does not exist', 2)
|
||||
if not site['enabled']:
|
||||
return
|
||||
for other in sites:
|
||||
if other['name'] == name or not other['enabled'] or other['port'] != port:
|
||||
continue
|
||||
prefix = f'port {port} is also the port of site {other["name"]}'
|
||||
if (other['tls'] == 'none') != (tls == 'none'):
|
||||
raise Failure(f'{prefix}, which is {"plain HTTP" if other["tls"] == "none" else "HTTPS"}', 2)
|
||||
if other['listen'] != site['listen']:
|
||||
raise Failure(f'{prefix}, which listens on {other["listen"]}', 2)
|
||||
if not hosts or not other['hosts']:
|
||||
raise Failure(f'{prefix}; sites sharing a port need host names', 2)
|
||||
common = sorted(set(hosts) & set(other['hosts']))
|
||||
if common:
|
||||
raise Failure(f'{prefix}, which has {", ".join(common)} too', 2)
|
||||
|
||||
|
||||
def save(cfg, name, values, create):
|
||||
"""Add the site with values, a dict of some of SETTINGS (create), or change those of an existing one; the site as
|
||||
shown(). An absent or empty private_key keeps the current one."""
|
||||
unknown = sorted(set(values) - set(SETTINGS))
|
||||
if unknown:
|
||||
raise Failure(f'unknown settings: {", ".join(unknown)}', 2)
|
||||
with database() as connection:
|
||||
rows = connection.execute('SELECT * FROM sites_list()').fetchall()
|
||||
current = next((row for row in rows if row['name'] == name), None)
|
||||
if create and current is not None:
|
||||
raise Failure(f'site {name} exists', 4)
|
||||
if not create:
|
||||
current = get(connection, name)
|
||||
site = {**SETTINGS, **({key: current[key] for key in SETTINGS} if current else {}), 'name': name}
|
||||
site.update({key: value for key, value in values.items() if key != 'private_key' or value})
|
||||
if isinstance(site['hosts'], list):
|
||||
site['hosts'] = [host.strip().lower() for host in site['hosts'] if isinstance(host, str) and host.strip()]
|
||||
if site['tls'] != 'file':
|
||||
site['certificate'] = site['private_key'] = ''
|
||||
if site['tls'] != 'ca':
|
||||
site['ca'] = ''
|
||||
check(cfg, rows, site)
|
||||
connection.execute('CALL site_put(%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s)',
|
||||
tuple(site[key] for key in ('name', *SETTINGS)))
|
||||
ensure_folders([name])
|
||||
return shown(get(connection, name))
|
||||
|
||||
|
||||
def remove(name, content):
|
||||
"""Remove the site, and with content its folder."""
|
||||
with database() as connection:
|
||||
get(connection, name)
|
||||
connection.execute('CALL site_remove(%s)', (name,))
|
||||
if content:
|
||||
shutil.rmtree(SITES_DIR / name, ignore_errors=True)
|
||||
return {'name': name, 'removed': True}
|
||||
|
||||
|
||||
def ensure_folders(names):
|
||||
"""Create the folders of the sites that miss one, readable by all."""
|
||||
SITES_DIR.mkdir(mode=0o755, parents=True, exist_ok=True)
|
||||
for name in names:
|
||||
(SITES_DIR / name).mkdir(mode=0o755, exist_ok=True)
|
||||
|
||||
|
||||
def resolve(name, relative):
|
||||
"""The path of the site's file or folder relative to its folder; Failure with status 2 for a path that leaves it,
|
||||
status 3 when the site has no folder."""
|
||||
if not isinstance(name, str) or not NAME.match(name):
|
||||
raise Failure(f'no site {name}', 3)
|
||||
base = SITES_DIR / name
|
||||
if not base.is_dir():
|
||||
raise Failure(f'site {name} has no folder', 3)
|
||||
if '\0' in relative or any(part in ('.', '..') for part in relative.split('/')):
|
||||
raise Failure(f'{relative!r} is not a path in the site\'s folder', 2)
|
||||
target = pathlib.Path(os.path.realpath(base / relative.strip('/')))
|
||||
if target != base and base not in target.parents:
|
||||
raise Failure(f'{relative!r} is not a path in the site\'s folder', 2)
|
||||
return target
|
||||
|
||||
|
||||
def relative_of(name, path):
|
||||
return str(path.relative_to(SITES_DIR / name))
|
||||
|
||||
|
||||
def entries(name, folder):
|
||||
"""The entries of a folder of the site: [{name, folder, size, modified}], folders first, then by name."""
|
||||
found = []
|
||||
for item in os.scandir(folder):
|
||||
try:
|
||||
info = item.stat(follow_symlinks=False)
|
||||
except OSError:
|
||||
continue
|
||||
found.append({'name': item.name, 'folder': stat.S_ISDIR(info.st_mode),
|
||||
'size': info.st_size if stat.S_ISREG(info.st_mode) else 0,
|
||||
'modified': datetime.datetime.fromtimestamp(info.st_mtime, datetime.timezone.utc)
|
||||
.isoformat(timespec='seconds')})
|
||||
return {'path': relative_of(name, folder) if folder != SITES_DIR / name else '',
|
||||
'entries': sorted(found, key=lambda entry: (not entry['folder'], entry['name']))}
|
||||
|
||||
|
||||
def receive(stream, length, target, limit):
|
||||
"""Write length bytes of the stream to the file target, replacing it in one step; Failure with status 2 over limit
|
||||
bytes. Its folders are created."""
|
||||
if length > limit:
|
||||
raise Failure(f'the file has {length} bytes, more than sites.upload_max_mb allows', 2)
|
||||
if target.is_dir():
|
||||
raise Failure(f'{target.name} is a folder', 2)
|
||||
target.parent.mkdir(mode=0o755, parents=True, exist_ok=True)
|
||||
descriptor, name = tempfile.mkstemp(dir=target.parent, prefix=f'.{target.name}.')
|
||||
try:
|
||||
with os.fdopen(descriptor, 'wb') as handle:
|
||||
left = length
|
||||
while left:
|
||||
data = stream.read(min(CHUNK, left))
|
||||
if not data:
|
||||
raise Failure('the client sent less than its Content-Length', 2)
|
||||
handle.write(data)
|
||||
left -= len(data)
|
||||
os.chmod(name, 0o644)
|
||||
os.replace(name, target)
|
||||
except BaseException:
|
||||
os.unlink(name)
|
||||
raise
|
||||
|
||||
|
||||
def extract(stream, length, folder, limit):
|
||||
"""Extract the zip archive of length bytes of the stream into the folder, created when missing; the number of files.
|
||||
|
||||
A member with an absolute path, a .. or a link, more than ARCHIVE_MEMBERS_MAX members or more than limit bytes
|
||||
unpacked refuse the whole archive, before anything is written.
|
||||
"""
|
||||
if length > limit:
|
||||
raise Failure(f'the archive has {length} bytes, more than sites.upload_max_mb allows', 2)
|
||||
folder.mkdir(mode=0o755, parents=True, exist_ok=True)
|
||||
with tempfile.TemporaryFile(dir=folder) as spool:
|
||||
left = length
|
||||
while left:
|
||||
data = stream.read(min(CHUNK, left))
|
||||
if not data:
|
||||
raise Failure('the client sent less than its Content-Length', 2)
|
||||
spool.write(data)
|
||||
left -= len(data)
|
||||
spool.seek(0)
|
||||
try:
|
||||
archive = zipfile.ZipFile(spool)
|
||||
except zipfile.BadZipFile:
|
||||
raise Failure('the file is not a zip archive', 2)
|
||||
with archive:
|
||||
members = archive.infolist()
|
||||
if len(members) > ARCHIVE_MEMBERS_MAX:
|
||||
raise Failure(f'the archive has more than {ARCHIVE_MEMBERS_MAX} members', 2)
|
||||
if sum(member.file_size for member in members) > limit:
|
||||
raise Failure('the archive unpacks to more than sites.upload_max_mb allows', 2)
|
||||
for member in members:
|
||||
parts = member.filename.replace('\\', '/').split('/')
|
||||
if member.filename.startswith('/') or '..' in parts or '\0' in member.filename or \
|
||||
stat.S_ISLNK(member.external_attr >> 16):
|
||||
raise Failure(f'the archive member {member.filename!r} is a link or leaves the folder', 2)
|
||||
count = 0
|
||||
for member in members:
|
||||
target = folder / member.filename.replace('\\', '/').strip('/')
|
||||
if member.is_dir():
|
||||
target.mkdir(mode=0o755, parents=True, exist_ok=True)
|
||||
continue
|
||||
target.parent.mkdir(mode=0o755, parents=True, exist_ok=True)
|
||||
with archive.open(member) as source:
|
||||
receive(source, member.file_size, target, limit)
|
||||
count += 1
|
||||
return count
|
||||
|
||||
|
||||
def make_folder(target):
|
||||
target.mkdir(mode=0o755, parents=True, exist_ok=True)
|
||||
|
||||
|
||||
def rename(name, source, relative):
|
||||
"""Move the file or folder source of the site to the path relative to its folder; Failure when that exists."""
|
||||
target = resolve(name, relative)
|
||||
if target.exists():
|
||||
raise Failure(f'{relative} exists', 4)
|
||||
target.parent.mkdir(mode=0o755, parents=True, exist_ok=True)
|
||||
os.replace(source, target)
|
||||
return target
|
||||
|
||||
|
||||
def delete(name, target):
|
||||
"""Remove the file or folder of the site, but not its folder."""
|
||||
if target == SITES_DIR / name:
|
||||
raise Failure('the site\'s folder itself cannot be removed', 2)
|
||||
if target.is_dir() and not target.is_symlink():
|
||||
shutil.rmtree(target)
|
||||
else:
|
||||
target.unlink()
|
||||
@@ -23,8 +23,9 @@ user who is not a service account when there are none.
|
||||
A group is a groupOfNames in ou=groups with a name, which follows the rules of user names, an
|
||||
optional description and at least one member; a group cannot be empty, so the last member
|
||||
cannot leave it and a user who is the last member of a group cannot be removed. ADMIN_GROUP grants the management of
|
||||
the router, and the allowed groups of the OpenID Provider's clients who may sign in to them; renaming a group renames it
|
||||
in those, and a group a client allows cannot be removed.
|
||||
the router, the allowed groups of the OpenID Provider's clients who may sign in to them, and the editors group of a web
|
||||
site who may change its files (vpnrouter_sites); renaming a group renames it in those, and a group a client allows or a
|
||||
site has cannot be removed.
|
||||
|
||||
A VPN user may have WireGuard peers, each a device with its public key and its IPs, in
|
||||
vpnRouterWireGuardPeer; vpn-router-setup writes a [Peer] of wg0.conf for each peer of a user who
|
||||
@@ -896,7 +897,8 @@ def group_add(connection, name, description, members):
|
||||
def group_update(connection, name, new_name=None, description=None):
|
||||
"""Rename the group or change its description ('' removes it); at least one must be given. Renaming ADMIN_GROUP
|
||||
switches the console login of its members off; renaming a group a client of the OpenID Provider allows renames it
|
||||
there."""
|
||||
there, and renaming the editors group of web sites renames it in the sites."""
|
||||
import vpnrouter_sites
|
||||
if new_name is None and description is None:
|
||||
raise Failure('give a new name or a description', 2)
|
||||
members = member_names(group_entry(connection, name))
|
||||
@@ -914,6 +916,10 @@ def group_update(connection, name, new_name=None, description=None):
|
||||
for client in allowing(connection, name):
|
||||
connection.modify_s(dn_of(client), [(ldap.MOD_DELETE, 'vpnRouterAllowedGroup', [group_dn(name).encode()]),
|
||||
(ldap.MOD_ADD, 'vpnRouterAllowedGroup', [group_dn(new_name).encode()])])
|
||||
try:
|
||||
vpnrouter_sites.rename_group(name, new_name)
|
||||
except vpnrouter_sites.DatabaseError as exc:
|
||||
raise Failure(f'group {name} is renamed {new_name}, but not in the web sites: {exc}')
|
||||
if name == ADMIN_GROUP:
|
||||
console_off(connection, members)
|
||||
name = new_name
|
||||
@@ -921,12 +927,19 @@ def group_update(connection, name, new_name=None, description=None):
|
||||
|
||||
|
||||
def group_remove(connection, name):
|
||||
"""Remove the group, unless a client of the OpenID Provider allows it; removing ADMIN_GROUP switches the console login
|
||||
of its members off."""
|
||||
"""Remove the group, unless a client of the OpenID Provider allows it or it is the editors group of web sites;
|
||||
removing ADMIN_GROUP switches the console login of its members off."""
|
||||
import vpnrouter_sites
|
||||
members = member_names(group_entry(connection, name))
|
||||
clients = allowing(connection, name)
|
||||
if clients:
|
||||
raise Failure(f'group {name} is an allowed group of the clients {", ".join(clients)}; take it out of them first', 2)
|
||||
try:
|
||||
sites = vpnrouter_sites.sites_of_group(name)
|
||||
except vpnrouter_sites.DatabaseError as exc:
|
||||
raise Failure(f'the web sites cannot be read: {exc}')
|
||||
if sites:
|
||||
raise Failure(f'group {name} is the editors group of the web sites {", ".join(sites)}; change them first', 2)
|
||||
connection.delete_s(group_dn(name))
|
||||
if name == ADMIN_GROUP:
|
||||
console_off(connection, members)
|
||||
|
||||
@@ -49,6 +49,11 @@ Pages:
|
||||
metrics_history())
|
||||
/logs the last lines of the journal of the router's units and of the files web.log_files lists
|
||||
(see logs())
|
||||
/sites the web sites Caddy serves: the settings of sites, list, add, change and remove (see sites()),
|
||||
and the files of each: list, download, upload, extract a zip archive, create a folder, rename and
|
||||
remove (see Handler.site_files()); a member of a site's editors group may use the site's files
|
||||
/dns the name server of dns.zone: its settings, the records it serves, and the manual records: add and
|
||||
remove (see dns())
|
||||
/postgresql
|
||||
the databases of each PostgreSQL cluster on the router with their sizes, as JSON (see
|
||||
postgresql_databases()); /config has the clusters' settings
|
||||
@@ -57,7 +62,7 @@ Pages:
|
||||
loopback address, where Caddy passes the requests of the HTTPS site; POST /oidc/login counts
|
||||
towards the limit of /auth calls
|
||||
|
||||
/status/full, /config, /metrics/history, /postgresql, /users, /service-accounts, /groups, /site-to-site, /backup, /remote-access, /ca, /logs, /shell and /account need an HS256 JWT in an Authorization: Bearer header signed
|
||||
/status/full, /config, /metrics/history, /postgresql, /users, /service-accounts, /groups, /site-to-site, /backup, /remote-access, /ca, /logs, /sites, /dns, /shell and /account need an HS256 JWT in an Authorization: Bearer header signed
|
||||
with vpnrouter_auth.WEB_KEY (see vpnrouter_auth.verify). A token of /auth names a user in sub;
|
||||
/account serves that user, and the other paths need a member of vpnrouter_users.ADMIN_GROUP. A
|
||||
token of vpn-router-token has the admin claim and names no user; it may call every path but
|
||||
@@ -78,7 +83,7 @@ whether each service runs, whether Docker is installed, how many of the site-to-
|
||||
and WireGuard. No configuration data, key or connection name is shown, and no address but the client's.
|
||||
|
||||
The service first initializes, without listening: it makes the signing key, the OpenID Provider's key
|
||||
(vpnrouter_oidc) and the refresh tokens' database (vpnrouter_auth), retrying every INIT_RETRY_SECONDS until all are ready and
|
||||
(vpnrouter_oidc), the refresh tokens' database (vpnrouter_auth) and the web sites' database (vpnrouter_sites), retrying every INIT_RETRY_SECONDS until all are ready and
|
||||
logging each failure. Then it listens. Its unit starts it only once wait-for-libpq has found libpq
|
||||
in the linker cache, where psycopg looks it up.
|
||||
|
||||
@@ -100,8 +105,10 @@ import fcntl
|
||||
import hashlib
|
||||
import ipaddress
|
||||
import json
|
||||
import mimetypes
|
||||
import os
|
||||
import pwd
|
||||
import re
|
||||
import secrets
|
||||
import select
|
||||
import shutil
|
||||
@@ -124,12 +131,14 @@ import ldap
|
||||
import vpnrouter
|
||||
import vpnrouter_auth
|
||||
import vpnrouter_backup
|
||||
import vpnrouter_dns
|
||||
import vpnrouter_metrics
|
||||
import vpnrouter_net
|
||||
import vpnrouter_oidc
|
||||
import vpnrouter_passkeys
|
||||
import vpnrouter_pki
|
||||
import vpnrouter_serve
|
||||
import vpnrouter_sites
|
||||
import vpnrouter_users
|
||||
from vpnrouter_status import established, established_each, openvpn_clients, run, swanctl, wireguard_peers
|
||||
|
||||
@@ -184,6 +193,15 @@ USER_STATUS = {2: 400, 3: 404, 4: 409, 5: 401}
|
||||
#: The HTTP status for each status of a vpnrouter_pki.Failure; any other is 500.
|
||||
PKI_STATUS = {2: 400, 3: 404, 4: 409}
|
||||
|
||||
#: The HTTP status for each status of a vpnrouter_sites.Failure; any other is 500.
|
||||
SITE_STATUS = {2: 400, 3: 404, 4: 409}
|
||||
|
||||
#: A path of a site's files: the site and the path in its folder.
|
||||
SITE_FILES = re.compile(r'^/sites/([^/]+)/files(?:/(.*))?$')
|
||||
|
||||
#: The bytes in a MiB, the unit of sites.upload_max_mb.
|
||||
MIB = 1024 * 1024
|
||||
|
||||
#: A file /pki/ serves: the certificate or the CRL of a CA, and its content type.
|
||||
PUBLISHED = {'.crt': 'application/pkix-cert', '.crl': 'application/pkix-crl'}
|
||||
|
||||
@@ -1128,6 +1146,149 @@ def logs(method, path, query):
|
||||
return 404, {'error': f'no route {method} {path}'}
|
||||
|
||||
|
||||
def section_change(name, body):
|
||||
"""Change any of the settings of the configuration's section as /backup changes its own; the section, or the HTTP
|
||||
status and the error document as a tuple."""
|
||||
settings = {option: body[option] for option in vpnrouter.SCHEMA[name] if option in body}
|
||||
try:
|
||||
with vpnrouter.config_lock():
|
||||
data = vpnrouter.read_settings()
|
||||
current = data.get(name)
|
||||
data[name] = {**(current if isinstance(current, dict) else {}), **settings}
|
||||
cfg = vpnrouter.with_defaults(data)
|
||||
vpnrouter.validate(cfg)
|
||||
vpnrouter.save_config(data)
|
||||
except vpnrouter.ConfigError as exc:
|
||||
return 400, {'error': str(exc)}
|
||||
except OSError as exc:
|
||||
return 500, {'error': f'the configuration cannot be written: {exc}'}
|
||||
except SystemExit as exc:
|
||||
return 500, {'error': str(exc.code)}
|
||||
run('systemctl', 'restart', '--no-block', 'vpn-router-setup.service')
|
||||
return cfg[name]
|
||||
|
||||
|
||||
def sites(method, path, query, body):
|
||||
"""Answer a /sites request but one of a site's files with vpnrouter_sites: the HTTP status and the JSON document.
|
||||
|
||||
GET /sites {"settings", "sites"}: the sites section of the configuration and the sites, each with its
|
||||
settings but private_key, with private_key_set, modified and its folder
|
||||
PUT /sites change any of the settings of the sites section
|
||||
POST /sites add a site: {"name", and any of "enabled", "hosts", "port", "listen", "tls", "ca",
|
||||
"certificate", "private_key", "index_file", "spa", "editors"}; an existing name is 409
|
||||
GET /sites/<name> the site
|
||||
PUT /sites/<name> change any of the settings of POST but name; an empty or absent private_key keeps the key
|
||||
DELETE /sites/<name> remove the site, with content=true also its folder
|
||||
|
||||
A change is applied by restarting vpn-router-setup without waiting for it.
|
||||
"""
|
||||
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
|
||||
return 400, {'error': 'the body must be a JSON object'}
|
||||
body = body or {}
|
||||
parts = path.strip('/').split('/')
|
||||
route = (method, len(parts))
|
||||
try:
|
||||
cfg = vpnrouter.load_config()
|
||||
if route == ('GET', 1):
|
||||
return 200, {'settings': cfg['sites'], 'sites': vpnrouter_sites.listing()}
|
||||
if route == ('PUT', 1):
|
||||
changed = section_change('sites', body)
|
||||
return changed if isinstance(changed, tuple) else (200, {'settings': changed})
|
||||
if route == ('POST', 1):
|
||||
name = body.get('name')
|
||||
document = vpnrouter_sites.save(cfg, name, {key: value for key, value in body.items() if key != 'name'},
|
||||
True)
|
||||
code = 201
|
||||
elif route == ('GET', 2):
|
||||
with vpnrouter_sites.database() as connection:
|
||||
return 200, vpnrouter_sites.shown(vpnrouter_sites.get(connection, parts[1]))
|
||||
elif route == ('PUT', 2):
|
||||
document, code = vpnrouter_sites.save(cfg, parts[1], body, False), 200
|
||||
elif route == ('DELETE', 2):
|
||||
document = vpnrouter_sites.remove(parts[1], query.get('content', [''])[0] == 'true')
|
||||
code = 200
|
||||
else:
|
||||
return 404, {'error': f'no route {method} {path}'}
|
||||
except vpnrouter_sites.Failure as failure:
|
||||
return SITE_STATUS.get(failure.status, 500), {'error': str(failure)}
|
||||
except vpnrouter_sites.DatabaseError as exc:
|
||||
return 500, {'error': str(exc)}
|
||||
except (ldap.LDAPError, OSError) as exc:
|
||||
return 500, {'error': f'the site cannot be changed: {exc}'}
|
||||
except SystemExit as exc:
|
||||
return 500, {'error': str(exc.code)}
|
||||
run('systemctl', 'restart', '--no-block', 'vpn-router-setup.service')
|
||||
return code, document
|
||||
|
||||
|
||||
def site_file_action(name, target, body):
|
||||
"""Answer a POST to a site's file or folder: {"folder": true} creates the folder, {"rename": "<path>"} moves the file
|
||||
or folder to the path in the site's folder; the HTTP status and the JSON document."""
|
||||
if not isinstance(body, dict):
|
||||
return 400, {'error': 'the body must be a JSON object'}
|
||||
try:
|
||||
if body.get('folder') is True:
|
||||
vpnrouter_sites.make_folder(target)
|
||||
return 201, {'path': vpnrouter_sites.relative_of(name, target), 'folder': True}
|
||||
if isinstance(body.get('rename'), str):
|
||||
moved = vpnrouter_sites.rename(name, target, body['rename'])
|
||||
return 200, {'path': vpnrouter_sites.relative_of(name, moved)}
|
||||
except vpnrouter_sites.Failure as failure:
|
||||
return SITE_STATUS.get(failure.status, 500), {'error': str(failure)}
|
||||
except OSError as exc:
|
||||
return 500, {'error': f'{exc.filename or target.name}: {exc.strerror}'}
|
||||
return 400, {'error': 'give {"folder": true} or {"rename": "<path>"}'}
|
||||
|
||||
|
||||
def dns(method, path, query, body):
|
||||
"""Answer a /dns request with vpnrouter_dns: the HTTP status and the JSON document.
|
||||
|
||||
GET /dns {"settings", "installed", "running", "records", "manual", "problem"}: the dns section of the
|
||||
configuration, whether PowerDNS is installed and runs, the records the name server serves,
|
||||
the manual records, and why the records cannot be read, null when they can
|
||||
PUT /dns change any of the settings of the dns section
|
||||
POST /dns/records add a manual record: {"name", "type", "content", "ttl"?}; name is relative to dns.zone, @
|
||||
for the zone itself, or absolute with a final dot; ttl is dns.ttl without it
|
||||
DELETE /dns/records remove the manual records of name= and type=, with content= only that one
|
||||
|
||||
A change is applied by restarting vpn-router-setup without waiting for it.
|
||||
"""
|
||||
if body == 'invalid' or (body is not None and not isinstance(body, dict)):
|
||||
return 400, {'error': 'the body must be a JSON object'}
|
||||
body = body or {}
|
||||
try:
|
||||
cfg = vpnrouter.load_config()
|
||||
if (method, path) == ('GET', '/dns'):
|
||||
served, manual, problem = [], [], None
|
||||
try:
|
||||
manual = vpnrouter_dns.manual()
|
||||
served = vpnrouter_dns.served(cfg['dns']['zone'].strip())
|
||||
except vpnrouter_dns.DatabaseError as exc:
|
||||
problem = str(exc)
|
||||
return 200, {'settings': cfg['dns'], 'installed': vpnrouter_dns.PDNS_SERVER.exists(),
|
||||
'running': unit_state(vpnrouter_dns.UNIT) == 'active', 'records': served, 'manual': manual,
|
||||
'problem': problem}
|
||||
if (method, path) == ('PUT', '/dns'):
|
||||
changed = section_change('dns', body)
|
||||
return changed if isinstance(changed, tuple) else (200, {'settings': changed})
|
||||
if (method, path) == ('POST', '/dns/records'):
|
||||
document, code = vpnrouter_dns.add(cfg, body), 201
|
||||
elif (method, path) == ('DELETE', '/dns/records'):
|
||||
content = query.get('content', [None])[0]
|
||||
document = vpnrouter_dns.remove(cfg, query.get('name', [''])[0], query.get('type', [''])[0], content)
|
||||
code = 200
|
||||
else:
|
||||
return 404, {'error': f'no route {method} {path}'}
|
||||
except vpnrouter_dns.Failure as failure:
|
||||
return (404 if failure.status == 3 else 400), {'error': str(failure)}
|
||||
except vpnrouter_dns.DatabaseError as exc:
|
||||
return 500, {'error': str(exc)}
|
||||
except SystemExit as exc:
|
||||
return 500, {'error': str(exc.code)}
|
||||
run('systemctl', 'restart', '--no-block', 'vpn-router-setup.service')
|
||||
return code, document
|
||||
|
||||
|
||||
def metrics_history(view):
|
||||
"""Answer GET /metrics/history?range=<view> with vpnrouter_metrics.history: the HTTP status and the JSON document.
|
||||
|
||||
@@ -1160,7 +1321,10 @@ def changes_admins(connection, method, parts, body):
|
||||
|
||||
|
||||
def refusal(claims, method, path, body):
|
||||
"""Why the holder of the claims may not make the request to a protected path, or '' when it may."""
|
||||
"""Why the holder of the claims may not make the request to a protected path, or '' when it may.
|
||||
|
||||
The files of a site may also be read and changed by a member of the site's editors group.
|
||||
"""
|
||||
name = claims.get('sub')
|
||||
if path == '/account' or path.startswith('/account/'):
|
||||
return '' if isinstance(name, str) else 'the token names no user'
|
||||
@@ -1171,6 +1335,9 @@ def refusal(claims, method, path, body):
|
||||
admins = vpnrouter_users.ADMIN_GROUP
|
||||
with vpnrouter_users.session() as connection:
|
||||
if not vpnrouter_users.is_admin(connection, name):
|
||||
files = SITE_FILES.match(path)
|
||||
if files and files.group(1) in vpnrouter_sites.editable(vpnrouter_users.groups_of(connection, name)):
|
||||
return ''
|
||||
return f'user {name} is not a member of {admins}'
|
||||
parts = path.strip('/').split('/')
|
||||
if changes_admins(connection, method, parts, body if isinstance(body, dict) else {}):
|
||||
@@ -1358,8 +1525,66 @@ class Handler(BaseHTTPRequestHandler):
|
||||
reason = refusal(claims, self.command, path, payload)
|
||||
except (ldap.LDAPError, OSError) as exc:
|
||||
return 500, f'the user directory failed: {exc}'
|
||||
except vpnrouter_sites.DatabaseError as exc:
|
||||
return 500, str(exc)
|
||||
return (403, reason) if reason else None
|
||||
|
||||
def site_files(self, name, relative, query, payload, send_body):
|
||||
"""Answer a request of a site's files: the HTTP status and the JSON document, or None once a file was sent.
|
||||
|
||||
GET /sites/<name>/files/<path> a folder: {"path", "entries": [{name, folder, size, modified}]}; a file:
|
||||
the file itself, as an attachment
|
||||
PUT /sites/<name>/files/<path> write the body to the file, its folders created; with unzip=true the body
|
||||
is a zip archive extracted into the folder of the path: {"path", "files"}
|
||||
POST /sites/<name>/files/<path> {"folder": true} creates the folder, {"rename": "<path>"} moves it
|
||||
DELETE /sites/<name>/files/<path> remove the file or folder
|
||||
|
||||
The body of a PUT needs a Content-Length of at most sites.upload_max_mb MiB.
|
||||
"""
|
||||
try:
|
||||
target = vpnrouter_sites.resolve(name, urllib.parse.unquote(relative))
|
||||
if self.command in ('GET', 'HEAD'):
|
||||
if target.is_dir():
|
||||
return 200, vpnrouter_sites.entries(name, target)
|
||||
if not target.is_file():
|
||||
return 404, {'error': f'{relative} does not exist'}
|
||||
with open(target, 'rb') as handle:
|
||||
size = os.fstat(handle.fileno()).st_size
|
||||
self.send_response(200)
|
||||
self.send_header('Content-Type', mimetypes.guess_type(target.name)[0] or 'application/octet-stream')
|
||||
self.send_header('Content-Length', str(size))
|
||||
self.send_header('Content-Disposition',
|
||||
f"attachment; filename*=UTF-8''{urllib.parse.quote(target.name)}")
|
||||
self.send_header('Cache-Control', 'no-store')
|
||||
self.end_headers()
|
||||
if send_body:
|
||||
shutil.copyfileobj(handle, self.wfile, vpnrouter_sites.CHUNK)
|
||||
return None
|
||||
if self.command == 'PUT':
|
||||
length = self.headers.get('Content-Length')
|
||||
if not length or not length.isdigit():
|
||||
return 411, {'error': 'the body needs a Content-Length'}
|
||||
limit = vpnrouter.load_config()['sites']['upload_max_mb'] * MIB
|
||||
if query.get('unzip', [''])[0] == 'true':
|
||||
count = vpnrouter_sites.extract(self.rfile, int(length), target, limit)
|
||||
return 200, {'path': vpnrouter_sites.relative_of(name, target), 'files': count}
|
||||
vpnrouter_sites.receive(self.rfile, int(length), target, limit)
|
||||
return 200, {'path': vpnrouter_sites.relative_of(name, target), 'files': 1}
|
||||
if self.command == 'POST':
|
||||
return site_file_action(name, target, payload)
|
||||
if self.command == 'DELETE':
|
||||
if not target.exists() and not target.is_symlink():
|
||||
return 404, {'error': f'{relative} does not exist'}
|
||||
vpnrouter_sites.delete(name, target)
|
||||
return 200, {'path': vpnrouter_sites.relative_of(name, target), 'removed': True}
|
||||
except vpnrouter_sites.Failure as failure:
|
||||
return SITE_STATUS.get(failure.status, 500), {'error': str(failure)}
|
||||
except OSError as exc:
|
||||
return 500, {'error': f'{exc.filename or relative}: {exc.strerror}'}
|
||||
except SystemExit as exc:
|
||||
return 500, {'error': str(exc.code)}
|
||||
return 405, {'error': f'{self.command} is not allowed on a site\'s files'}
|
||||
|
||||
def exact(self, size):
|
||||
"""The next size bytes from the client; ConnectionError when it closes first."""
|
||||
data = self.rfile.read(size)
|
||||
@@ -1546,15 +1771,19 @@ class Handler(BaseHTTPRequestHandler):
|
||||
if self.command not in ('GET', 'HEAD'):
|
||||
self.close_connection = True
|
||||
directory = next((name for name in ('users', 'service-accounts', 'groups', 'account', 'site-to-site', 'backup',
|
||||
'ca', 'shell', 'remote-access', 'logs')
|
||||
'ca', 'shell', 'remote-access', 'logs', 'sites', 'dns')
|
||||
if path == f'/{name}' or path.startswith(f'/{name}/')), None)
|
||||
protected = path in ('/status/full', '/config', '/metrics/history', '/postgresql') or directory is not None
|
||||
signing = path in ('/auth', '/auth/passkey', '/auth/passkey/complete', '/auth/refresh', '/auth/logout')
|
||||
oidc = (path == '/.well-known/openid-configuration' or path.startswith('/oidc/')) and \
|
||||
self.server.server_address[0] == vpnrouter_net.LOOPBACK
|
||||
files = SITE_FILES.match(path) if directory == 'sites' else None
|
||||
query = urllib.parse.parse_qs(urllib.parse.urlsplit(self.path).query)
|
||||
state = current() if protected or signing else {}
|
||||
claims = self.claims(state['key']) if protected and state['key'] else None
|
||||
payload = self.request_body() if (protected or signing) and self.command not in ('GET', 'HEAD') else None
|
||||
# The body of a file upload is read as a stream, after the request is allowed.
|
||||
payload = self.request_body() if (protected or signing) and self.command not in ('GET', 'HEAD') and \
|
||||
not (files and self.command == 'PUT') else None
|
||||
if claims is not None:
|
||||
self.principal = 'admin' if claims.get('admin') is True else claims.get('sub')
|
||||
if (protected or signing) and not state['key']:
|
||||
@@ -1610,8 +1839,20 @@ class Handler(BaseHTTPRequestHandler):
|
||||
code, document = ca(self.command, path, payload)
|
||||
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
|
||||
elif directory == 'logs':
|
||||
code, document = logs(self.command, path, urllib.parse.parse_qs(urllib.parse.urlsplit(self.path).query))
|
||||
code, document = logs(self.command, path, query)
|
||||
body, kind = json.dumps(document).encode() + b'\n', 'application/json'
|
||||
elif files:
|
||||
answered = self.site_files(files.group(1), files.group(2) or '', query, payload, send_body)
|
||||
if answered is None:
|
||||
return
|
||||
code, document = answered
|
||||
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
|
||||
elif directory == 'sites':
|
||||
code, document = sites(self.command, path, query, payload)
|
||||
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
|
||||
elif directory == 'dns':
|
||||
code, document = dns(self.command, path, query, payload)
|
||||
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
|
||||
elif directory == 'service-accounts':
|
||||
code, document = service_accounts(self.command, path, payload)
|
||||
body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json'
|
||||
@@ -1700,8 +1941,8 @@ def log(message):
|
||||
|
||||
|
||||
def initialize():
|
||||
"""Make the signing key, the OpenID Provider's key and the refresh tokens' database, retrying every
|
||||
INIT_RETRY_SECONDS until all are ready."""
|
||||
"""Make the signing key, the OpenID Provider's key, the refresh tokens' database and the web sites' database,
|
||||
retrying every INIT_RETRY_SECONDS until all are ready."""
|
||||
reported = None
|
||||
while True:
|
||||
phase = 'reading the configuration'
|
||||
@@ -1716,7 +1957,12 @@ def initialize():
|
||||
phase = 'connecting to the token database'
|
||||
with vpnrouter_auth.database(cfg):
|
||||
pass
|
||||
except (OSError, SystemExit, vpnrouter_auth.DatabaseError) as exc:
|
||||
phase = 'creating the database of the web sites'
|
||||
vpnrouter_sites.provision()
|
||||
phase = 'connecting to the database of the web sites'
|
||||
with vpnrouter_sites.database():
|
||||
pass
|
||||
except (OSError, SystemExit, vpnrouter_auth.DatabaseError, vpnrouter_sites.DatabaseError) as exc:
|
||||
error = f'{phase} failed: {exc.code if isinstance(exc, SystemExit) else exc}'
|
||||
if error != reported:
|
||||
log(f'{error}; retrying every {INIT_RETRY_SECONDS} seconds')
|
||||
|
||||
+51
@@ -0,0 +1,51 @@
|
||||
#!/usr/bin/python3
|
||||
"""The auth proxy of rclone's WebDAV server of the web sites' folders (vpn-router-webdav.service).
|
||||
|
||||
rclone runs it for each new user and password with {"user", "pass"} on standard input. The user signs in with the
|
||||
password of the router's directory, followed by the TOTP code for an enrolled user. A member of
|
||||
vpnrouter_users.ADMIN_GROUP sees the folders of all sites, any other user those of the sites whose editors group
|
||||
the user is a member of (vpnrouter_sites.editable), each as a folder named after its site. The answer on standard output
|
||||
is rclone's combine backend of those folders; a failed sign-in, a user with no site or any failure is a message on
|
||||
standard error and the exit status 1, which rclone refuses the user with.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
|
||||
|
||||
import ldap
|
||||
|
||||
import vpnrouter_ldap
|
||||
import vpnrouter_sites
|
||||
import vpnrouter_users
|
||||
|
||||
|
||||
def main():
|
||||
try:
|
||||
request = json.load(sys.stdin)
|
||||
name, password = request['user'], request['pass']
|
||||
vpnrouter_users.login(name, password, None)
|
||||
connection = vpnrouter_ldap.manager()
|
||||
try:
|
||||
groups = vpnrouter_users.groups_of(connection, name)
|
||||
finally:
|
||||
connection.unbind_s()
|
||||
if vpnrouter_users.ADMIN_GROUP in groups:
|
||||
sites = [site['name'] for site in vpnrouter_sites.listing()]
|
||||
else:
|
||||
sites = vpnrouter_sites.editable(groups)
|
||||
except (ValueError, KeyError, TypeError, OSError, ldap.LDAPError, vpnrouter_users.Failure,
|
||||
vpnrouter_sites.DatabaseError) as exc:
|
||||
sys.exit(f'webdav-auth: refused: {exc}')
|
||||
if not sites:
|
||||
sys.exit(f'webdav-auth: refused: {name} may change no site')
|
||||
vpnrouter_sites.ensure_folders(sites)
|
||||
upstreams = ' '.join(f'{site}={vpnrouter_sites.SITES_DIR / site}' for site in sites)
|
||||
json.dump({'type': 'combine', '_root': '', 'upstreams': upstreams}, sys.stdout)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
Executable
+157
@@ -0,0 +1,157 @@
|
||||
#!/usr/bin/python3
|
||||
"""Manage the web sites Caddy serves from the command line, with vpnrouter_sites.
|
||||
|
||||
A site has a name, which also names its folder in /var/lib/vpn-router/sites, host names, a port, the interfaces whose
|
||||
firewall opens it, TLS (none, acme, ca with --ca, or file with --certificate and --key), the file served for a folder,
|
||||
whether any path without a file is served that file of the site (--spa), and the group whose members change its files.
|
||||
A change is applied by restarting vpn-router-setup without waiting for it. The answer is text for people on standard
|
||||
output, and a failure is a message on standard error. With --json the answer is JSON on standard output, and a failure
|
||||
is {"error": ...} there too. The exit status says what failed: 2 the request is invalid, 3 the site does not exist, 4
|
||||
the site exists, 1 anything else. -h or --help shows the commands and their options.
|
||||
|
||||
vpn-router-site [--json] list
|
||||
vpn-router-site [--json] show <name>
|
||||
vpn-router-site [--json] add <name> [options]
|
||||
vpn-router-site [--json] update <name> [options]
|
||||
vpn-router-site [--json] remove <name> [--content]
|
||||
|
||||
options: [--host H ...] [--no-hosts] [--port P] [--listen internal|external|both] [--tls none|acme|ca|file]
|
||||
[--ca NAME] [--certificate FILE] [--key FILE] [--index FILE] [--spa | --no-spa]
|
||||
[--enabled | --no-enabled] [--editors GROUP]
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, '/usr/lib/vpn-router')
|
||||
|
||||
import ldap
|
||||
|
||||
import vpnrouter
|
||||
import vpnrouter_sites
|
||||
from vpnrouter_sites import Failure
|
||||
|
||||
#: The help line of each command.
|
||||
HELP = {
|
||||
'list': 'list the sites',
|
||||
'show': 'show a site',
|
||||
'add': 'add a site',
|
||||
'update': 'change the settings of a site',
|
||||
'remove': 'remove a site; its folder is kept without --content',
|
||||
}
|
||||
|
||||
|
||||
class Parser(argparse.ArgumentParser):
|
||||
"""An argument parser whose errors are Failures with status 2, so that --json answers them as JSON."""
|
||||
|
||||
def error(self, message):
|
||||
raise Failure(f'{message}; see {self.prog} --help', 2)
|
||||
|
||||
|
||||
def parse(argv):
|
||||
parser = Parser(prog='vpn-router-site', description='Manage the web sites of the router.')
|
||||
parser.add_argument('--json', action='store_true', help='answer in JSON')
|
||||
commands = parser.add_subparsers(dest='command', required=True, metavar='command', parser_class=Parser)
|
||||
for command, text in HELP.items():
|
||||
sub = commands.add_parser(command, help=text, description=text[0].upper() + text[1:] + '.')
|
||||
if command != 'list':
|
||||
sub.add_argument('name', help='the site name')
|
||||
if command in ('add', 'update'):
|
||||
sub.add_argument('--host', dest='hosts', action='append', metavar='HOST',
|
||||
help='a host name; give it once for each name, replacing the current ones')
|
||||
sub.add_argument('--no-hosts', action='store_true', help='answer any host name')
|
||||
sub.add_argument('--port', type=int, metavar='PORT', help='the TCP port')
|
||||
sub.add_argument('--listen', choices=vpnrouter_sites.LISTEN, help='the interfaces whose firewall opens the port')
|
||||
sub.add_argument('--tls', choices=vpnrouter_sites.TLS, help='where the certificate comes from')
|
||||
sub.add_argument('--ca', metavar='NAME', help='the issuing CA of tls ca')
|
||||
sub.add_argument('--certificate', metavar='FILE', help='the PEM certificate chain of tls file')
|
||||
sub.add_argument('--key', metavar='FILE', help='the PEM private key of tls file')
|
||||
sub.add_argument('--index', metavar='FILE', help='the file served for a folder')
|
||||
sub.add_argument('--spa', action=argparse.BooleanOptionalAction, help='serve the index of the site for any '
|
||||
'path without a file')
|
||||
sub.add_argument('--enabled', action=argparse.BooleanOptionalAction, help='serve the site')
|
||||
sub.add_argument('--editors', metavar='GROUP', help="the group whose members change the files; '' for none")
|
||||
if command == 'remove':
|
||||
sub.add_argument('--content', action='store_true', help='remove the site\'s folder too')
|
||||
return parser.parse_args(argv)
|
||||
|
||||
|
||||
def values_of(options):
|
||||
"""The settings the options give."""
|
||||
values = {key: getattr(options, option) for key, option in (('port', 'port'), ('listen', 'listen'), ('tls', 'tls'),
|
||||
('ca', 'ca'), ('index_file', 'index'), ('spa', 'spa'),
|
||||
('enabled', 'enabled'), ('editors', 'editors'))
|
||||
if getattr(options, option) is not None}
|
||||
if options.no_hosts:
|
||||
values['hosts'] = []
|
||||
elif options.hosts:
|
||||
values['hosts'] = options.hosts
|
||||
for key, option in (('certificate', 'certificate'), ('private_key', 'key')):
|
||||
path = getattr(options, option)
|
||||
if path is not None:
|
||||
try:
|
||||
with open(path, encoding='utf-8') as handle:
|
||||
values[key] = handle.read()
|
||||
except OSError as exc:
|
||||
raise Failure(f'{path} cannot be read: {exc.strerror}', 2)
|
||||
return values
|
||||
|
||||
|
||||
def run(options):
|
||||
"""The answer of the command the options name, and whether it changed a site."""
|
||||
command = options.command
|
||||
if command == 'list':
|
||||
return {'sites': vpnrouter_sites.listing()}, False
|
||||
if command == 'show':
|
||||
with vpnrouter_sites.database() as connection:
|
||||
return vpnrouter_sites.shown(vpnrouter_sites.get(connection, options.name)), False
|
||||
if command == 'remove':
|
||||
return vpnrouter_sites.remove(options.name, options.content), True
|
||||
cfg = vpnrouter.load_config()
|
||||
return vpnrouter_sites.save(cfg, options.name, values_of(options), command == 'add'), True
|
||||
|
||||
|
||||
def text_of(answer):
|
||||
"""The answer as text for people."""
|
||||
if answer.get('removed'):
|
||||
return f'site {answer["name"]} removed'
|
||||
sites = answer['sites'] if 'sites' in answer else [answer]
|
||||
if not sites:
|
||||
return 'no sites'
|
||||
head = ('SITE', 'ENABLED', 'HOSTS', 'PORT', 'LISTEN', 'TLS', 'EDITORS')
|
||||
rows = [(site['name'], 'yes' if site['enabled'] else 'no', ', '.join(site['hosts']) or '*', str(site['port']),
|
||||
site['listen'], site['tls'] + (f' {site["ca"]}' if site['tls'] == 'ca' else ''), site['editors'] or '-')
|
||||
for site in sites]
|
||||
widths = [max(len(row[column]) for row in (head, *rows)) for column in range(len(head))]
|
||||
return '\n'.join(' '.join(cell.ljust(width) for cell, width in zip(row, widths)).rstrip() for row in (head, *rows))
|
||||
|
||||
|
||||
def main(argv):
|
||||
# A request that cannot be parsed is answered in JSON when --json is among the arguments.
|
||||
as_json = '--json' in argv
|
||||
try:
|
||||
options = parse(argv)
|
||||
as_json = options.json
|
||||
answer, changed = run(options)
|
||||
except Failure as failure:
|
||||
message, status = str(failure), failure.status
|
||||
except vpnrouter_sites.DatabaseError as exc:
|
||||
message, status = str(exc), 1
|
||||
except (ldap.LDAPError, OSError) as exc:
|
||||
message, status = f'the user directory failed: {exc}', 1
|
||||
else:
|
||||
if changed:
|
||||
subprocess.run(['systemctl', 'restart', '--no-block', 'vpn-router-setup.service'])
|
||||
print(json.dumps(answer) if as_json else text_of(answer))
|
||||
return 0
|
||||
if as_json:
|
||||
print(json.dumps({'error': message}))
|
||||
else:
|
||||
print(f'vpn-router-site: {message}', file=sys.stderr)
|
||||
return status
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main(sys.argv[1:]))
|
||||
@@ -0,0 +1,95 @@
|
||||
# bash completion for vpn-router-site -*- shell-script -*-
|
||||
|
||||
# The names of the sites, of the groups and of the general-purpose issuing CAs, read as root; nothing for other users.
|
||||
_vpn_router_site_names()
|
||||
{
|
||||
vpn-router-site --json list 2>/dev/null |
|
||||
python3 -c 'import json, sys; print("\n".join(site["name"] for site in json.load(sys.stdin)["sites"]))' 2>/dev/null
|
||||
}
|
||||
|
||||
_vpn_router_site_groups()
|
||||
{
|
||||
vpn-router-group --json list 2>/dev/null |
|
||||
python3 -c 'import json, sys; print("\n".join(group["name"] for group in json.load(sys.stdin)["groups"]))' 2>/dev/null
|
||||
}
|
||||
|
||||
_vpn_router_site_cas()
|
||||
{
|
||||
python3 -c 'import sys; sys.path.insert(0, "/usr/lib/vpn-router"); import vpnrouter
|
||||
print("\n".join(vpnrouter.load_config()["pki"]["issuing_cas"]))' 2>/dev/null
|
||||
}
|
||||
|
||||
_vpn_router_site()
|
||||
{
|
||||
local cur prev words cword split
|
||||
_init_completion -s || return
|
||||
|
||||
# The command is the first word that is not an option; the arguments are the words after it but option values.
|
||||
local i command='' args=0
|
||||
for ((i = 1; i < cword; i++)); do
|
||||
if [[ -z $command ]]; then
|
||||
[[ ${words[i]} != -* ]] && command=${words[i]}
|
||||
continue
|
||||
fi
|
||||
case ${words[i]} in
|
||||
--host | --port | --listen | --tls | --ca | --certificate | --key | --index | --editors) ((i++)) ;;
|
||||
-*) ;;
|
||||
*) ((args++)) ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [[ -z $command ]]; then
|
||||
if [[ $cur == -* ]]; then
|
||||
COMPREPLY=($(compgen -W '--json --help' -- "$cur"))
|
||||
else
|
||||
COMPREPLY=($(compgen -W 'list show add update remove' -- "$cur"))
|
||||
fi
|
||||
return
|
||||
fi
|
||||
|
||||
case $prev in
|
||||
--listen)
|
||||
COMPREPLY=($(compgen -W 'internal external both' -- "$cur"))
|
||||
return
|
||||
;;
|
||||
--tls)
|
||||
COMPREPLY=($(compgen -W 'none acme ca file' -- "$cur"))
|
||||
return
|
||||
;;
|
||||
--ca)
|
||||
COMPREPLY=($(compgen -W "$(_vpn_router_site_cas)" -- "$cur"))
|
||||
return
|
||||
;;
|
||||
--editors)
|
||||
COMPREPLY=($(compgen -W "$(_vpn_router_site_groups)" -- "$cur"))
|
||||
return
|
||||
;;
|
||||
--certificate | --key)
|
||||
_filedir
|
||||
return
|
||||
;;
|
||||
--host | --port | --index)
|
||||
return
|
||||
;;
|
||||
esac
|
||||
$split && return
|
||||
|
||||
if [[ $cur == -* ]]; then
|
||||
local options='--help'
|
||||
case $command in
|
||||
add | update) options+=' --host --no-hosts --port --listen --tls --ca --certificate --key --index --spa
|
||||
--no-spa --enabled --no-enabled --editors' ;;
|
||||
remove) options+=' --content' ;;
|
||||
esac
|
||||
COMPREPLY=($(compgen -W "$options" -- "$cur"))
|
||||
return
|
||||
fi
|
||||
|
||||
# The site, which add takes new.
|
||||
if ((args == 0)) && [[ $command != @(list|add) ]]; then
|
||||
COMPREPLY=($(compgen -W "$(_vpn_router_site_names)" -- "$cur"))
|
||||
fi
|
||||
} &&
|
||||
complete -F _vpn_router_site vpn-router-site
|
||||
|
||||
# ex: filetype=sh
|
||||
@@ -80,6 +80,20 @@
|
||||
"oidc": {
|
||||
"enabled": false
|
||||
},
|
||||
"sites": {
|
||||
"enabled": false,
|
||||
"upload_max_mb": 512,
|
||||
"webdav": false,
|
||||
"webdav_port": 7445
|
||||
},
|
||||
"dns": {
|
||||
"enabled": false,
|
||||
"zone": "",
|
||||
"nameserver": "",
|
||||
"addresses": [],
|
||||
"listen": "external",
|
||||
"ttl": 300
|
||||
},
|
||||
"pki": {
|
||||
"root_name": "VPN Router Root CA",
|
||||
"root_days": 7300,
|
||||
|
||||
@@ -0,0 +1,87 @@
|
||||
-- The zone the router's name server (PowerDNS, gpgsql backend) answers for (vpnrouter_dns), in the vpn-router-dns
|
||||
-- database, whose domains and records tables are PowerDNS's own schema.
|
||||
-- The whole file runs as one transaction, serialized by an advisory lock, so that two processes can load it at once.
|
||||
|
||||
SELECT pg_advisory_xact_lock(hashtext('vpn-router dns schema'));
|
||||
|
||||
-- A record an administrator added to the zone, besides those of the zone itself and of the sites.
|
||||
CREATE TABLE IF NOT EXISTS dns_manual_records (
|
||||
name text NOT NULL,
|
||||
type text NOT NULL,
|
||||
content text NOT NULL,
|
||||
ttl integer NOT NULL,
|
||||
PRIMARY KEY (name, type, content)
|
||||
);
|
||||
|
||||
-- The digest of the records last written to the zone and the serial of its SOA.
|
||||
CREATE TABLE IF NOT EXISTS dns_state (
|
||||
zone text PRIMARY KEY,
|
||||
digest text NOT NULL,
|
||||
serial bigint NOT NULL
|
||||
);
|
||||
|
||||
CREATE OR REPLACE FUNCTION dns_manual_list()
|
||||
RETURNS TABLE (name text, type text, content text, ttl integer)
|
||||
LANGUAGE sql STABLE AS $$
|
||||
SELECT name, type, content, ttl FROM dns_manual_records ORDER BY name, type, content;
|
||||
$$;
|
||||
|
||||
-- Add the record, or set its TTL.
|
||||
CREATE OR REPLACE PROCEDURE dns_manual_put(p_name text, p_type text, p_content text, p_ttl integer)
|
||||
LANGUAGE sql AS $$
|
||||
INSERT INTO dns_manual_records VALUES (p_name, p_type, p_content, p_ttl)
|
||||
ON CONFLICT (name, type, content) DO UPDATE SET ttl = p_ttl;
|
||||
$$;
|
||||
|
||||
-- Remove the records of the name and type, all of them without content.
|
||||
CREATE OR REPLACE PROCEDURE dns_manual_remove(p_name text, p_type text, p_content text)
|
||||
LANGUAGE sql AS $$
|
||||
DELETE FROM dns_manual_records WHERE name = p_name AND type = p_type AND (p_content IS NULL OR content = p_content);
|
||||
$$;
|
||||
|
||||
-- Make the zone the only one served and give it the records, its SOA first, whose content has {serial} for the serial.
|
||||
-- The serial grows, to at least the current Unix time, only when the records differ from those last written.
|
||||
CREATE OR REPLACE PROCEDURE dns_apply(p_zone text, p_soa text, p_ttl integer, p_names text[], p_types text[],
|
||||
p_contents text[], p_ttls integer[])
|
||||
LANGUAGE plpgsql AS $$
|
||||
DECLARE
|
||||
v_domain integer;
|
||||
v_serial bigint;
|
||||
v_digest text := md5(concat_ws('|', p_soa, p_ttl, array_to_string(p_names, ','), array_to_string(p_types, ','),
|
||||
array_to_string(p_contents, ','), array_to_string(p_ttls, ',')));
|
||||
BEGIN
|
||||
DELETE FROM domains WHERE name <> p_zone;
|
||||
DELETE FROM dns_state WHERE zone <> p_zone;
|
||||
SELECT id INTO v_domain FROM domains WHERE name = p_zone;
|
||||
IF v_domain IS NULL THEN
|
||||
INSERT INTO domains (name, type) VALUES (p_zone, 'NATIVE') RETURNING id INTO v_domain;
|
||||
ELSIF EXISTS (SELECT 1 FROM dns_state WHERE zone = p_zone AND digest = v_digest) THEN
|
||||
RETURN;
|
||||
END IF;
|
||||
SELECT greatest(coalesce(max(serial) + 1, 1), extract(epoch FROM now())::bigint) INTO v_serial
|
||||
FROM dns_state WHERE zone = p_zone;
|
||||
DELETE FROM records WHERE domain_id = v_domain;
|
||||
INSERT INTO records (domain_id, name, type, content, ttl, prio, disabled, auth)
|
||||
VALUES (v_domain, p_zone, 'SOA', replace(p_soa, '{serial}', v_serial::text), p_ttl, 0, false, true);
|
||||
INSERT INTO records (domain_id, name, type, content, ttl, prio, disabled, auth)
|
||||
SELECT v_domain, r.name, r.type, r.content, r.ttl, 0, false, true
|
||||
FROM unnest(p_names, p_types, p_contents, p_ttls) AS r(name, type, content, ttl);
|
||||
INSERT INTO dns_state VALUES (p_zone, v_digest, v_serial)
|
||||
ON CONFLICT (zone) DO UPDATE SET digest = v_digest, serial = v_serial;
|
||||
END;
|
||||
$$;
|
||||
|
||||
-- Serve no zone.
|
||||
CREATE OR REPLACE PROCEDURE dns_clear()
|
||||
LANGUAGE sql AS $$
|
||||
DELETE FROM domains;
|
||||
DELETE FROM dns_state;
|
||||
$$;
|
||||
|
||||
-- The records of the zone as they are served.
|
||||
CREATE OR REPLACE FUNCTION dns_served(p_zone text)
|
||||
RETURNS TABLE (name text, type text, content text, ttl integer)
|
||||
LANGUAGE sql STABLE AS $$
|
||||
SELECT r.name, r.type, r.content, r.ttl FROM records r JOIN domains d ON d.id = r.domain_id
|
||||
WHERE d.name = p_zone ORDER BY r.type <> 'SOA', r.name, r.type, r.content;
|
||||
$$;
|
||||
@@ -0,0 +1,85 @@
|
||||
-- The web sites Caddy serves (vpnrouter_sites), in the vpn-router-sites database.
|
||||
-- The whole file runs as one transaction, serialized by an advisory lock, so that two processes can load it at once.
|
||||
|
||||
SELECT pg_advisory_xact_lock(hashtext('vpn-router sites schema'));
|
||||
|
||||
-- A static web site: its host names, port, interfaces and TLS, and the group whose members change its files.
|
||||
CREATE TABLE IF NOT EXISTS sites (
|
||||
name text PRIMARY KEY,
|
||||
enabled boolean NOT NULL,
|
||||
hosts text[] NOT NULL,
|
||||
port integer NOT NULL,
|
||||
listen text NOT NULL CHECK (listen IN ('internal', 'external', 'both')),
|
||||
tls text NOT NULL CHECK (tls IN ('none', 'acme', 'ca', 'file')),
|
||||
ca text NOT NULL,
|
||||
certificate text NOT NULL,
|
||||
private_key text NOT NULL,
|
||||
index_file text NOT NULL,
|
||||
spa boolean NOT NULL,
|
||||
editors text NOT NULL,
|
||||
modified timestamptz NOT NULL
|
||||
);
|
||||
|
||||
-- Every site, by name; the private key is left out.
|
||||
CREATE OR REPLACE FUNCTION sites_list()
|
||||
RETURNS TABLE (name text, enabled boolean, hosts text[], port integer, listen text, tls text, ca text,
|
||||
certificate text, index_file text, spa boolean, editors text, modified timestamptz)
|
||||
LANGUAGE sql STABLE AS $$
|
||||
SELECT name, enabled, hosts, port, listen, tls, ca, certificate, index_file, spa, editors, modified
|
||||
FROM sites ORDER BY name;
|
||||
$$;
|
||||
|
||||
-- The site with its private key.
|
||||
CREATE OR REPLACE FUNCTION site_get(p_name text)
|
||||
RETURNS TABLE (name text, enabled boolean, hosts text[], port integer, listen text, tls text, ca text,
|
||||
certificate text, private_key text, index_file text, spa boolean, editors text, modified timestamptz)
|
||||
LANGUAGE sql STABLE AS $$
|
||||
SELECT name, enabled, hosts, port, listen, tls, ca, certificate, private_key, index_file, spa, editors, modified
|
||||
FROM sites WHERE name = p_name;
|
||||
$$;
|
||||
|
||||
-- The enabled sites with their private keys, by name, for Caddy's configuration and certificate files.
|
||||
CREATE OR REPLACE FUNCTION sites_served()
|
||||
RETURNS TABLE (name text, enabled boolean, hosts text[], port integer, listen text, tls text, ca text,
|
||||
certificate text, private_key text, index_file text, spa boolean, editors text, modified timestamptz)
|
||||
LANGUAGE sql STABLE AS $$
|
||||
SELECT name, enabled, hosts, port, listen, tls, ca, certificate, private_key, index_file, spa, editors, modified
|
||||
FROM sites WHERE enabled ORDER BY name;
|
||||
$$;
|
||||
|
||||
-- Add the site, or replace all its settings.
|
||||
CREATE OR REPLACE PROCEDURE site_put(p_name text, p_enabled boolean, p_hosts text[], p_port integer, p_listen text,
|
||||
p_tls text, p_ca text, p_certificate text, p_private_key text, p_index_file text,
|
||||
p_spa boolean, p_editors text)
|
||||
LANGUAGE sql AS $$
|
||||
INSERT INTO sites VALUES (p_name, p_enabled, p_hosts, p_port, p_listen, p_tls, p_ca, p_certificate, p_private_key,
|
||||
p_index_file, p_spa, p_editors, now())
|
||||
ON CONFLICT (name) DO UPDATE SET enabled = p_enabled, hosts = p_hosts, port = p_port, listen = p_listen,
|
||||
tls = p_tls, ca = p_ca, certificate = p_certificate, private_key = p_private_key, index_file = p_index_file,
|
||||
spa = p_spa, editors = p_editors, modified = now();
|
||||
$$;
|
||||
|
||||
CREATE OR REPLACE PROCEDURE site_remove(p_name text)
|
||||
LANGUAGE sql AS $$
|
||||
DELETE FROM sites WHERE name = p_name;
|
||||
$$;
|
||||
|
||||
-- The names of the sites whose files the members of the groups may change.
|
||||
CREATE OR REPLACE FUNCTION sites_editable(p_groups text[])
|
||||
RETURNS TABLE (name text)
|
||||
LANGUAGE sql STABLE AS $$
|
||||
SELECT name FROM sites WHERE editors <> '' AND editors = ANY (p_groups) ORDER BY name;
|
||||
$$;
|
||||
|
||||
-- The names of the sites whose editors are the group.
|
||||
CREATE OR REPLACE FUNCTION sites_of_group(p_group text)
|
||||
RETURNS TABLE (name text)
|
||||
LANGUAGE sql STABLE AS $$
|
||||
SELECT name FROM sites WHERE editors = p_group ORDER BY name;
|
||||
$$;
|
||||
|
||||
-- Give the sites of a renamed group its new name.
|
||||
CREATE OR REPLACE PROCEDURE sites_rename_group(p_old text, p_new text)
|
||||
LANGUAGE sql AS $$
|
||||
UPDATE sites SET editors = p_new, modified = now() WHERE editors = p_old;
|
||||
$$;
|
||||
@@ -1,13 +1,13 @@
|
||||
# Written by vpn-router-setup from /etc/vpn-router/vpn-router.json; edits are overwritten.
|
||||
# Written by vpn-router-setup from /etc/vpn-router/vpn-router.json and the web sites; edits are overwritten.
|
||||
{
|
||||
auto_https disable_redirects
|
||||
{%- if https_email %}
|
||||
email {{ https_email }}
|
||||
{%- endif %}
|
||||
{%- if acme_cas %}
|
||||
{%- if pki_cas %}
|
||||
skip_install_trust
|
||||
pki {
|
||||
{%- for ca in acme_cas %}
|
||||
{%- for ca in pki_cas %}
|
||||
ca {{ ca.id }} {
|
||||
name "{{ ca.name }}"
|
||||
root {
|
||||
@@ -23,7 +23,7 @@
|
||||
}
|
||||
{%- endif %}
|
||||
}
|
||||
{% if web_enabled or 'https' in pki_publish or acme_cas %}
|
||||
{% if https and (web_enabled or 'https' in pki_publish or acme_cas) %}
|
||||
{{ local_fqdn }}:{{ https_web_port }} {
|
||||
{%- if 'https' in pki_publish %}
|
||||
handle_path /pki/* {
|
||||
@@ -32,9 +32,9 @@
|
||||
}
|
||||
{%- endif %}
|
||||
{%- for ca in acme_cas %}
|
||||
handle /acme/{{ ca.id }}/* {
|
||||
handle /acme/{{ ca }}/* {
|
||||
acme_server {
|
||||
ca {{ ca.id }}
|
||||
ca {{ ca }}
|
||||
}
|
||||
}
|
||||
{%- endfor %}
|
||||
@@ -47,7 +47,7 @@
|
||||
}
|
||||
}
|
||||
{% endif %}
|
||||
{%- if 'http-80' in pki_publish %}
|
||||
{%- if https and 'http-80' in pki_publish %}
|
||||
http://{{ local_fqdn }} {
|
||||
handle_path /pki/* {
|
||||
root * {{ pki_publish_dir }}
|
||||
@@ -58,7 +58,7 @@ http://{{ local_fqdn }} {
|
||||
}
|
||||
}
|
||||
{% endif %}
|
||||
{%- if 'https-default' in pki_publish %}
|
||||
{%- if https and 'https-default' in pki_publish %}
|
||||
{{ local_fqdn }}:443 {
|
||||
handle_path /pki/* {
|
||||
root * {{ pki_publish_dir }}
|
||||
@@ -69,13 +69,39 @@ http://{{ local_fqdn }} {
|
||||
}
|
||||
}
|
||||
{% endif %}
|
||||
{%- if manager_port %}
|
||||
{%- if https and manager_port %}
|
||||
{{ local_fqdn }}:{{ https_manager_port }} {
|
||||
reverse_proxy {{ loopback }}:{{ manager_port }}
|
||||
}
|
||||
{% endif %}
|
||||
{%- if metrics_enabled %}
|
||||
{%- if https and metrics_enabled %}
|
||||
{{ local_fqdn }}:{{ metrics_https_port }} {
|
||||
reverse_proxy {{ loopback }}:{{ metrics_port }}
|
||||
}
|
||||
{% endif %}
|
||||
{%- if webdav %}
|
||||
{{ local_fqdn }}:{{ webdav_port }} {
|
||||
reverse_proxy {{ loopback }}:{{ webdav_loopback_port }}
|
||||
}
|
||||
{% endif %}
|
||||
{%- for site in sites %}
|
||||
# Site {{ site.name }}
|
||||
{{ site.addresses | join(', ') }} {
|
||||
root * {{ site.root }}
|
||||
{%- if site.tls == 'ca' %}
|
||||
tls {
|
||||
issuer internal {
|
||||
ca {{ site.ca }}
|
||||
}
|
||||
}
|
||||
{%- elif site.tls == 'file' %}
|
||||
tls {{ site.cert }} {{ site.key }}
|
||||
{%- endif %}
|
||||
{%- if site.spa %}
|
||||
try_files {path} /{{ site.index }}
|
||||
{%- endif %}
|
||||
file_server {
|
||||
index {{ site.index }}
|
||||
}
|
||||
}
|
||||
{% endfor %}
|
||||
Reference in new issue
Block a user