diff --git a/ADR.md b/ADR.md index f1c0847..2df25cc 100644 --- a/ADR.md +++ b/ADR.md @@ -1065,3 +1065,20 @@ Consequences: Completions follow the code without being edited for each new sett Tab that offers names runs a command or Python, which takes a fraction of a second. The files are not conffiles; an administrator overrides one in `/etc/bash_completion.d` or the user's own completions directory. + +## ADR-39: /logs shows the last lines of listed units and files, capped per request + +Status: Accepted + +Context: The Router Manager should show the router's logs without SSH. The web service runs as +root, so a path or a unit taken from a request could expose any file or journal on the router, +and a log of gigabytes cannot be sent whole to a browser. + +Decision: `/logs` serves the journal of a fixed list of units, the package's own and the services +it runs, or the whole journal, read with `journalctl`, and the files `web.log_files` lists, +which `vpn-config-edit` sets; a unit or a path outside the lists is refused. A request answers +at most 1000 lines, 200 by default, and a file is read from its last MiB only. The answer is +read at each request; nothing is followed or kept. + +Consequences: Only administrators read logs, and only those listed. Showing a new file needs a +configuration change, not code. Long histories stay with SSH and `journalctl`. diff --git a/README.md b/README.md index d13332d..2353f02 100644 --- a/README.md +++ b/README.md @@ -130,6 +130,7 @@ must have the type shown below; an unknown section or setting is an error. | `web` | `status_content` | string | `basic` | What `/` and `/status` show: `basic`, `standard` or `full` (see Status web service) | | `web` | `shell` | boolean | `false` | `/shell` opens a login shell for a member of `admins` with console login (see The shell) | | `web` | `shell_idle_minutes` | integer | `15` | Minutes without a keystroke after which a shell is closed, 1 to 1440 | +| `web` | `log_files` | list | | Absolute paths of the log files `/logs` shows, at most 32 (see The /logs API) | | `web` | `shell_auth_timeout` | integer | `15` | Minutes after the last use of a shell in which a new shell opens without signing in again, 0 (always sign in) to 1440 | | `ssh` | `ldap_login` | boolean | `false` | sshd accepts the directory password of the console accounts in the directory (see Console login and SSH) | | `https` | `enabled` | boolean | `true` | Caddy serves the status web service and the Router Manager over HTTPS for `wan.local_fqdn` | @@ -940,6 +941,7 @@ curl http://10.0.0.4:17081/ | `/groups` | The groups of users; needs a token | | `/site-to-site` | The site-to-site connections: list, add, change and remove; needs a token | | `/account` | The account of the signed-in user: its details, its password and its TOTP enrolment; needs a user's token | +| `/logs` | The last lines of the journal of the router's units and of the files of `web.log_files`; needs a token | | `/.well-known/openid-configuration`, `/oidc/...` | The OpenID Provider, on `127.0.0.1` only, where Caddy passes the requests (see OpenID Provider) | The status holds the state of `vpn-router-setup`, `slapd` and each PostgreSQL cluster unit @@ -963,7 +965,7 @@ curl http://10.0.0.4:7081/status ### Tokens and signing in -`/config`, `/users`, `/service-accounts`, `/groups`, `/site-to-site`, `/backup`, `/remote-access`, `/ca`, `/shell` and `/account` need a JWT in an `Authorization: Bearer` header, signed with +`/config`, `/users`, `/service-accounts`, `/groups`, `/site-to-site`, `/backup`, `/remote-access`, `/ca`, `/logs`, `/shell` and `/account` need a JWT in an `Authorization: Bearer` header, signed with HS256 by the key the web service makes on its first run, `/etc/vpn-router/web.key` (root only). The key never leaves the router; while it does not exist these paths and `/auth` are not served (404). A missing, expired or wrongly signed token, or one of another algorithm, gets 401 with @@ -1078,6 +1080,27 @@ router: - **A cluster that is not online:** `databases` and `size` are `null`. - **Collection:** the web service reads it with `psql` as the `postgres` user at each request. +### The /logs API + +`/logs`, with the same token as `/config`, shows the last lines of the journal and of log files, +read at each request: + +| Request | Answer | +| --- | --- | +| `GET /logs` | `{"units", "files", "max_lines"}`: the units whose journal it shows, the files of `web.log_files` and the most lines one request answers, 1000 | +| `GET /logs/journal?unit=&lines=` | `{"lines": [...]}`: the last lines of the unit's journal, or of the whole journal without `unit`, with ISO times | +| `GET /logs/file?path=&lines=` | `{"lines": [...]}`: the last lines of a file `web.log_files` lists | + +The units are the package's own (`vpn-router-setup`, `-web`, `-metrics`, `-pki`, `-backup`, +`-caddy`), `vpn-router-manager`, `caddy`, `slapd`, `nslcd`, `postgresql`, `strongswan`, +`wg-quick@wg0`, `openvpn-server@vpn-router` and `ssh`. `lines` is 1 to 1000, 200 without it. A +file is read from at most its last MiB, so very long lines give fewer lines. A unit or a file not +in the lists is 404, as is a listed file that does not exist. + +```sh +vpn-config-edit set web.log_files /var/log/syslog,/var/log/auth.log +``` + ### The /users API `/users` manages users with the same token as `/config`, with the module `vpn-router-user` uses diff --git a/debian-package/debian/changelog b/debian-package/debian/changelog index 1f48146..8c12f1e 100644 --- a/debian-package/debian/changelog +++ b/debian-package/debian/changelog @@ -1,3 +1,16 @@ +vpn-router (0.30.0-1) unstable; urgency=medium + + feature: the web service serves the last lines of the journal of the router's units and of the log files web.log_files lists, at /logs. + + * GET /logs lists the units whose journal and the files whose lines it + serves; GET /logs/journal?unit=&lines= and /logs/file?path=&lines= + answer the last lines, 200 by default and at most 1000, a file read + from its last MiB. Units and files outside the lists are refused. + * The new setting web.log_files lists the absolute paths of the log + files, at most 32. + + -- Slawomir Koszewski Mon, 26 Oct 2026 19:00:00 +0100 + vpn-router (0.29.0-1) unstable; urgency=medium feature: vpn-config-edit set and remove take --restart, which applies the change with vpn-router-setup and waits for it. diff --git a/debian-package/src/usr/lib/vpn-router/vpnrouter.py b/debian-package/src/usr/lib/vpn-router/vpnrouter.py index c607a9b..d6e3947 100644 --- a/debian-package/src/usr/lib/vpn-router/vpnrouter.py +++ b/debian-package/src/usr/lib/vpn-router/vpnrouter.py @@ -58,7 +58,8 @@ SCHEMA = { 'openvpn': {'enabled': False, 'address_pool': '', 'port': 1194, 'protocol': 'udp'}, 'docker': {'mode': 'auto', 'exposed_ports': []}, 'web': {'enabled': True, 'listen': DEFAULT_LISTEN, 'port': 7081, 'database': '', 'trusted_proxies': [], - 'status_content': 'basic', 'shell': False, 'shell_idle_minutes': 15, 'shell_auth_timeout': 15}, + 'status_content': 'basic', 'shell': False, 'shell_idle_minutes': 15, 'shell_auth_timeout': 15, + 'log_files': []}, 'ssh': {'ldap_login': False}, 'https': {'enabled': True, 'email': '', 'web_port': 7444, 'manager_port': 7443, 'caddy_source': 'ubuntu'}, 'oidc': {'enabled': False}, @@ -126,6 +127,9 @@ ACME_HTTP_PORT = 80 #: The default port of HTTPS, where Caddy publishes the CA files when pki.publish has https-default. HTTPS_DEFAULT_PORT = 443 +#: The most files web.log_files lists. +LOG_FILES_MAX = 32 + #: Where vpn-router-setup installs Caddy from, https.caddy_source: the Ubuntu archive or Caddy's own repository. CADDY_SOURCES = ('ubuntu', 'upstream') @@ -518,6 +522,13 @@ def validate(cfg): for item in _get_list(cfg, 'web', 'trusted_proxies'): _check_address(item, 'web.trusted_proxies') + log_files = _get_list(cfg, 'web', 'log_files') + if len(log_files) > LOG_FILES_MAX: + raise ConfigError(f'web.log_files: at most {LOG_FILES_MAX} files') + for item in log_files: + if not item.startswith('/') or '/../' in f'{item}/': + raise ConfigError(f'web.log_files: {item!r} is not an absolute path') + caddy_source = _get(cfg, 'https', 'caddy_source') if caddy_source not in CADDY_SOURCES: raise ConfigError(f'https.caddy_source: {caddy_source!r} is not one of {", ".join(CADDY_SOURCES)}') diff --git a/debian-package/src/usr/lib/vpn-router/web b/debian-package/src/usr/lib/vpn-router/web index bc576b5..f97449f 100644 --- a/debian-package/src/usr/lib/vpn-router/web +++ b/debian-package/src/usr/lib/vpn-router/web @@ -47,6 +47,8 @@ Pages: /metrics/history the metrics the metrics service keeps, for a view: hour, day, month or year (see metrics_history()) + /logs the last lines of the journal of the router's units and of the files web.log_files lists + (see logs()) /postgresql the databases of each PostgreSQL cluster on the router with their sizes, as JSON (see postgresql_databases()); /config has the clusters' settings @@ -55,7 +57,7 @@ Pages: loopback address, where Caddy passes the requests of the HTTPS site; POST /oidc/login counts towards the limit of /auth calls -/status/full, /config, /metrics/history, /postgresql, /users, /service-accounts, /groups, /site-to-site, /backup, /remote-access, /ca, /shell and /account need an HS256 JWT in an Authorization: Bearer header signed +/status/full, /config, /metrics/history, /postgresql, /users, /service-accounts, /groups, /site-to-site, /backup, /remote-access, /ca, /logs, /shell and /account need an HS256 JWT in an Authorization: Bearer header signed with vpnrouter_auth.WEB_KEY (see vpnrouter_auth.verify). A token of /auth names a user in sub; /account serves that user, and the other paths need a member of vpnrouter_users.ADMIN_GROUP. A token of vpn-router-token has the admin claim and names no user; it may call every path but @@ -157,6 +159,16 @@ REFRESH_SECONDS = 60 POSTGRESQL_SETTINGS = ('server_version', 'listen_addresses', 'port', 'max_connections', 'shared_buffers', 'effective_cache_size', 'work_mem', 'password_encryption', 'ssl', 'timezone') +#: The units whose journal /logs/journal shows, besides the whole journal: the package's own and the services it runs. +JOURNAL_UNITS = ('vpn-router-setup', 'vpn-router-web', 'vpn-router-metrics', 'vpn-router-pki', 'vpn-router-backup', + 'vpn-router-caddy', 'vpn-router-manager', 'caddy', 'slapd', 'nslcd', 'postgresql', 'strongswan', + 'wg-quick@wg0', 'openvpn-server@vpn-router', 'ssh') + +#: The most lines /logs answers, the number it answers without lines, and the most bytes it reads from the end of a file. +LOG_LINES_MAX = 1000 +LOG_LINES_DEFAULT = 200 +LOG_BYTES_MAX = 1024 * 1024 + #: The largest request body accepted, in bytes; a CSR or a certificate chain in PEM is a few kilobytes. BODY_LIMIT = 65536 @@ -1059,6 +1071,63 @@ def account(name, method, path, body): return 200, document +def tail(path, count): + """The last count lines of a file, read from at most LOG_BYTES_MAX bytes at its end, invalid UTF-8 replaced.""" + with open(path, 'rb') as handle: + size = handle.seek(0, os.SEEK_END) + start = max(0, size - LOG_BYTES_MAX) + handle.seek(start) + lines = handle.read().decode('utf-8', 'replace').splitlines() + # A read that starts inside the file starts inside a line, which is left out. + return (lines[1:] if start and lines else lines)[-count:] + + +def logs(method, path, query): + """Answer a /logs request: the HTTP status and the JSON document. + + GET /logs {"units": JOURNAL_UNITS, "files": web.log_files, "max_lines": LOG_LINES_MAX} + GET /logs/journal {"lines": [...]}: the last lines of the journal, of the unit unit= names, one of + JOURNAL_UNITS, or of the whole journal without it + GET /logs/file {"lines": [...]}: the last lines of the file path= names, one of web.log_files + + lines= is the number of lines, 1 to LOG_LINES_MAX, LOG_LINES_DEFAULT without it. + """ + if method not in ('GET', 'HEAD'): + return 405, {'error': f'{method} is not allowed on /logs'} + try: + files = [item.strip() for item in vpnrouter.load_config()['web']['log_files'] if item.strip()] + except SystemExit as exc: + return 500, {'error': str(exc.code)} + if path == '/logs': + return 200, {'units': list(JOURNAL_UNITS), 'files': files, 'max_lines': LOG_LINES_MAX} + text = query.get('lines', [str(LOG_LINES_DEFAULT)])[0] + if not text.isdigit() or not 1 <= int(text) <= LOG_LINES_MAX: + return 400, {'error': f'lines must be a whole number from 1 to {LOG_LINES_MAX}'} + count = int(text) + if path == '/logs/journal': + unit = query.get('unit', [''])[0] + if unit and unit not in JOURNAL_UNITS: + return 404, {'error': f'{unit} is not one of the units whose journal /logs shows'} + try: + result = subprocess.run(['journalctl', '--no-pager', '--quiet', '--output=short-iso', f'--lines={count}', + *([f'--unit={unit}.service'] if unit else [])], capture_output=True, text=True, + errors='replace', timeout=30) + except (OSError, subprocess.TimeoutExpired) as exc: + return 500, {'error': f'journalctl cannot be run: {exc}'} + if result.returncode != 0: + return 500, {'error': result.stderr.strip() or f'journalctl ended with status {result.returncode}'} + return 200, {'lines': result.stdout.splitlines()[-count:]} + if path == '/logs/file': + name = query.get('path', [''])[0] + if name not in files: + return 404, {'error': f'{name!r} is not one of web.log_files'} + try: + return 200, {'lines': tail(name, count)} + except OSError as exc: + return (404 if isinstance(exc, FileNotFoundError) else 500), {'error': f'{name} cannot be read: {exc.strerror}'} + return 404, {'error': f'no route {method} {path}'} + + def metrics_history(view): """Answer GET /metrics/history?range= with vpnrouter_metrics.history: the HTTP status and the JSON document. @@ -1477,7 +1546,7 @@ class Handler(BaseHTTPRequestHandler): if self.command not in ('GET', 'HEAD'): self.close_connection = True directory = next((name for name in ('users', 'service-accounts', 'groups', 'account', 'site-to-site', 'backup', - 'ca', 'shell', 'remote-access') + 'ca', 'shell', 'remote-access', 'logs') if path == f'/{name}' or path.startswith(f'/{name}/')), None) protected = path in ('/status/full', '/config', '/metrics/history', '/postgresql') or directory is not None signing = path in ('/auth', '/auth/passkey', '/auth/passkey/complete', '/auth/refresh', '/auth/logout') @@ -1540,6 +1609,9 @@ class Handler(BaseHTTPRequestHandler): elif directory == 'ca': code, document = ca(self.command, path, payload) body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json' + elif directory == 'logs': + code, document = logs(self.command, path, urllib.parse.parse_qs(urllib.parse.urlsplit(self.path).query)) + body, kind = json.dumps(document).encode() + b'\n', 'application/json' elif directory == 'service-accounts': code, document = service_accounts(self.command, path, payload) body, kind = json.dumps(document, indent=2).encode() + b'\n', 'application/json' diff --git a/debian-package/src/usr/share/doc/vpn-router/vpn-router.json.example b/debian-package/src/usr/share/doc/vpn-router/vpn-router.json.example index c363063..159f62a 100644 --- a/debian-package/src/usr/share/doc/vpn-router/vpn-router.json.example +++ b/debian-package/src/usr/share/doc/vpn-router/vpn-router.json.example @@ -64,7 +64,8 @@ "status_content": "basic", "shell": false, "shell_idle_minutes": 15, - "shell_auth_timeout": 15 + "shell_auth_timeout": 15, + "log_files": ["/var/log/syslog", "/var/log/auth.log"] }, "ssh": { "ldap_login": false