feature: the router keeps its metrics in PostgreSQL, serves them for an OpenTelemetry Collector and pushes them over OTLP.

ADR.md

@@ -467,8 +467,8 @@ e-mail address can be added later without changing their classes; VPN users get
 used.
 
 Consequences: The schema check stays strict. New attributes extend `vpnRouterAccount`, and their
-OIDs continue the arcs. Nothing reads `vpnRouterScope` yet. The OID must never be reused for
-anything else.
+OIDs continue the arcs. The scope `service` marks a service account (ADR-23). The OID must never
+be reused for anything else.
 
 ## ADR-20: Users manage their own account through /account, proving who they are again
 
@@ -523,3 +523,57 @@ settings set it, so `internal` keeps passwords off the internet. Caddy's admin e
 on `localhost:2019`, which `caddy reload` uses. Port 80 and the HTTPS ports are not available
 to Docker containers while HTTPS is on. The Manager's HTTPS port is a setting of the router, not
 of the manager.
+
+## ADR-22: Metrics are kept in PostgreSQL and served in the Prometheus format, pushed with OTLP
+
+Status: Accepted
+
+Context: The router should keep its CPU, memory, disk, network, load and VPN client metrics for
+hour, day, month and year views, as Proxmox VE does, and expose them to OpenTelemetry. Ubuntu
+noble has no OpenTelemetry Python SDK and no OpenTelemetry Collector package. OTLP itself is
+push-only; a collector reads a scraped endpoint in the Prometheus format with its `prometheus`
+receiver. The router already runs PostgreSQL for the refresh tokens (ADR-15).
+
+Decision: A service of its own, `vpn-router-metrics`, takes a sample every minute with `psutil`
+(`python3-psutil`) and the VPN readers it shares with the web service (`vpnrouter_status`). It
+keeps the samples in the router's database, in three detail tables (`metrics_system`,
+`metrics_network`, `metrics_vpn`). Each detail table has hourly, daily, monthly and yearly summary
+tables, with the average and the maximum of each value per period, written when the clock passes a
+period from the level below with `INSERT ... ON CONFLICT DO NOTHING`, so a stopped service catches
+up. Garbage collection runs on start and once a day, not with every write, and deletes the rows
+older than the retention of their level, `metrics.keep_*`, PostgreSQL intervals. The service serves
+the last sample as `/metrics` in the Prometheus text format with `prometheus_client`
+(`python3-prometheus-client`) and pushes it as OTLP/HTTP JSON, built with the standard library, to
+`metrics.otlp_endpoint`. The names follow the OpenTelemetry semantic conventions. The web service
+answers the views from the tables at `/metrics/history`, protected like `/config`, for the Router
+Manager.
+
+Consequences: The metrics live in the database of the refresh tokens and are dropped with it on
+purge. A view shows only what its level keeps, so a shorter retention empties the older part of a
+view. A month view needs the hourly rows and a year view the monthly rows. `/metrics` has only the
+last sample; history is not scraped.
+
+## ADR-23: /metrics takes HTTP Basic authentication of a service account that setup owns
+
+Status: Accepted
+
+Context: An OpenTelemetry Collector's `prometheus` receiver authenticates with HTTP Basic, not with
+the router's tokens, which last minutes (ADR-3). The scraper should not be able to sign in to the
+API or the Manager, and its password should be checked and locked out like any other. The account
+belongs to the package's metrics, not to the people using the router.
+
+Decision: A service account is a Manager-only account (ADR-18) with `vpnRouterScope` `service`.
+`vpn-router-setup` owns it: while metrics are enabled it adds the account `metrics.service_account`
+names, `metrics` by default, with a random password it writes to `/etc/vpn-router/metrics.secret`
+(root only), and sets a new random password, ending a lockout, when the file is missing or no
+longer holds the password. `vpn-router-user` and `/users` cannot add one. `/auth` refuses it
+with the message of a wrong password. `/metrics` accepts HTTP Basic only for that account, and
+only when it is a service account; the password is checked with a bind as the account, so the
+password policy's lockout applies (ADR-14). TLS is Caddy's, on `metrics.https_port` (ADR-21);
+plain HTTP is served on the interfaces `metrics.listen` names, `internal` by default.
+
+Consequences: Every scrape binds to slapd once. The collector's operator copies the password from
+the router; deleting the file and running setup changes it. A user who already has the name is not
+taken over; setup fails until `metrics.service_account` names another. A disabled account stays
+disabled. A scraper on the external interface over plain HTTP sends the password in the clear, so
+`metrics.listen` stays `internal` unless HTTPS is used.