fix: update remote LXD scripts to use OVN backed NAT-ed networks.

This commit is contained in:
slawek committed 2026-11-22 19:27:53 +01:00
1 parent da889b4865
commit 5fff009320
9 files changed
+500 -19

No files matched your search

+24
View File
@@ -1570,3 +1570,27 @@ site-to-site or IKEv2 is ready.
Consequences: A core-only server needs one interface and gets its firewall at its first setup
run. The readiness checks (`routing_ready`, `s2s_ready`, `p2s_ready`, `wg_ready`,
`openvpn_ready`) carry the switches, so every consumer of them follows the components.
## ADR-58: Manual tests get numbered dev builds from a signed apt repository in a test project
Status: Accepted
Context: The server's package reaches production servers from an apt repository, and the Manager
checks for and installs its updates through apt (ADR-24, ADR-53). A package copied into a test
server and installed from a file tests none of that, and a rebuild of an unreleased changelog
entry keeps the same version, so apt sees no update.
Decision: `tests/remote/lxd-project.py` makes an LXD project for manual tests with networks of its
own (`features.networks`), a WAN and two LANs, and a container `repo` that serves a signed apt
repository, `resolute main`, built with `apt-ftparchive` and signed by its own key. The networks
are OVN networks on an uplink bridge of the remote, as OVN is the only network type LXD creates
inside a project; they belong to the project and go with it. `tests/remote/dev-build.py` builds the newest
changelog entry as `<upstream>~dev<N>-<revision>`, `N` counted in `tmp/test-number`, by passing
`DEB_VERSION` to the build, which rewrites the version in the image only, and publishes the build
in that repository. `tests/remote/vm.py` gives a VM of the project that repository as its apt
source. The automated tests (`run.py`) keep their own project for one run and install from a
file.
Consequences: Every dev build is an apt update of the one before, and the release of the entry,
which sorts above all of its `~dev` builds, is an update of them all. The changelog is not
changed by a dev build. Dev builds go to `out/dev/`, so `out/` holds only release builds.
+3
View File
@@ -24,6 +24,9 @@ WORKDIR /build
COPY debian-package/ src/
COPY docs/ src/docs/
COPY --from=web /web/dist src/web/dist/
# DEB_VERSION, when set, replaces the version of the newest changelog entry in the image only.
ARG DEB_VERSION
RUN if [ -n "$DEB_VERSION" ]; then sed -i "1s/([^)]*)/($DEB_VERSION)/" src/debian/changelog; fi
RUN cd src && dpkg-buildpackage -us -uc -b
RUN lintian --no-tag-display-limit /build/*.changes || true
+4
View File
@@ -49,6 +49,10 @@ linux-manager (0.41.0-1) unstable; urgency=medium
LXD remote, and tests/remote/vm.py starts and removes one server
VM there to work with, reached through ports 7080, 7081 and 2222 (SSH)
of the LXD host, by default at the address of the remote's URL.
* tests/remote/lxd-project.py creates and removes a project for manual
tests with its own OVN networks, a WAN and two LANs, and a signed apt
repository; tests/remote/dev-build.py publishes numbered dev builds, version
<upstream>~devN-<revision>, to it, and vm.py installs from it there.
-- Slawomir Koszewski <slawek@koszewscy.waw.pl> Sat, 21 Nov 2026 23:39:03 +0100
+80
View File
@@ -89,6 +89,86 @@ tests/remote/vm.py --remove
| `--config` | | `core.json` | The configuration from `configs/` |
| `--listen` | `LXD_LISTEN` | the remote URL's host | An address of the LXD host the proxies listen on |
In a test project made by `lxd-project.py`, `vm.py` installs the package from the project's
repository instead of `out/`, so later dev builds reach the VM as updates. LXD refuses NAT proxy
devices in a project with networks of its own, so there the ports lead through a network forward
on the project's `wan`, whose listen address LXD allocates from the uplink's `--routes`;
`--listen` does not apply, and `--remove` deletes the forward with the VM.
## A test project with dev builds
`lxd-project.py create` makes a project for manual tests that stays until
`lxd-project.py remove`; `run.py` keeps making and removing its own projects. The project has:
- networks of its own (`features.networks`), OVN networks on the physical uplink `--uplink`:
`wan`, with NAT and DHCP on a subnet LXD picks, and `lana` and `lanb`, on subnets LXD picks
without DHCP; OVN gives each network's router an address, which LAN hosts don't use as their
gateway;
- a default profile with a root disk on the pool and `eth0` on `wan`;
- the container `repo`, whose nginx serves a signed apt repository of the dev builds,
`resolute main` for amd64 and arm64, at its pinned address.
What the other scripts need, the repository's URL among it, is written to
`tmp/lxd-project-<project>.json`, and the repository's key to `tmp/lxd-project-<project>.gpg`.
`remove` deletes the project's instances, its networks, the project and those files.
The remote needs OVN, set up as LXD's standalone OVN guide describes, and a host bridge on the
LAN, whose host address stays on the bridge. `uplink setup` creates on it, once per remote, the
physical network `--uplink` that the OVN networks use as uplink:
- `--gateway`: the LAN's gateway, as address/prefix;
- `--dns`: the LAN's name servers, comma-separated;
- `--ovn-ranges`: free LAN addresses outside the LAN's DHCP pool, as `FIRST-LAST`, one for the
router of each OVN network;
- `--routes`: a free LAN subnet, in CIDR, whose addresses network forwards listen on.
OVN answers ARP for its routers' and forwards' addresses on the LAN (`ovn.ingress_mode`
`l2proxy`, LXD's default), so the LAN needs no route. `uplink setup` leaves an existing network
of that name as it is; `uplink remove` deletes it.
```sh
tests/remote/lxd-project.py uplink setup
```
```sh
tests/remote/lxd-project.py uplink remove
```
```sh
tests/remote/lxd-project.py create
```
```sh
tests/remote/lxd-project.py remove
```
| Option | Variable | Default | Meaning |
|---|---|---|---|
| `--remote` | `LXD_REMOTE` | required | The `lxc` remote |
| `--pool` | `LXD_POOL` | required by `create` | The storage pool of the project's root disks |
| `--project` | `LXD_PROJECT` | `lm-dev` | The project |
| `--image` | `LXD_IMAGE` | `ubuntu:26.04` | The image of the repository container |
| `--uplink` | `LXD_UPLINK` | `uplink` | The physical uplink network of the OVN networks |
| `--parent` | `LXD_UPLINK_PARENT` | required by `uplink setup` | The host bridge on the LAN |
| `--gateway` | `LXD_UPLINK_GATEWAY` | required by `uplink setup` | The LAN's gateway, address/prefix |
| `--dns` | `LXD_UPLINK_DNS` | required by `uplink setup` | The LAN's name servers |
| `--ovn-ranges` | `LXD_UPLINK_OVN_RANGES` | required by `uplink setup` | LAN addresses for OVN routers |
| `--routes` | `LXD_UPLINK_ROUTES` | required by `uplink setup` | A LAN subnet for network forwards |
`dev-build.py` builds and publishes the next dev build. Its version is the newest changelog
entry's with `~dev<N>` before the Debian revision, `0.41.0~dev7-1` for `0.41.0-1`, so the release
of that entry is newer than its dev builds. `N` is kept in `tmp/test-number` and goes up with each
build that succeeds. `./build.sh` builds it, with `CONTAINER_RUNTIME` and `DOCKER_CONTEXT` from the
environment; the build is moved to `out/dev/`, copied into the repository and published there. A
VM of the project then gets it with Check update and Upgrade in the Manager, or `apt-get update`
and `apt-get upgrade`.
```sh
tests/remote/dev-build.py
```
`--project` (`LXD_PROJECT`, `lm-dev` by default) names the project.
## Scope of the tests
Not covered by `run.py`: IKEv2 client access, HTTPS with Let's Encrypt, the OpenID Provider and
+6
View File
@@ -0,0 +1,6 @@
#cloud-config
package_update: true
packages:
- nginx
- apt-utils
- gnupg
+72
View File
@@ -0,0 +1,72 @@
#!/usr/bin/env python3
"""Build the next numbered dev build and publish it to a test project's repository: dev-build.py.
The test number in tmp/test-number goes up by one for each build that succeeds. The build's version
is the newest changelog entry's with ~devN before its Debian revision, 0.41.0-1 becoming
0.41.0~dev7-1, so the release of that entry is newer than all its dev builds. ./build.sh builds it
with CONTAINER_RUNTIME and DOCKER_CONTEXT from the environment; the build is moved to out/dev/,
copied into the repository container of the project (--project or LXD_PROJECT, lm-dev by default)
made by lxd-project.py, and published there.
"""
import argparse
import json
import os
import pathlib
import re
import subprocess
import sys
HERE = pathlib.Path(__file__).resolve().parent
ROOT = HERE.parent.parent
NUMBER = ROOT / 'tmp' / 'test-number'
CHANGELOG = ROOT / 'debian-package' / 'debian' / 'changelog'
def main():
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
parser.add_argument('--project', default=os.environ.get('LXD_PROJECT', 'lm-dev'),
help='the test project made by lxd-project.py (LXD_PROJECT, default lm-dev)')
args = parser.parse_args()
state_file = ROOT / 'tmp' / f'lxd-project-{args.project}.json'
if not state_file.is_file():
sys.exit(f'no {state_file}; create the project with lxd-project.py create')
state = json.loads(state_file.read_text(encoding='utf-8'))
# The newest entry's first line is "package (version) distribution; urgency=...".
match = re.match(r'(\S+) \(([^)]+)\)', CHANGELOG.read_text(encoding='utf-8'))
if not match:
sys.exit(f'no version in the first line of {CHANGELOG}')
package, version = match.groups()
number = int(NUMBER.read_text(encoding='utf-8').strip()) + 1 if NUMBER.is_file() else 1
upstream, dash, revision = version.rpartition('-')
dev = f'{upstream}~dev{number}-{revision}' if dash else f'{version}~dev{number}'
print(f'building {package} {dev}')
if subprocess.run(['./build.sh', '--build-arg', f'DEB_VERSION={dev}'], cwd=ROOT).returncode != 0:
sys.exit(f'the build of {dev} failed; the test number stays {number - 1}')
NUMBER.write_text(f'{number}\n', encoding='utf-8')
# The dev build leaves out/ to out/dev/, so out/ keeps the release builds.
target = ROOT / 'out' / 'dev'
target.mkdir(exist_ok=True)
for artifact in (ROOT / 'out').glob(f'{package}_{dev}_*'):
artifact.replace(target / artifact.name)
deb = target / f'{package}_{dev}_all.deb'
if not deb.is_file():
sys.exit(f'the build made no {deb.name}')
where = ['--project', state['project']]
pushed = subprocess.run(['lxc', 'file', 'push', str(deb), f'{state["remote"]}:repo/var/www/html/pool/main/{deb.name}',
'--mode', '0644', *where], capture_output=True, text=True)
if pushed.returncode != 0:
sys.exit(f'could not copy {deb.name} into the repository: {pushed.stderr.strip()}')
published = subprocess.run(['lxc', 'exec', f'{state["remote"]}:repo', *where, '-T', '--',
'linux-manager-repo-publish'], capture_output=True, text=True)
if published.returncode != 0:
sys.exit(f'the repository was not published: {published.stderr.strip()}')
print(f'{package} {dev} is published at {state["repository"]} (resolute main)')
if __name__ == '__main__':
main()
+221
View File
@@ -0,0 +1,221 @@
#!/usr/bin/env python3
"""Create or remove an LXD project for manual tests: lxd-project.py create|remove|uplink setup|remove.
uplink setup creates the physical uplink network once per remote, uplink remove deletes it (see
uplink).
create makes, on the remote (--remote or LXD_REMOTE), the project (--project or LXD_PROJECT,
lm-dev by default) with its root disks on --pool (LXD_POOL) and networks of its own, OVN networks
on the uplink --uplink (LXD_UPLINK, uplink by default): wan, with NAT and DHCP on a subnet LXD
picks, and lana and lanb, on subnets LXD picks without DHCP. The project's default profile puts
eth0 on wan. The container repo serves a signed apt repository of the dev builds over HTTP; dev-build.py publishes into it and vm.py installs from it. What they need is
written to tmp/lxd-project-PROJECT.json, with the repository's key in tmp/lxd-project-PROJECT.gpg.
remove deletes the project's instances, its networks, the project and those files.
"""
import argparse
import json
import os
import pathlib
import re
import subprocess
import sys
import time
HERE = pathlib.Path(__file__).resolve().parent
ROOT = HERE.parent.parent
NETWORKS = ('wan', 'lana', 'lanb')
def lxc(*args, project, check=True, input=None):
"""Run lxc with the arguments in the project; raise CalledProcessError when it fails, unless check is false."""
return subprocess.run(['lxc', *args, '--project', project], input=input, capture_output=True, text=True,
check=check)
def create(args, state, key):
"""Create the project with its networks and default profile, and the repository container."""
subprocess.run(['lxc', 'project', 'create', f'{args.remote}:{args.project}', '-s', args.pool,
'-c', 'features.images=false', '-c', 'features.networks=true'],
capture_output=True, text=True, check=True)
print(f'project {args.project} created; creating its OVN networks on {args.uplink}')
lxc('network', 'create', f'{args.remote}:wan', '--type=ovn', f'network={args.uplink}', 'ipv4.address=auto',
'ipv4.nat=true', 'ipv4.dhcp=true', 'ipv6.address=none', project=args.project)
# OVN gives every network's router an address; without DHCP the LAN hosts keep their own gateways.
for name in ('lana', 'lanb'):
lxc('network', 'create', f'{args.remote}:{name}', '--type=ovn', f'network={args.uplink}',
'ipv4.address=auto', 'ipv4.nat=true', 'ipv4.dhcp=false', 'ipv6.address=none', project=args.project)
# The project's default profile has the root disk on the pool of project create's -s.
lxc('profile', 'device', 'add', f'{args.remote}:default', 'eth0', 'nic', 'network=wan', 'name=eth0',
project=args.project)
print(f'launching the repository container {args.remote}:repo (project {args.project})')
lxc('launch', args.image, f'{args.remote}:repo',
'-c', 'cloud-init.user-data=' + (HERE / 'cloud-init' / 'repo.yaml').read_text(encoding='utf-8'),
project=args.project)
def inside(*command):
return subprocess.run(['lxc', 'exec', f'{args.remote}:repo', '--project', args.project, '-T', '--', *command],
capture_output=True, text=True)
# Wait for cloud-init (status 2 is done with recoverable errors), then for the address of eth0.
for _ in range(120):
if inside('cloud-init', 'status', '--wait').returncode in (0, 2):
break
time.sleep(5)
else:
raise RuntimeError('the repository container did not boot')
for _ in range(24):
match = re.search(r' inet (\d+\.\d+\.\d+\.\d+)/', inside('ip', '-4', '-o', 'addr', 'show', 'dev', 'eth0').stdout)
if match:
break
time.sleep(5)
else:
raise RuntimeError('the repository container has no address on eth0')
address = match.group(1)
lxc('config', 'device', 'override', f'{args.remote}:repo', 'eth0', f'ipv4.address={address}',
project=args.project)
# The signing key, the publishing script and the empty indexes.
for step, command in (('key generation', ('gpg', '--batch', '--pinentry-mode', 'loopback', '--passphrase', '',
'--quick-generate-key', 'Linux Manager test repository', 'default',
'default', 'never')),
('key export', ('gpg', '--batch', '--yes', '--export', '-o', '/root/linux-manager-test.gpg'))):
result = inside(*command)
if result.returncode != 0:
raise RuntimeError(f'the {step} failed in repo: {result.stderr.strip()}')
lxc('file', 'push', str(HERE / 'repo' / 'publish.sh'), f'{args.remote}:repo/usr/local/sbin/linux-manager-repo-publish',
'--mode', '0755', project=args.project)
result = inside('linux-manager-repo-publish')
if result.returncode != 0:
raise RuntimeError(f'the repository was not published: {result.stderr.strip()}')
lxc('file', 'pull', f'{args.remote}:repo/root/linux-manager-test.gpg', str(key), project=args.project)
state.write_text(json.dumps({'remote': args.remote, 'project': args.project, 'networks': list(NETWORKS),
'repository': f'http://{address}/', 'key': str(key)}, indent=2) + '\n',
encoding='utf-8')
print(f'project {args.project} is ready on {args.remote}: networks {", ".join(NETWORKS)}, '
f'repository http://{address}/')
print(f'state: {state}')
def remove(args, state, key):
"""Delete the project's instances, the networks, the project and the local files; True when all went."""
ok = True
listing = lxc('list', f'{args.remote}:', '-c', 'n', '-f', 'csv', project=args.project, check=False)
for name in listing.stdout.split():
result = lxc('delete', '--force', f'{args.remote}:{name}', project=args.project, check=False)
if result.returncode != 0:
ok = False
print(f'could not delete {name}: {result.stderr.strip()}', file=sys.stderr)
# The default profile's eth0 keeps wan in use.
lxc('profile', 'device', 'remove', f'{args.remote}:default', 'eth0', project=args.project, check=False)
for name in NETWORKS:
if lxc('network', 'show', f'{args.remote}:{name}', project=args.project, check=False).returncode != 0:
continue
result = lxc('network', 'delete', f'{args.remote}:{name}', project=args.project, check=False)
if result.returncode != 0:
ok = False
print(f'could not delete network {name}: {result.stderr.strip()}', file=sys.stderr)
result = subprocess.run(['lxc', 'project', 'delete', f'{args.remote}:{args.project}'], capture_output=True,
text=True)
if result.returncode != 0:
ok = False
print(f'could not delete project {args.project}: {result.stderr.strip()}', file=sys.stderr)
if ok:
state.unlink(missing_ok=True)
key.unlink(missing_ok=True)
print(f'removed project {args.project} and its networks from {args.remote}')
return ok
def uplink(args):
"""Create the physical uplink network of the OVN networks on a host bridge of the LAN, or delete it.
setup creates --uplink in the default project with parent --parent, the LAN's gateway
--gateway (address/prefix), its name servers --dns, the LAN addresses OVN routers take
(--ovn-ranges, FIRST-LAST) and the LAN subnets for network forwards (--routes, CIDR); OVN
answers ARP for those on the LAN (ovn.ingress_mode l2proxy, the default). It leaves an
existing network of that name as it is. remove deletes the network.
"""
network = f'{args.remote}:{args.uplink}'
exists = lxc('network', 'show', network, project='default', check=False).returncode == 0
if args.operation == 'remove':
if exists:
lxc('network', 'delete', network, project='default')
print(f'{args.remote} has no network {args.uplink}')
return
if exists:
print(f'{args.remote} has the network {args.uplink} already')
return
missing = [option for option, value in (('--parent', args.parent), ('--gateway', args.gateway), ('--dns', args.dns),
('--ovn-ranges', args.ovn_ranges), ('--routes', args.routes)) if not value]
if missing:
sys.exit(f'uplink setup needs {", ".join(missing)}')
lxc('network', 'create', network, '--type=physical', f'parent={args.parent}', f'ipv4.gateway={args.gateway}',
f'dns.nameservers={args.dns}', f'ipv4.ovn.ranges={args.ovn_ranges}', f'ipv4.routes={args.routes}',
project='default')
print(f'{args.uplink} on {args.remote}: physical on {args.parent}, gateway {args.gateway}, OVN routers '
f'{args.ovn_ranges}, forwards {args.routes}')
def main():
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
parser.add_argument('action', choices=('create', 'remove', 'uplink'))
parser.add_argument('operation', nargs='?', choices=('setup', 'remove'), help='with uplink: setup or remove')
parser.add_argument('--remote', default=os.environ.get('LXD_REMOTE'), help='the lxc remote (LXD_REMOTE)')
parser.add_argument('--pool', default=os.environ.get('LXD_POOL'),
help='the storage pool of the project, for create (LXD_POOL)')
parser.add_argument('--project', default=os.environ.get('LXD_PROJECT', 'lm-dev'),
help='the project (LXD_PROJECT, default lm-dev)')
parser.add_argument('--image', default=os.environ.get('LXD_IMAGE', 'ubuntu:26.04'),
help='the image of the repository container (LXD_IMAGE, default ubuntu:26.04)')
parser.add_argument('--uplink', default=os.environ.get('LXD_UPLINK', 'uplink'),
help='the physical uplink network of the OVN networks (LXD_UPLINK, default uplink)')
parser.add_argument('--parent', default=os.environ.get('LXD_UPLINK_PARENT'),
help='uplink setup: the host bridge on the LAN (LXD_UPLINK_PARENT)')
parser.add_argument('--gateway', default=os.environ.get('LXD_UPLINK_GATEWAY'),
help='uplink setup: the LAN gateway as address/prefix (LXD_UPLINK_GATEWAY)')
parser.add_argument('--dns', default=os.environ.get('LXD_UPLINK_DNS'),
help='uplink setup: the LAN name servers, comma-separated (LXD_UPLINK_DNS)')
parser.add_argument('--ovn-ranges', default=os.environ.get('LXD_UPLINK_OVN_RANGES'),
help='uplink setup: LAN addresses for OVN routers, FIRST-LAST (LXD_UPLINK_OVN_RANGES)')
parser.add_argument('--routes', default=os.environ.get('LXD_UPLINK_ROUTES'),
help='uplink setup: LAN subnets for network forwards, CIDR (LXD_UPLINK_ROUTES)')
args = parser.parse_args()
if not args.remote:
parser.error('--remote or LXD_REMOTE is required')
if (args.action == 'uplink') != (args.operation is not None):
parser.error('setup or remove follows uplink, and only uplink')
if args.action == 'uplink':
try:
uplink(args)
except subprocess.CalledProcessError as error:
sys.exit(f'{" ".join(error.cmd[:4])} failed: {error.stderr.strip()}')
return
state = ROOT / 'tmp' / f'lxd-project-{args.project}.json'
key = ROOT / 'tmp' / f'lxd-project-{args.project}.gpg'
if args.action == 'remove':
sys.exit(0 if remove(args, state, key) else 1)
if not args.pool:
parser.error('--pool or LXD_POOL is required')
if subprocess.run(['lxc', 'project', 'show', f'{args.remote}:{args.project}'], capture_output=True).returncode == 0:
sys.exit(f'project {args.project} exists on {args.remote}; remove it first or choose another --project')
state.parent.mkdir(parents=True, exist_ok=True)
try:
create(args, state, key)
except subprocess.CalledProcessError as error:
sys.exit(f'{" ".join(error.cmd[:4])} failed: {error.stderr.strip()}\n'
f'remove what was made with: {sys.argv[0]} remove --remote {args.remote} --project {args.project}')
except RuntimeError as error:
sys.exit(f'{error}\nremove what was made with: {sys.argv[0]} remove --remote {args.remote} '
f'--project {args.project}')
if __name__ == '__main__':
main()
+26
View File
@@ -0,0 +1,26 @@
#!/bin/sh
# Regenerates the indexes of the test repository that nginx serves from /var/www/html: the
# packages of pool/main in dists/resolute/main for amd64 and arm64, and the Release file signed
# with root's key as InRelease.
set -e
cd /var/www/html
mkdir -p pool/main
for ARCH in amd64 arm64; do
mkdir -p "dists/resolute/main/binary-$ARCH"
apt-ftparchive packages pool/main > "dists/resolute/main/binary-$ARCH/Packages"
gzip -9 -k -f "dists/resolute/main/binary-$ARCH/Packages"
done
# The old Release and InRelease would be listed in the new Release.
rm -f dists/resolute/Release dists/resolute/InRelease
apt-ftparchive \
-o APT::FTPArchive::Release::Origin=linux-manager-test \
-o APT::FTPArchive::Release::Label=linux-manager-test \
-o APT::FTPArchive::Release::Suite=resolute \
-o APT::FTPArchive::Release::Codename=resolute \
-o "APT::FTPArchive::Release::Architectures=amd64 arm64" \
-o APT::FTPArchive::Release::Components=main \
release dists/resolute > /tmp/linux-manager-test-Release
mv /tmp/linux-manager-test-Release dists/resolute/Release
gpg --batch --yes --clear-sign -o dists/resolute/InRelease dists/resolute/Release
+64 -19
View File
@@ -3,12 +3,16 @@
The VM is launched on the remote (--remote or LXD_REMOTE) in --project with that project's
default profile. It gets the newest out/linux-manager_*_all.deb with the administrator admin and
a generated password, and a configuration from configs/ (core.json by default), whose @ETH0@ is
the interface of the VM's default route, @CIDR0@ its network and @PSK@ a generated key. The
password and the key are written to tmp/vm-NAME.env. The VM's leased address is pinned on its
NIC, and NAT proxy devices forward ports 7080 and 7081 of the host address --listen (or LXD_LISTEN,
by default the IPv4 address of the host of the remote's URL) to the same ports of the VM, and port
2222 to its SSH port. --remove deletes the VM and that file.
a generated password; in a test project made by lxd-project.py, which tmp/lxd-project-PROJECT.json
describes, it gets the package instead from that project's repository, as its apt source, so later
dev builds reach it as updates. It also gets a configuration from configs/ (core.json by
default), whose @ETH0@ is the interface of the VM's default route, @CIDR0@ its network and @PSK@
a generated key. The password and the key are written to tmp/vm-NAME.env. The VM's leased
address is pinned on its NIC, and ports 7080 and 7081 of a listen address lead to the same ports
of the VM, and port 2222 to its SSH port. In a test project the listen address is a network
forward's on the project's WAN, allocated from its uplink's routes; elsewhere NAT proxy devices
listen on the host address --listen (or LXD_LISTEN, by default the IPv4 address of the host of the
remote's URL). --remove deletes the VM, its forward and that file.
"""
import argparse
@@ -51,13 +55,17 @@ def main():
help='the image of the VM (LXD_IMAGE, default ubuntu:26.04)')
parser.add_argument('--config', default='core.json', help='the configuration from configs/ (default core.json)')
parser.add_argument('--listen', default=os.environ.get('LXD_LISTEN'),
help='the LXD host address the proxies listen on (LXD_LISTEN, default the IPv4 address '
'of the remote URL\'s host)')
help='outside a test project, the LXD host address the proxies listen on (LXD_LISTEN, '
'default the IPv4 address of the remote URL\'s host)')
parser.add_argument('--remove', action='store_true', help='delete the VM and its env file')
args = parser.parse_args()
if not args.remote:
parser.error('--remote or LXD_REMOTE is required')
if not args.remove and not args.listen:
project_file = ROOT / 'tmp' / f'lxd-project-{args.project}.json'
project = json.loads(project_file.read_text(encoding='utf-8')) if project_file.is_file() else None
if project and project['remote'] != args.remote:
sys.exit(f'{project_file} is of remote {project["remote"]}, not {args.remote}')
if not project and not args.remove and not args.listen:
# The default listen address is the IPv4 address of the remote's host in its lxc URL.
remotes = json.loads(subprocess.run(['lxc', 'remote', 'list', '--format', 'json'], capture_output=True,
text=True, check=True).stdout)
@@ -73,9 +81,20 @@ def main():
capture_output=True).returncode == 0
env = ROOT / 'tmp' / f'vm-{args.name}.env'
# In a test project the VM is reached through a network forward on the project's WAN, tagged with user.vm.
wan = f'{args.remote}:{project["networks"][0]}' if project else None
def forward_of_vm():
listing = subprocess.run(['lxc', 'network', 'forward', 'list', wan, '-f', 'json', '--project', args.project],
capture_output=True, text=True, check=True).stdout
return next((forward['listen_address'] for forward in json.loads(listing) or []
if forward.get('config', {}).get('user.vm') == args.name), None)
if args.remove:
if not exists:
sys.exit(f'no VM {where}')
if project and (listen := forward_of_vm()):
subprocess.run(['lxc', 'network', 'forward', 'delete', wan, listen, '--project', args.project], check=True)
subprocess.run(['lxc', 'delete', '--force', f'{args.remote}:{args.name}', '--project', args.project],
check=True)
env.unlink(missing_ok=True)
@@ -88,8 +107,9 @@ def main():
if not config.is_file():
parser.error(f'no configs/{args.config}')
debs = sorted((ROOT / 'out').glob('linux-manager_*_all.deb'))
if not debs:
if not project and not debs:
sys.exit('no out/linux-manager_*_all.deb; run ./build.sh first')
source = f'the repository {project["repository"]}' if project else debs[-1].name
password, psk = secrets.token_hex(16), secrets.token_hex(24)
env.parent.mkdir(parents=True, exist_ok=True)
@@ -97,7 +117,8 @@ def main():
env.chmod(0o600)
env.write_text(f'ADMIN_USERNAME=admin\nADMIN_PASSWORD={password}\nPSK={psk}\n', encoding='utf-8')
print(f'launching {where} from {args.image} with {debs[-1].name}; proxies listen on {args.listen}')
reached = f'a forward on {project["networks"][0]}' if project else f'proxies on {args.listen}'
print(f'launching {where} from {args.image} with {source}; reached through {reached}')
subprocess.run(['lxc', 'launch', args.image, f'{args.remote}:{args.name}', '--vm', '-c', 'limits.cpu=2',
'-c', 'limits.memory=4GiB',
'-c', 'cloud-init.user-data=' + (HERE / 'cloud-init' / 'server.yaml').read_text(encoding='utf-8'),
@@ -121,7 +142,7 @@ def main():
sys.exit(f'{args.name} has no default route or address; remove it with --remove')
nic, interface = route.group(1), ipaddress.IPv4Interface(address.group(1))
# Pin the leased address on the profile's NIC and proxy the host's --listen ports to the VM.
# Pin the leased address on the profile's NIC, then forward or proxy the ports to the VM.
mac = inside(args, 'cat', f'/sys/class/net/{nic}/address').stdout.strip()
expanded = subprocess.run(['lxc', 'config', 'show', '--expanded', f'{args.remote}:{args.name}',
'--project', args.project], capture_output=True, text=True, check=True).stdout
@@ -131,19 +152,43 @@ def main():
sys.exit(f'no LXD device of {args.name} has the MAC address {mac} of {nic}')
subprocess.run(['lxc', 'config', 'device', 'override', f'{args.remote}:{args.name}', device,
f'ipv4.address={interface.ip}', '--project', args.project], check=True)
for host_port, vm_port in PROXIES:
subprocess.run(['lxc', 'config', 'device', 'add', f'{args.remote}:{args.name}', f'proxy-{vm_port}', 'proxy',
f'listen=tcp:{args.listen}:{host_port}', f'connect=tcp:{interface.ip}:{vm_port}', 'nat=true',
if project:
# An OVN network refuses NAT proxies; the forward's listen address is allocated from the uplink's routes.
subprocess.run(['lxc', 'network', 'forward', 'create', wan, '--allocate=ipv4', f'user.vm={args.name}',
'--project', args.project], check=True)
args.listen = forward_of_vm()
if not args.listen:
sys.exit(f'no forward of {args.name} on {wan}; remove the VM with --remove')
for host_port, vm_port in PROXIES:
subprocess.run(['lxc', 'network', 'forward', 'port', 'add', wan, args.listen, 'tcp', str(host_port),
str(interface.ip), str(vm_port), '--project', args.project], check=True)
else:
for host_port, vm_port in PROXIES:
subprocess.run(['lxc', 'config', 'device', 'add', f'{args.remote}:{args.name}', f'proxy-{vm_port}',
'proxy', f'listen=tcp:{args.listen}:{host_port}', f'connect=tcp:{interface.ip}:{vm_port}',
'nat=true', '--project', args.project], check=True)
# Install the package with the test administrator.
if not push(args, debs[-1], '/tmp/linux-manager.deb', '0644'):
sys.exit(f'could not copy {debs[-1].name} into {args.name}')
# Install the package with the test administrator, from the test project's repository when there is one.
if project:
sources = (f'Types: deb\nURIs: {project["repository"]}\nSuites: resolute\nComponents: main\n'
f'Signed-By: /etc/apt/keyrings/linux-manager-test.gpg\n')
if (inside(args, 'mkdir', '-p', '/etc/apt/keyrings').returncode != 0 or
not push(args, project['key'], '/etc/apt/keyrings/linux-manager-test.gpg', '0644') or
inside(args, 'tee', '/etc/apt/sources.list.d/linux-manager-test.sources', input=sources).returncode != 0):
sys.exit(f'could not add the repository of project {args.project} to {args.name}')
updated = inside(args, 'apt-get', 'update')
if updated.returncode != 0:
sys.exit(f'apt-get update failed in {args.name}: {updated.stderr.strip()}')
package = 'linux-manager'
else:
if not push(args, debs[-1], '/tmp/linux-manager.deb', '0644'):
sys.exit(f'could not copy {debs[-1].name} into {args.name}')
package = '/tmp/linux-manager.deb'
preseed = (f'linux-manager linux-manager/admin_username string admin\n'
f'linux-manager linux-manager/admin_password password {password}\n')
if inside(args, 'debconf-set-selections', input=preseed).returncode != 0:
sys.exit('debconf-set-selections failed')
result = inside(args, 'env', 'DEBIAN_FRONTEND=noninteractive', 'apt-get', 'install', '-y', '/tmp/linux-manager.deb')
result = inside(args, 'env', 'DEBIAN_FRONTEND=noninteractive', 'apt-get', 'install', '-y', package)
log = ROOT / 'tmp' / f'vm-{args.name}-install.log'
log.write_text(result.stdout + result.stderr, encoding='utf-8')
if result.returncode != 0: