fix: a module of the tfvars store says whether it discovers its configurator, as configuratorDiscovery.

ADR.md

@@ -157,3 +157,39 @@ store is first used.
 Consequences: The schema of an instance's files is the schema of its newest revision; files
 imported without a schema, or written before revisions, have none until a configurator writes
 them. Revisions grow with every write; the README describes the SQL that removes them.
+
+## ADR-8: A Vite plugin sets the color scheme before the first paint
+
+Status: Accepted
+
+Context: `appTheme` puts the class of the color scheme on `<html>` from React, after the scripts
+of the application have loaded. Until then a page in the dark scheme is white, which shows on
+every load, and on every opening of a page that embeds an application in an iframe.
+
+Decision: The `colorScheme()` plugin adds an inline script and style to `index.html`, so the class
+is set and the dark background painted before the application loads. The script reads MUI's
+storage key `mui-mode` and follows `prefers-color-scheme` for the system mode, as MUI does, and
+the style uses MUI's default dark background, which `appTheme` keeps. The plugin is part of the
+Vite helpers, which every application already uses, so the applications need no code of their own
+in `index.html`.
+
+Consequences: The storage key and the dark background are repeated in the plugin; a theme that
+changes either has to change the plugin too. An application of the toolkit adds `colorScheme()`
+to the plugins of its `vite.config.ts`.
+
+## ADR-9: The checks run in a container built from the files git knows, with a scratch PostgreSQL
+
+Status: Accepted
+
+Context: The checks are faster on a remote Docker engine than on the developer's machine. The
+store's tests need a PostgreSQL server whose role creates databases, and a remote engine sees
+neither the checkout nor its `node_modules`.
+
+Decision: `test-container.mjs` sends a tar archive of the files git tracks and the untracked files
+it does not ignore as the build context of `test.Dockerfile`, whose build runs the checks that need
+no server, and runs the tests in a container next to a PostgreSQL container on a network of their
+own, both removed afterwards. The image's `package` stage holds the packed toolkit, which the
+applications install to check against the checkout.
+
+Consequences: The tests need no PostgreSQL server of the developer's. A file git ignores is not
+part of the checks. The checks run on Linux, whatever the developer's system.

CHANGELOG.md

@@ -1,5 +1,27 @@
 # Changelog
 
+## 0.7.2
+
+fix: a module of the tfvars store says whether it discovers its configurator, as
+configuratorDiscovery.
+
+- `TfModule` has `configuratorDiscovery`, and `tf_modules` the column `configurator_discovery`:
+  whether a module without `configurator_url` finds its configurator by the `config.json` of its
+  folder or one above it. A module registered before keeps it on; `modules.put` sets it.
+- `tf_module_put` takes the new argument; its earlier signature is dropped.
+
+## 0.7.1
+
+fix: a page in the dark color scheme, also one in an iframe, is no longer white while it loads,
+and test-container.mjs runs the checks with Docker.
+
+- The `colorScheme()` plugin of `/vite` sets the color scheme of `appTheme` and the dark background
+  on the page before its first paint; an application adds it to the plugins of its
+  `vite.config.ts`.
+- `./test-container.mjs` runs the typecheck, the lint, the package build and the tests in a
+  container, on the engine of the docker CLI's context, with a scratch PostgreSQL;
+  `--package <folder>` writes the packed toolkit to the folder.
+
 ## 0.7.0
 
 feature: every write of an instance's variable files is kept as a revision with the schema version

README.md

@@ -19,15 +19,15 @@ published version and imports `@koszewscy/sk-react-toolkit/server`, `/shared`, `
 { "dependencies": { "@koszewscy/sk-react-toolkit": "^0.2.0" } }
 ```
 
-The web application adds the toolkit's resolve options and font plugin to its `vite.config.ts`, so
-React, MUI and Emotion come from its own `node_modules`:
+The web application adds the toolkit's resolve options, so React, MUI and Emotion come from its
+own `node_modules`, and its font and color scheme plugins to its `vite.config.ts`:
 
 ```ts
-import { font, fontChoice, TOOLKIT_RESOLVE } from '@koszewscy/sk-react-toolkit/vite'
+import { colorScheme, font, fontChoice, TOOLKIT_RESOLVE } from '@koszewscy/sk-react-toolkit/vite'
 import { defineConfig, loadEnv } from 'vite'
 
 export default defineConfig(({ mode }) => ({
-  plugins: [react(), font(fontChoice(loadEnv(mode, import.meta.dirname)))],
+  plugins: [react(), font(fontChoice(loadEnv(mode, import.meta.dirname))), colorScheme()],
   resolve: TOOLKIT_RESOLVE,
 }))
 ```
@@ -94,6 +94,7 @@ API types imported by both, without the code of either: `CredentialField`, `Cred
 | Export | Purpose |
 | --- | --- |
 | `font(choice)`, `fontChoice(env)`, `FONTS` | the font plugin of `VITE_FONT`: `cdn` (Google Fonts links, the default), `bundled` (`@fontsource/roboto`, a dependency of the application) or `none` |
+| `colorScheme()` | the plugin that sets the color scheme of `appTheme`, chosen in the theme menu or by the system, on the page before its first paint, so a dark page, also one in an iframe, does not show white while it loads |
 | `TOOLKIT_RESOLVE` | the resolve options that keep one copy of React, MUI and Emotion |
 
 ## Credential store
@@ -177,7 +178,7 @@ which edit their variables. `createTfvarsStore(pool)` takes a `pg` pool and appl
 
 | Table | Rows |
 | --- | --- |
-| `tf_modules` | `name`, `root`, the folder of the Terraform code, `description` and `configurator_url` |
+| `tf_modules` | `name`, `root`, the folder of the Terraform code, `description`, `configurator_url` and `configurator_discovery`, whether a module without `configurator_url` finds its configurator by the `config.json` of its folder or one above it; a module registered before the column keeps it on, a new one has it off unless given |
 | `tf_instances` | `module`, `name` and `state_schema`, the schema of the instance's state in the pg backend, unique, and `created` |
 | `tf_variable_files` | `module`, `instance`, `name` (`*.tfvars.json`), `content` and `previous`, the object it replaced, as `jsonb`, and `updated` |
 | `tf_variable_revisions` | `module`, `instance`, `revision`, numbered from 1 for each instance, `files`, all its files after a write as a `jsonb` object by name, `schema`, the schema version of their layout or null, `producer` and `saved` |
@@ -239,6 +240,13 @@ database `postgres`. `npm run build` (`build.mjs`) builds the published package
 `dist/`: the code compiled to JavaScript with its declarations, the SQL of the server, README.md, CHANGELOG.md and a
 `package.json` whose exports name the compiled files.
 
+`./test-container.mjs` runs the same checks with Docker, on the engine of the docker CLI's
+context, which may be remote: it sends the files git tracks, and the untracked files it does not
+ignore, as the build context of `test.Dockerfile`, whose build runs the typecheck, the lint and
+the package build, and runs the tests against a scratch PostgreSQL in a container on a private
+network, removed afterwards. `--package <folder>` writes the packed toolkit to the folder instead,
+for the tests of an application against the checkout.
+
 ## Releases
 
 `./release.mjs` commits, tags the version of the newest CHANGELOG.md entry and pushes `main` with

package-lock.json

@@ -1,12 +1,12 @@
 {
   "name": "@koszewscy/sk-react-toolkit",
-  "version": "0.7.0",
+  "version": "0.7.2",
   "lockfileVersion": 3,
   "requires": true,
   "packages": {
     "": {
       "name": "@koszewscy/sk-react-toolkit",
-      "version": "0.7.0",
+      "version": "0.7.2",
       "devDependencies": {
         "@emotion/react": "^11.14.0",
         "@emotion/styled": "^11.14.1",

package.json

@@ -1,7 +1,7 @@
 {
   "name": "@koszewscy/sk-react-toolkit",
-  "version": "0.7.0",
-  "description": "Shared code of the React and Fastify web applications: web components, server helpers and the Vite font plugin.",
+  "version": "0.7.2",
+  "description": "Shared code of the React and Fastify web applications: web components, server helpers and the Vite font and color scheme plugins.",
   "private": true,
   "type": "module",
   "engines": {

server/sql/tfvars_store.sql

@@ -4,12 +4,24 @@
 -- again.
 
 CREATE TABLE IF NOT EXISTS tf_modules (
-  name             text PRIMARY KEY CHECK (name ~ '^[a-z][a-z0-9-]*$'),
-  root             text NOT NULL CHECK (root <> ''),
-  description      text NOT NULL DEFAULT '',
-  configurator_url text
+  name                   text PRIMARY KEY CHECK (name ~ '^[a-z][a-z0-9-]*$'),
+  root                   text NOT NULL CHECK (root <> ''),
+  description            text NOT NULL DEFAULT '',
+  configurator_url       text,
+  configurator_discovery boolean NOT NULL DEFAULT false
 );
 
+-- A module registered before configurator_discovery keeps finding its configurator.
+DO $$
+BEGIN
+  IF NOT EXISTS (SELECT 1 FROM information_schema.columns
+                 WHERE table_schema = current_schema() AND table_name = 'tf_modules' AND column_name = 'configurator_discovery') THEN
+    ALTER TABLE tf_modules ADD COLUMN configurator_discovery boolean NOT NULL DEFAULT true;
+    ALTER TABLE tf_modules ALTER COLUMN configurator_discovery SET DEFAULT false;
+  END IF;
+END
+$$;
+
 CREATE TABLE IF NOT EXISTS tf_instances (
   module       text NOT NULL REFERENCES tf_modules (name),
   name         text NOT NULL CHECK (name ~ '^[a-z][a-z0-9-]*$'),
@@ -51,23 +63,36 @@ WHERE NOT EXISTS (SELECT 1 FROM tf_variable_revisions r WHERE r.module = f.modul
 GROUP BY f.module, f.instance
 ON CONFLICT DO NOTHING;
 
--- The modules, by name.
+-- The modules, by name; its columns are those of the table, so a table of other columns replaces it.
+DO $$
+BEGIN
+  IF EXISTS (SELECT 1 FROM pg_proc p JOIN pg_namespace n ON n.oid = p.pronamespace
+             WHERE n.nspname = current_schema() AND p.proname = 'tf_modules_list'
+               AND NOT 'configurator_discovery' = ANY (coalesce(p.proargnames, '{}'))) THEN
+    DROP FUNCTION tf_modules_list();
+  END IF;
+END
+$$;
+
 CREATE OR REPLACE FUNCTION tf_modules_list()
-RETURNS TABLE (name text, root text, description text, configurator_url text)
+RETURNS TABLE (name text, root text, description text, configurator_url text, configurator_discovery boolean)
 LANGUAGE sql STABLE
 SET search_path FROM CURRENT
 AS $$
-  SELECT m.name, m.root, m.description, m.configurator_url FROM tf_modules m ORDER BY m.name
+  SELECT m.name, m.root, m.description, m.configurator_url, m.configurator_discovery FROM tf_modules m ORDER BY m.name
 $$;
 
+DROP PROCEDURE IF EXISTS tf_module_put(text, text, text, text);
+
 -- Creates a module, or changes the one of that name.
-CREATE OR REPLACE PROCEDURE tf_module_put(module_name text, module_root text, module_description text, module_configurator_url text)
+CREATE OR REPLACE PROCEDURE tf_module_put(module_name text, module_root text, module_description text, module_configurator_url text, module_configurator_discovery boolean)
 LANGUAGE sql
 SET search_path FROM CURRENT
 AS $$
-  INSERT INTO tf_modules (name, root, description, configurator_url)
-  VALUES (module_name, module_root, module_description, module_configurator_url)
-  ON CONFLICT (name) DO UPDATE SET root = excluded.root, description = excluded.description, configurator_url = excluded.configurator_url
+  INSERT INTO tf_modules (name, root, description, configurator_url, configurator_discovery)
+  VALUES (module_name, module_root, module_description, module_configurator_url, module_configurator_discovery)
+  ON CONFLICT (name) DO UPDATE SET root = excluded.root, description = excluded.description,
+    configurator_url = excluded.configurator_url, configurator_discovery = excluded.configurator_discovery
 $$;
 
 -- Deletes a module; the foreign key of tf_instances refuses it while the module has instances.

server/tfvarsStore.test.ts

@@ -35,16 +35,38 @@ const status = async (promise: Promise<unknown>) => {
 }
 
 test('modules are created, changed, listed and deleted', async () => {
-  await store.modules.put({ name: 'router', root: 'router/terraform', description: 'VPN router', configuratorUrl: null })
-  await store.modules.put({ name: 'router', root: 'router/terraform', description: 'VPN router', configuratorUrl: 'http://localhost:7083/' })
-  await store.modules.put({ name: 'app', root: 'app', description: '', configuratorUrl: null })
+  await store.modules.put({ name: 'router', root: 'router/terraform', description: 'VPN router', configuratorUrl: null, configuratorDiscovery: true })
+  await store.modules.put({ name: 'router', root: 'router/terraform', description: 'VPN router', configuratorUrl: 'http://localhost:7083/', configuratorDiscovery: false })
+  await store.modules.put({ name: 'app', root: 'app', description: '', configuratorUrl: null, configuratorDiscovery: true })
   assert.deepEqual(await store.modules.list(), [
-    { name: 'app', root: 'app', description: '', configuratorUrl: null },
-    { name: 'router', root: 'router/terraform', description: 'VPN router', configuratorUrl: 'http://localhost:7083/' },
+    { name: 'app', root: 'app', description: '', configuratorUrl: null, configuratorDiscovery: true },
+    { name: 'router', root: 'router/terraform', description: 'VPN router', configuratorUrl: 'http://localhost:7083/', configuratorDiscovery: false },
   ])
   await store.modules.delete('app')
   assert.deepEqual((await store.modules.list()).map((module) => module.name), ['router'])
-  assert.equal(await status(store.modules.put({ name: 'Bad', root: 'x', description: '', configuratorUrl: null })), 400)
+  assert.equal(await status(store.modules.put({ name: 'Bad', root: 'x', description: '', configuratorUrl: null, configuratorDiscovery: false })), 400)
+})
+
+test('a module registered before configurator_discovery keeps discovering its configurator; a new one does not', async () => {
+  // The table, the function and the procedure as they were before the column, in a schema of their own.
+  await pool.query(`CREATE SCHEMA earlier;
+    CREATE TABLE earlier.tf_modules (name text PRIMARY KEY, root text NOT NULL, description text NOT NULL DEFAULT '', configurator_url text);
+    INSERT INTO earlier.tf_modules VALUES ('router', 'router/terraform', '', NULL);
+    CREATE FUNCTION earlier.tf_modules_list() RETURNS TABLE (name text, root text, description text, configurator_url text)
+      LANGUAGE sql AS $$ SELECT m.name, m.root, m.description, m.configurator_url FROM earlier.tf_modules m $$;
+    CREATE PROCEDURE earlier.tf_module_put(a text, b text, c text, d text) LANGUAGE sql AS $$ SELECT 1 $$;`)
+  const earlierPool = new pg.Pool({ database, options: '-c search_path=earlier' })
+  try {
+    const earlier = createTfvarsStore(earlierPool)
+    await earlier.modules.put({ name: 'app', root: 'app', description: '', configuratorUrl: null, configuratorDiscovery: false })
+    assert.deepEqual((await earlier.modules.list()).map((module) => [module.name, module.configuratorDiscovery]), [['app', false], ['router', true]])
+    const { rows } = await earlierPool.query("SELECT count(*)::int AS count FROM pg_proc p JOIN pg_namespace n ON n.oid = p.pronamespace WHERE n.nspname = 'earlier' AND p.proname = 'tf_module_put'")
+    assert.equal(rows[0].count, 1)
+    // Another store applies the file again over the migrated schema, which keeps the modules as they are.
+    assert.deepEqual((await createTfvarsStore(earlierPool).modules.list()).map((module) => [module.name, module.configuratorDiscovery]), [['app', false], ['router', true]])
+  } finally {
+    await earlierPool.end()
+  }
 })
 
 test('instances are created and refused for a missing module or a name in use', async () => {

server/tfvarsStore.ts

@@ -152,10 +152,10 @@ export function createTfvarsStore(pool: Pool): TfvarsStore {
   }
   return {
     modules: {
-      list: async () => (await query<{ name: string; root: string; description: string; configurator_url: string | null }>('SELECT * FROM tf_modules_list()', []))
-        .map((row) => ({ name: row.name, root: row.root, description: row.description, configuratorUrl: row.configurator_url })),
+      list: async () => (await query<{ name: string; root: string; description: string; configurator_url: string | null; configurator_discovery: boolean }>('SELECT * FROM tf_modules_list()', []))
+        .map((row) => ({ name: row.name, root: row.root, description: row.description, configuratorUrl: row.configurator_url, configuratorDiscovery: row.configurator_discovery })),
       async put(module) {
-        await query('CALL tf_module_put($1, $2, $3, $4)', [module.name, module.root, module.description, module.configuratorUrl])
+        await query('CALL tf_module_put($1, $2, $3, $4, $5)', [module.name, module.root, module.description, module.configuratorUrl, module.configuratorDiscovery])
       },
       async delete(name) {
         await query('CALL tf_module_delete($1)', [name])

shared/tfvars.ts

@@ -5,6 +5,11 @@ export type TfModule = {
   description: string
   /** The address of the module's configurator, the application that edits the variables of its instances. */
   configuratorUrl: string | null
+  /**
+   * Whether a module without configuratorUrl finds its configurator by the config.json of its folder or one above it;
+   * without either, the module has no configurator.
+   */
+  configuratorDiscovery: boolean
 }
 
 /** A deployment of a module, whose state is kept in the schema stateSchema of the pg backend. */

test-container.mjs

@@ -0,0 +1,87 @@
+#!/usr/bin/env node
+
+// Runs the checks of the toolkit with Docker, on the engine of the docker CLI's context: builds test.Dockerfile from
+// the files of the checkout git knows, then runs the tests against a scratch PostgreSQL on a private network.
+
+import { spawnSync } from 'node:child_process';
+import { resolve } from 'node:path';
+import { parseArgs } from 'node:util';
+
+const NAME = 'sk-react-toolkit-test';
+const POSTGRES_IMAGE = 'postgres:18';
+
+const usage = `Usage: test-container.mjs [-h|--help] [--package <folder>]
+
+  Builds test.Dockerfile, which runs the typecheck, the lint and the package build, and runs the
+  tests in a container with a scratch PostgreSQL. The build context is the files git tracks and the
+  untracked files it does not ignore.
+
+  --package <folder>  write the packed toolkit, the .tgz file npm pack makes, to the folder instead
+                      of running the tests`;
+
+const { values } = parseArgs({
+  options: {
+    package: { type: 'string' },
+    help: { type: 'boolean', short: 'h', default: false },
+  },
+});
+
+if (values.help) {
+  console.log(usage);
+  process.exit(0);
+}
+
+function fail(message) {
+  console.error(`test-container: ${message}`);
+  process.exit(1);
+}
+
+// Runs docker, failing with its message when it cannot start; answers the result.
+function docker(args, options = {}) {
+  const result = spawnSync('docker', args, { stdio: 'inherit', ...options });
+  if (result.error) fail(`docker: ${result.error.message}`);
+  return result;
+}
+
+const files = spawnSync('git', ['ls-files', '-z', '--cached', '--others', '--exclude-standard'], { cwd: import.meta.dirname, maxBuffer: 1 << 26 });
+if (files.status !== 0) fail(`git ls-files failed: ${files.stderr}`);
+// macOS tar adds extended attributes and AppleDouble files, which a Linux engine refuses.
+const macOptions = process.platform === 'darwin' ? ['--no-xattrs', '--no-mac-metadata'] : [];
+const context = spawnSync('tar', [...macOptions, '--null', '-T', '-', '-cf', '-'], { cwd: import.meta.dirname, input: files.stdout, maxBuffer: 1 << 30 });
+if (context.status !== 0) fail(`tar failed: ${context.stderr}`);
+
+if (values.package !== undefined) {
+  const output = `type=local,dest=${resolve(values.package)}`;
+  const built = docker(['build', '-f', 'test.Dockerfile', '--target', 'package', '--output', output, '-'], { input: context.stdout, stdio: ['pipe', 'inherit', 'inherit'] });
+  process.exit(built.status ?? 1);
+}
+
+const built = docker(['build', '-f', 'test.Dockerfile', '--target', 'check', '-t', NAME, '-'], { input: context.stdout, stdio: ['pipe', 'inherit', 'inherit'] });
+if (built.status !== 0) process.exit(built.status ?? 1);
+
+// Ctrl-C stops the running docker command; the cleanup below still runs.
+process.on('SIGINT', () => {});
+
+const network = `${NAME}-${process.pid}`;
+const database = `${NAME}-pg-${process.pid}`;
+let status = 1;
+try {
+  if (docker(['network', 'create', network], { stdio: 'ignore' }).status !== 0) throw new Error(`cannot create the network ${network}`);
+  const started = docker(['run', '-d', '--rm', '--name', database, '--network', network, '-e', 'POSTGRES_PASSWORD=test', POSTGRES_IMAGE], { stdio: ['ignore', 'ignore', 'inherit'] });
+  if (started.status !== 0) throw new Error(`cannot start ${POSTGRES_IMAGE}`);
+  // The server answers on TCP once its initialization has finished.
+  let ready = false;
+  for (let attempt = 0; attempt < 60 && !ready; attempt++) {
+    ready = docker(['exec', database, 'pg_isready', '-q', '-h', '127.0.0.1', '-U', 'postgres'], { stdio: 'ignore' }).status === 0;
+    if (!ready) Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, 1000);
+  }
+  if (!ready) throw new Error('PostgreSQL did not start within 60 seconds');
+  const environment = ['PGHOST=' + database, 'PGUSER=postgres', 'PGPASSWORD=test', 'PGDATABASE=postgres'].flatMap((variable) => ['-e', variable]);
+  status = docker(['run', '--rm', '--network', network, ...environment, NAME]).status ?? 1;
+} catch (error) {
+  console.error(`test-container: ${error.message}`);
+} finally {
+  docker(['rm', '-f', database], { stdio: 'ignore' });
+  docker(['network', 'rm', network], { stdio: 'ignore' });
+}
+process.exit(status);

test.Dockerfile

@@ -0,0 +1,16 @@
+# The toolkit's checks on Linux: the build runs the typecheck, the lint and the package build, the
+# container the tests; the package stage holds the packed toolkit for the tests of an application.
+
+ARG NODE_VERSION=26
+
+FROM node:${NODE_VERSION}-trixie-slim AS check
+WORKDIR /src
+COPY package.json package-lock.json ./
+RUN npm ci --no-fund --no-audit
+COPY . .
+RUN npm run typecheck && npm run lint && npm run build
+RUN mkdir /pack && npm pack ./dist --pack-destination /pack
+CMD ["npm", "test"]
+
+FROM scratch AS package
+COPY --from=check /pack/ /

vite/index.ts

@@ -38,6 +38,31 @@ export function font(choice: Font): Plugin {
   }
 }
 
+/**
+ * Sets the color scheme of appTheme on the page before its first paint, so a dark page, and a page in an iframe of a
+ * dark page, is not white until the application renders: adds to index.html a script that sets the class dark or
+ * light on <html>, from MUI's mui-mode in localStorage or, for system, from prefers-color-scheme, and a style that
+ * gives those classes their color-scheme and the dark scheme MUI's dark background.
+ */
+export function colorScheme(): Plugin {
+  const script = `(function () {
+  try {
+    var mode = localStorage.getItem('mui-mode') || 'system';
+    var dark = mode === 'dark' || (mode === 'system' && window.matchMedia('(prefers-color-scheme: dark)').matches);
+    document.documentElement.classList.add(dark ? 'dark' : 'light');
+  } catch (e) {}
+})();`
+  return {
+    name: 'color-scheme',
+    transformIndexHtml() {
+      return [
+        { tag: 'style', children: 'html.light { color-scheme: light; }\nhtml.dark { color-scheme: dark; background-color: #121212; }', injectTo: 'head' },
+        { tag: 'script', children: script, injectTo: 'head' },
+      ]
+    },
+  }
+}
+
 /** VITE_FONT of env, the variables Vite's loadEnv answers, cdn when it is not set; another value throws. */
 export function fontChoice(env: Record<string, string>): Font {
   const choice = env.VITE_FONT ?? 'cdn'