5 Commits
18 changed files with 552 additions and 58 deletions

No files matched your search

+75
View File
@@ -118,3 +118,78 @@ Consequences: Every application of the store works on the same rows and refuses
conflicts, and changes of the tables reach them with a release of the toolkit. The objects' keys
are in `jsonb`'s order. The store is tested against a real PostgreSQL server, which the release
workflow does not have, so `npm test` runs on the developer's machine.
## ADR-6: A fourth entry point, /shared, exports the shared types and constants
Status: Accepted
Context: An application keeps the types of its API in one file that its server and its web
application both import. When those types name types of the toolkit, such as `TfModule`, the file
must import them from an entry point both sides can resolve: `/server` brings Fastify and `pg`
into the web application's type check, `/web` brings React and MUI into the server's.
Decision: `shared/index.ts` is the entry point `/shared`, built and exported like the other three.
It exports the types and constants of `shared/`, the credential types and the types and name
patterns of the tfvars store, and no code that needs a peer dependency. `/server` and `/web` keep
exporting them too.
Consequences: An application's shared API types import from `/shared` without the code of either
side. A type added to `shared/` is exported from `/shared` as well as from the side that uses it.
## ADR-7: Every write of an instance's variable files is a revision with its schema and producer
Status: Accepted
Context: A configurator changes the layout of the variable files it writes from one version to
the next, and Cloud Platform has to tell whether an instance's files are of the layout its
module's configurator writes before it runs Terraform with them. The files written earlier have
to stay available, to restore or compare them.
Decision: `tf_variable_revisions` holds every write of an instance's files: all files after the
write, the schema version of their layout as the writer gives it, null when it does not know it,
the producer, the writing application with its version, and the time. `write` and `remove` take
the schema and the producer and record the revision in the transaction of the change, under the
lock of the files; a write of no file records none. The current files stay in `tf_variable_files`,
so reading them and their version is unchanged. Revisions are kept until removed by hand with
SQL. Files without revisions are recorded as revision 1 of an unknown producer and schema when the
store is first used.
Consequences: The schema of an instance's files is the schema of its newest revision; files
imported without a schema, or written before revisions, have none until a configurator writes
them. Revisions grow with every write; the README describes the SQL that removes them.
## ADR-8: A Vite plugin sets the color scheme before the first paint
Status: Accepted
Context: `appTheme` puts the class of the color scheme on `<html>` from React, after the scripts
of the application have loaded. Until then a page in the dark scheme is white, which shows on
every load, and on every opening of a page that embeds an application in an iframe.
Decision: The `colorScheme()` plugin adds an inline script and style to `index.html`, so the class
is set and the dark background painted before the application loads. The script reads MUI's
storage key `mui-mode` and follows `prefers-color-scheme` for the system mode, as MUI does, and
the style uses MUI's default dark background, which `appTheme` keeps. The plugin is part of the
Vite helpers, which every application already uses, so the applications need no code of their own
in `index.html`.
Consequences: The storage key and the dark background are repeated in the plugin; a theme that
changes either has to change the plugin too. An application of the toolkit adds `colorScheme()`
to the plugins of its `vite.config.ts`.
## ADR-9: The checks run in a container built from the files git knows, with a scratch PostgreSQL
Status: Accepted
Context: The checks are faster on a remote Docker engine than on the developer's machine. The
store's tests need a PostgreSQL server whose role creates databases, and a remote engine sees
neither the checkout nor its `node_modules`.
Decision: `test-container.mjs` sends a tar archive of the files git tracks and the untracked files
it does not ignore as the build context of `test.Dockerfile`, whose build runs the checks that need
no server, and runs the tests in a container next to a PostgreSQL container on a network of their
own, both removed afterwards. The image's `package` stage holds the packed toolkit, which the
applications install to check against the checkout.
Consequences: The tests need no PostgreSQL server of the developer's. A file git ignores is not
part of the checks. The checks run on Linux, whatever the developer's system.
+52
View File
@@ -1,5 +1,57 @@
# Changelog
## 0.7.1
fix: a page in the dark color scheme, also one in an iframe, is no longer white while it loads,
and test-container.mjs runs the checks with Docker.
- The `colorScheme()` plugin of `/vite` sets the color scheme of `appTheme` and the dark background
on the page before its first paint; an application adds it to the plugins of its
`vite.config.ts`.
- `./test-container.mjs` runs the typecheck, the lint, the package build and the tests in a
container, on the engine of the docker CLI's context, with a scratch PostgreSQL;
`--package <folder>` writes the packed toolkit to the folder.
## 0.7.0
feature: every write of an instance's variable files is kept as a revision with the schema version
of their layout and the application that wrote them.
- `tf_variable_revisions` keeps all files of an instance after each write, with `schema`, null when
not known, `producer` and `saved`; revisions are numbered from 1 for each instance and kept until
removed by hand.
- `write(version, contents, meta)` and `remove(version, name, meta)` take `meta`, `{ schema,
producer }`, and record the revision in the same transaction; a write of no file records none.
- `read()` answers the newest revision as `revision`; `revisions()` lists them, newest first, and
`revision(n)` answers one with its files. `TfvarsRevision` is exported from `/server` and
`/shared`.
- The files of an instance without revisions become its revision 1, of the producer `unknown` and
no schema.
- The README describes the SQL that purges some or all revisions of a module's instance.
## 0.6.0-2
fix: release.mjs reads a summary wrapped over several lines.
- `release.mjs` takes the summary of the newest entry as the paragraph after its heading and joins
its lines with single spaces into the commit message.
## 0.6.0
feature: RowDialog validates a row as part of a larger value, such as a whole document with the row put in.
- `RowDialog` takes `body`, the value posted to `validatePath` for the row, and `select`, the
row's problems by field and message taken from the answer.
- `useValidation` and `RowDialog` take an answer of any shape.
## 0.5.0
feature: the entry point /shared exports the types and constants a server and its web application share.
- `@koszewscy/sk-react-toolkit/shared` exports the credential types, `TfModule`, `TfInstance`,
`TfvarsFileStatus` and the name patterns of the tfvars store, for the API types of an
application imported by both its server and its web application.
## 0.4.0
feature: the tfvars store keeps Terraform modules, their instances and the instances' variable files in PostgreSQL.
+58 -11
View File
@@ -8,7 +8,7 @@ source: Node 26 runs the server helpers, and the application's Vite build compil
The package is `@koszewscy/sk-react-toolkit`, published to the npm registry of Gitea,
`https://gitea.koszewscy.waw.pl/api/packages/koszewscy/npm/`. An application depends on a
published version and imports `@koszewscy/sk-react-toolkit/server`, `/web` and `/vite`. Its
published version and imports `@koszewscy/sk-react-toolkit/server`, `/shared`, `/web` and `/vite`. Its
`.npmrc` names the registry of the scope:
```ini
@@ -19,15 +19,15 @@ published version and imports `@koszewscy/sk-react-toolkit/server`, `/web` and `
{ "dependencies": { "@koszewscy/sk-react-toolkit": "^0.2.0" } }
```
The web application adds the toolkit's resolve options and font plugin to its `vite.config.ts`, so
React, MUI and Emotion come from its own `node_modules`:
The web application adds the toolkit's resolve options, so React, MUI and Emotion come from its
own `node_modules`, and its font and color scheme plugins to its `vite.config.ts`:
```ts
import { font, fontChoice, TOOLKIT_RESOLVE } from '@koszewscy/sk-react-toolkit/vite'
import { colorScheme, font, fontChoice, TOOLKIT_RESOLVE } from '@koszewscy/sk-react-toolkit/vite'
import { defineConfig, loadEnv } from 'vite'
export default defineConfig(({ mode }) => ({
plugins: [react(), font(fontChoice(loadEnv(mode, import.meta.dirname)))],
plugins: [react(), font(fontChoice(loadEnv(mode, import.meta.dirname))), colorScheme()],
resolve: TOOLKIT_RESOLVE,
}))
```
@@ -66,6 +66,13 @@ code; `vite` for the Vite helpers.
| `CredentialStore`, `credentialProblems`, `credentialValues`, `registerCredentialRoutes`, `POSTGRESQL_KIND` | the credential store; see [Credential store](#credential-store) |
| `createTfvarsStore`, `versionOf`, `tfvarsText`, `TfModule`, `TfInstance`, `TfvarsFileStatus`, `TF_NAME_PATTERN`, `TFVARS_FILE_PATTERN`, `STATE_SCHEMA_PATTERN` | the Terraform modules, their instances and their variable files in PostgreSQL; see [Tfvars store](#tfvars-store) |
## sk-react-toolkit/shared
The types and constants of `/server` and `/web` that a server and its web application share, for
API types imported by both, without the code of either: `CredentialField`, `CredentialKind`,
`CredentialProfile`, `CredentialsDocument`, `TfModule`, `TfInstance`, `TfvarsFileStatus`,
`TF_NAME_PATTERN`, `TFVARS_FILE_PATTERN` and `STATE_SCHEMA_PATTERN`.
## sk-react-toolkit/web
| Export | Purpose |
@@ -77,7 +84,7 @@ code; `vite` for the Vite helpers.
| `ThemeMenu` | the button cycling light, dark and system theme |
| `SwitcherMenu`, `SwitcherItem` | a button naming the item chosen, whose menu chooses another, such as a page or a connection |
| `TextInput` | a text field with clear and show/hide buttons |
| `RowDialog`, `useValidation` | a dialog that edits a value validated by the server as it changes, and the validation hook |
| `RowDialog`, `useValidation` | a dialog that edits a value validated by the server as it changes, and the validation hook; with `body` and `select` the dialog posts another value, such as a whole document with the row put in, and takes the row's problems from the answer |
| `createBrowserSettings(keys, event)` | switches kept in localStorage and followed across tabs |
| `CredentialsButton` | the dialog of a credential store; see [Credential store](#credential-store) |
| `TfModule`, `TfInstance`, `TfvarsFileStatus`, `TF_NAME_PATTERN`, `TFVARS_FILE_PATTERN`, `STATE_SCHEMA_PATTERN` | the types and name patterns of the [Tfvars store](#tfvars-store) |
@@ -87,6 +94,7 @@ code; `vite` for the Vite helpers.
| Export | Purpose |
| --- | --- |
| `font(choice)`, `fontChoice(env)`, `FONTS` | the font plugin of `VITE_FONT`: `cdn` (Google Fonts links, the default), `bundled` (`@fontsource/roboto`, a dependency of the application) or `none` |
| `colorScheme()` | the plugin that sets the color scheme of `appTheme`, chosen in the theme menu or by the system, on the page before its first paint, so a dark page, also one in an iframe, does not show white while it loads |
| `TOOLKIT_RESOLVE` | the resolve options that keep one copy of React, MUI and Emotion |
## Credential store
@@ -173,6 +181,7 @@ which edit their variables. `createTfvarsStore(pool)` takes a `pg` pool and appl
| `tf_modules` | `name`, `root`, the folder of the Terraform code, `description` and `configurator_url` |
| `tf_instances` | `module`, `name` and `state_schema`, the schema of the instance's state in the pg backend, unique, and `created` |
| `tf_variable_files` | `module`, `instance`, `name` (`*.tfvars.json`), `content` and `previous`, the object it replaced, as `jsonb`, and `updated` |
| `tf_variable_revisions` | `module`, `instance`, `revision`, numbered from 1 for each instance, `files`, all its files after a write as a `jsonb` object by name, `schema`, the schema version of their layout or null, `producer` and `saved` |
Names of modules and instances are `TF_NAME_PATTERN`, of variable files `TFVARS_FILE_PATTERN`,
of state schemas `STATE_SCHEMA_PATTERN`.
@@ -180,11 +189,42 @@ of state schemas `STATE_SCHEMA_PATTERN`.
- `modules.list()`, `modules.put(module)`, which creates or changes, and `modules.delete(name)`.
- `instances.list(module)`, `instances.create(module, name, stateSchema)` and
`instances.delete(module, name)`, which deletes its variable files too.
- `variables(module, instance)`: `read()` answers the files, their sizes and times and the
version, `versionOf` the files, the name and the SHA-256 of each in name order. `write(version,
contents)` writes files given by name as JSON text and `remove(version, name)` deletes one; both
lock the table, compare the version and change in one transaction, and answer `false`, changing
nothing, when the files changed since the version.
- `variables(module, instance)`: `read()` answers the files, their sizes and times, the version,
`versionOf` the files, the name and the SHA-256 of each in name order, and the newest revision.
`write(version, contents, meta)` writes files given by name as JSON text and `remove(version,
name, meta)` deletes one; both lock the table, compare the version, change and record all files
as the next revision with `meta`, `{ schema, producer }`, in one transaction, and answer
`false`, changing nothing, when the files changed since the version. A write of no file records
no revision. `revisions()` lists the revisions, newest first, without their files, and
`revision(n)` answers one with its files, or `null`.
Every revision is kept. The files of an instance without any revision, as those written before
revisions existed, are recorded as its revision 1, of the producer `unknown` and no schema, when
the store is first used.
### Purging revisions
Revisions are removed by hand, with `psql` connected to the database of the store as the owner of
its tables, in the schema the applications use (`SET search_path TO <schema>;` when it is not the
default). Each statement removes the revisions of one instance, here `router`/`platform`:
```sql
-- All revisions but the newest.
DELETE FROM tf_variable_revisions
WHERE module = 'router' AND instance = 'platform'
AND revision < (SELECT max(revision) FROM tf_variable_revisions WHERE module = 'router' AND instance = 'platform');
-- The revisions before revision 10.
DELETE FROM tf_variable_revisions WHERE module = 'router' AND instance = 'platform' AND revision < 10;
-- All revisions of the instance.
DELETE FROM tf_variable_revisions WHERE module = 'router' AND instance = 'platform';
```
Leaving out `AND instance = ...` applies a statement to every instance of the module. The next
write numbers its revision after the newest one left. An instance whose revisions are all removed
gets its current files back as revision 1, of the producer `unknown` and no schema, when the store
is next first used, so its schema is unknown until the next write by its configurator.
A change the tables refuse throws `HttpError`: 409 for a name or a state schema in use and for a
module that has instances, 404 for a module or an instance that does not exist, 400 for a name or
@@ -200,6 +240,13 @@ database `postgres`. `npm run build` (`build.mjs`) builds the published package
`dist/`: the code compiled to JavaScript with its declarations, the SQL of the server, README.md, CHANGELOG.md and a
`package.json` whose exports name the compiled files.
`./test-container.mjs` runs the same checks with Docker, on the engine of the docker CLI's
context, which may be remote: it sends the files git tracks, and the untracked files it does not
ignore, as the build context of `test.Dockerfile`, whose build runs the typecheck, the lint and
the package build, and runs the tests against a scratch PostgreSQL in a container on a private
network, removed afterwards. `--package <folder>` writes the packed toolkit to the folder instead,
for the tests of an application against the checkout.
## Releases
`./release.mjs` commits, tags the version of the newest CHANGELOG.md entry and pushes `main` with
+2 -2
View File
@@ -10,7 +10,7 @@ import { join } from 'node:path';
const ROOT = import.meta.dirname;
const DIST = join(ROOT, 'dist');
const ENTRIES = ['server', 'web', 'vite'];
const ENTRIES = ['server', 'shared', 'web', 'vite'];
function fail(message) {
console.error(`build: ${message}`);
@@ -23,7 +23,7 @@ if (tsc.error) fail(`tsc: ${tsc.error.message}; run npm ci first`);
if (tsc.status !== 0) process.exit(tsc.status ?? 1);
// The declarations import the other declarations by their .js names, as the compiled JavaScript does.
for (const entry of [...ENTRIES, 'shared']) {
for (const entry of ENTRIES) {
for (const file of readdirSync(join(DIST, entry)).filter((name) => name.endsWith('.d.ts'))) {
const path = join(DIST, entry, file);
writeFileSync(path, readFileSync(path, 'utf8').replace(/(from\s+|import\s+)(['"])(\.{1,2}\/[^'"]+)\.tsx?\2/g, '$1$2$3.js$2'));
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "@koszewscy/sk-react-toolkit",
"version": "0.4.0",
"version": "0.7.1",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "@koszewscy/sk-react-toolkit",
"version": "0.4.0",
"version": "0.7.1",
"devDependencies": {
"@emotion/react": "^11.14.0",
"@emotion/styled": "^11.14.1",
+3 -2
View File
@@ -1,7 +1,7 @@
{
"name": "@koszewscy/sk-react-toolkit",
"version": "0.4.0",
"description": "Shared code of the React and Fastify web applications: web components, server helpers and the Vite font plugin.",
"version": "0.7.1",
"description": "Shared code of the React and Fastify web applications: web components, server helpers and the Vite font and color scheme plugins.",
"private": true,
"type": "module",
"engines": {
@@ -13,6 +13,7 @@
},
"exports": {
"./server": "./server/index.ts",
"./shared": "./shared/index.ts",
"./web": "./web/index.ts",
"./vite": "./vite/index.ts"
},
+12 -7
View File
@@ -10,8 +10,9 @@ const CHANGELOG = `${import.meta.dirname}/CHANGELOG.md`;
const usage = `Usage: release.mjs [-h|--help] [-n|--dry-run] [--no-push] [-m|--message <message>]
-m, --message commit message; default: the summary line of the newest CHANGELOG.md entry,
the line between blank lines right after its "## x.y.z" or "## x.y.z-n" heading
-m, --message commit message; default: the summary of the newest CHANGELOG.md entry, the
paragraph between blank lines right after its "## x.y.z" or "## x.y.z-n"
heading, its lines joined with single spaces
The tag is the version of the heading: x.y.z for a release of the code, which must be the version
of package.json and which the build workflow publishes, x.y.z-n, n from 2, for a later release
@@ -48,8 +49,8 @@ function git(...args) {
if (status !== 0) process.exit(status ?? 1);
}
// The newest entry: its version from the first "## x.y.z" or "## x.y.z-n" heading, n from 2, and its summary line when
// the heading is followed by a blank line, one line that is not a "-" item, and another blank line.
// The newest entry: its version from the first "## x.y.z" or "## x.y.z-n" heading, n from 2, and its summary when the
// heading is followed by a blank line and a paragraph that is not a "-" item; the paragraph's lines are joined with spaces.
const lines = readFileSync(CHANGELOG, 'utf8').split('\n');
const heading = lines.findIndex((line) => line.startsWith('## '));
if (heading === -1) fail(`${CHANGELOG} has no "## x.y.z" entry`);
@@ -65,12 +66,16 @@ const packageVersion = JSON.parse(readFileSync(`${import.meta.dirname}/package.j
if (revision === undefined && version !== packageVersion) {
fail(`${CHANGELOG}: ${tag} is not the version of package.json, ${packageVersion}`);
}
const [above, line, below] = lines.slice(heading + 1, heading + 4);
const summary = above?.trim() === '' && below?.trim() === '' && line?.trim() && !line.startsWith('-') && !line.startsWith('#') ? line.trim() : undefined;
const end = lines.findIndex((line, index) => index > heading + 1 && line.trim() === '');
const paragraph = end === -1 ? [] : lines.slice(heading + 2, end);
const summary =
lines[heading + 1]?.trim() === '' && paragraph.length > 0 && !paragraph[0].startsWith('-') && !paragraph.some((line) => line.startsWith('#'))
? paragraph.map((line) => line.trim()).join(' ')
: undefined;
const message = values.message ?? summary;
if (!message) {
fail(`no commit message: pass -m, or put a summary line between blank lines right after the newest heading of ${CHANGELOG}`);
fail(`no commit message: pass -m, or put a summary paragraph between blank lines right after the newest heading of ${CHANGELOG}`);
}
if (spawnSync('git', ['rev-parse', '-q', '--verify', `refs/tags/${tag}`], { cwd: import.meta.dirname }).status === 0) {
+12 -2
View File
@@ -4,6 +4,16 @@ export { HttpError, errorHandler, notFoundHandler, strict } from './http.ts'
export { QUIET, createLog, fail, timestampsOn, type Log } from './log.ts'
export { exitOnSignals, gitVersion, readJsonObject } from './process.ts'
export { decrypt, encrypt, encryptionKey } from './secrets.ts'
export type { TfInstance, TfModule, TfvarsFileStatus } from '../shared/tfvars.ts'
export type { TfInstance, TfModule, TfvarsFileStatus, TfvarsRevision } from '../shared/tfvars.ts'
export { STATE_SCHEMA_PATTERN, TFVARS_FILE_PATTERN, TF_NAME_PATTERN } from '../shared/tfvars.ts'
export { createTfvarsStore, tfvarsText, versionOf, type InstanceVariables, type TfvarsDocument, type TfvarsFile, type TfvarsStore } from './tfvarsStore.ts'
export {
createTfvarsStore,
tfvarsText,
versionOf,
type InstanceVariables,
type RevisionMeta,
type TfvarsDocument,
type TfvarsFile,
type TfvarsRevisionFiles,
type TfvarsStore,
} from './tfvarsStore.ts'
+58 -2
View File
@@ -1,6 +1,7 @@
-- The store of Terraform modules, their instances and the instances' variable files: the tables tf_modules,
-- tf_instances and tf_variable_files, and the functions and procedures the store calls. The store applies the file on
-- its first use in the schema its connection's search path names; every statement can run again.
-- tf_instances, tf_variable_files and tf_variable_revisions, and the functions and procedures the store calls. The
-- store applies the file on its first use in the schema its connection's search path names; every statement can run
-- again.
CREATE TABLE IF NOT EXISTS tf_modules (
name text PRIMARY KEY CHECK (name ~ '^[a-z][a-z0-9-]*$'),
@@ -28,6 +29,28 @@ CREATE TABLE IF NOT EXISTS tf_variable_files (
FOREIGN KEY (module, instance) REFERENCES tf_instances (module, name) ON DELETE CASCADE
);
-- Every write of an instance's variable files: all its files after the write, by name, the schema version of their
-- layout, null when not known, and the application that wrote them.
CREATE TABLE IF NOT EXISTS tf_variable_revisions (
module text NOT NULL,
instance text NOT NULL,
revision integer NOT NULL CHECK (revision > 0),
files jsonb NOT NULL CHECK (jsonb_typeof(files) = 'object'),
schema integer CHECK (schema > 0),
producer text NOT NULL CHECK (producer <> ''),
saved timestamptz NOT NULL DEFAULT now(),
PRIMARY KEY (module, instance, revision),
FOREIGN KEY (module, instance) REFERENCES tf_instances (module, name) ON DELETE CASCADE
);
-- The files of an instance that has no revision are its revision 1, of an unknown schema and producer.
INSERT INTO tf_variable_revisions (module, instance, revision, files, schema, producer, saved)
SELECT f.module, f.instance, 1, jsonb_object_agg(f.name, f.content), NULL, 'unknown', max(f.updated)
FROM tf_variable_files f
WHERE NOT EXISTS (SELECT 1 FROM tf_variable_revisions r WHERE r.module = f.module AND r.instance = f.instance)
GROUP BY f.module, f.instance
ON CONFLICT DO NOTHING;
-- The modules, by name.
CREATE OR REPLACE FUNCTION tf_modules_list()
RETURNS TABLE (name text, root text, description text, configurator_url text)
@@ -115,3 +138,36 @@ SET search_path FROM CURRENT
AS $$
DELETE FROM tf_variable_files WHERE module = module_name AND instance = instance_name AND name = file_name
$$;
-- Records the variable files of an instance as its next revision, with the schema of their layout and their producer;
-- called with the lock of tf_variable_files_lock held.
CREATE OR REPLACE PROCEDURE tf_variable_revision_add(module_name text, instance_name text, revision_schema integer, revision_producer text)
LANGUAGE sql
SET search_path FROM CURRENT
AS $$
INSERT INTO tf_variable_revisions (module, instance, revision, files, schema, producer)
SELECT module_name, instance_name,
(SELECT coalesce(max(r.revision), 0) + 1 FROM tf_variable_revisions r WHERE r.module = module_name AND r.instance = instance_name),
(SELECT coalesce(jsonb_object_agg(f.name, f.content), '{}'::jsonb) FROM tf_variable_files f WHERE f.module = module_name AND f.instance = instance_name),
revision_schema, revision_producer
$$;
-- The revisions of an instance, newest first, with the names of their files.
CREATE OR REPLACE FUNCTION tf_variable_revisions_list(module_name text, instance_name text)
RETURNS TABLE (revision integer, schema integer, producer text, saved timestamptz, names text[])
LANGUAGE sql STABLE
SET search_path FROM CURRENT
AS $$
SELECT r.revision, r.schema, r.producer, r.saved, ARRAY(SELECT jsonb_object_keys(r.files) ORDER BY 1)
FROM tf_variable_revisions r WHERE r.module = module_name AND r.instance = instance_name ORDER BY r.revision DESC
$$;
-- A revision of an instance with its files.
CREATE OR REPLACE FUNCTION tf_variable_revision_get(module_name text, instance_name text, revision_number integer)
RETURNS TABLE (revision integer, schema integer, producer text, saved timestamptz, files jsonb)
LANGUAGE sql STABLE
SET search_path FROM CURRENT
AS $$
SELECT r.revision, r.schema, r.producer, r.saved, r.files
FROM tf_variable_revisions r WHERE r.module = module_name AND r.instance = instance_name AND r.revision = revision_number
$$;
+41 -7
View File
@@ -60,29 +60,62 @@ test('instances are created and refused for a missing module or a name in use',
assert.equal(await status(store.modules.delete('router')), 409)
})
const META = { schema: 1, producer: 'Test Configurator 1.0.0' }
test('variable files are written with their version and refused against another version', async () => {
const variables = store.variables('router', 'test')
const empty = await variables.read()
assert.deepEqual(empty.files, [])
assert.equal(empty.version, '')
assert.equal(await variables.write(empty.version, { 'platform.tfvars.json': tfvarsText({ a: 1 }), 'secrets.tfvars.json': tfvarsText({ s: 'x' }) }), true)
assert.equal(empty.revision, null)
assert.equal(await variables.write(empty.version, { 'platform.tfvars.json': tfvarsText({ a: 1 }), 'secrets.tfvars.json': tfvarsText({ s: 'x' }) }, META), true)
const written = await variables.read()
assert.deepEqual(written.files.map((file) => [file.name, file.content.toString('utf8')]), [
['platform.tfvars.json', tfvarsText({ a: 1 })],
['secrets.tfvars.json', tfvarsText({ s: 'x' })],
])
assert.deepEqual(written.status.map((file) => file.previous), [false, false])
assert.equal(await variables.write(empty.version, { 'platform.tfvars.json': tfvarsText({ a: 2 }) }), false)
assert.equal(await variables.write(written.version, { 'platform.tfvars.json': tfvarsText({ a: 2 }) }), true)
assert.equal(await variables.write(empty.version, { 'platform.tfvars.json': tfvarsText({ a: 2 }) }, META), false)
assert.equal(await variables.write(written.version, { 'platform.tfvars.json': tfvarsText({ a: 2 }) }, META), true)
const changed = await variables.read()
assert.deepEqual(changed.status.map((file) => file.previous), [true, false])
assert.notEqual(changed.version, written.version)
assert.equal(await variables.remove(written.version, 'secrets.tfvars.json'), false)
assert.equal(await variables.remove(changed.version, 'secrets.tfvars.json'), true)
assert.equal(await variables.remove(written.version, 'secrets.tfvars.json', META), false)
assert.equal(await variables.remove(changed.version, 'secrets.tfvars.json', META), true)
assert.deepEqual((await variables.read()).files.map((file) => file.name), ['platform.tfvars.json'])
assert.deepEqual((await store.variables('router', 'platform').read()).files, [])
assert.equal(await status(variables.write((await variables.read()).version, { 'notes.json': tfvarsText({}) })), 400)
assert.equal(await status(store.variables('router', 'missing').write('', { 'a.tfvars.json': tfvarsText({}) })), 404)
assert.equal(await status(variables.write((await variables.read()).version, { 'notes.json': tfvarsText({}) }, META)), 400)
assert.equal(await status(store.variables('router', 'missing').write('', { 'a.tfvars.json': tfvarsText({}) }, META)), 404)
})
test('every write is a revision with its files, schema and producer', async () => {
const variables = store.variables('router', 'test')
assert.deepEqual((await variables.revisions()).map((revision) => [revision.revision, revision.schema, revision.producer, revision.names]), [
[3, 1, META.producer, ['platform.tfvars.json']],
[2, 1, META.producer, ['platform.tfvars.json', 'secrets.tfvars.json']],
[1, 1, META.producer, ['platform.tfvars.json', 'secrets.tfvars.json']],
])
const current = await variables.read()
assert.equal(await variables.write(current.version, {}, META), true)
assert.equal(await variables.write(current.version, { 'platform.tfvars.json': tfvarsText({ a: 3 }) }, { schema: null, producer: 'manage.mjs import' }), true)
const newest = (await variables.read()).revision
assert.deepEqual(newest && [newest.revision, newest.schema, newest.producer], [4, null, 'manage.mjs import'])
const first = await variables.revision(1)
assert.deepEqual(first?.files.map((file) => [file.name, file.content.toString('utf8')]), [
['platform.tfvars.json', tfvarsText({ a: 1 })],
['secrets.tfvars.json', tfvarsText({ s: 'x' })],
])
assert.equal(await variables.revision(9), null)
assert.deepEqual(await store.variables('router', 'platform').revisions(), [])
})
test('the files of an instance without revisions become its revision 1 of an unknown producer', async () => {
await pool.query(`INSERT INTO tf_variable_files (module, instance, name, content) VALUES ('router', 'platform', 'a.tfvars.json', '{"b": 1}')`)
await pool.query('DELETE FROM tf_variable_revisions')
const again = createTfvarsStore(pool)
const revisions = await again.variables('router', 'platform').revisions()
assert.deepEqual(revisions.map((revision) => [revision.revision, revision.schema, revision.producer, revision.names]), [[1, null, 'unknown', ['a.tfvars.json']]])
assert.deepEqual((await again.variables('router', 'test').revisions()).map((revision) => revision.revision), [1])
})
test('an instance is deleted with its variable files, then its module', async () => {
@@ -90,6 +123,7 @@ test('an instance is deleted with its variable files, then its module', async ()
await store.instances.delete('router', 'platform')
assert.deepEqual(await store.instances.list('router'), [])
assert.equal((await pool.query('SELECT count(*)::int AS n FROM tf_variable_files')).rows[0].n, 0)
assert.equal((await pool.query('SELECT count(*)::int AS n FROM tf_variable_revisions')).rows[0].n, 0)
await store.modules.delete('router')
assert.deepEqual(await store.modules.list(), [])
})
+81 -17
View File
@@ -1,25 +1,42 @@
import { createHash } from 'node:crypto'
import { readFileSync } from 'node:fs'
import type { Pool, PoolClient } from 'pg'
import type { TfInstance, TfModule, TfvarsFileStatus } from '../shared/tfvars.ts'
import type { TfInstance, TfModule, TfvarsFileStatus, TfvarsRevision } from '../shared/tfvars.ts'
import { HttpError } from './http.ts'
/** A variable file of an instance: its name and its content, the object jsonb keeps written as JSON text. */
export type TfvarsFile = { name: string; content: Buffer }
/** The variable files of an instance, by name, with their version, versionOf the files. */
export type TfvarsDocument = { files: TfvarsFile[]; status: TfvarsFileStatus[]; version: string }
/**
* The variable files of an instance, by name, with their version, versionOf the files, and their newest revision, null
* for an instance never written.
*/
export type TfvarsDocument = { files: TfvarsFile[]; status: TfvarsFileStatus[]; version: string; revision: TfvarsRevision | null }
/** A revision with its files, by name. */
export type TfvarsRevisionFiles = TfvarsRevision & { files: TfvarsFile[] }
/** What a write records with the files: the schema version of their layout, null when not known, and the producer. */
export type RevisionMeta = { schema: number | null; producer: string }
/** The variable files of one instance. */
export type InstanceVariables = {
read(): Promise<TfvarsDocument>
/**
* Writes the given files, name and JSON text of an object, when the files are still those of version, keeping each
* replaced content as its previous one; answers false, writing nothing, when they changed since.
* replaced content as its previous one, and records all files as the next revision with meta; answers false, writing
* nothing, when they changed since. A write of no file records no revision.
*/
write(version: string, contents: Record<string, string>): Promise<boolean>
/** Deletes the file name when the files are still those of version; answers false, deleting nothing, otherwise. */
remove(version: string, name: string): Promise<boolean>
write(version: string, contents: Record<string, string>, meta: RevisionMeta): Promise<boolean>
/**
* Deletes the file name when the files are still those of version, and records the remaining files as the next
* revision with meta; answers false, deleting nothing, otherwise.
*/
remove(version: string, name: string, meta: RevisionMeta): Promise<boolean>
/** The revisions, newest first. */
revisions(): Promise<TfvarsRevision[]>
/** The revision of the number with its files; null when there is none. */
revision(revision: number): Promise<TfvarsRevisionFiles | null>
}
/** The modules, the instances and their variable files in PostgreSQL. */
@@ -56,6 +73,15 @@ const SQL = new URL('./sql/tfvars_store.sql', import.meta.url)
type FileRow = { name: string; content: object; previous: boolean; updated: Date }
type RevisionRow = { revision: number; schema: number | null; producer: string; saved: Date; names: string[] }
const revisionOf = (row: Omit<RevisionRow, 'names'>, names: string[]): TfvarsRevision =>
({ revision: row.revision, schema: row.schema, producer: row.producer, saved: row.saved.toISOString(), names })
/** The files of an object of revision files, by name, written as JSON text. */
const filesOf = (files: Record<string, object>): TfvarsFile[] =>
Object.keys(files).sort().map((name) => ({ name, content: Buffer.from(tfvarsText(files[name]), 'utf8') }))
/**
* The store of sql/tfvars_store.sql in the database of pool, applied at the first use in the schema of the
* connection's search path. A refused change is an HttpError: 409 for a name in use or a module that has instances,
@@ -72,27 +98,49 @@ export function createTfvarsStore(pool: Pool): TfvarsStore {
throw refused(error)
}
}
const document = (rows: FileRow[]): TfvarsDocument => {
const document = (rows: FileRow[], newest: RevisionRow | undefined): TfvarsDocument => {
const files = rows.map((row) => ({ name: row.name, content: Buffer.from(tfvarsText(row.content), 'utf8') }))
return {
files,
status: rows.map((row, index) => ({ name: row.name, bytes: files[index].content.length, modified: row.updated.toISOString(), previous: row.previous })),
version: versionOf(files),
revision: newest === undefined ? null : revisionOf(newest, newest.names),
}
}
/** Runs change in a transaction that holds the lock of the variable files, when they are still those of version. */
const locked = async (module: string, instance: string, version: string, change: (client: PoolClient) => Promise<void>) => {
/** The files and the newest revision of an instance, read in one snapshot. */
const read = async (module: string, instance: string): Promise<TfvarsDocument> => {
await ready()
const client = await pool.connect()
try {
await client.query('BEGIN ISOLATION LEVEL REPEATABLE READ READ ONLY')
const files = await client.query<FileRow>('SELECT * FROM tf_variable_files_read($1, $2)', [module, instance])
const revisions = await client.query<RevisionRow>('SELECT * FROM tf_variable_revisions_list($1, $2) LIMIT 1', [module, instance])
await client.query('COMMIT')
return document(files.rows, revisions.rows[0])
} catch (error) {
await client.query('ROLLBACK')
throw refused(error)
} finally {
client.release()
}
}
/**
* Runs change in a transaction that holds the lock of the variable files, when they are still those of version, and
* records the files after it as the next revision with meta.
*/
const locked = async (module: string, instance: string, version: string, meta: RevisionMeta, change: (client: PoolClient) => Promise<void>) => {
await ready()
const client = await pool.connect()
try {
await client.query('BEGIN')
await client.query('CALL tf_variable_files_lock()')
const { rows } = await client.query<FileRow>('SELECT * FROM tf_variable_files_read($1, $2)', [module, instance])
if (document(rows).version !== version) {
if (document(rows, undefined).version !== version) {
await client.query('ROLLBACK')
return false
}
await change(client)
await client.query('CALL tf_variable_revision_add($1, $2, $3, $4)', [module, instance, meta.schema, meta.producer])
await client.query('COMMIT')
return true
} catch (error) {
@@ -124,15 +172,31 @@ export function createTfvarsStore(pool: Pool): TfvarsStore {
},
},
variables: (module, instance) => ({
read: async () => document(await query<FileRow>('SELECT * FROM tf_variable_files_read($1, $2)', [module, instance])),
write: (version, contents) => locked(module, instance, version, async (client) => {
for (const [name, text] of Object.entries(contents)) {
await client.query('CALL tf_variable_file_put($1, $2, $3, $4::jsonb)', [module, instance, name, text])
read: () => read(module, instance),
write: async (version, contents, meta) => {
if (Object.keys(contents).length === 0) {
return (await read(module, instance)).version === version
}
}),
remove: (version, name) => locked(module, instance, version, async (client) => {
return locked(module, instance, version, meta, async (client) => {
for (const [name, text] of Object.entries(contents)) {
await client.query('CALL tf_variable_file_put($1, $2, $3, $4::jsonb)', [module, instance, name, text])
}
})
},
remove: (version, name, meta) => locked(module, instance, version, meta, async (client) => {
await client.query('CALL tf_variable_file_delete($1, $2, $3)', [module, instance, name])
}),
revisions: async () => (await query<RevisionRow>('SELECT * FROM tf_variable_revisions_list($1, $2)', [module, instance]))
.map((row) => revisionOf(row, row.names)),
revision: async (revision) => {
const [row] = await query<Omit<RevisionRow, 'names'> & { files: Record<string, object> }>(
'SELECT * FROM tf_variable_revision_get($1, $2, $3)', [module, instance, revision])
if (row === undefined) {
return null
}
const files = filesOf(row.files)
return { ...revisionOf(row, files.map((file) => file.name)), files }
},
}),
}
}
+3
View File
@@ -0,0 +1,3 @@
export type { CredentialField, CredentialKind, CredentialProfile, CredentialsDocument } from './credentials.ts'
export type { TfInstance, TfModule, TfvarsFileStatus, TfvarsRevision } from './tfvars.ts'
export { STATE_SCHEMA_PATTERN, TFVARS_FILE_PATTERN, TF_NAME_PATTERN } from './tfvars.ts'
+14
View File
@@ -26,6 +26,20 @@ export type TfvarsFileStatus = {
previous: boolean
}
/** A write of an instance's variable files, without their content. */
export type TfvarsRevision = {
/** The number of the write, from 1 for each instance. */
revision: number
/** The schema version of the files' layout, as their producer gave it; null when not known. */
schema: number | null
/** The application that wrote the files, with its version. */
producer: string
/** ISO 8601. */
saved: string
/** The names of the files of the revision. */
names: string[]
}
/** The names of modules and instances: a lower-case letter, then lower-case letters, digits and hyphens. */
export const TF_NAME_PATTERN = /^[a-z][a-z0-9-]*$/
+87
View File
@@ -0,0 +1,87 @@
#!/usr/bin/env node
// Runs the checks of the toolkit with Docker, on the engine of the docker CLI's context: builds test.Dockerfile from
// the files of the checkout git knows, then runs the tests against a scratch PostgreSQL on a private network.
import { spawnSync } from 'node:child_process';
import { resolve } from 'node:path';
import { parseArgs } from 'node:util';
const NAME = 'sk-react-toolkit-test';
const POSTGRES_IMAGE = 'postgres:18';
const usage = `Usage: test-container.mjs [-h|--help] [--package <folder>]
Builds test.Dockerfile, which runs the typecheck, the lint and the package build, and runs the
tests in a container with a scratch PostgreSQL. The build context is the files git tracks and the
untracked files it does not ignore.
--package <folder> write the packed toolkit, the .tgz file npm pack makes, to the folder instead
of running the tests`;
const { values } = parseArgs({
options: {
package: { type: 'string' },
help: { type: 'boolean', short: 'h', default: false },
},
});
if (values.help) {
console.log(usage);
process.exit(0);
}
function fail(message) {
console.error(`test-container: ${message}`);
process.exit(1);
}
// Runs docker, failing with its message when it cannot start; answers the result.
function docker(args, options = {}) {
const result = spawnSync('docker', args, { stdio: 'inherit', ...options });
if (result.error) fail(`docker: ${result.error.message}`);
return result;
}
const files = spawnSync('git', ['ls-files', '-z', '--cached', '--others', '--exclude-standard'], { cwd: import.meta.dirname, maxBuffer: 1 << 26 });
if (files.status !== 0) fail(`git ls-files failed: ${files.stderr}`);
// macOS tar adds extended attributes and AppleDouble files, which a Linux engine refuses.
const macOptions = process.platform === 'darwin' ? ['--no-xattrs', '--no-mac-metadata'] : [];
const context = spawnSync('tar', [...macOptions, '--null', '-T', '-', '-cf', '-'], { cwd: import.meta.dirname, input: files.stdout, maxBuffer: 1 << 30 });
if (context.status !== 0) fail(`tar failed: ${context.stderr}`);
if (values.package !== undefined) {
const output = `type=local,dest=${resolve(values.package)}`;
const built = docker(['build', '-f', 'test.Dockerfile', '--target', 'package', '--output', output, '-'], { input: context.stdout, stdio: ['pipe', 'inherit', 'inherit'] });
process.exit(built.status ?? 1);
}
const built = docker(['build', '-f', 'test.Dockerfile', '--target', 'check', '-t', NAME, '-'], { input: context.stdout, stdio: ['pipe', 'inherit', 'inherit'] });
if (built.status !== 0) process.exit(built.status ?? 1);
// Ctrl-C stops the running docker command; the cleanup below still runs.
process.on('SIGINT', () => {});
const network = `${NAME}-${process.pid}`;
const database = `${NAME}-pg-${process.pid}`;
let status = 1;
try {
if (docker(['network', 'create', network], { stdio: 'ignore' }).status !== 0) throw new Error(`cannot create the network ${network}`);
const started = docker(['run', '-d', '--rm', '--name', database, '--network', network, '-e', 'POSTGRES_PASSWORD=test', POSTGRES_IMAGE], { stdio: ['ignore', 'ignore', 'inherit'] });
if (started.status !== 0) throw new Error(`cannot start ${POSTGRES_IMAGE}`);
// The server answers on TCP once its initialization has finished.
let ready = false;
for (let attempt = 0; attempt < 60 && !ready; attempt++) {
ready = docker(['exec', database, 'pg_isready', '-q', '-h', '127.0.0.1', '-U', 'postgres'], { stdio: 'ignore' }).status === 0;
if (!ready) Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, 1000);
}
if (!ready) throw new Error('PostgreSQL did not start within 60 seconds');
const environment = ['PGHOST=' + database, 'PGUSER=postgres', 'PGPASSWORD=test', 'PGDATABASE=postgres'].flatMap((variable) => ['-e', variable]);
status = docker(['run', '--rm', '--network', network, ...environment, NAME]).status ?? 1;
} catch (error) {
console.error(`test-container: ${error.message}`);
} finally {
docker(['rm', '-f', database], { stdio: 'ignore' });
docker(['network', 'rm', network], { stdio: 'ignore' });
}
process.exit(status);
+16
View File
@@ -0,0 +1,16 @@
# The toolkit's checks on Linux: the build runs the typecheck, the lint and the package build, the
# container the tests; the package stage holds the packed toolkit for the tests of an application.
ARG NODE_VERSION=26
FROM node:${NODE_VERSION}-trixie-slim AS check
WORKDIR /src
COPY package.json package-lock.json ./
RUN npm ci --no-fund --no-audit
COPY . .
RUN npm run typecheck && npm run lint && npm run build
RUN mkdir /pack && npm pack ./dist --pack-destination /pack
CMD ["npm", "test"]
FROM scratch AS package
COPY --from=check /pack/ /
+25
View File
@@ -38,6 +38,31 @@ export function font(choice: Font): Plugin {
}
}
/**
* Sets the color scheme of appTheme on the page before its first paint, so a dark page, and a page in an iframe of a
* dark page, is not white until the application renders: adds to index.html a script that sets the class dark or
* light on <html>, from MUI's mui-mode in localStorage or, for system, from prefers-color-scheme, and a style that
* gives those classes their color-scheme and the dark scheme MUI's dark background.
*/
export function colorScheme(): Plugin {
const script = `(function () {
try {
var mode = localStorage.getItem('mui-mode') || 'system';
var dark = mode === 'dark' || (mode === 'system' && window.matchMedia('(prefers-color-scheme: dark)').matches);
document.documentElement.classList.add(dark ? 'dark' : 'light');
} catch (e) {}
})();`
return {
name: 'color-scheme',
transformIndexHtml() {
return [
{ tag: 'style', children: 'html.light { color-scheme: light; }\nhtml.dark { color-scheme: dark; background-color: #121212; }', injectTo: 'head' },
{ tag: 'script', children: script, injectTo: 'head' },
]
},
}
}
/** VITE_FONT of env, the variables Vite's loadEnv answers, cdn when it is not set; another value throws. */
export function fontChoice(env: Record<string, string>): Font {
const choice = env.VITE_FONT ?? 'cdn'
+10 -5
View File
@@ -6,12 +6,16 @@ import { useValidation } from './useValidation.ts'
/**
* A dialog that edits one row. The server validates the value as it changes (validatePath); onSave writes it and
* the dialog closes when it succeeds. fields renders the inputs from the value, a setter and the problems by field;
* below shows extra information from the validation, such as subnet placements.
* below shows extra information from the validation, such as subnet placements. body gives the value posted to
* validatePath, such as a whole document with the row put in, and select the problems by field and the message of
* the row taken from the answer; without them the row itself is posted and the answer is its validation.
*/
export default function RowDialog<T, V extends Validation = Validation>(props: {
export default function RowDialog<T, V = Validation>(props: {
title: string
initial: T
validatePath: string
body?: (value: T) => unknown
select?: (result: V) => Validation
saveLabel: string
onSave: (value: T) => Promise<void>
onCancel: () => void
@@ -21,9 +25,10 @@ export default function RowDialog<T, V extends Validation = Validation>(props: {
const [value, setValue] = useState<T>(props.initial)
const [saving, setSaving] = useState(false)
const [error, setError] = useState<string | null>(null)
const validation = useValidation<V>(props.validatePath, value)
const problems = validation.result?.problems ?? {}
const message = validation.result?.message ?? ''
const validation = useValidation<V>(props.validatePath, props.body ? props.body(value) : value)
const selected = validation.result === null ? null : props.select ? props.select(validation.result) : validation.result as Validation
const problems = selected?.problems ?? {}
const message = selected?.message ?? ''
const blocked = validation.pending || Object.keys(problems).length > 0 || message !== ''
async function submit(event: FormEvent) {
+1 -1
View File
@@ -5,7 +5,7 @@ import { request, type Validation } from './api.ts'
* Asks the server to validate value at path (a .../validate endpoint) 250 ms after it last
* changed. pending is true until the answer for the current value arrives; until then the previous result stays.
*/
export function useValidation<T extends Validation>(path: string | null, value: unknown): { result: T | null; error: string; pending: boolean } {
export function useValidation<T = Validation>(path: string | null, value: unknown): { result: T | null; error: string; pending: boolean } {
const key = path === null ? '' : `${path}\n${JSON.stringify(value)}`
const [answer, setAnswer] = useState<{ key: string; result: T | null; error: string } | null>(null)