From d0a652d673e564354097ee1a3c2085a406daf157 Mon Sep 17 00:00:00 2001 From: Kuba Sunderland-Ober Date: Fri, 25 Sep 2026 04:11:09 +0200 Subject: [PATCH 01/53] scripts: survey_tooling.mjs, a before-and-after measure of the tooling --- docs/Documentation/Tools.md | 11 + scripts/survey_tooling.mjs | 431 ++++++++++++++++++++++++++++++++++++ 2 files changed, 442 insertions(+) create mode 100644 scripts/survey_tooling.mjs diff --git a/docs/Documentation/Tools.md b/docs/Documentation/Tools.md index 64945db2..745f01fb 100644 --- a/docs/Documentation/Tools.md +++ b/docs/Documentation/Tools.md @@ -558,6 +558,17 @@ It shares [`census_attributes.mjs`](#census-attributes)'s export and cache, and Normalises literal en-dash / em-dash characters in markdown source under `docs/` to the ASCII source forms markdown-it's typographer converts at build time (`--` for en-dash, `---` for em-dash). The site forbids literal `–` / `—` in source --- this is the canonical fixer if any slip back in. Skips fenced code blocks and inline code spans, and preserves each file's existing line endings. `--check` reports what it would change and exits non-zero without writing, so it can serve as a gate. +### survey_tooling.mjs +{: #survey-tooling } + + node scripts/survey_tooling.mjs # the summary, then every listing + node scripts/survey_tooling.mjs --summary # the summary only + node scripts/survey_tooling.mjs --root # measure another checkout + +Measures the repository's own tooling for repetition and structure: code duplicated between files, found token by token so that two copies differing only in names still match; top-level functions defined under one name in several files; how the command-line tools read their arguments; packages imported without being declared in `package.json`; and the import graph --- the imports that cross from one directory to another, the files nothing imports, and the most imported modules. `builder/PLAN-TOOLING-REVIEW.md` records its summary at the commit the tooling review started from, and the review's last phase runs it again to compare. + +It is not a gate, and nothing runs it: take a measurement before and after a piece of refactoring. It reads only the files git tracks, so a scratch file never changes a number. `--root` measures another checkout, such as a worktree at an older commit that does not contain the script. `perf/` is measured, but it is counted separately in the summary and left out of the listings unless `--include-perf` is given. Exits 0, or 2 on a bad argument or a folder that is not a git checkout. + ### tbbuild.mjs {: #tbbuild } diff --git a/scripts/survey_tooling.mjs b/scripts/survey_tooling.mjs new file mode 100644 index 00000000..1e025833 --- /dev/null +++ b/scripts/survey_tooling.mjs @@ -0,0 +1,431 @@ +#!/usr/bin/env node +// Measure the repository's own tooling for repetition and structure. +// +// node scripts/survey_tooling.mjs # the summary, then every listing +// node scripts/survey_tooling.mjs --summary # the summary only +// node scripts/survey_tooling.mjs --root # measure another checkout +// node scripts/survey_tooling.mjs --top 300 # list more clone regions +// node scripts/survey_tooling.mjs --include-perf # list what involves perf/ too +// +// Not a gate, and nothing runs it. It is a measurement taken by hand before and +// after a piece of refactoring work: builder/PLAN-TOOLING-REVIEW.md records its +// summary at the commit the tooling review started from, and the review's last +// phase runs it again to compare. It reads only the files git tracks, so a +// scratch file never moves a number; and --root lets this copy of the script +// measure a checkout that does not contain it, such as a worktree at that +// baseline commit. +// +// ---------------------------------------------------------- what it measures +// +// Clones. A token-level detector in the manner of PMD's CPD. Each file is +// tokenised with acorn, and identifiers and literals are reduced to their kind, +// so two copies that differ only in names or strings still match. Every run of +// --window equal tokens that occurs in two places is grown into the longest +// region the two places share. Comments and layout are not tokens, so a +// formatter pass moves nothing here. A window that occurs in more than +// MAX_OCCURRENCES places is skipped as boilerplate, such as an import block, +// rather than reported as that many copies of each other. +// +// Top-level functions defined under the same name in two or more files. A +// lead, not a verdict: two `pad` helpers may do different things. Two `opt` +// helpers that nearly agree are how the review found one argument parser in +// three versions. +// +// Command-line handling: the files that read process.argv, how many take +// node:util's parseArgs, and how many define a private flag/opt/die helper. +// +// Undeclared packages: bare import specifiers, other than Node's own modules, +// that package.json does not declare. Such a package is installed only because +// something else depends on it, and an update of that something can remove it. +// +// The import graph: edges that cross a top-level directory, the files nothing +// imports (entry points, scripts injected into a page, or dead code), and the +// most imported modules. +// +// perf/ is a lab notebook (decision 1 in PLAN-TOOLING-REVIEW.md): it is +// measured, but the summary counts it separately and the listings leave it out +// unless --include-perf is given, because nothing in it is maintained. + +import { execFileSync } from "node:child_process"; +import { readFileSync } from "node:fs"; +import { builtinModules } from "node:module"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; +import { parseArgs } from "node:util"; +import * as acorn from "acorn"; +import * as walk from "acorn-walk"; + +const TOOLING_DIRS = ["builder", "scripts", "book", "eval", "wisdom", "test", "perf"]; +const VENDORED = [/^book\/lib\/paged\.browser\.js$/, /^builder\/vendor\//]; +const LAB = "perf/"; +const MAX_OCCURRENCES = 40; +const ARG_HELPERS = new Set(["flag", "opt", "die"]); + +const USAGE = "usage: node scripts/survey_tooling.mjs [--root DIR] [--summary] " + + "[--window N] [--top N] [--include-perf]"; + +let opts; +try { + ({ values: opts } = parseArgs({ + options: { + root: { type: "string" }, + summary: { type: "boolean", default: false }, + window: { type: "string", default: "60" }, + top: { type: "string", default: "45" }, + "include-perf": { type: "boolean", default: false }, + help: { type: "boolean", default: false }, + }, + })); +} catch (err) { + console.error(`${err.message}\n${USAGE}`); + process.exit(2); +} +if (opts.help) { + console.log(USAGE); + process.exit(0); +} +const WINDOW = positiveInt("window", opts.window); +const TOP = positiveInt("top", opts.top); +const ROOT = path.resolve(opts.root ?? fileURLToPath(new URL("..", import.meta.url))); +const listed = (f) => opts["include-perf"] || !f.startsWith(LAB); + +function positiveInt(name, raw) { + const n = Number(raw); + if (!Number.isInteger(n) || n < 1) { + console.error(`--${name} expects a positive integer, got: ${raw}\n${USAGE}`); + process.exit(2); + } + return n; +} + +function gitFiles(...args) { + try { + return execFileSync("git", ["ls-files", ...args], { cwd: ROOT, encoding: "utf8", stdio: ["ignore", "pipe", "pipe"] }) + .split("\n").filter(Boolean); + } catch (err) { + const reason = String(err.stderr ?? "").trim() || err.message; + console.error(`cannot list the files git tracks under ${ROOT}: ${reason}`); + process.exit(2); + } +} + +// Top-level directory, with scripts/lib/ told apart from the tools that use it. +const areaOf = (f) => (f.startsWith("scripts/lib/") ? "scripts/lib" : f.split("/")[0]); + +// ------------------------------------------------------------------ parsing + +const files = gitFiles("--", ...TOOLING_DIRS) + .filter((f) => /\.(mjs|js|cjs)$/.test(f) && !VENDORED.some((re) => re.test(f))); + +const ACORN_OPTIONS = { + ecmaVersion: "latest", allowHashBang: true, allowAwaitOutsideFunction: true, + allowReturnOutsideFunction: true, locations: true, +}; + +const parsed = new Map(); // file -> { tokens: [{ kind, line, text }], ast } +for (const f of files) { + const src = readFileSync(path.join(ROOT, f), "utf8"); + // A page script injected into a browser is not a module; try it as both. + let ast = null; + let sourceType = null; + for (const type of ["module", "script"]) { + try { + ast = acorn.parse(src, { ...ACORN_OPTIONS, sourceType: type }); + sourceType = type; + break; + } catch { + // try the other source type + } + } + if (!ast) { + console.error(`skipped, acorn cannot parse it: ${f}`); + continue; + } + const tokens = []; + for (const tok of acorn.tokenizer(src, { ...ACORN_OPTIONS, sourceType })) { + tokens.push({ kind: tokenKind(tok), line: tok.loc.start.line, text: src.slice(tok.start, tok.end) }); + } + parsed.set(f, { tokens, ast }); +} + +// Identifiers and literals compare by kind only; keywords and punctuation as themselves. +function tokenKind(tok) { + switch (tok.type.label) { + case "name": return "name"; + case "privateId": return "name"; + case "string": return "string"; + case "num": return "number"; + case "template": return "template"; + case "regexp": return "regexp"; + default: return tok.type.keyword ?? tok.type.label; + } +} + +// ------------------------------------------------------------------- clones + +const kindIds = new Map(); +const seqs = new Map(); // file -> Int32Array of token kinds +for (const [f, p] of parsed) { + seqs.set(f, Int32Array.from(p.tokens, (t) => { + let id = kindIds.get(t.kind); + if (id === undefined) kindIds.set(t.kind, (id = kindIds.size + 1)); + return id; + })); +} +const order = new Map([...parsed.keys()].map((f, i) => [f, i])); + +function findClones() { + // A polynomial rolling hash over every window; windows that share a hash are + // candidate pairs, confirmed token by token before a region is grown. + const B = 1000003; + let bPowWindow = 1; + for (let i = 0; i < WINDOW; i++) bPowWindow = Math.imul(bPowWindow, B) >>> 0; + const buckets = new Map(); + for (const [f, s] of seqs) { + let h = 0; + for (let i = 0; i < s.length; i++) { + h = (Math.imul(h, B) + s[i]) >>> 0; + if (i >= WINDOW) h = (h - Math.imul(s[i - WINDOW], bPowWindow)) >>> 0; + if (i < WINDOW - 1) continue; + let occurrences = buckets.get(h); + if (!occurrences) buckets.set(h, (occurrences = [])); + occurrences.push([f, i - WINDOW + 1]); + } + } + + const regions = []; + const grown = new Set(); + for (const occurrences of buckets.values()) { + if (occurrences.length < 2 || occurrences.length > MAX_OCCURRENCES) continue; + for (let x = 0; x < occurrences.length; x++) { + for (let y = x + 1; y < occurrences.length; y++) { + const region = grow(occurrences[x], occurrences[y], grown); + if (region) regions.push(region); + } + } + } + return regions.sort((p, q) => q.length - p.length || order.get(p.a) - order.get(q.a) || p.aStart - q.aStart); +} + +// Grow the region two equal windows belong to, once per region: every window +// inside it backs up to the same start, and `grown` remembers that start. +function grow([fa, ia], [fb, ib], grown) { + if (order.get(fa) > order.get(fb) || (fa === fb && ia > ib)) [fa, ia, fb, ib] = [fb, ib, fa, ia]; + if (fa === fb && ib - ia < WINDOW) return null; // a window overlapping itself + const a = seqs.get(fa); + const b = seqs.get(fb); + for (let k = 0; k < WINDOW; k++) if (a[ia + k] !== b[ib + k]) return null; // a hash collision + let back = 0; + while (ia - back > 0 && ib - back > 0 && a[ia - back - 1] === b[ib - back - 1]) back++; + const aStart = ia - back; + const bStart = ib - back; + const key = `${fa}:${aStart}|${fb}:${bStart}`; + if (grown.has(key)) return null; + grown.add(key); + let length = 0; + while (aStart + length < a.length && bStart + length < b.length && a[aStart + length] === b[bStart + length]) length++; + // Code that repeats within itself matches its own continuation; keep the two + // regions from overlapping. + if (fa === fb) length = Math.min(length, bStart - aStart); + const ta = parsed.get(fa).tokens; + const tb = parsed.get(fb).tokens; + return { + a: fa, b: fb, length, aStart, + aLines: [ta[aStart].line, ta[aStart + length - 1].line], + bLines: [tb[bStart].line, tb[bStart + length - 1].line], + head: ta.slice(aStart, aStart + 14).map((t) => t.text).join(" "), + }; +} + +// ------------------------------------------------- declarations and imports + +const functionDefs = new Map(); // name -> [{ file, line, lines }] +const argvReaders = new Set(); +const parseArgsUsers = new Set(); +const argHelperFiles = new Set(); +const importers = new Map([...parsed.keys()].map((f) => [f, new Set()])); +const edges = []; +const bareImports = new Map(); // package -> Set of files + +for (const [f, { ast }] of parsed) { + for (const statement of ast.body) { + const d = statement.type.startsWith("Export") ? statement.declaration : statement; + if (!d) continue; + if (d.type === "FunctionDeclaration" && d.id) addFunction(d.id.name, f, d); + if (d.type === "VariableDeclaration") { + for (const v of d.declarations) { + if (v.id.type === "Identifier" && /Function/.test(v.init?.type ?? "")) addFunction(v.id.name, f, v); + } + } + } + walk.full(ast, (node) => { + if (node.type === "MemberExpression" && node.object.name === "process" && node.property.name === "argv") { + argvReaders.add(f); + } + if (node.type === "ImportDeclaration" && /^(node:)?util$/.test(node.source.value) && + node.specifiers.some((s) => s.imported?.name === "parseArgs")) { + parseArgsUsers.add(f); + } + const specifier = moduleSpecifier(node); + if (typeof specifier === "string") addImport(f, specifier); + }); +} + +function addFunction(name, file, node) { + const defs = functionDefs.get(name) ?? []; + defs.push({ file, line: node.loc.start.line, lines: node.loc.end.line - node.loc.start.line + 1 }); + functionDefs.set(name, defs); + if (ARG_HELPERS.has(name)) argHelperFiles.add(file); +} + +// The module a node loads, if it loads one by a literal name: import and +// export ... from, import(), require(), and new Worker(new URL(...)). +function moduleSpecifier(node) { + switch (node.type) { + case "ImportDeclaration": + case "ExportAllDeclaration": + case "ExportNamedDeclaration": + return node.source?.value; + case "ImportExpression": + return node.source.type === "Literal" ? node.source.value : undefined; + case "CallExpression": + return node.callee.name === "require" && node.arguments[0]?.type === "Literal" + ? node.arguments[0].value : undefined; + case "NewExpression": { + const url = node.callee.name === "Worker" ? node.arguments[0] : undefined; + return url?.type === "NewExpression" && url.arguments[0]?.type === "Literal" + ? url.arguments[0].value : undefined; + } + default: + return undefined; + } +} + +function addImport(file, specifier) { + if (specifier.startsWith(".")) { + const target = path.posix.normalize(path.posix.join(path.posix.dirname(file), specifier)); + edges.push([file, target]); + importers.get(target)?.add(file); + return; + } + if (specifier.startsWith("node:") || specifier.startsWith("/")) return; + const parts = specifier.split("/"); + const pkg = specifier.startsWith("@") ? parts.slice(0, 2).join("/") : parts[0]; + if (builtinModules.includes(pkg)) return; + if (!bareImports.has(pkg)) bareImports.set(pkg, new Set()); + bareImports.get(pkg).add(file); +} + +const manifest = JSON.parse(readFileSync(path.join(ROOT, "package.json"), "utf8")); +const declared = new Set(["dependencies", "devDependencies", "optionalDependencies", "peerDependencies"] + .flatMap((field) => Object.keys(manifest[field] ?? {}))); +const undeclared = [...bareImports].filter(([pkg]) => !declared.has(pkg)); + +// ------------------------------------------------------------------ summary + +const outsideLab = (f) => !f.startsWith(LAB); +const clones = findClones(); +const repeatedNames = [...functionDefs] + .filter(([, defs]) => new Set(defs.map((d) => d.file)).size >= 2); +const repeatedOutsideLab = repeatedNames + .filter(([, defs]) => new Set(defs.filter((d) => outsideLab(d.file)).map((d) => d.file)).size >= 2); +const toolsOutsideLab = [...argvReaders].filter(outsideLab); +const totalTokens = [...seqs.values()].reduce((sum, s) => sum + s.length, 0); + +const summary = [ + ["files surveyed", files.length], + ["tokens", totalTokens], + [`clone regions of ${WINDOW}+ tokens`, clones.length], + [" not involving perf/", clones.filter((c) => outsideLab(c.a) && outsideLab(c.b)).length], + ["top-level function names defined in 2+ files", repeatedNames.length], + [" counting only files outside perf/", repeatedOutsideLab.length], + ["files outside perf/ that read process.argv", toolsOutsideLab.length], + [" of which import node:util parseArgs", toolsOutsideLab.filter((f) => parseArgsUsers.has(f)).length], + ["files outside perf/ defining flag, opt or die", [...argHelperFiles].filter(outsideLab).length], + ["undeclared packages imported outside perf/", undeclared.filter(([, fs]) => [...fs].some(outsideLab)).length], +]; + +console.log(`# Tooling survey: ${ROOT}\n`); +const width = Math.max(...summary.map(([label]) => label.length)); +for (const [label, value] of summary) console.log(`${label.padEnd(width)} ${value}`); +if (opts.summary) process.exit(0); + +// ----------------------------------------------------------------- listings + +console.log("\n## Clone regions by pair of areas (tokens counted once per region)\n"); +const byPair = new Map(); +for (const c of clones) { + const key = [areaOf(c.a), areaOf(c.b)].sort().join(" <-> "); + const agg = byPair.get(key) ?? { regions: 0, tokens: 0 }; + agg.regions++; + agg.tokens += c.length; + byPair.set(key, agg); +} +for (const [key, agg] of [...byPair].sort((p, q) => q[1].tokens - p[1].tokens)) { + console.log(`${String(agg.regions).padStart(5)} regions ${String(agg.tokens).padStart(7)} tokens ${key}`); +} + +console.log("\n## Files most involved in clone regions (tokens, counted on each side)\n"); +const byFile = new Map(); +for (const c of clones) for (const f of [c.a, c.b]) byFile.set(f, (byFile.get(f) ?? 0) + c.length); +for (const [f, n] of [...byFile].filter(([f]) => listed(f)).sort((p, q) => q[1] - p[1]).slice(0, 30)) { + console.log(`${String(n).padStart(7)} ${f} (of ${seqs.get(f).length})`); +} + +const shownClones = clones.filter((c) => listed(c.a) && listed(c.b)); +console.log(`\n## The ${Math.min(TOP, shownClones.length)} largest of ${shownClones.length} clone regions\n`); +for (const c of shownClones.slice(0, TOP)) { + console.log(`${String(c.length).padStart(5)} tokens ${c.a}:${c.aLines.join("-")} ~ ${c.b}:${c.bLines.join("-")}`); + console.log(` ${c.head.slice(0, 150)}`); +} + +const shownNames = (opts["include-perf"] ? repeatedNames : repeatedOutsideLab) + .map(([name, defs]) => [name, defs.filter((d) => listed(d.file))]) + .sort((p, q) => q[1].length - p[1].length || p[0].localeCompare(q[0])); +console.log(`\n## ${shownNames.length} top-level function names defined in 2+ files\n`); +for (const [name, defs] of shownNames) { + console.log(`${String(defs.length).padStart(3)}x ${name.padEnd(26)} ${defs.map((d) => `${d.file}:${d.line}(${d.lines})`).join(" ")}`); +} + +console.log("\n## Undeclared packages\n"); +if (!undeclared.length) console.log("none"); +for (const [pkg, fs] of undeclared) console.log(`${pkg.padEnd(24)} ${[...fs].join(" ")}`); + +console.log("\n## Import edges that cross an area\n"); +const crossing = new Map(); +for (const [from, to] of edges) { + if (areaOf(from) === areaOf(to) || !listed(from) || !listed(to)) continue; + const key = `${areaOf(from)} -> ${areaOf(to)}`; + const list = crossing.get(key) ?? []; + list.push(`${from} -> ${to}`); + crossing.set(key, list); +} +for (const [key, list] of [...crossing].sort((p, q) => q[1].length - p[1].length)) { + console.log(`${String(list.length).padStart(4)} ${key}`); + for (const edge of list.slice(0, 12)) console.log(` ${edge}`); + if (list.length > 12) console.log(` ... and ${list.length - 12} more`); +} + +// Names only, so a lead: a file named in a .bat file may be run by it, or may +// only be mentioned in one of its comments. +console.log("\n## Files nothing imports, and where their names appear\n"); +const mentions = gitFiles() + .filter((f) => /\.(bat|ya?ml|json|md|mjs|js|ps1)$/.test(f) && !f.startsWith("docs/Reference/")) + .map((f) => [f, readFileSync(path.join(ROOT, f), "utf8")]); +for (const f of [...parsed.keys()].filter((f) => listed(f) && importers.get(f).size === 0)) { + const base = path.posix.basename(f); + const namedBy = mentions.filter(([g, text]) => g !== f && text.includes(base)).map(([g]) => g); + const run = namedBy.filter((g) => /\.(bat|ya?ml)$/.test(g) || g === "package.json"); + const code = namedBy.filter((g) => /\.(mjs|js|ps1)$/.test(g)); + const docs = namedBy.filter((g) => g.endsWith(".md")); + const where = run.length ? `named by ${run.join(", ")}` + : code.length ? `named in code: ${code.slice(0, 3).join(", ")}` + : docs.length ? `named only in ${docs.length} document(s)` + : "NAMED NOWHERE"; + console.log(` ${f.padEnd(48)} ${where}`); +} + +console.log("\n## The most imported modules\n"); +for (const [f, from] of [...importers].filter(([f]) => listed(f)).sort((p, q) => q[1].size - p[1].size).slice(0, 20)) { + console.log(`${String(from.size).padStart(4)} ${f}`); +} From 9062484145c7c3a04e5b1bcf9964f62830303c58 Mon Sep 17 00:00:00 2001 From: Kuba Sunderland-Ober Date: Fri, 25 Sep 2026 04:11:09 +0200 Subject: [PATCH 02/53] book: delete four superseded pdf-lib shims that only perf/ loaded --- book/lib/fast-dict-array.mjs | 328 ----------------------------- book/lib/fast-dict-iter.mjs | 81 ------- book/lib/fast-indirect-objects.mjs | 2 +- book/lib/fast-parse-dict.mjs | 87 -------- book/lib/fast-parse-object.mjs | 2 +- book/lib/fast-refs-class.mjs | 17 +- book/lib/fast-refs.mjs | 140 ------------ book/lib/fast-sync-load.mjs | 4 +- book/render-book.mjs | 12 +- perf/measure.mjs | 80 ++----- perf/phase0-measure.mjs | 2 +- 11 files changed, 35 insertions(+), 720 deletions(-) delete mode 100644 book/lib/fast-dict-array.mjs delete mode 100644 book/lib/fast-dict-iter.mjs delete mode 100644 book/lib/fast-parse-dict.mjs delete mode 100644 book/lib/fast-refs.mjs diff --git a/book/lib/fast-dict-array.mjs b/book/lib/fast-dict-array.mjs deleted file mode 100644 index 5f709859..00000000 --- a/book/lib/fast-dict-array.mjs +++ /dev/null @@ -1,328 +0,0 @@ -// Replace PDFDict's backing Map with a flat alternating array -// [k0, v0, k1, v1, ...]. -// -// Motivation. The sampling heap profile of the process phase (see -// "Profiling pdf-lib heap allocation" in perf/README.md) put `Map` -// constructors and `Map.prototype.set` at 50 % of total allocations -// -- ~63 MB combined -- with ~80 % of that traffic coming from one -// site: fastParseDict's per-dict accumulator -// ([fast-parse-dict.mjs:62](docs/lib/fast-parse-dict.mjs:62)). -// -// const dict = new Map(); // 24 MB of Map() constructors -// while (...) { -// const key = this.parseName(); -// const value = this.parseObject(); -// dict.set(key, value); // 38 MB of Map.set entries -// } -// ... PDFDict.fromMapWithContext(dict, this.context); -// -// Each parsed dict pays for one Map header + one hash-table backing -// arena + one bucket allocation per entry. PDF dicts are tiny (typical -// has <= 10 entries, often 2-3), so the hash-table overhead is pure -// loss vs a linear scan -- and the Map's amortized O(1) lookup buys -// nothing because nobody iterates a parsed dict enough times for the -// hash to pay back. -// -// The fix: store entries in a flat array. One allocation per dict -// (the array itself; the inline alternating layout avoids any per- -// entry bucket alloc). Lookup is a linear scan, which beats Map.get -// at this size class on every V8 microbench I've seen. -// -// Mechanism. We do three things: -// -// 1. Patch PDFDict.prototype.{keys, values, entries, set, get, has, -// delete, asMap, clone, toString, sizeInBytes, copyBytesInto} so -// `this.dict` is read as a flat array instead of a Map. -// sizeInBytes / copyBytesInto subsume fast-dict-iter.mjs (no -// Map.forEach + thisArg context object needed; iteration is just -// `for (let i = 0; i < arr.length; i += 2)`). -// -// 2. Patch PDFDict.withContext, PDFDict.fromMapWithContext, and the -// parallel fromMapWithContext / withContextAndPages helpers on -// PDFCatalog / PDFPageTree / PDFPageLeaf, plus PDFPageLeaf's -// clone() which constructs `new Map()` directly. Each of these is -// rewritten to produce / accept a flat array; the Map argument is -// converted at the seam (rare-path cost, only a few dicts per -// document hit these factories). -// -// 3. Patch PDFObjectParser.prototype.parseDict so the parser's hot -// inner loop accumulates into a flat array directly (no Map(), no -// Map.set). The Type-sentinel dispatch at the tail becomes a -// short linear scan over the array; on dicts that have a /Type -// entry it's the first or second key (PDF convention), so the -// scan is effectively O(1). This subsumes fast-parse-dict.mjs. -// -// Compatibility. Every consumer of `dict.dict.X` inside pdf-lib -// (ViewerPreferences, AppearanceCharacteristics, PDFAcroField, -// PDFAcroChoice, PDFAcroText, PDFAcroForm, PDFAnnotation, -// PDFWidgetAnnotation, BorderStyle, PDFStreamWriter, PDFCrossRefStream, -// PDFObjectCopier, PDFXRefStreamParser, etc.) goes through -// PDFDict.prototype methods (.set / .get / .has / .delete / .entries / -// .lookup), all of which we re-implement to read the array. Nobody in -// the codebase touches `dict.dict` expecting a Map iterator -- grep -// confirmed. `asMap()` still returns a fresh `new Map(...)` for any -// caller that genuinely wants a Map view. -// -// This shim is mutually exclusive with --fast-parse-dict and -// --fast-dict-iter: both are subsumed and would re-install the -// Map-based methods if loaded afterwards. measure.mjs enforces this. -// -// Side-effecting import. Import once before any pdf-lib operation: -// -// import "./lib/fast-dict-array.mjs"; -// -// Idempotent -- repeated imports do nothing after the first. - -import { createRequire } from 'node:module'; - -const require = createRequire(import.meta.url); -const PDFDict = require('pdf-lib/cjs/core/objects/PDFDict.js').default; -const PDFCatalog = require('pdf-lib/cjs/core/structures/PDFCatalog.js').default; -const PDFPageTree = require('pdf-lib/cjs/core/structures/PDFPageTree.js').default; -const PDFPageLeaf = require('pdf-lib/cjs/core/structures/PDFPageLeaf.js').default; -const PDFName = require('pdf-lib/cjs/core/objects/PDFName.js').default; -const PDFNull = require('pdf-lib/cjs/core/objects/PDFNull.js').default; -const PDFObjectParser = require('pdf-lib/cjs/core/parser/PDFObjectParser.js').default; -const CharCodes = require('pdf-lib/cjs/core/syntax/CharCodes.js').default; - -// Captured canonical PDFNames for the parser's Type-dispatch tail. -// Pool-dedup ([PDFName.js:18,100]) guarantees reference equality with -// whatever the parser sees inside the dict. -const TypeName = PDFName.of('Type'); -const CatalogName = PDFName.of('Catalog'); -const PagesName = PDFName.of('Pages'); -const PageName = PDFName.of('Page'); - -// Map -> flat array. Called at the seam from the factories below; not -// on the hot parse path. -function mapToArray(map) { - const arr = new Array(map.size * 2); - let i = 0; - for (const [k, v] of map) { arr[i++] = k; arr[i++] = v; } - return arr; -} - -// Linear scan for the index of `key` in [k0, v0, k1, v1, ...]; returns -// the key-slot index, or -1 if absent. -function indexOfKey(arr, key) { - for (let i = 0, len = arr.length; i < len; i += 2) { - if (arr[i] === key) return i; - } - return -1; -} - -if (!PDFDict.prototype.__fastDictArrayInstalled) { - - // ---- PDFDict.prototype -------------------------------------------- - - PDFDict.prototype.keys = function () { - const arr = this.dict; - const out = new Array(arr.length >> 1); - for (let i = 0, j = 0, len = arr.length; i < len; i += 2, j++) out[j] = arr[i]; - return out; - }; - - PDFDict.prototype.values = function () { - const arr = this.dict; - const out = new Array(arr.length >> 1); - for (let i = 1, j = 0, len = arr.length; i < len; i += 2, j++) out[j] = arr[i]; - return out; - }; - - PDFDict.prototype.entries = function () { - const arr = this.dict; - const out = new Array(arr.length >> 1); - for (let i = 0, j = 0, len = arr.length; i < len; i += 2, j++) { - out[j] = [arr[i], arr[i + 1]]; - } - return out; - }; - - PDFDict.prototype.set = function (key, value) { - const arr = this.dict; - const idx = indexOfKey(arr, key); - if (idx >= 0) { - arr[idx + 1] = value; - } else { - arr.push(key, value); - } - }; - - PDFDict.prototype.get = function (key, preservePDFNull) { - if (preservePDFNull === undefined) preservePDFNull = false; - const arr = this.dict; - const idx = indexOfKey(arr, key); - if (idx < 0) return undefined; - const value = arr[idx + 1]; - if (value === PDFNull && !preservePDFNull) return undefined; - return value; - }; - - PDFDict.prototype.has = function (key) { - const arr = this.dict; - const idx = indexOfKey(arr, key); - if (idx < 0) return false; - const value = arr[idx + 1]; - return value !== undefined && value !== PDFNull; - }; - - PDFDict.prototype.delete = function (key) { - const arr = this.dict; - const idx = indexOfKey(arr, key); - if (idx < 0) return false; - arr.splice(idx, 2); - return true; - }; - - PDFDict.prototype.asMap = function () { - const arr = this.dict; - const m = new Map(); - for (let i = 0, len = arr.length; i < len; i += 2) m.set(arr[i], arr[i + 1]); - return m; - }; - - PDFDict.prototype.clone = function (context) { - const ctx = context || this.context; - const cloned = this.dict.slice(); - return new PDFDict(cloned, ctx); - }; - - PDFDict.prototype.toString = function () { - const arr = this.dict; - let s = '<<\n'; - for (let i = 0, len = arr.length; i < len; i += 2) { - s += arr[i].toString() + ' ' + arr[i + 1].toString() + '\n'; - } - return s + '>>'; - }; - - PDFDict.prototype.sizeInBytes = function () { - const arr = this.dict; - let size = 5; - for (let i = 0, len = arr.length; i < len; i += 2) { - size += arr[i].sizeInBytes() + arr[i + 1].sizeInBytes() + 2; - } - return size; - }; - - PDFDict.prototype.copyBytesInto = function (buffer, offset) { - const initialOffset = offset; - buffer[offset++] = CharCodes.LessThan; - buffer[offset++] = CharCodes.LessThan; - buffer[offset++] = CharCodes.Newline; - const arr = this.dict; - for (let i = 0, len = arr.length; i < len; i += 2) { - offset += arr[i].copyBytesInto(buffer, offset); - buffer[offset++] = CharCodes.Space; - offset += arr[i + 1].copyBytesInto(buffer, offset); - buffer[offset++] = CharCodes.Newline; - } - buffer[offset++] = CharCodes.GreaterThan; - buffer[offset++] = CharCodes.GreaterThan; - return offset - initialOffset; - }; - - // ---- PDFDict factories -------------------------------------------- - - PDFDict.withContext = function (context) { - return new PDFDict([], context); - }; - PDFDict.fromMapWithContext = function (map, context) { - return new PDFDict(mapToArray(map), context); - }; - - // ---- Subclass factories ------------------------------------------- - // PDFCatalog.withContextAndPages builds a fresh 2-entry Map; just - // hand it the equivalent 2-entry array. - - PDFCatalog.withContextAndPages = function (context, pages) { - return new PDFCatalog( - [PDFName.of('Type'), CatalogName, PagesName, pages], - context, - ); - }; - PDFCatalog.fromMapWithContext = function (map, context) { - return new PDFCatalog(mapToArray(map), context); - }; - - PDFPageTree.fromMapWithContext = function (map, context) { - return new PDFPageTree(mapToArray(map), context); - }; - - PDFPageLeaf.fromMapWithContext = function (map, context, autoNormalizeCTM) { - return new PDFPageLeaf(mapToArray(map), context, autoNormalizeCTM); - }; - // PDFPageLeaf.prototype.clone constructs `new Map()` explicitly, - // then copies via this.entries() + clone.set(); since clone.set is - // PDFDict.prototype.set (now array-aware), it works as long as - // fromMapWithContext receives an empty Map and converts it. - // mapToArray(new Map()) yields []; nothing to patch here. - - // ---- PDFObjectParser.prototype.parseDict -------------------------- - // Subsumes fast-parse-dict.mjs: no `new Map()`, no `dict.set(...)` - // in the hot inner loop. The Type-sentinel dispatch at the tail is - // a short linear scan; PDF convention places /Type first, so it's - // effectively O(1) per dict. - - // Initial capacity for the per-dict accumulator. NOT a scratch - // buffer (the array isn't reused across calls -- it's allocated - // fresh each dict, filled with parsed entries, and handed to the - // PDFDict constructor where it lives as `pdfDict.dict` for the - // document's lifetime). Just a pre-sized initial capacity that - // skips push-grow's reallocation chain. - // - // Histogram from the book parse (see instrument-parsedict.mjs): - // 5-entry dicts dominate (52 %, exactly 10 push slots), 4-entry - // next (28 %, 8 slots), long tail to 7-8 entries. INITIAL_SLOTS = - // 10 is exact-fit for the median case; smaller dicts (2/3/4 - // entries) waste a few slots, larger ones (7+) take one growth - // via push. Cuts ~70 bytes of FixedArray-header allocation per - // dict vs INITIAL_SLOTS=16 -- on 261 k dict invocations that - // adds up. - const INITIAL_SLOTS = 10; - PDFObjectParser.prototype.parseDict = function fastParseDictArray() { - const bytes = this.bytes; - bytes.assertNext(CharCodes.LessThan); - bytes.assertNext(CharCodes.LessThan); - this.skipWhitespaceAndComments(); - const arr = new Array(INITIAL_SLOTS); - let len = 0; - while (!bytes.done() && - bytes.peek() !== CharCodes.GreaterThan && - bytes.peekAhead(1) !== CharCodes.GreaterThan) { - const key = this.parseName(); - const value = this.parseObject(); - if (len < INITIAL_SLOTS) { - arr[len] = key; - arr[len + 1] = value; - } else { - // Rare overflow path: set length to current len so push - // appends at the right offset, then grow naturally. - arr.length = len; - arr.push(key, value); - } - len += 2; - this.skipWhitespaceAndComments(); - } - this.skipWhitespaceAndComments(); - bytes.assertNext(CharCodes.GreaterThan); - bytes.assertNext(CharCodes.GreaterThan); - arr.length = len; - - // Type-sentinel dispatch. Inline-scan for TypeName; in practice - // it's at arr[0] or arr[2]. - let Type; - for (let i = 0; i < len; i += 2) { - if (arr[i] === TypeName) { Type = arr[i + 1]; break; } - } - if (Type === CatalogName) return new PDFCatalog(arr, this.context); - if (Type === PagesName) return new PDFPageTree(arr, this.context); - if (Type === PageName) return new PDFPageLeaf(arr, this.context); - return new PDFDict(arr, this.context); - }; - - PDFDict.prototype.__fastDictArrayInstalled = true; - // Mark the subsumed shims as installed so a redundant load is a no-op. - PDFDict.prototype.__fastDictIterInstalled = true; - PDFObjectParser.prototype.__fastParseDictInstalled = true; -} diff --git a/book/lib/fast-dict-iter.mjs b/book/lib/fast-dict-iter.mjs deleted file mode 100644 index 1d2a6cb8..00000000 --- a/book/lib/fast-dict-iter.mjs +++ /dev/null @@ -1,81 +0,0 @@ -// Replace pdf-lib's PDFDict.sizeInBytes and PDFDict.copyBytesInto -- both of -// which materialize a fresh Array of [key, value] tuples via this.entries() -// on every call -- with versions that iterate the underlying Map in place. -// -// The upstream entries() helper -// ([PDFDict.js:22](node_modules/pdf-lib/cjs/core/objects/PDFDict.js:22)) is: -// -// PDFDict.prototype.entries = function () { -// return Array.from(this.dict.entries()); -// }; -// -// Per call that is: one MapIterator + one outer Array + one fresh -// [key, value] tuple per entry (allocated by the iterator itself). The save -// path fires both consumers on every dict (sizeInBytes to measure first, -// then copyBytesInto to write), so on the book that's ~100 k Array.from -// calls feeding the GC; PDFDict.entries was the largest non-GC row in the -// process profile (~10 % of process self-time) and (garbage collector) sat -// at the top. -// -// Map.prototype.forEach((value, key) => ...) calls back with positional -// arguments and never allocates a tuple. The two consumers don't need the -// tuple form -- they immediately destructure -- so swapping is local. -// -// We do NOT touch PDFDict.prototype.entries itself: clone() and toString() -// still call it and rely on the Array-of-tuples contract. Those paths fire -// rarely (clone on incremental updates only, toString in debug output) and -// aren't worth the contract churn. -// -// Side-effecting import. Import once before any pdf-lib save: -// -// import "./lib/fast-dict-iter.mjs"; -// -// Idempotent -- repeated imports do nothing after the first. - -import { createRequire } from 'node:module'; - -const require = createRequire(import.meta.url); -const PDFDict = require('pdf-lib/cjs/core/objects/PDFDict.js').default; -const CharCodes = require('pdf-lib/cjs/core/syntax/CharCodes.js').default; - -// Callbacks are module-level (not closures) so Map.forEach reuses the same -// function reference on every call instead of allocating a fresh context -// per invocation. Per-call state is threaded through forEach's `thisArg` -// (one small object alloc per call, instead of one closure context plus -// one heap cell for the captured `offset` mutation). -function _sizeInBytesEntry(value, key) { - this.s += key.sizeInBytes() + value.sizeInBytes() + 2; -} - -function _copyBytesIntoEntry(value, key) { - const buf = this.buf; - let off = this.off; - off += key.copyBytesInto(buf, off); - buf[off++] = CharCodes.Space; - off += value.copyBytesInto(buf, off); - buf[off++] = CharCodes.Newline; - this.off = off; -} - -if (!PDFDict.prototype.__fastDictIterInstalled) { - PDFDict.prototype.sizeInBytes = function () { - const ctx = { s: 5 }; - this.dict.forEach(_sizeInBytesEntry, ctx); - return ctx.s; - }; - - PDFDict.prototype.copyBytesInto = function (buffer, offset) { - const initialOffset = offset; - buffer[offset++] = CharCodes.LessThan; - buffer[offset++] = CharCodes.LessThan; - buffer[offset++] = CharCodes.Newline; - const ctx = { buf: buffer, off: offset }; - this.dict.forEach(_copyBytesIntoEntry, ctx); - offset = ctx.off; - buffer[offset++] = CharCodes.GreaterThan; - buffer[offset++] = CharCodes.GreaterThan; - return offset - initialOffset; - }; - - PDFDict.prototype.__fastDictIterInstalled = true; -} diff --git a/book/lib/fast-indirect-objects.mjs b/book/lib/fast-indirect-objects.mjs index 9058414f..ebb740d6 100644 --- a/book/lib/fast-indirect-objects.mjs +++ b/book/lib/fast-indirect-objects.mjs @@ -18,7 +18,7 @@ // indirect objects, discarding each intermediate arena to GC. // // PDFRefs are overwhelmingly gen=0 (revisions / incremental updates -// are the only gen!=0 producers, and they're rare). fast-refs.mjs +// are the only gen!=0 producers, and they're rare). fast-refs-class.mjs // already exploits this on the key side -- a dense array indexed by // objectNumber for the PDFRef pool, Map fallback for gen!=0. This // shim does the same on the value side for PDFContext.indirectObjects. diff --git a/book/lib/fast-parse-dict.mjs b/book/lib/fast-parse-dict.mjs deleted file mode 100644 index 203549c8..00000000 --- a/book/lib/fast-parse-dict.mjs +++ /dev/null @@ -1,87 +0,0 @@ -// Hoist the four sentinel PDFName.of calls out of -// PDFObjectParser.prototype.parseDict. -// -// The upstream parseDict -// ([PDFObjectParser.js:141](node_modules/pdf-lib/cjs/core/parser/PDFObjectParser.js:141)) -// ends every dict it parses with a Type-dispatch tail: -// -// var Type = dict.get(PDFName.of('Type')); -// if (Type === PDFName.of('Catalog')) return PDFCatalog.fromMapWithContext(...); -// else if (Type === PDFName.of('Pages')) return PDFPageTree.fromMapWithContext(...); -// else if (Type === PDFName.of('Page')) return PDFPageLeaf.fromMapWithContext(...); -// else return PDFDict.fromMapWithContext(...); -// -// That's 4 PDFName.of calls per dict, even on the overwhelming -// majority (resource dicts, font descriptors, content-stream dicts) -// that have no /Type entry at all. With --fast-decode-name in -// effect each call collapses to a Map.get on fastCache, but -// fastOf is still the #4 row in process.cpuprofile (~80 ms, -// 5.2 %). -// -// PDFName instances are pool-deduped -// ([PDFName.js:18,100](node_modules/pdf-lib/cjs/core/objects/PDFName.js:18)) -// so the sentinel "Type" / "Catalog" / "Pages" / "Page" PDFNames -// are reference-stable for the entire load. Capture them once at -// shim-load time and substitute direct constants for the four -// PDFName.of calls inside parseDict. The rest of the function -// body is preserved verbatim -- same loop, same dict.set, same -// dispatch shape. -// -// Mechanism: PDFObjectParser isn't re-exported by pdf-lib's index, -// so we reach in through the CJS internals via createRequire (same -// shape as fast-parse-number.mjs / fast-dict-iter.mjs). Mutating -// PDFObjectParser.prototype.parseDict is global -- every parser -// instance created after this shim loads picks it up. -// -// Side-effecting import. Import once before PDFDocument.load runs: -// -// import "./lib/fast-parse-dict.mjs"; -// -// Idempotent -- repeated imports do nothing after the first. - -import { createRequire } from 'node:module'; - -const require = createRequire(import.meta.url); -const PDFObjectParser = require('pdf-lib/cjs/core/parser/PDFObjectParser.js').default; -const PDFName = require('pdf-lib/cjs/core/objects/PDFName.js').default; -const PDFDict = require('pdf-lib/cjs/core/objects/PDFDict.js').default; -const PDFCatalog = require('pdf-lib/cjs/core/structures/PDFCatalog.js').default; -const PDFPageTree = require('pdf-lib/cjs/core/structures/PDFPageTree.js').default; -const PDFPageLeaf = require('pdf-lib/cjs/core/structures/PDFPageLeaf.js').default; -const CharCodes = require('pdf-lib/cjs/core/syntax/CharCodes.js').default; - -// Capture canonical PDFName instances. Pool-dedup guarantees the -// parser would have built === these even if the original parseDict -// were still in play. -const TypeName = PDFName.of('Type'); -const CatalogName = PDFName.of('Catalog'); -const PagesName = PDFName.of('Pages'); -const PageName = PDFName.of('Page'); - -if (!PDFObjectParser.prototype.__fastParseDictInstalled) { - PDFObjectParser.prototype.parseDict = function fastParseDict() { - const bytes = this.bytes; - bytes.assertNext(CharCodes.LessThan); - bytes.assertNext(CharCodes.LessThan); - this.skipWhitespaceAndComments(); - const dict = new Map(); - while (!bytes.done() && - bytes.peek() !== CharCodes.GreaterThan && - bytes.peekAhead(1) !== CharCodes.GreaterThan) { - const key = this.parseName(); - const value = this.parseObject(); - dict.set(key, value); - this.skipWhitespaceAndComments(); - } - this.skipWhitespaceAndComments(); - bytes.assertNext(CharCodes.GreaterThan); - bytes.assertNext(CharCodes.GreaterThan); - const Type = dict.get(TypeName); - if (Type === CatalogName) return PDFCatalog.fromMapWithContext(dict, this.context); - if (Type === PagesName) return PDFPageTree.fromMapWithContext(dict, this.context); - if (Type === PageName) return PDFPageLeaf.fromMapWithContext(dict, this.context); - return PDFDict.fromMapWithContext(dict, this.context); - }; - - PDFObjectParser.prototype.__fastParseDictInstalled = true; -} diff --git a/book/lib/fast-parse-object.mjs b/book/lib/fast-parse-object.mjs index e573dc44..33bcbfa4 100644 --- a/book/lib/fast-parse-object.mjs +++ b/book/lib/fast-parse-object.mjs @@ -36,7 +36,7 @@ // // Mechanism: PDFObjectParser isn't re-exported from pdf-lib's index, // so we reach in through the CJS internals via createRequire (same -// shape as fast-parse-dict.mjs). Mutating +// shape as fast-sync-load.mjs). Mutating // PDFObjectParser.prototype.parseObject is global -- every parser // instance created after this shim loads picks it up. // diff --git a/book/lib/fast-refs-class.mjs b/book/lib/fast-refs-class.mjs index c1c11e29..f88b29d4 100644 --- a/book/lib/fast-refs-class.mjs +++ b/book/lib/fast-refs-class.mjs @@ -1,6 +1,8 @@ -// fast-refs variant: use a class-style constructor for stable hidden class. +// Replaces PDFRef.of: a dense-array cache for gen=0 refs, built with a +// class-style constructor for a stable hidden class. // -// fast-refs.mjs builds PDFRef instances with +// The shim this replaced, fast-refs.mjs (since deleted; see +// "fast-refs-class" in perf/notes/08-pdf-lib.md), built PDFRef instances with // `Object.create(PDFRef.prototype) + fresh.objectNumber = ... + fresh.gen = ...`. // V8 treats objects built that way as transitioning through intermediate // hidden-class maps as each property is added, and the result is roughly @@ -29,17 +31,20 @@ // raw, aligned to 16 B by V8 -- versus 12 + 2*4 = 20 B raw, aligned to // 24 B for a 2-slot instance. Saves 8 B per gen=0 PDFRef * ~226 k unique // = ~1.8 MB heap on the book. -// -// Mutually exclusive with --fast-refs in the harness. import { PDFRef } from 'pdf-lib'; -// ---- helpers (same as fast-refs.mjs, see commentary there) ------------- +// ---- helpers ----------------------------------------------------------- +// Write n's decimal representation into buffer starting at offset. +// No allocations. Returns the number of bytes written. n must be a +// non-negative integer. function _writeUint(buffer, offset, n) { if (n < 10) { buffer[offset] = 0x30 + n; return 1; } + // Count digits. let m = n, d = 0; while (m > 0) { d++; m = (m / 10) | 0; } + // Write digits backwards. for (let i = d - 1; i >= 0; i--) { buffer[offset + i] = 0x30 + (n % 10); n = (n / 10) | 0; @@ -47,6 +52,8 @@ function _writeUint(buffer, offset, n) { return d; } +// Non-allocating decimal digit count for non-negative integers. +// Ladder catches the common small-number cases without arithmetic. function _digitCount(n) { if (n < 10) return 1; if (n < 100) return 2; diff --git a/book/lib/fast-refs.mjs b/book/lib/fast-refs.mjs deleted file mode 100644 index beeb76ad..00000000 --- a/book/lib/fast-refs.mjs +++ /dev/null @@ -1,140 +0,0 @@ -// Replace pdf-lib's PDFRef.of pool lookup with a dense-array cache -// for the generation=0 case (the overwhelmingly common one), AND -// drop the per-instance `tag` string entirely. -// -// The upstream implementation -// (node_modules/pdf-lib/cjs/core/objects/PDFRef.js) keys its pool by -// a freshly-built string ` R` on every call: -// -// var tag = objectNumber + " " + generationNumber + " R"; -// var instance = pool.get(tag); -// -// On the book we see ~1.2 M PDFRef.of calls per load, 82 % of them -// with gen=0; each call allocates the tag string before Map.get can -// hash it. That's ~330 ms of self-time on the process-phase profile -// plus measurable GC pressure. -// -// Shim part 1: dense array indexed by objectNumber for the gen=0 branch. -// Plain array indexing, no string alloc, no Map hash. On a gen=0 cache -// miss we construct the PDFRef directly via -// `Object.create(PDFRef.prototype)` plus manual field init, skipping -// both the ENFORCER check and the upstream `pool.set(tag, instance)`. -// -// Shim part 2: drop the per-instance `tag` field. Upstream caches -// ` R` on each PDFRef so toString / sizeInBytes / -// copyBytesInto can read it back. After fast-array-onebuf shipped, -// the heap profile showed PDFParser.parseIndirectObjectHeader sitting -// at 13.7 MB (25 % of total). The attribution chain (via -// perf/find-heap-callers.mjs): -// -// parseIndirectObjectHeader → skipJibberish (14.2 MB) -// → matchIndirectObjectHeader (try/catch wrapper) -// → parseIndirectObjectHeader → fastOf -// -// skipJibberish runs after every successful indirect object parse and -// speculatively calls matchIndirectObjectHeader to detect the next -// `N M obj` header. On valid PDFs the speculation always succeeds, so -// fastOf fires once per indirect-object boundary, populating the -// dense-array cache. The subsequent "real" parseIndirectObject then -// hits the cache. V8 inlines fastOf at this call site (small + hot -// from speculation) so the attribution lands on the caller -- 13.7 MB -// of which was the tag-string allocation (`objectNumber + ' 0 R'`): -// V8 builds 1-2 intermediate concat strings + the final ~25-35 B -// tag, ~150 k times. -// -// Eliminating the `tag` field collapses all of that. The prototype -// methods now compute their results from objectNumber / generationNumber -// directly. copyBytesInto writes digits straight into the output buffer -// with a no-allocation _writeUint helper; sizeInBytes returns -// digitCount(obj) + digitCount(gen) + 3 (for " " + " R"); toString -// builds on demand (only used for debug, no caching needed). -// -// gen != 0 PDFRefs constructed via the upstream path still have `tag` -// set by the upstream constructor -- our overrides ignore the field, -// so the tag string is allocated-then-wasted. gen != 0 is ~18 % of refs -// at ~50 K instances; the waste is bounded and not worth patching the -// constructor for. -// -// gen != 0 cache lookups (pdf-lib's xref-stream bookkeeping where -// "generation" encodes an in-ObjStm index per PDF 1.5 spec, see -// PDFXRefStreamParser.js:74-80) still pass through the original -// PDFRef.of -- their Map pool is harmless at gen!=0's volume. -// -// Side-effecting import. Import once before any pdf-lib operation. -// Idempotent. - -import { PDFRef } from "pdf-lib"; - -// Write n's decimal representation into buffer starting at offset. -// No allocations. Returns the number of bytes written. n must be a -// non-negative integer. -function _writeUint(buffer, offset, n) { - if (n < 10) { buffer[offset] = 0x30 + n; return 1; } - // Count digits. - let m = n, d = 0; - while (m > 0) { d++; m = (m / 10) | 0; } - // Write digits backwards. - for (let i = d - 1; i >= 0; i--) { - buffer[offset + i] = 0x30 + (n % 10); - n = (n / 10) | 0; - } - return d; -} - -// Non-allocating decimal digit count for non-negative integers. -// Ladder catches the common small-number cases without arithmetic. -function _digitCount(n) { - if (n < 10) return 1; - if (n < 100) return 2; - if (n < 1000) return 3; - if (n < 10000) return 4; - if (n < 100000) return 5; - if (n < 1000000) return 6; - let d = 0; - while (n > 0) { d++; n = (n / 10) | 0; } - return d; -} - -if (!PDFRef.__fastPoolInstalled) { - const original = PDFRef.of; - const pool0 = []; - PDFRef.of = function fastOf(objectNumber, generationNumber) { - if (generationNumber === undefined || generationNumber === 0) { - const existing = pool0[objectNumber]; - if (existing) return existing; - // Direct construction -- skip ENFORCER check, skip upstream pool.set, - // skip the per-instance `tag` string (the prototype methods now - // compute their results from objectNumber / generationNumber). - const fresh = Object.create(PDFRef.prototype); - fresh.objectNumber = objectNumber; - fresh.generationNumber = 0; - pool0[objectNumber] = fresh; - return fresh; - } - return original.call(PDFRef, objectNumber, generationNumber); - }; - - // Replace the upstream prototype methods to ignore `tag` entirely. - // Works for both gen=0 (tag is absent) and gen!=0 (tag is set by - // upstream's constructor but ignored). - - PDFRef.prototype.toString = function () { - return this.objectNumber + ' ' + this.generationNumber + ' R'; - }; - - PDFRef.prototype.sizeInBytes = function () { - return _digitCount(this.objectNumber) + _digitCount(this.generationNumber) + 3; - }; - - PDFRef.prototype.copyBytesInto = function (buffer, offset) { - const start = offset; - offset += _writeUint(buffer, offset, this.objectNumber); - buffer[offset++] = 0x20; // ' ' - offset += _writeUint(buffer, offset, this.generationNumber); - buffer[offset++] = 0x20; // ' ' - buffer[offset++] = 0x52; // 'R' - return offset - start; - }; - - PDFRef.__fastPoolInstalled = true; -} diff --git a/book/lib/fast-sync-load.mjs b/book/lib/fast-sync-load.mjs index 1109247d..f05bd760 100644 --- a/book/lib/fast-sync-load.mjs +++ b/book/lib/fast-sync-load.mjs @@ -77,8 +77,8 @@ const { Keywords } = require('pdf-lib/cjs/core/syntax/Keywords.js'); const { toUint8Array, copyStringIntoBuffer, last } = require('pdf-lib/cjs/utils/index.js'); // Pool-deduped PDFName instances are reference-stable for the whole -// load (see fast-parse-dict.mjs for the same trick). Capture the three -// sentinels parseIndirectObject's Type-dispatch needs. +// load. Capture the three sentinels parseIndirectObject's Type-dispatch +// needs. const TypeName = PDFName.of('Type'); const ObjStmName = PDFName.of('ObjStm'); const XRefName = PDFName.of('XRef'); diff --git a/book/render-book.mjs b/book/render-book.mjs index 427586b0..c2fd8231 100644 --- a/book/render-book.mjs +++ b/book/render-book.mjs @@ -43,7 +43,7 @@ import { PDFDocument } from 'pdf-lib'; // copyBytesInto compute from objectNumber / generationNumber // directly via _writeUint + _digitCount helpers). Replaces the // `Object.create(PDFRef.prototype) + property writes` pattern of -// the older fast-refs.mjs shim, which V8 routes through the +// the older fast-refs shim (since deleted), which V8 routes through the // slow-property path: PDFRef ended up at ~60 B/instance vs // PDFName's ~31 B (`new PDFName(...)`-built). The constructor // gives V8 a stable hidden class from the first instance and @@ -54,8 +54,8 @@ import { PDFDocument } from 'pdf-lib'; // trimmed parseIndirectObjectHeader by ~4.3 MB. Same prototype // methods, same instanceof semantics; the only change is the // construction style. See "fast-refs-class" in -// perf/notes/08-pdf-lib.md. fast-refs.mjs stays in the tree as -// an A/B baseline (mutex-checked in measure.mjs). +// perf/notes/08-pdf-lib.md, which also records the measurements +// for fast-refs; git history has its code. // fast-inflate -- swaps pako.inflate for node:zlib.inflateSync // on the one pdf-lib call site that uses it // (PDFCrossRefStreamParser during load). Negligible cost shift, @@ -92,9 +92,9 @@ import { PDFDocument } from 'pdf-lib'; // beyond fast-dict-array. See "One-buffer PDFDict" in // perf/notes/08-pdf-lib.md. // -// Earlier dict-shape shims (fast-dict-array, fast-dict-iter, -// fast-parse-dict) stay in the tree as A/B baselines but are -// mutually exclusive with --fast-dict-onebuf in measure.mjs. +// The earlier dict-shape shims it replaced (fast-dict-array, +// fast-dict-iter, fast-parse-dict) were deleted; their measurements +// are in perf/notes/08-pdf-lib.md, and git history has the code. // fast-parse-object -- replace PDFObjectParser.prototype.parseObject // with a first-byte-dispatch version that gates the three // matchKeyword (true / false / null) scans behind a byte check. diff --git a/perf/measure.mjs b/perf/measure.mjs index 99eec99a..fd47f4a6 100644 --- a/perf/measure.mjs +++ b/perf/measure.mjs @@ -29,12 +29,12 @@ // [--no-detach-pages] [--instrument] [--time-hooks] // [--incremental] [--chrome-outline] [--timing] // [--clone-count] [--render-only] -// [--fast-refs] [--parallel-deflate] +// [--parallel-deflate] // [--fast-decode-name] [--fast-number-to-string] // [--fast-size-in-bytes] [--fast-inflate] -// [--fast-parse-number] [--fast-parse-dict] +// [--fast-parse-number] // [--fast-parse-object] [--fast-sync-load] -// [--fast-dict-array] [--fast-indirect-objects] +// [--fast-indirect-objects] // [--fast-pdfnumber-pool] // // --render-only bails out after the render phase. Skips meta extraction, @@ -106,11 +106,12 @@ // 512 for finer-grained attribution on short phases. Composable with // --cpu-profile-process; both share one inspector session. // -// --fast-refs replaces PDFRef.of's string-keyed Map lookup with a -// dense-array cache for the gen=0 case (82 % of ~1.2 M calls on the -// book). Eliminates the per-call ` R` string allocation -// and Map hash. gen != 0 calls (pdf-lib's xref-stream bookkeeping -// for compressed objects) pass through unchanged. +// --fast-refs, --fast-parse-dict, --fast-dict-iter and --fast-dict-array +// loaded shims that production had replaced with fast-refs-class and +// fast-dict-onebuf. They were deleted in the tooling review, and their +// flags with them: pdf-lib is pinned at its final release, so the +// comparisons they served are settled. The measurements are in +// notes/08-pdf-lib.md, and git history has the code. // // --parallel-deflate replaces pdfDoc.save() with parallelSave from // book/lib/parallel-deflate.mjs: object streams are pre-deflated in @@ -149,14 +150,6 @@ // PDF flows through these; hundreds of thousands of calls per load // on the book. Production runs through it. // -// --fast-parse-dict hoists the four sentinel PDFName.of calls -// (Type / Catalog / Pages / Page) out of the type-dispatch tail -// in PDFObjectParser.prototype.parseDict. The dispatch fires -// per-dict (tens of thousands on the book) and even with -// --fast-decode-name each lookup is still a Map.get on fastCache. -// Pool-dedup makes the canonical PDFNames reference-stable, so -// captured constants replace the four calls verbatim. -// // --fast-parse-object replaces PDFObjectParser.prototype.parseObject // with a first-byte-dispatch version that gates the three // matchKeyword (true / false / null) scans behind a byte check. @@ -166,18 +159,6 @@ // rewind costs on every invocation. Same semantics, dispatch // reordered by observed frequency in dict-value position. // -// --fast-dict-array replaces PDFDict's backing Map with a flat -// alternating array [k0, v0, k1, v1, ...]. The sampling heap profile -// showed `new Map()` + `Map.prototype.set` accounting for half the -// process-phase allocations (~63 MB combined), 80 % of that traffic -// from the parser's per-dict accumulator. The flat array is one -// allocation per dict, no hash-table arena; lookups are linear scans -// but PDF dicts are tiny (typically <= 10 entries). Subsumes -// --fast-parse-dict and --fast-dict-iter (the parser's hot loop -// accumulates into the array directly; sizeInBytes / copyBytesInto -// iterate in place). Now superseded by --fast-dict-onebuf; kept as -// an A/B baseline. -// // --fast-dict-onebuf collapses the per-dict array allocation into // ONE long-lived mainBuf shared across every committed PDFDict // entry. A small per-parser temp array acts as a stack of parseDict @@ -187,10 +168,9 @@ // header. Owned dicts (factory-created post-parse) append to main // and mutate in place / COW to the tail. PDFContext is a singleton // in our pipeline (one PDFDocument.load per process); a second -// distinct context throws. Mutually exclusive with --fast-dict-array -// and the other dict-shape shims. ~57 % cumulative heap reduction -// since the original Map-backed PDFDict (152 -> 66 MB). Production -// runs through it. +// distinct context throws. ~57 % cumulative heap reduction since +// the original Map-backed PDFDict (152 -> 66 MB). Production runs +// through it. // // --fast-indirect-objects replaces PDFContext.indirectObjects // (Map) with a dense array indexed by @@ -281,7 +261,6 @@ let timing = false; let cloneCount = false; let renderOnly = false; let tracing = false; -let fastRefs = false; let fastRefsClass = false; let parallelDeflate = false; let fastDecodeName = false; @@ -289,12 +268,9 @@ let fastNumberToString = false; let fastSizeInBytes = false; let fastInflate = false; let fastParseNumber = false; -let fastDictIter = false; -let fastParseDict = false; let fastParseObject = false; let fastParseName = false; let fastSyncLoad = false; -let fastDictArray = false; let fastIndirectObjects = false; let fastPdfnumberPool = false; let fastDictOnebuf = false; @@ -325,7 +301,6 @@ for (let i = 0; i < args.length; i++) { else if (a === '--render-only') renderOnly = true; else if (a === '--tracing') tracing = true; else if (a === '--no-affinity') { /* handled in pin-cpu.mjs */ } - else if (a === '--fast-refs') fastRefs = true; else if (a === '--fast-refs-class') fastRefsClass = true; else if (a === '--parallel-deflate') parallelDeflate = true; else if (a === '--fast-decode-name') fastDecodeName = true; @@ -333,12 +308,9 @@ for (let i = 0; i < args.length; i++) { else if (a === '--fast-size-in-bytes') fastSizeInBytes = true; else if (a === '--fast-inflate') fastInflate = true; else if (a === '--fast-parse-number') fastParseNumber = true; - else if (a === '--fast-dict-iter') fastDictIter = true; - else if (a === '--fast-parse-dict') fastParseDict = true; else if (a === '--fast-parse-object') fastParseObject = true; else if (a === '--fast-parse-name') fastParseName = true; else if (a === '--fast-sync-load') fastSyncLoad = true; - else if (a === '--fast-dict-array') fastDictArray = true; else if (a === '--fast-indirect-objects') fastIndirectObjects = true; else if (a === '--fast-pdfnumber-pool') fastPdfnumberPool = true; else if (a === '--fast-dict-onebuf') fastDictOnebuf = true; @@ -389,14 +361,6 @@ if (heapProfileProcess && renderOnly) { console.error('--heap-profile-process is incompatible with --render-only (the process phase is skipped).'); process.exit(2); } -if (fastDictArray && (fastParseDict || fastDictIter)) { - console.error('--fast-dict-array subsumes --fast-parse-dict and --fast-dict-iter (Map-backed shims). Pick one shape.'); - process.exit(2); -} -if (fastDictOnebuf && (fastDictArray || fastParseDict || fastDictIter)) { - console.error('--fast-dict-onebuf subsumes the other dict-shape shims (different storage shape). Pick one.'); - process.exit(2); -} if (measurePass && !fastDictOnebuf) { console.error('--measure-pass requires --fast-dict-onebuf (the only shim that consumes setExpectedDictSlots so far).'); process.exit(2); @@ -420,14 +384,6 @@ if (instrumentSlotTypes && (incremental || renderOnly)) { // Install the dense-array cache for PDFRef.of's gen=0 path before any // pdf-lib operation. Side-effecting import; idempotent. -if (fastRefs && fastRefsClass) { - console.error('--fast-refs and --fast-refs-class are mutually exclusive (both shim PDFRef.of).'); - process.exit(2); -} -if (fastRefs) { - await import('../book/lib/fast-refs.mjs'); - console.log('[harness] fast-refs: PDFRef.of dense-array cache for gen=0'); -} if (fastRefsClass) { await import('../book/lib/fast-refs-class.mjs'); console.log('[harness] fast-refs-class: PDFRef.of dense-array cache + class-constructor shape'); @@ -452,14 +408,6 @@ if (fastParseNumber) { await import('../book/lib/fast-parse-number.mjs'); console.log('[harness] fast-parse-number: direct-integer accumulator for parseRawNumber/parseRawInt'); } -if (fastDictIter) { - await import('../book/lib/fast-dict-iter.mjs'); - console.log('[harness] fast-dict-iter: in-place Map.forEach for PDFDict.sizeInBytes/copyBytesInto'); -} -if (fastParseDict) { - await import('../book/lib/fast-parse-dict.mjs'); - console.log('[harness] fast-parse-dict: hoist Type/Catalog/Pages/Page sentinel PDFNames out of parseDict'); -} if (fastParseObject) { await import('../book/lib/fast-parse-object.mjs'); console.log('[harness] fast-parse-object: first-byte dispatch in parseObject, gate true/false/null matchKeyword behind byte check'); @@ -472,10 +420,6 @@ if (fastSyncLoad) { await import('../book/lib/fast-sync-load.mjs'); console.log('[harness] fast-sync-load: synchronify PDFParser load path, strip waitForTick machinery'); } -if (fastDictArray) { - await import('../book/lib/fast-dict-array.mjs'); - console.log('[harness] fast-dict-array: PDFDict backed by flat alternating array (subsumes fast-parse-dict + fast-dict-iter)'); -} if (fastIndirectObjects) { await import('../book/lib/fast-indirect-objects.mjs'); console.log('[harness] fast-indirect-objects: PDFContext.indirectObjects dense-array cache for gen=0 PDFRefs'); diff --git a/perf/phase0-measure.mjs b/perf/phase0-measure.mjs index 28b9d2a0..9cc802ae 100644 --- a/perf/phase0-measure.mjs +++ b/perf/phase0-measure.mjs @@ -25,7 +25,7 @@ import { performance } from 'node:perf_hooks'; import { createRequire } from 'node:module'; // Production-equivalent shim wiring (same order as book/render-book.mjs). -await import('../book/lib/fast-refs.mjs'); +await import('../book/lib/fast-refs-class.mjs'); await import('../book/lib/fast-inflate.mjs'); await import('../book/lib/fast-parse-number.mjs'); await import('../book/lib/fast-decode-name.mjs'); From 26eefeeb92f2ff16707758daa064005b0ff29017 Mon Sep 17 00:00:00 2001 From: Kuba Sunderland-Ober Date: Fri, 25 Sep 2026 04:11:09 +0200 Subject: [PATCH 03/53] perf: say that the book build loads detach-pages.js --- perf/README.md | 6 ++++++ perf/detach-pages.js | 5 +++++ 2 files changed, 11 insertions(+) diff --git a/perf/README.md b/perf/README.md index 0705e495..36e561ff 100644 --- a/perf/README.md +++ b/perf/README.md @@ -18,6 +18,12 @@ optimisation, what was tried and failed -- lives split across the seven phase files in [`notes/`](notes/). The current state is summarised at the bottom of this file. +One file here is not lab-only: `detach-pages.js` is loaded into every +book render by `book.bat` and by the deploy workflow +(`.github/workflows/tbdocs-gh-pages.yml`). Edit it as production code. +It stays in this folder because the lab's own scripts load it from here +too. + ## Profiling `paged.browser.js`: canonical command The command we reach for whenever CPU-profiling paged.js: diff --git a/perf/detach-pages.js b/perf/detach-pages.js index 0a6689b8..beddc71b 100644 --- a/perf/detach-pages.js +++ b/perf/detach-pages.js @@ -1,3 +1,8 @@ +// Production code, despite its folder: book.bat and the deploy workflow +// (.github/workflows/tbdocs-gh-pages.yml) load it into every book render +// through render-book.mjs's --additional-script. It stays in perf/ because +// the lab's scripts load it from here too. +// // Paged.Handler that physically removes each finalized page from the // layout tree as soon as paged.js finishes laying it out, then restores // them in original order at afterRendered before page.pdf() runs. From 86a70107ae2122d5b81a350d4aea3d63259680a6 Mon Sep 17 00:00:00 2001 From: Kuba Sunderland-Ober Date: Fri, 25 Sep 2026 04:11:09 +0200 Subject: [PATCH 04/53] builder: PLAN-TOOLING-REVIEW.md, strategy and decisions for the review --- builder/PLAN-TOOLING-REVIEW.md | 289 +++++++++++++++++++++++++++++++++ 1 file changed, 289 insertions(+) create mode 100644 builder/PLAN-TOOLING-REVIEW.md diff --git a/builder/PLAN-TOOLING-REVIEW.md b/builder/PLAN-TOOLING-REVIEW.md new file mode 100644 index 00000000..85d2b335 --- /dev/null +++ b/builder/PLAN-TOOLING-REVIEW.md @@ -0,0 +1,289 @@ +# Tooling review: strategy, decisions and status + +A software-engineering review of the repository's own tooling: `tbdocs`, the gates, the +compiler harness, the book renderer and the smaller tools. It asks about factoring and +repetition, and about sound design against hacks. That makes it a different kind of review +from [REVIEW-c9f2dfe0-1b6922b.md](REVIEW-c9f2dfe0-1b6922b.md), which asked whether one +range of commits was correct. + +The findings will be written to `REVIEW-TOOLING-fe9ce12b.md`. The commit plan is added to +this file once they are in. + +## Status + +- 2026-09-25: strategy agreed; review passes started against `fe9ce12b` on `staging`. +- 2026-09-25: all fourteen passes reported; verification running. Decisions 1 and 5 were + settled further while the passes ran (below), and the four superseded pdf-lib shims were + deleted. The passes also raised a theme the review will lead with: markdown is repeatedly + processed as text, each tool deciding privately what counts as code. + +## Decisions + +Made on 2026-09-25, before the review started. + +1. **`perf/` is a lab notebook, and nothing moves out of it.** It is not reviewed, linted + or formatted. *Amended during the review:* the book build loads one file from it at run + time, `perf/detach-pages.js`. Moving that file would break five lab scripts that load it + from beside themselves, and touch several pages, so it stays; its header and + `perf/README.md` now say that the book build depends on it. Four superseded pdf-lib + shims that only `perf/` loaded (`fast-refs`, `fast-dict-array`, `fast-dict-iter`, + `fast-parse-dict`) were deleted from `book/lib/` rather than moved: pdf-lib is pinned at + its final release, so the comparisons they served are settled, their measurements are + in `perf/notes/08-pdf-lib.md`, and git keeps the code. +2. **Fix in place first.** Splitting a large module is a separate pass after the in-place + work, taken only where the evidence says good practice calls for it. The review records + split candidates; it does not propose splits as fixes. +3. **The before-and-after tree comparison becomes a committed tool**, because every future + `builder/` refactor needs it. +4. **A linter and a formatter run before every commit**, as a matter of course, with a gate + in `test.bat` and both CI workflows as the backstop. **The linter comes first** (Phase + 0): it is most useful while code is being consolidated, when moved and deleted code + leaves unused imports and undeclared names behind. **Formatting is a separate, last + pass** (Phase 6), once every fix from the review is in. The review's citations stay + valid while the fixes are made, and the one formatting commit touches only code that + survived them. +5. **`impexp.py` and `impexp.mjs` both stay**: both are published for readers' + convenience. **`build_fonts.py` stays**: a JavaScript port was evaluated recently and is + blocked by harfbuzzjs ([WIP.Fonts.md](../WIP.Fonts.md)). **The standalone link checker + stays, as a thin wrapper**: decided on A4's evidence. `scripts/check_links.mjs` keeps + its command line and its own reading of a tree from disk, and calls the functions + `builder/check.mjs` exports for everything else; `check_links_diff.mjs` then compares + one implementation reading a tree two ways. +6. **A gate checks that both CI workflows run the same gates as the wrappers**, rather than + generating the workflows and wrappers from one list. + +## Scope + +| Area | Size | Treatment | +|---|---|---| +| `builder/` (tbdocs) | ~16.5k lines | full review | +| `scripts/`, `scripts/lib/` | ~18.7k | full review | +| `book/` renderer and pdf-lib patches | ~4.4k, not counting the 33k-line paged.js fork | full review | +| `test/addin/`, `eval/`, `wisdom/` | 1.5k, 1.0k, 2.4k | lighter review | +| the `.bat` wrappers and both workflows | ~0.9k | reviewed together, as the list of gates | +| `docs/assets/js/` | 0.4k | lighter review; it ships to readers | +| `perf/` | 8.8k, 49 files | not reviewed (decision 1) | +| vendored code: `book/lib/paged.browser.js`, `builder/vendor/` | | not reviewed; only whether our changes to it are recorded | + +## Baseline survey + +Taken at `fe9ce12b` with [`scripts/survey_tooling.mjs`](../scripts/survey_tooling.mjs), to +be re-run when the work is done: a token-level clone detector (identifiers and literals +normalised, windows of 60 tokens) and an import graph over the 185 first-party JavaScript +files, `perf/` included and vendored code excluded. `--summary` prints the first six rows; +`--root` measures a checkout of `fe9ce12b` itself, which does not contain the script. + +| Measure | At `fe9ce12b` | +|---|---| +| clone regions of 60+ tokens | 680, of which 318 do not involve `perf/` | +| top-level function names defined in two or more files | 77, of which 57 outside `perf/` | +| command-line tools outside `perf/` that read `process.argv` | 32, none using `node:util` `parseArgs` | +| tools with a private copy of `flag`/`opt`/`die` | 6, with `opt` in three different versions | +| packages imported but not declared | 2: `picocolors` (installed for `@babel/code-frame`), `pako` (for `pdf-lib`) | +| clone regions by pair of areas: `builder`/`scripts`, `scripts`/`scripts`, `builder`/`builder` | 55, 95, 39 | + +Observed by hand at the same commit: + +| Observation | At `fe9ce12b` | +|---|---| +| places that state which gates run | 5 that execute (three `.bat` files, two workflows), plus Tools.md | +| lint or format tooling | none | +| line endings | LF in the repository; CRLF in 118 of 140 working-tree files under `core.autocrlf=true`; no `.gitattributes` | +| quote style | double in `builder/`, `scripts/`, `test/`, `eval/`; single in `book/`, `wisdom/`, `perf/` | + +## How the review is run + +### Passes + +The last review split its passes by area, and that misses repetition between areas, which +is where much of what the survey found sits. So there are two kinds of pass. An **area +pass** asks about design inside one part. A **lens pass** follows one concern across the +whole tree. All passes run on Sonnet; the orchestrator reconciles them and judges. + +| Pass | Subject | Scope | +|---|---|---| +| A1 | build orchestration and scheduling | `builder/`: `tbdocs`, `scheduler`, `sab-scheduler`, `sab-broadcast`, `cpu-worker`, `worker-pool`, `serve`, `gantt`, `build-info`, `discover`, `data`, `paths` | +| A2 | output stages and auxiliary outputs | `builder/`: `write`, `offline`, `offline-rewrite`, `redirects`, `sitemap`, `search`, `compress`, `scss`, `vendor-assets`, `counts`, `publish-policy`, `page-baseline`, `symbol-baseline`, `symbols` | +| A3 | the Markdown dialect and page templates | `builder/`: `render`, `highlight`, `highlight-theme`, `template`, `seo`, `nav` | +| A4 | link and integrity checks, and the two-checker question | `builder/`: `link-check`, `check`, `check-tree`; `scripts/`: `check_links`, `check_links_diff`, `crawl_check`; `test/fixtures/` | +| A5 | site-reading gates, accessibility, diagrams | `scripts/lib/axe-scan`, `check_a11y`, `check_a11y_fingerprint`, `check_axe_patch_equiv`, `pick_a11y_sample`, `sweep_a11y`, `check_dot_fit`, `build_dot_metrics`, `check_tree_fresh`; `builder/`: `dot`, `dot-metrics` | +| A6 | the gates as a system, and the `test.bat` gates | the seven `.bat` files, both workflows, `check_gate_lists`, `check_publish_policy`, `check_regex_safety` with `lib/regex-fold`, `check_code_regions` with `lib/markdown-files`, `check_page_baseline`, `check_book_coverage`, `check_symbol_index`, `convert_em_dash_separators` | +| A7 | the compiler harness | `scripts/lib/tb-*` with `tb-launch.ps1`, `tbbuild`, `tbrun`, `addin_test`, `check_tb_registry`, `test/addin/` | +| A8 | sample compiling and the package API | `check_examples`, `lib/tb-fences`, `gen_attribute_probes`, `builder/census_attributes`, `build_package_api`, `lib/twin-api`, `lib/tb-packages` | +| A9 | the book pipeline | `book/render-book`, `book/lib/` except the fork, `builder/book`, `builder/pdf`, `book.bat`, and whatever the book build loads from `perf/` | +| A10 | smaller tools and the site's scripts | `eval/`, `wisdom/`, `scripts/impexp.mjs`, `scripts/build_fonts.py`, `docs/assets/js/` | +| L1 | command-line and process conventions | every entry point outside `perf/` | +| L2 | paths, configuration, file walking, dependencies | the whole tree outside `perf/` | +| L3 | browsers, child processes, and rewriting HTML and Markdown as text | the whole tree outside `perf/` | +| L4 | the survey's duplicate-code leads | the clone list, outside `perf/` | + +A verifier, also on Sonnet, then re-reads every R1 and R2 citation against the source. The +orchestrator merges findings that more than one pass reported, and writes the review. + +### What counts as a finding + +Five kinds: + +- **dup**: two or more implementations of one thing. Worst when the copies have already + diverged, because the divergence is a latent bug. +- **hack**: a workaround that fails one of the three tests below. +- **struct**: a module doing several unrelated jobs, a module in the wrong place, a + dependency pointing the wrong way (production code loading from `perf/`), or a missing + seam where a test would need one. +- **conv**: conventions that differ between tools for no reason: flags, help, exit codes, + output streams, error reporting. +- **dead**: superseded code, unused exports, dead flags, comments naming files that have + moved. + +**A workaround is acceptable when it passes three tests.** It is *contained*: in one place, +behind one interface. It is *guarded*: if what it works around changes, something fails +loudly; the axe patch and `check_axe_patch_equiv.mjs` are the model. It has a *stated +exit*: the condition under which it can be removed. A workaround that fails any of the +three is a finding, and so is one whose recorded reason no longer holds. + +**A recorded decision is not a finding.** This codebase explains itself in header comments, +in the `WIP.*.md` casebooks and in `builder/PLAN-*.md`. Before calling anything a hack or a +duplication, look for its recorded reason. Report it only if the reason does not cover what +the code actually does, or its premise has expired, and cite where the reason is. + +**Severity is by cost:** + +- **R1**: has already produced a divergence or a defect, or will on the next ordinary + change: copies that disagree, a list maintained by hand in several places. +- **R2**: makes every change in its area slower or riskier: a helper edited in several + places, a module too large to hold in mind, a missing seam. +- **R3**: local untidiness. + +Size alone is not a finding. A large module is reported as a *split candidate*, with +evidence of what its size costs (decision 2). + +### Rules for every pass + +- **Read-only.** Create, modify or delete nothing in the repository. Scratch files go only + in the session scratchpad. +- **Run nothing that writes outside the scratchpad, starts a browser or starts an IDE.** No + `build.bat`, `serve.bat`, `check.bat`, `test.bat` or `book.bat`, and no + `node builder/tbdocs.mjs`: it writes `docs/_site*` and can rewrite the committed + baselines. No `examples.bat`, `addin-test.bat`, `tbbuild`, `tbrun`, `build_package_api`, + `census_attributes`, `gen_attribute_probes` or `check_tb_registry`: a harness run starts + an IDE and changes the registry, and two must never run at once. No network, no + `npm install`. +- Allowed: reading, `grep`, `git log` / `show` / `blame`, and small Node scripts in the + scratchpad to test a hypothesis, including importing a repository module that does + nothing on import. These gates are read-only and may be run on their own: + `check_gate_lists`, `check_publish_policy`, `check_code_regions`, `check_page_baseline`, + `check_book_coverage`, `check_symbol_index`, `check_regex_safety`, and + `check_examples --census`. +- **Cite `path:line` and the enclosing function or constant.** Line numbers will move + before the last fixes land. +- Stay in scope. Anything seen outside it goes under *Cross-area leads*. +- Be complete on findings and brief in prose. + +### Output format for a pass + +``` +## -- + +### Findings +-. [R1|R2|R3] [dup|hack|struct|conv|dead] + Where: , ... + Recorded reason: | none found (looked in: ...) + Cost: + Fix in place: + Verify by: + Size: S | M | L + +### Split candidates + -- + +### Verified sound + + +### Cross-area leads + +``` + +## Oracles + +The existing gates check the site, not how the tools behave inside, so each refactor is +compared before and after: + +| Tool | Comparison | +|---|---| +| `tbdocs` | Build before and after into scratch `--dest` folders with `--no-fetch-assets`; the three trees must match byte for byte. The first step is to show that two builds of one commit already match. Known differences to exclude: `assets/images/gantt.svg`, which holds the build's own timings, and the commit on the PDF title page. This becomes a committed tool (decision 3). | +| gates | The same output on the real tree; and a changed gate must still fail when its original defect is put back. | +| harness | The `examples.bat` summary unchanged (1,119 samples) and `addin-test.bat` green, against the local BETA 983. Never two harness runs at once. | +| book | Page count, outline and extracted text unchanged; the PDF's bytes include timestamps. | +| CI | A workflow dispatch on the fork, for any commit that changes a workflow. | + +## Execution + +Each phase lands as commits on `staging`, the working branch, which is merged upstream when a +chunk of work is done. + +**Phase 0: process and oracles**, before any finding is fixed. + +- The tree comparison tool (decision 3). +- The CI-roster gate (decision 6), with probes that make it fail on a deliberately + mismatched workflow. +- The linter, lint rules only (decision 4): + - Biome, pinned to an exact version, is the candidate: one package, `npm install` still + enough to run everything. Confirm it on the tree before adopting it, counting findings + and false positives on the mixed Node and browser code. ESLint is the fallback. + - Scope: `builder/`, `scripts/`, `book/`, `eval/`, `wisdom/`, `test/`, + `docs/assets/js/`. Excluded: `perf/`, the vendored code, generated JSON (the two + baselines, `package-api.json`, `inter-metrics.json`), `package-lock.json`, and every + Markdown, SCSS, YAML and `.bat` file. + - Correctness rules only. No style rules until Phase 6. + - Findings with a mechanical fix are fixed here. A rule whose findings need design work + starts disabled, and the phase that fixes them enables it. + - A gate in `test.bat` and both workflows; a `pre-commit` hook in a committed + `.githooks/`, checking the staged files; the rule in WIP.md and Tools.md. Enabling the + hook in a clone sets `core.hooksPath`, and that change to git configuration is + confirmed before it is made. + +**Phase 1: remove and relocate.** Move `census_attributes.mjs` out of `builder/`; declare +the two packages; delete the dead code the review identifies. Done ahead of it, during the +review: the four superseded pdf-lib shims deleted, and `perf/detach-pages.js` marked as +code the book build loads (decision 1). + +**Phase 2: shared code, in place, with no change in behaviour.** Shared modules for what +the review finds repeated, adopted one tool at a time: argument parsing on `node:util` +`parseArgs` that preserves every tool's current flags and exit codes, paths and +configuration, the gates' self-test scaffolding, browser launching, and the helpers +`builder/` defines twice. + +**Phase 3: conventions users see.** Flags, exit codes and help text brought into line, +with Tools.md updated in the same commit. + +**Phase 4: splits.** A separate pass over the split candidates, taken only where the +evidence supports it (decision 2). + +**Phase 5: documentation and measurement.** The Builder.md module map, Tools.md and +WIP.Build.md brought up to date; the survey re-run and compared with the baseline above. + +**Phase 6: formatting** (decision 4), once every fix from the review is in. + +- The formatter configured to the majority style and pinned to an exact version, with any + style lint rules alongside it. +- Line endings settled so the check means the same on a CRLF Windows checkout and an LF CI + checkout; otherwise every local run flags 118 files. Either a `.gitattributes` for the + formatted file types or the formatter's own setting, shown to agree on both platforms. +- One mechanical commit, listed in `.git-blame-ignore-revs`. The tree comparison shows + what it changed in the output; only the two published site scripts should differ. +- The gate and the hook extended to check formatting; WIP.md and Tools.md updated. + +## The bar for each commit + +- `build.bat && check.bat && test.bat` clean; after Phase 0, lint clean; after Phase 6, + format clean. +- The commit's oracle from the table above, with every difference it shows explained in + the commit message. +- A commit that changes a gate shows the gate still failing on its reintroduced defect. +- One concern per commit, with a concise one-line message. + +## Open questions + +None of the plan's own. Both that it started with are settled: the two link checkers +(decision 5), and the survey script, which is committed. The decisions the review raises are +listed in `REVIEW-TOOLING-fe9ce12b.md`. From e0d127f0f471d11fa9ecba43080dbf2f4232fb7b Mon Sep 17 00:00:00 2001 From: Kuba Sunderland-Ober Date: Fri, 25 Sep 2026 05:09:43 +0200 Subject: [PATCH 05/53] wisdom: close the blockquote fence in a Len.md staging section --- wisdom/data/findings/staging.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/wisdom/data/findings/staging.md b/wisdom/data/findings/staging.md index 94002ac4..9d14efb7 100644 --- a/wisdom/data/findings/staging.md +++ b/wisdom/data/findings/staging.md @@ -14242,8 +14242,8 @@ _Date range: 2023-09-19 to 2023-09-20_ > In twinBASIC x64 builds, `Len()` and `LenB()` return different values for UDTs whose members are pointer-sized or otherwise architecture-dependent. `Len()` returns a character or element count that does not reflect the actual byte footprint on x64, while `LenB()` returns the correct byte size. Code ported from VB6 or 32-bit twinBASIC that passes `Len()` as a byte count to `CopyMemory` or uses it to size a buffer with `ReDim` will silently allocate the wrong amount of memory on x64. Always use `LenB()` when the intent is to obtain the byte size of a UDT for memory operations. When dimensioning a byte array to hold a UDT for `CopyMemory`, subtract 1 from the result because array indices are zero-based: > > ```tb -ReDim arr(LenB(udt) - 1) -``` +> ReDim arr(LenB(udt) - 1) +> ``` _Source threads: 1314412625714479125 · confidence: high_ _Date range: 2024-12-06_ From bb2bf38e6537edd2e3e515fbbdcc85d62692b109 Mon Sep 17 00:00:00 2001 From: Kuba Sunderland-Ober Date: Fri, 25 Sep 2026 05:09:43 +0200 Subject: [PATCH 06/53] builder: the tooling review at fe9ce12b, with its evidence --- builder/REVIEW-TOOLING-fe9ce12b.md | 618 ++++++++++++++++++ builder/REVIEW-TOOLING-fe9ce12b/README.md | 18 + builder/REVIEW-TOOLING-fe9ce12b/V1.md | 80 +++ builder/REVIEW-TOOLING-fe9ce12b/V2.md | 69 ++ builder/REVIEW-TOOLING-fe9ce12b/V3.md | 100 +++ builder/REVIEW-TOOLING-fe9ce12b/V4.md | 134 ++++ builder/REVIEW-TOOLING-fe9ce12b/ledger.md | 577 ++++++++++++++++ .../markdown-inventory.md | 273 ++++++++ 8 files changed, 1869 insertions(+) create mode 100644 builder/REVIEW-TOOLING-fe9ce12b.md create mode 100644 builder/REVIEW-TOOLING-fe9ce12b/README.md create mode 100644 builder/REVIEW-TOOLING-fe9ce12b/V1.md create mode 100644 builder/REVIEW-TOOLING-fe9ce12b/V2.md create mode 100644 builder/REVIEW-TOOLING-fe9ce12b/V3.md create mode 100644 builder/REVIEW-TOOLING-fe9ce12b/V4.md create mode 100644 builder/REVIEW-TOOLING-fe9ce12b/ledger.md create mode 100644 builder/REVIEW-TOOLING-fe9ce12b/markdown-inventory.md diff --git a/builder/REVIEW-TOOLING-fe9ce12b.md b/builder/REVIEW-TOOLING-fe9ce12b.md new file mode 100644 index 00000000..0c14480c --- /dev/null +++ b/builder/REVIEW-TOOLING-fe9ce12b.md @@ -0,0 +1,618 @@ +# Review of the tooling at `fe9ce12b` + +Scope: all first-party tooling — `builder/`, `scripts/`, `book/`, `eval/`, `wisdom/`, `test/`, the `.bat` wrappers and both CI workflows (`perf/` and vendored code excluded) · reviewed 2026-09-25 · line numbers refer to `fe9ce12b` + +This is a different kind of review from [REVIEW-c9f2dfe0-1b6922b.md](REVIEW-c9f2dfe0-1b6922b.md), which asked whether one range of commits was correct. This one asks about factoring and repetition, and about sound design against workarounds, across the whole of the repository's own tooling. Its charter, rubric and decisions are in [PLAN-TOOLING-REVIEW.md](PLAN-TOOLING-REVIEW.md). + +--- + +## Verdict + +The tooling holds up well where it matters most. Fourteen review passes and four independent verifiers read `builder/`, `scripts/`, `book/`, `eval/`, `wisdom/`, `test/` and the wrappers directly against `fe9ce12b`, and confirmed nearly everything they found. The scheduler's barrier and SAB layout, the compiler harness's process discipline, the axe patch and the dot-metrics WASM patch, the gates' probe-and-report shape, and the paged.js fork's divergence record are all sound, for the reasons given under Verified sound below. + +Where the tooling is weak, it is weak in the same two ways, repeated across many files. First, small helper functions — URL builders, HTML escapers, baseline serializers, argv parsers, repo-root derivations, directory walkers — exist in two, three, or for HTML escaping seven copies, and several of the copies have already disagreed with each other, not merely risked it: a link checker with a narrower attribute table than its sibling, three CLI parsers that return `undefined` instead of a stated default on a trailing flag, two frontmatter readers with no BOM handling next to a third that has it, two install-finders that recognise an install by different files. Second, a smaller number of workarounds fail the review's own three-part test for an acceptable one — contained, guarded, with a stated exit. The clearest cases are the three post-render HTML rewrites that depend on an invariant (code content is always entity-escaped) that is true for three of the four ways text can reach a `` block and false for the fourth, and the pdf-lib shims, pinned against accidental version drift but asserting nothing about what they overwrite. + +The single most important theme, raised by the repository's owner partway through the review, threads through both weaknesses: markdown is handled as text, not as a parsed structure, and almost every tool that touches it re-derives, by hand, what counts as a fence, a heading, or a code span. The costs range from a reproducible bug — two fence-opener predicates in the same file disagree on one input — to a genuine defect in a file the repository commits: a missing blockquote marker in `wisdom/data/findings/staging.md` breaks a fence, and the tool that re-serializes that file's sections has no way to notice. Closing that gap with one shared, markdown-it-based module is the review's leading recommendation for the fix phases that follow; the remaining findings are smaller individually but numerous, and the plan schedules them across its phases, described in [PLAN-TOOLING-REVIEW.md](PLAN-TOOLING-REVIEW.md#execution). + +--- + +## Provenance + +Fourteen passes, all on Sonnet, read the tree at `fe9ce12b` without changing it: reading, `git log`/`blame`, scratch scripts in the session scratchpad, and the read-only gates, but never a build, a browser, or the compiler harness: + +| Pass | Subject | +|---|---| +| A1 | Build orchestration and scheduling | +| A2 | Output stages and auxiliary outputs | +| A3 | The Markdown dialect and page templates | +| A4 | Link and integrity checks, and the two-checker question | +| A5 | Site-reading gates, accessibility, diagrams | +| A6 | The gates as a system, and `test.bat` | +| A7 | The compiler harness | +| A8 | Sample compiling and the package API | +| A9 | The book pipeline | +| A10 | Smaller tools and the site's scripts | +| L1 | Command-line and process conventions, every entry point | +| L2 | Paths, configuration, file walking, dependencies | +| L3 | Browsers, child processes, rewriting HTML and Markdown as text | +| L4 | The survey's duplicate-code leads | + +Four verifiers, also on Sonnet, then re-read every R1 and R2 citation against the source as it is at `fe9ce12b` (`git show fe9ce12b:`, so commits made meanwhile could not move anything under them): + +| Verifier | Scope | Findings checked | Verdict | +|---|---|---|---| +| V1 | `builder/` (A1–A3, A9 subset, A2/L2/L4 overlaps) | 23 | All confirmed; several severity notes, five items noticed in passing | +| V2 | Gates, links, a11y, CI, small tools (A4–A6, A10, L1–L2, L4 overlaps) | 29 | 25 confirmed, 4 corrected (A4-3, A5-1, L2-5, L2-6) | +| V3 | Harness, samples, book (A7–A9, L2, L4 overlaps) | 15 | 10 confirmed, 5 corrected (A7-1, A7-3, A7-4, A8-2, A9-5) | +| V4 | L3 (browsers, processes, text rewrites) | 6 | All confirmed; one nuance (L3-3), one conflict ruled (A3-6) | + +The orchestrator ran six checks of its own, beyond what the verifiers covered, and reports them because each one changed a finding's stated impact or resolved a conflict the passes left open: + +- **The census Overridable check against the BETA 983 package cache** — traced A8-1's claimed misclassification against the real exported package tree and found zero attributed `Overridable` members exist there today, so the divergence is real but dormant. +- **The tbrun regex** — compared tbrun.mjs's failed-build pattern with tb-ide.mjs's `BUILD_FAILED` by reading (A7-1). The orchestrator first reported the narrower pattern as matching two of the five failure shapes; it is case-insensitive and matches three, as V3 found. The two it misses are the ones the finding names. +- **The Gantt sections** — confirmed A1-1's claim that Check-phase tasks and `vendorAssets` are dropped from the Gantt chart on every build. +- **The BOM scan of `docs/`** — confirmed A10-2's BOM gap is dormant: zero of 912 markdown files begin with a UTF-8 BOM today. +- **The staging.md `parseStaging` run** — ran the real parser against the real, committed 15,553-line file and found the inventory's claim of live section-metadata mis-pairing not confirmed; see Where the passes were wrong. +- **The Wisdom.md:304–305 fence** — confirmed `check_gate_lists.mjs`'s `splitSections` genuinely mis-splits on a heading-shaped line inside a fenced example at that location, dormant only because the phantom section states no gate count. + +The working records — the orchestrator's ledger, the four verifiers' files and the markdown inventory — are committed beside this document, in [`REVIEW-TOOLING-fe9ce12b/`](REVIEW-TOOLING-fe9ce12b/README.md). + +Four commits were made on `staging` during the review, all before this document: + +- **`d0a652d6`** — `scripts/survey_tooling.mjs`, the before-and-after clone-detection and import-graph tool the plan's baseline survey and this document's appendix both depend on (decision 3). +- **`90624841`** — deleted the four superseded pdf-lib shims (`fast-refs`, `fast-dict-array`, `fast-dict-iter`, `fast-parse-dict`) that only `perf/` loaded, approved by the user during the review. Resolves **A9-4**, **A9-6**, **A9-9**. +- **`26eefeeb`** — documented, in `perf/detach-pages.js` and `perf/README.md`, that `book.bat` and the deploy workflow load that file at run time — the amendment to decision 1 (nothing else moves out of `perf/`). +- **`86a70107`** — `PLAN-TOOLING-REVIEW.md` itself, recording the review's strategy and the decisions made as the passes ran. + +--- + +## Themes + +### Markdown is processed as text, each tool deciding privately what counts as code + +This is the theme the repository's owner raised while the passes were running, and it is the one worth fixing first. A dedicated inventory pass ([`REVIEW-TOOLING-fe9ce12b/markdown-inventory.md`](REVIEW-TOOLING-fe9ce12b/markdown-inventory.md)) found **15 sites** that scan or rewrite markdown by hand. Five rewrite text: `render.mjs`'s `maskCodeRegions`/`maskInlineCode` and its `stashCodeFences`/`rewriteAdmonitions` (behind **A3-1**), `convert_em_dash_separators.mjs` (behind **L3-4**/**L4-7**), `check_examples.mjs`'s marker splice, and wisdom's `parseStaging`/`serializeStaging` (behind **L3-3**). Ten only scan: `census_attributes.mjs` and `gen_attribute_probes.mjs` reading `Attributes.md` line by line, `check_gate_lists.mjs`'s section splitter, `counts.mjs`'s two raw-source counts, wisdom's two frontmatter readers (behind **A10-2**), two readers in `eval/`, and one test that reads markdown returned by the compiler's hover. Of the 15, **eight have no code awareness of any kind** — not even a hand-written fence check — and six more have a hand-written fence or frontmatter rule with a known, named gap. Exactly one site (`check_examples.mjs`'s marker splice) locates its edit through a markdown-it-derived line number and then re-verifies the line before touching it; it is the model the rest should follow, alongside the token-traversing plugins in `render.mjs` and `check_code_regions.mjs`'s own gate, which already consult markdown-it's token stream instead of re-deriving it (`scripts/lib/tb-fences.mjs`, `builder/discover.mjs`'s `gray-matter` use). + +Two live problems came out of tracing this by hand rather than by inference. `check_gate_lists.mjs`'s `splitSections` genuinely splits `docs/Documentation/Wisdom.md`'s `### staging.md format` section into two phantom sections at `Wisdom.md:305`, because that line is a fenced *example* of a `## ` heading, not a real one — dormant only because the phantom section happens to state no gate count. And the real, committed `wisdom/data/findings/staging.md` has a genuine content defect at lines 14245–14246: a fenced sample nested inside a `> [!NOTE]` admonition is missing its blockquote continuation marker on the fence's content line, so markdown-it does not close the fence where the source intends — it runs on for 52 lines. The inventory's stronger claim, that this already causes `parseStaging` to attach the wrong metadata to the wrong section, is **not confirmed** — see Where the passes were wrong. + +Markdown-it itself was tested directly, empirically, rather than assumed. Block-level tokens (`fence`, `code_block`, `html_block`) have a `.map` giving the exact source line range, including a fence nested inside a blockquote or a list item, prefix markers included in the raw slice and stripped in `.content` — both available from one parse. Inline tokens have no offsets at all (`.map` is always `null`), so a correct inline code-span splitter still has to be the same manual backtick/tilde-run scan `maskInlineCode` and `splitInlineCode` already implement independently — markdown-it can supply which lines an inline run occupies, never where within them. Frontmatter is not merely unhandled by markdown-it: it is **actively misparsed** unless split off first — a leading `---` becomes an `hr`, and the block that follows is then read as a setext H2 heading, so frontmatter must be split off before markdown-it sees a page, as `discover.mjs` already does. Timing over all 912 markdown files under `docs/` (3.98 MiB): a block-only parse (bypassing inline/linkify/replacements/smartquotes) takes 33.7 ms total, about 0.037 ms/file; a full parse takes 234–257 ms, about 0.26–0.28 ms/file — roughly seven times cheaper for block-only, verified to genuinely skip inline splitting (every inline token's `.children` stays the empty-array default). + +One more fact belongs to this theme, found independently while tracing the frontmatter gap: the build runs **two YAML parsers**. `gray-matter@4.0.3` bundles its own nested `js-yaml@3.14.2` (`node_modules/gray-matter/node_modules/js-yaml`), while `builder/data.mjs`, `builder/tbdocs.mjs` and `scripts/check_publish_policy.mjs` import the top-level `js-yaml@4.1.1` directly for `_config.yml`/`_book.yml`. Page frontmatter and site configuration are parsed by two major versions of the same library. See Decisions for you (a). + +### Command-line handling + +Thirty-six entry points (32 argv readers plus 4 argument-less gates) were inventoried by L1, and the pattern across them is consistent: every tool parses its own flags by hand, and several of the hand-written parsers have already shipped the same class of bug more than once. **L1-1** through **L1-13** catalogue it in full (see Findings); the sharper instances are a positional-option helper reimplemented seven times in four variants, one of which returns `undefined` instead of its stated default on a trailing value flag (**L1-2**); a theme/viewport validator that exists in one script specifically because an unvalidated value once mislabeled a report, and is missing from two siblings that do the same kind of work, one of them the axe-upgrade gate itself (**L1-1**); and a positional-argument detector that fails outright on a boolean flag placed before a path (**L1-3**). The area passes hit the same shape independently: **A5-1** found the identical manually written argv loop, diverged three ways, in eight accessibility and diagram scripts; **A7-5** found three incompatible `opt()`/`die()` shapes inside the compiler harness alone, with a concrete, traced misdiagnosis (a `NaN` timeout produces the wrong error message, not a timeout error); **A10-1** found the same split among the four parsers in `eval/`. The exception is `impexp.mjs`, whose verb table, named exit codes and single usage-error path L1 names as the model to follow. **L1-13** names the reason none of this was caught: nothing tests any tool's own argument handling, anywhere in the repository. See Decisions for you (e) for the proposed convergence. + +### Copies that have already diverged + +The review's severity rubric reserves R1 for a duplication that has already produced a disagreement, not merely risked one, and a sizeable share of the R1 tier is exactly that: **A4-3**'s link-attribute table is five pairs against the build checker's 21 tags and 26 pairs, and the post-release checker that uses the narrower one already misses seven kinds of link the build-time checker catches; **A3-3**/**L4-6**'s two URL helpers disagree on a forced leading slash, on protocol-relative `//host`, and on null input, in three separately confirmed ways; **A7-1**'s failed-build detector misses two of the five failure shapes its own sibling module lists, so a failed compile can report success; **A8-1**'s attribute-keyword lists already differ (dormant only because the current package cache never exercises the gap); **A10-2**'s three independent frontmatter readers disagree on BOM handling and on type coercion; **L2-1**'s two output-tree exclusion lists are missing the same prefix that a prior fix (`check_tree_fresh.mjs`) already closed once, in a different pair of files; **L2-2**'s two twinBASIC-install finders recognise an install by different files, so a partly unpacked install passes one and fails the other; **L4-10**'s two `logicalLines` implementations differ on BOM handling and on block comments. Every one of these copies has already diverged; two of the divergences (**A8-1**, **A10-2**) happen to change nothing on today's content. Each is presented in full under Findings, tier 1. + +### Dead code left by the retired `_diff`/`_triage` tools + +Commit `644d6bdb` deleted `_diff.mjs`, `_triage.mjs`, `_sitemap_diff.mjs` and their siblings, and the tree still shows two kinds of trace of them: functions that only they called, and comments that still name them. **A1-4**'s exported `makeTimer` has zero callers anywhere in the repository, while `offline.mjs` keeps a private, byte-identical copy whose own comment cites the deleted tools as the reason it can't import the exported one. **A2-1** (absorbing **A1-5** and **L4-4**) is five more dead paths from the same cause — `writeOfflinePages`, an unread `precomputed` parameter, an unreachable fallback branch, `writeSearchData`, `extractSitemapUrls` — plus a 24-name re-export block in `offline.mjs` with zero importers for any of the 24. **A2-2** (absorbing **A9-10**'s comment half) is five files whose comments still name the deleted tools, one of which (`pdf.mjs`) uses the stale comment to justify an export, `extractImagePaths`, that also has zero callers. **A9-4** and **A9-9** were two more instances of the same shape in `book/lib/`'s pdf-lib shims; both are now moot, resolved by the shim deletion in `90624841`. + +### Gate scaffolding and conventions + +The gates are individually well built — see Verified sound — but the scaffolding around them repeats itself and has already drifted in one place. **A6-1** (absorbing **L1-11** and **L4-13**) is a self-test accumulator, a report loop, and an `uncaughtException` handler, each copied across three or four gate scripts, with one of the three already re-indenting its output differently from its two siblings. **A6-2** is a genuine three-way disagreement, inside the repository itself, about the history of a past incident (`test.bat`'s own comment tells a different story than `check_gate_lists.mjs`'s header and `Tools.md`, neither of which corroborates `test.bat`'s specific counts) — in the comment that introduces the gate built to stop exactly that kind of drift. **A5-2** and **A6-3** are three tools that break the documented 0/1/2 exit convention (`build_dot_metrics.mjs`, `pick_a11y_sample.mjs`, `convert_em_dash_separators.mjs`); `pick_a11y_sample.mjs` runs in `check.bat` today, so a crash there reads as a coverage gap rather than a crash. **A6-4** is the structural gap decision 6 exists to close: nothing today reads either CI workflow to confirm it runs the same gates as the wrappers, though the two workflows' shared steps are in fact byte-identical, differing only in three already-recorded ways. + +### Dependencies and pinning + +**A1-2** is an undeclared direct dependency (`picocolors`, imported by `tbdocs.mjs` and `scheduler.mjs` and reachable today only through `puppeteer → cosmiconfig → parse-json → @babel/code-frame`); it works by accident of the lockfile, not by declaration. `book/lib/fast-inflate.mjs` imports `pako` the same way, through `pdf-lib`. Separately, `docs/Documentation/Builder.md`'s own "Dependencies" section is stale in three ways at once — it omits `recheck` entirely, states `wasm-graphviz` as `^1.21` when the installed version is `^1.29.1`, and says `axe-core` is the only exact pin when four packages are pinned exactly (`axe-core`, `pdf-lib`, `puppeteer`, `recheck`) — a repeat of a drift the previous review already fixed once, in `74b3395`. Each individual pin has its own recorded technical reason; nothing states the general policy behind the choice of exact versus caret, so a caret on a dependency whose output the build depends on reads as an oversight rather than a decision. See Decisions for you (g). The pdf-lib shims (**A9-1**, **A9-2**) round out the theme from the workaround side, immediately below. + +### Workarounds judged by the three tests + +The plan's bar for an acceptable workaround is that it is contained, guarded, and has a stated exit. Two workarounds pass emphatically and are the model for the rest: the axe patch (`scripts/lib/axe-scan.mjs`'s `SOURCE_PATCHES`, with `check_axe_patch_equiv.mjs` as its guard and "until axe ships a modern build" as its stated exit) and the dot-metrics WASM patch (signature match, a read-back check, a real layout check, and a `WeakSet` guard). Against that model, several workarounds found by the passes fall short. The pdf-lib shims (**A9-1**) are contained — one file per shim — but guarded only by an exact version pin, which catches accidental drift and nothing else: no shim asserts anything about the pdf-lib internals it depends on, and there is no equivalence test against stock pdf-lib output anywhere (**A9-2**). The three whole-page HTML rewrites in `render.mjs` and `template.mjs` (**A3-6**) depend on an invariant — code content is always entity-escaped before these run — that holds for three of the four ways text can end up inside ``/`
` and not the fourth (raw, hand-authored HTML, which markdown-it passes through unescaped by design), is not stated at any of the three call sites, and is outside what `check_code_regions.mjs` checks. WIP.Build.md already warns, about a neighbouring case, "Do not rely on that accident"; nothing connects that warning to these three. Wisdom's `parseStaging` (**L3-3**) promises in its header comment that it never silently drops reviewer content, and breaks the promise whenever a bare `---` line sits inside a code fence in a section: the rest of that section's body and its `finding_ids` line are dropped without a word.
+
+---
+
+## Findings
+
+Findings are grouped by severity after the verifiers' corrections, not as first reported. Where a finding was reported by more than one pass, every source ID is kept so it can be traced back to [`REVIEW-TOOLING-fe9ce12b/ledger.md`](REVIEW-TOOLING-fe9ce12b/ledger.md).
+
+### Tier 1 — already diverged, or one ordinary change from it
+
+#### A1-1 — R1, struct
+**The build's own Gantt chart silently drops the Check phase and `vendorAssets` from every build, and its "Other" bucket is dead.**
+
+- Where: `builder/gantt.mjs:39-49` (`mainSections`, only pushes `Seeds`/`Spine`/`Render`/`Write`), `:12` (`COLORS.Other`, unreferenced); `builder/tbdocs.mjs:1270-1282` (`GANTT_SECTION`/`GANTT_SECTION_ORDER` list `Check`); `vendorAssets` (`tbdocs.mjs:539-562`, `runOnMain:true`, no `GANTT_SECTION` entry).
+- Recorded reason: none found; `docs/Documentation/Builder.md:410` states the opposite — that a section-less task falls into a generic "Other" bucket.
+- Cost: reaches the published Build Info page. `checkBook`, `checkReport` and `vendorAssets` durations stretch the chart's time axis with no visible bar to show for it.
+- Fix in place: add `Check` and `Other` to `gantt.mjs`'s `mainSections`; give `vendorAssets` a `GANTT_SECTION` entry.
+- Verify by: the built `gantt.svg` names `checkBook`, `checkReport` and `vendorAssets` after the fix and not before (the byte comparison of the trees excludes this file, because it records timings); correct `Builder.md:410`.
+- Size: S.
+- Verified: confirmed (V1), and checked directly by the orchestrator (see Provenance).
+
+#### A1-2 — R1, hack
+**`picocolors` is imported directly in two files but never declared; it works only because it is an unlisted transitive dependency.**
+
+- Where: `builder/tbdocs.mjs:35`, `builder/scheduler.mjs:5`; reachable today via `puppeteer → cosmiconfig → parse-json → @babel/code-frame` (confirmed in the lockfile).
+- Recorded reason: none.
+- Cost: the day an update stops that chain installing it, every build — local, CI and deploy — stops at import with "Cannot find package 'picocolors'", for a reason unrelated to the colour output that uses it.
+- Fix in place: declare `picocolors` `^1.1.1` directly in `package.json`.
+- Verify by: `npm ls picocolors` before/after; build unaffected.
+- Size: S.
+- Verified: confirmed (V1).
+
+#### A3-1 / L3-4 / L4-7 — R1, dup
+**Two fence-opening predicates 1,550 lines apart in the same file disagree on whether a backtick-fenced block with a backtick in its info string is a valid opener, and a third file, and a fourth function, repeat the same gap.**
+
+- Where: `builder/render.mjs:141-175` `maskCodeRegions` (open test `:148`, no info-string check) vs. `:1704-1730` `stashCodeFences` (`:1713`, CommonMark-correct refusal); `scripts/convert_em_dash_separators.mjs:59-60` `FENCE_OPEN_RE` is byte-for-byte identical to `maskCodeRegions`'s pattern and shares the same gap; `render.mjs:180-204` `maskInlineCode` and `convert_em_dash_separators.mjs:74-99` `splitInlineCode` independently implement the same inline code-span algorithm (confirmed behaviourally equivalent today across 16 edge cases, not yet diverged).
+- Recorded reason: none; the three were written independently.
+- Cost: reproduced directly. A fence opened with `` ```abc`def `` is masked (protected) by `maskCodeRegions` but left exposed to `stashCodeFences`, so a `> [!NOTE]` inside it is rewritten into a live HTML admonition. `check_code_regions.mjs` is structurally blind to this class. Not observed in `docs/` today.
+- Fix in place: one shared, CommonMark-correct fence-opener predicate and one shared inline-code splitter, in the markdown module proposed under Decisions for you (a); both `render.mjs` functions and `convert_em_dash_separators.mjs` migrate onto it.
+- Verify by: the reproduction case above, turned into a probe in `check_code_regions.mjs`; tree comparison.
+- Size: M.
+- Verified: A3-1 confirmed by direct reproduction (V1); L3-4 confirmed, byte-identical regex (V4), which itself notes the merged finding should inherit A3-1's R1 rather than average down; L4-7 confirmed equivalent today (V1).
+
+#### A3-3 / L4-6 — R1, dup
+**Two URL helpers, ported into two files, have diverged in three separately confirmed ways.**
+
+- Where: `builder/seo.mjs:121-148` `absoluteUrl`/`relativeUrl` (config-arg, forces a leading slash at `:145-148`, no space encoding, `isAbsoluteUrl`'s scheme-only regex at `:160-162` misses `//host`) vs. `builder/template.mjs:917-936` (baseurl-arg, leaves a bare value unchanged at `:922`, encodes spaces at `:920`, its `:919` condition explicitly excludes `//host`); non-string/null input also handled differently (`null` vs. `""`).
+- Recorded reason: none.
+- Cost: no call site hits the gap today — this site's `baseurl` is always `""`, which masks the `//host` disagreement — but two independently maintained URL helpers is exactly the shape the rubric reserves for R1.
+- Fix in place: one `builder/url.mjs`.
+- Verify by: every URL in both trees, byte-identical before/after.
+- Size: M.
+- Verified: confirmed, all three disagreements traced directly (V1).
+
+#### A4-3 — R1, dup
+**The only post-release, real-HTTP link checker keeps its own five-pair attribute table instead of using the build checker's 21-tag, 26-pair table, and already misses seven kinds of link the build-time checker catches.**
+
+- Where: `scripts/crawl_check.mjs:91-108` (own if/else chain: `a`/`href`, `link`/`href`, `img`/`src`, `script`/`src`, `iframe`/`src`) vs. `builder/link-check.mjs:38-60` `LINK_ATTR_TABLE` (21 tags, 26 flattened pairs, 9 distinct attribute names — corrected from an initial count of 20).
+- Recorded reason: none found.
+- Cost: live, on the deployed site — `crawl_check.mjs` never follows `srcset`, `poster`, `cite`, `formaction`, `action`, `data` or `longdesc` links, which the build-time checker does cover.
+- Fix in place: export `LINK_ATTR_TABLE` (and `splitSrcset`) from `link-check.mjs`, and drive `crawl_check.mjs`'s tag handler from it, keeping its own HTTP concerns (concurrency, redirects, HEAD-then-GET). This is separate from the owner's decision on the two filesystem link checkers, which does not cover `crawl_check.mjs`.
+- Verify by: `crawl_check.mjs`'s findings before/after over a page using one of the missed attributes.
+- Size: S.
+- Verified: corrected (V2) — table size corrected to 21/26; severity raised from the pass's original R2 to R1 because the gap is live.
+
+#### A5-1 — R1, dup/conv
+**Argv-parsing is hand-written in eight scripts and has already diverged three ways.**
+
+- Where: `scripts/check_a11y.mjs:63-79` (no `--help` case, falls to "unknown arg", exit 2) vs. `scripts/pick_a11y_sample.mjs:144-147` and `scripts/sweep_a11y.mjs:106-110` (stderr, exit 0); `scripts/check_dot_fit.mjs:47` and `scripts/build_dot_metrics.mjs:55` (bare `.includes()`, silently ignores a typo'd flag); `scripts/check_a11y_fingerprint.mjs:115-121` and `scripts/check_axe_patch_equiv.mjs:43-45` (stdout, exit 0); `scripts/check_tree_fresh.mjs:76-90,82` (a third stdout example, in scope but left out of the pass's original count of seven).
+- Recorded reason: none.
+- Cost: a typo'd flag does nothing, silently, in two scripts; feeds the wider L1 convention gap.
+- Fix in place: the shared argv module proposed under command-line handling (Decisions for you (e)).
+- Verify by: each script's current flags and exit codes, preserved exactly.
+- Size: L (the fix is the shared module itself).
+- Verified: corrected (V2) — count raised from seven scripts to eight.
+
+#### A6-1 / L1-11 / L4-13 — R1, dup
+**A self-test accumulator, its report loop, an `uncaughtException` crash handler, and a `withBaseline` test fixture are each copied across three or four gate scripts, and have already diverged once.**
+
+- Where: `check_page_baseline.mjs:38-44,128-139`; `check_book_coverage.mjs:81-87,158-170`; `check_symbol_index.mjs:40-45,361-370`; the crash handler additionally byte-identical in `check_publish_policy.mjs:29`; missing entirely from `check_tb_registry.mjs`, whose only error path ends at exit 1 for both a crash and a real failure; `withBaseline` duplicated between `check_page_baseline.mjs:46-55` and `check_symbol_index.mjs:314-323`.
+- Recorded reason: none.
+- Cost: `check_book_coverage.mjs:162` is the only one of the three that re-indents a multi-line detail string — already diverged. A shared fix must keep probes unconditional and take the probe-failure exit code as a parameter, since two other gates (`check_gate_lists`, `check_regex_safety`) use probes to guard a separate sweep rather than as the whole gate.
+- Fix in place: one shared self-test/report/crash-handler helper, and one shared `withBaseline`.
+- Verify by: each gate's current probe count preserved (18, 6, 11, 12, 46, 22, 12, per the Sound tally).
+- Size: M.
+- Verified: confirmed (V2).
+
+#### A6-2 — R1, dup
+**Three places in the repository disagree about the history of the same past incident, and the odd one out is the comment that introduces the gate built to stop exactly that kind of drift.**
+
+- Where: `test.bat:34-43` (frames it as a fix that introduced three more wrong numbers) vs. `check_gate_lists.mjs`'s header (~30-44) and `Tools.md:435,437` (agree with each other: the numbers were already wrong in the commit that shipped the gate green). `test.bat`'s specific counts are corroborated by neither other account.
+- Recorded reason: three competing recorded reasons is itself the finding.
+- Cost: a maintainer reading `test.bat`'s comment gets an uncorroborated story.
+- Fix in place: trim `test.bat`'s comment to a citation into `check_gate_lists.mjs`'s header, matching the file's other seven comments.
+- Verify by: re-read after the edit; no behaviour change.
+- Size: S.
+- Verified: confirmed, the three-way disagreement traced exactly (V2).
+
+#### A7-1 — R1, dup
+**`tbrun`'s failed-build detector matches 3 of the 5 failure shapes its own sibling module lists, and still misses the two that matter.**
+
+- Where: `scripts/tbrun.mjs:327` (`/^\[(BUILD\]\s+failed|LINKER\]\s+FAILED)\b/i`) vs. `scripts/lib/tb-ide.mjs:730-734` `BUILD_FAILED` (5 shapes).
+- Recorded reason: the round-8 incident that motivated `tb-ide.mjs`'s own list is recorded; `tbrun.mjs`'s regex was never brought into line with it.
+- Cost: a build that fails with `"[BUILD] ERROR"` or `"[LINKER] compilation (codegen) error"` is reported by `tbrun` as a success.
+- Fix in place: export `tb-ide.mjs`'s `BUILD_FAILED` and use it from `tbrun.mjs`.
+- Verify by: a probe project that fails each of the 5 ways; `tbrun` exits non-zero for all 5.
+- Size: S.
+- Verified: corrected (V3) — the orchestrator first reported "2 of 5"; tbrun's `/i` flag catches both listed casings of the BUILD shape, so it is 3 of 5. The consequence — the two missed shapes still produce a false success — is unchanged.
+
+#### A8-1 — R1, dup
+**`census_attributes.mjs`'s modifier-keyword list already disagrees with the two other places that classify the same twinBASIC source, and is missing `Overridable`.**
+
+- Where: `builder/census_attributes.mjs` `MODS` (`:138-148`, also missing `Iterator`, `Dim`) vs. `scripts/lib/twin-api.mjs:123-125` and `scripts/lib/tb-fences.mjs:340-343` (which additionally has `Async`, `PtrSafe`, and others).
+- Recorded reason: none — three independently maintained keyword lists.
+- Cost: originally reported as a live misclassification (`"Public Overridable Sub"` falling through to the variable-declaration path). Traced against the real BETA 983 package cache: 31 `Overridable` `Sub`/`Function`/`Property` lines exist there, and zero have an attribute, so the divergence is real but dormant today, not live. Two further divergences (`blankStrings`'s missing `""` escape handling, and a per-line vs. cross-line block-comment state) are structurally confirmed, not shown to misfire on real source.
+- Fix in place: one shared keyword-classifier module, paired with the ride-along probes **A8-4** also calls for.
+- Verify by: re-run against the exported package tree; census output unchanged except newly recognised `Overridable` members.
+- Size: M.
+- Verified: confirmed as an already-diverged keyword list (V3); the impact claim corrected — see Where the passes were wrong.
+
+#### A10-2 — R1, dup
+**Three independent frontmatter readers — two in wisdom, one in the build — disagree on BOM handling and on type coercion.**
+
+- Where: `wisdom/extract/sitemap.mjs:69-87` `parseFrontmatter` (no BOM strip) vs. `builder/discover.mjs:94-117` (`stripBom`, added after the AppGlobalClassObject incident) and `wisdom/extract/prep.mjs:343-388` (adds array and boolean/number coercion sitemap.mjs's version lacks).
+- Recorded reason: none for the BOM gap.
+- Cost: dormant — a scan of all 912 `docs/` markdown files at `fe9ce12b` confirmed zero begin with a BOM today. `WIP.md` itself documents that Windows editors add one "without being asked."
+- Fix in place: route all three through the shared frontmatter parser proposed under Decisions for you (a).
+- Verify by: every page's parsed frontmatter, deep-equal before/after.
+- Size: M.
+- Verified: confirmed, dormancy confirmed by direct scan rather than inference (V2); independent of the "different dialects" point below (see Where the passes were wrong).
+
+#### L1-1 — R1, dup
+**A theme/viewport validator exists in one accessibility script, built specifically after an unvalidated value once mislabelled a report, and is missing from two siblings that do the identical kind of dispatch — one of which is the axe-upgrade gate.**
+
+- Where: `scripts/check_a11y.mjs:82-95` `pick()` vs. `scripts/sweep_a11y.mjs:120-121` and `scripts/check_a11y_fingerprint.mjs:134-135` (both bare `themeArg === "both" ? THEMES : [themeArg]`).
+- Recorded reason: the fix exists in one place; it was never propagated.
+- Cost: traced end to end — `axe-scan.mjs`'s `buildMatrix` builds the report label straight from the unvalidated string, and `gotoPage` applies it with no validation either; the dark CSS is scoped to `[data-theme=dark]` only, so an unrecognised value silently renders light while the report claims otherwise, reproducing the original bug in two tools that never received the fix.
+- Fix in place: hoist `pick()` into `axe-scan.mjs`, or validate inside `buildMatrix`.
+- Verify by: `--theme drak` fails loudly in all three tools.
+- Size: S.
+- Verified: confirmed, full consequence traced (V2).
+
+#### L1-2 — R1, dup
+**The flag-value helper `opt()` is reimplemented seven times in four variants, and the most common variant returns `undefined`, not its stated default, for a trailing value flag.**
+
+- Where: (A) returns `undefined` — `census_attributes.mjs:86`, `check_examples.mjs:93`, `tbbuild.mjs:61` (3 copies); (B) falls back to the default, also for an explicit empty value — `addin_test.mjs:68`, `tbrun.mjs:105-108` (2); (C) one-arg, no default — `build_package_api.mjs:56`; (D) inline/unnamed — `check_publish_policy.mjs:31-33`. `NaN` confirmed at `tbbuild.mjs:68,70` (port, timeout) and `check_examples.mjs:102-104` (jobs, port, batch).
+- Recorded reason: none.
+- Cost: a trailing `--port` or `--timeout` silently becomes `NaN` instead of the documented default.
+- Fix in place: the shared `numberOption()` in the cli.mjs module (Decisions for you (e)).
+- Verify by: a trailing value flag now errors instead of becoming `NaN`, for every affected tool.
+- Size: M.
+- Verified: confirmed, all 7 copies found and classified (V2).
+
+#### L1-3 — R1, dup/hack
+**Positional-argument detection disagrees between two harness tools, and one fails outright on a boolean flag placed before the positional argument.**
+
+- Where: `scripts/tbrun.mjs:112-116` (explicit `VALUE_FLAGS` list) vs. `scripts/tbbuild.mjs:62` (order-dependent: rejects a token whose *predecessor* starts with `--`, with no distinction between a boolean switch and a value-taking flag).
+- Recorded reason: none.
+- Cost: `tbbuild --keep proj` reports a usage error, because `--keep`'s own `--` prefix rejects the token after it.
+- Fix in place: the shared flag registry in the cli.mjs module.
+- Verify by: `tbbuild --keep proj` succeeds after the fix.
+- Size: S.
+- Verified: confirmed, traced exactly (V2, by trace rather than execution).
+
+#### L1-4 — R1, hack
+**`tbdocs.mjs` throws on an unknown argument, and the resulting exit code is the same bit already documented elsewhere in the same function as meaning "link check failed."**
+
+- Where: `builder/tbdocs.mjs:189-191` (throw), `:1616-1635` (`main()`/`.catch`, uniform exit 1), `:1568-1570` (the link-check bit-0 meaning).
+- Recorded reason: none.
+- Cost: a parse error and a real link-check failure are indistinguishable by exit code, in the flagship, CI-invoked tool.
+- Fix in place: exit 2 for parse errors, distinct from the build-failure bitmask.
+- Verify by: a malformed flag exits 2; a real link-check failure still sets bit 0.
+- Size: S.
+- Verified: confirmed, full trace (V2).
+
+#### L2-1 — R1, dup
+**Two more hand-kept lists of "which folders are output trees" repeat a defect class `check_tree_fresh.mjs` was already fixed for once, and one of the two has a traced live consequence.**
+
+- Where: `builder/serve.mjs:138` `IGNORED_PREFIXES` (no `_site-basepath*` entry — a `--dest docs/_site-basepath` run mid-`serve.bat` is not filtered, causing a spurious rebuild); `eval/build_corpus.mjs:59-71` `EXCLUDED_PATHS` (lists `docs/_site-basepath` explicitly, but its prefix test does not match `-offline`/`-pdf` variants, since the next character is `-` not `/`). `60bb6f5` edited `serve.mjs`'s list the same day `scripts/lib/markdown-files.mjs`'s general `isOutputTree` was added, without switching to it.
+- Recorded reason: none.
+- Cost: a spurious rebuild while `serve.bat` is running; incomplete exclusion in the evaluator's corpus builder.
+- Fix in place: both use `isOutputTree`.
+- Verify by: a `--dest docs/_site-basepath` build during `serve.bat` produces no spurious rebuild; `build_corpus.mjs` excludes all three basepath variants.
+- Size: S.
+- Verified: confirmed, both halves (V1).
+
+#### L2-2 — R1, dup
+**`census_attributes.mjs` reimplements the twinBASIC-install finder that a shared module's own header exists specifically to prevent a private copy of, and the two have already diverged.**
+
+- Where: `census_attributes.mjs:99-119` `findInstall` (validates via a `packages/` directory) vs. `scripts/lib/tb-install.mjs:19-35` `findIde` (validates via `twinBASIC.exe`); the home-directory fallback (`os.homedir()` when `USERPROFILE` is unset) exists only in the private copy.
+- Recorded reason: `tb-install.mjs`'s own header states the shared-module rationale this private copy undercuts.
+- Cost: the two recognise an install by different files, so a partly unpacked install passes one check and fails the other; they also disagree when `USERPROFILE` is unset.
+- Fix in place: `census_attributes.mjs` uses `findIde`, keeping its `packages/` validation; fold the home-directory fallback into `tb-install.mjs` itself.
+- Verify by: both functions agree on the same Desktop layout, including an unset `USERPROFILE`.
+- Size: S.
+- Verified: confirmed (V3).
+
+#### L3-1 — R1, dup
+**`check_a11y.mjs` launches its browser with no try/finally, and the exact exception path it leaks on is already named in a comment one line above the unguarded code.**
+
+- Where: `scripts/check_a11y.mjs:167` (launch), `:218` (`browser.close()`, success path only), `:244-247` (`main().catch`, exits 2 without closing); `:229`'s own comment ("though PAGE_STATES throws before it gets that far"); every sibling using the same library guards it (`check_a11y_fingerprint.mjs:234,238-248`; `sweep_a11y.mjs:209,214-274`; `check_axe_patch_equiv.mjs:99,104-116`).
+- Recorded reason: none for the missing guard; the throw path is independently confirmed real — `axe-scan.mjs`'s `PAGE_STATES` appliers throw by design on a failed assertion.
+- Cost: any `PAGE_STATES` assertion failure leaves Chromium running for the rest of the job on Linux, which is where CI runs this gate. Whether Windows reaps it with its parent was not tested.
+- Fix in place: a shared `withBrowser(fn)` helper in `axe-scan.mjs`, adopted by all four scripts.
+- Verify by: a deliberately failing assertion leaves no child process running, before/after.
+- Size: S.
+- Verified: confirmed by code reading; no browser was actually started to reproduce it (V4).
+
+#### L3-3 — R1, hack
+**Wisdom's section splitter contradicts its own header comment — "we never silently drop reviewer content" — by dropping exactly that, with no fence awareness of any kind.**
+
+- Where: `wisdom/extract/merger.mjs:114-129` `parseStaging` (splits on any line exactly equal to `---`); `:162-168` `parseSection` (returns `null` for a chunk not starting `"## "`); `:147-148`/`152-153` (the caller drops a `null` result silently); `:111-112` (the contradicted comment).
+- Recorded reason: the comment states the opposite of what the code does.
+- Cost: reproduced directly on a synthetic file — a bare `---` inside a fenced block does not make the whole section disappear, but it does silently cut off the rest of that section's own body and its entire metadata line (`finding_ids`/`confidence`): the tail chunk does not start with `## `, so it is dropped. Ground truth on the real, 15,553-line committed `staging.md`: 1,160 bare `---` lines produce 1,160 chunks, and all 1,160 start with `"## "` — zero dropped today, so the defect is dormant on the current file (a separate, real content slip at the same file's lines 14245–14246 is discussed under Where the passes were wrong and Decisions for you (f)).
+- Fix in place: parse `staging.md` through the shared markdown module's fence-aware section splitter (Decisions for you (a)) instead of a bare line-equality test.
+- Verify by: replay of the real `staging.md`, zero dropped or mis-paired sections, before and after.
+- Size: M.
+- Verified: confirmed, with the nuance above (V4).
+
+#### L4-10 — R1, dup
+**Twin-BASIC source-line splitting is implemented twice, and the two have already diverged on BOM handling and on block comments.**
+
+- Where: `scripts/lib/tb-fences.mjs:423-445` `logicalLines` (private) vs. `scripts/lib/twin-api.mjs:51-86` `logicalLines` (exported). Twin-api strips a leading BOM and threads a multi-line `/* */` comment across physical lines; tb-fences does neither — it has **no `/* */` handling at all**.
+- Recorded reason: tb-fences's quote-aware comment stripping (its own stated reason, `:423`) is preserved in twin-api's version too — confirmed directly, both are quote-aware for the same structural reason. Not a drop-in swap, though: twin-api keeps blank lines (tb-fences drops them), uses 1-based `line` vs. tb-fences's 0-based `at`, never trims (tb-fences does), and recognises `Rem` (tb-fences does not) — four behaviour changes a real replacement must absorb deliberately.
+- Cost: a `/* */` comment spanning two physical lines is not stripped by tb-fences at all; its trailing declaration is left unreachable by tb-fences' own `^`-anchored classifier regexes.
+- Fix in place: consolidate on one `logicalLines`, absorbing the four behaviour differences deliberately, with a probe for a `/* */` span.
+- Verify by: `check_examples.mjs`'s 74-probe `runProbes` suite unchanged; a new probe for a block comment.
+- Size: M.
+- Verified: confirmed, including the full replacement-question analysis run for exactly this purpose (V3).
+
+### Tier 2 — makes every change in the area slower or riskier
+
+#### A1-3 / L4-1 — R2, dup
+**A "run handler, time it, report" block is repeated three times in the same file.** Where: `builder/cpu-worker.mjs:360-377, 423-440, 469-486` (a fourth block at `:497-540` is genuinely different — a distinct message shape and a deliberate SAB-then-postMessage ordering that closes a race, related to **A1-7**). Recorded reason: none. Cost: a fix must be applied by hand three times. Fix in place: one shared `runTimed()` helper for the three identical paths. Verify by: tree comparison. Size: M. Verified: confirmed, including the fourth path's genuine difference (V1).
+
+#### A1-4 — R2, dead
+**An exported timer helper has zero callers anywhere, and the private copy that is actually used justifies itself by citing tools that no longer exist.** Where: `builder/tbdocs.mjs:196-209` `makeTimer` (exported, unused) vs. `builder/offline.mjs:98-111` (private, identical, called at `:151`); the comment at `:95-97` cites the tools `644d6bdb` deleted. Recorded reason: expired — the cited consumers are gone. Cost: a future editor cannot tell which copy is live. Fix in place: delete the unused export, or have `offline.mjs` import the one copy; correct the comment. Verify by: grep for callers; tree comparison. Size: S. Verified: confirmed (V1). One of the five members of the **L4-8** low-level-util cluster (see Tier 3).
+
+#### A1-6 — R2, conv
+**Exit-code bits are set at seven sites with bare magic-number literals, and bit 0 is set independently by five of them.** Where: `builder/tbdocs.mjs:560, 1469, 1470, 1568-1573, 1598, 1610`. Recorded reason: none. Cost: today's five bit-0 sites are safe only because they execute in a fixed source order before the OR'd sites; nothing enforces that order, so a reordering silently clobbers a bit. Fix in place: a `failBuild(bit)` helper with named constants. Verify by: tree comparison; a deliberately reordered defect still fails loudly. Size: M. Verified: confirmed (V1).
+
+#### A2-1 / A1-5 / L4-4 — R2, dead
+**Five dead code paths from the retired `_diff`/`_triage` tools remain live in `offline.mjs`, plus a 24-name re-export block with zero importers.** Where: `builder/offline.mjs:244-289` `writeOfflinePages` (zero callers, a clone of the live pre-pass at `cpu-worker.mjs:183-201`), `:117` (unread `precomputed` param), `:207,213,359-394` (`buildSitePaths`'s unreachable fallback — `tbdocs.mjs:705-706` always sets `sitePaths`), `builder/search.mjs:18-24` `writeSearchData`, `builder/sitemap.mjs:70-74` `extractSitemapUrls`, `offline.mjs:55-88` (re-export block, confirmed zero importers for every one of the 24 names). Recorded reason: none live. Cost: dead code that looks essential to an editor who doesn't check. Fix in place: delete all five paths and the re-export block. Verify by: repo-wide grep for callers = 0; tree comparison. Size: M. Verified: confirmed, severity corrected from the pass's own hedged R1 to R2, since nothing here has actually diverged, only gone unreachable (V1).
+
+#### A2-2 / A9-10 — R2, dead
+**Comments in five files still name tools deleted in `644d6bdb`, and one uses the stale reference to justify a dead export.** Where: `offline-rewrite.mjs:411`, `search.mjs:65-66`, `sitemap.mjs:67-68,97`, `redirects.mjs:35-36`, `pdf.mjs:6-9,99-107` (justifies `extractImagePaths`, `pdf.mjs:146-158`, zero callers; `deriveBookOutputs`, by contrast, is live, called from `writePdf`). Recorded reason: describes tools that no longer exist. Cost: a stale comment claiming a real consumer is what lets dead code survive review. Fix in place: delete the comments and the dead export. Verify by: grep for the deleted tool names outside git history. Size: S. Verified: confirmed (V1); zero callers for `extractImagePaths` confirmed repo-wide.
+
+#### A2-3 / L2-4 / L4-5 — R2, dup
+**A baseline reader is byte-identical in two files, and the six-branch drift-guard state machine around it is retyped rather than shared; the accompanying hand-written writer has its own small divergence.** Where: `page-baseline.mjs:83-90` vs. `symbol-baseline.mjs:45-52` (`readBaseline`); `checkPageBaseline:117-176` vs. `checkSymbolBaseline:75-125` (the six-branch structure); `symbol-baseline.mjs:55-58` `writeBaseline`. Recorded reason: `WIP.Build.md` ("the page-count guard's shape applied to URLs") explains why the two look alike, not why the branch bodies weren't factored together. Cost: any fix to the drift-guard logic must be made twice by hand. Fix in place: one shared baseline-drift-guard module; `writeBaseline` delegates to `JSON.stringify(x, null, 2) + "\n"`. Verify by: byte-identical baseline files before/after; the gate still fails on reintroduced drift. Size: M. Verified: confirmed (V1). Conflict resolved: `writeBaseline` is byte-identical to `JSON.stringify(...,null,2)+"\n"` for every non-empty list, confirmed by execution — it differs only on an empty list (`[\n\n  ]` vs. `[]`), an accidental artifact rather than the "one URL per line" design L4 first read into it.
+
+#### A2-4 / A3-5 (escapeRegExp) — R2, dup
+**A regex-escaping helper is defined identically in three files, and a fourth tool had to be specially built to tolerate the duplication.** Where: `render.mjs:2235-2237`, `offline-rewrite.mjs:239-241` (exported), `book.mjs:212-214` `escapeRegExpBook`. Recorded reason: none for the duplication; `scripts/lib/regex-fold.mjs:58-63` explicitly recognises the escaper "by shape rather than by name" because of these exact copies — evidence for the finding, not a defence of it. Cost: a fourth copy is one keystroke away. Fix in place: one exported `escapeRegExp`. Verify by: tree comparison. Size: S. Verified: confirmed (V1).
+
+#### A2-7 — R2, dup
+**An ASCII-only, NBSP-preserving whitespace trim is implemented twice, with two different techniques, repeating the shape of a whitespace-in-code defect that has already shipped once.** Where: `compress.mjs:73-84` `collapseWhitespace` (regex-based) vs. `search.mjs:251-261` `stripAsciiWhitespace`/`isAsciiWs` (charCode-scan-based); both independently preserve NBSP. Recorded reason: none for the duplication. Cost: `WIP.Build.md` documents a real, named prior defect in exactly this area of `compress.mjs` ("Whitespace inside inline code is content"); two implementations double the chance of a second one. Fix in place: one shared whitespace-collapse helper. Verify by: tree comparison (search index and compressed HTML unchanged). Size: S. Verified: confirmed (V1).
+
+#### A3-2 / L3-5 — R2, dup
+**Seven HTML-escaper copies exist across two classes, "escapeHtml" collides three ways as a name across the build, and one function mixes both classes for sibling token types.** Where: minimal (`&<>`) — `render.mjs:2230-2233` `escapeHtmlMinimal`, `highlight.mjs:251-254` `escapeHtml` (recorded reason: matches Rouge, which escapes only `&<>`), `gantt.mjs:213` `esc`; full (`&<>"'`) — `render.mjs:2225-2228` `escapeHtml`, `template.mjs:993-998` `escText` and `:999-1001` `escAttr` (byte-identical to `escText` under a different exported name), `sitemap.mjs:106-113` `xmlEscape`; `render.mjs:1437-1450` `headingTocHtml` uses the full escaper for `text` tokens and the minimal one for `code_inline` tokens, within the same function. Recorded reason: `highlight.mjs`'s own 3-character choice is explained (Rouge parity); `render.mjs`'s internal mixing is not. Cost: dormant (no TOC'd heading has an apostrophe today), but a maintainer grepping for "escapeHtml" can pick the wrong module's copy and silently under-escape. Fix in place: one minimal and one full escaper in a shared module; `headingTocHtml` escapes all inline-token text consistently. Verify by: TOC rendering unchanged for the existing corpus; a probe heading with an apostrophe. Size: S. Verified: confirmed — A3-2 narrowed to `render.mjs`'s internal mixing, since `highlight.mjs`'s own choice is separately justified (V1); L3-5's broader catalogue, including the `escText`/`escAttr` duplicate, confirmed independently (V4).
+
+#### A3-6 — R2, hack
+**Three whole-page HTML rewrites have no code guard and depend on an invariant that is true for three of four ways text can reach ``/`
`, and false for the fourth.** Where: `render.mjs:74-80` `padEmptyCells`, `:351-354` `normaliseVoidTags`, `template.mjs:714-727` `injectAnchorHeadings` (its `HEADING_REGEX` at `:694` has no ``/`
` leading alternative). Recorded reason: none at the three sites; `WIP.Build.md`'s "Never rewrite markdown source without knowing what is code" section never mentions these three, as compliant examples or as a stated exception. Cost: the fourth path — a raw, hand-authored `
` or heading tag in markdown source, structurally open via `html: true` and never overridden by any `md.renderer.rules` assignment — lets text end up in ``/`
` unescaped, though `git grep` across all 912 pages finds none today. This codebase has already paid for exactly this class of mistake once, in `book.mjs`'s chapter-transform rewrites, whose incident report concludes "do not rely on that accident." Fix in place: state the invariant, and extend it with the code-guarded pattern already used elsewhere (`replaceOutsideCode`, or the ``/`
` leading-alternation shape), or a probe. Verify by: `check_code_regions.mjs` extended to the post-render chain. Size: M. Verified: confirmed, with a conflict ruled — L3 had called these three "sound" on the reasoning that every code path escapes first; true for three of four paths, not the fourth. Both readings are partly right; A3-6 stands as written (V4).
+
+#### A3-7 — R2, struct
+**One template module holds a date formatter, URL helpers, and a fourth, independent escape-helper implementation, beyond its own templating job.** Where: `template.mjs:940-989` (strftime tables and `formatDate`/`parseDate`), `:917-936` (URL helpers), `:991-998` (a fourth escape-helper block, under its own "§5.15 escape helpers" header). Recorded reason: none. Cost: several unrelated jobs in one module, the rubric's own definition of a struct finding. Fix in place: see Split candidates. Size: L. Verified: confirmed, reinforced by a fourth job V1 found beyond the pass's own citation.
+
+#### A4-2 — R2, dup
+**The structured-findings translation is written twice, though the human-readable half is already shared.** Where: `builder/check.mjs:422-449` `findingsFor` vs. `scripts/check_links.mjs:602-634` `buildFindings` (same broken/forbidden/html/a11y/dupIds/remoteAssets logic, differing only in the path-relativiser and a null-guard); both already import `formatLinkReport`/`formatIntegrityReport` from `link-check.mjs`. Recorded reason: both comments (`check.mjs:410-414`, `check_links.mjs:599-601`) tie the parallel shape to `check_links_diff.mjs` cross-checking two independently written implementations — legitimate when written. Cost: superseded by the user's 2026-09-25 decision to make `check_links.mjs` a thin wrapper over `builder/check.mjs` (decision 5); after that refactor there is one implementation behind two front ends, and both comments need rewriting. Fix in place: as part of the Phase 2 thin-wrapper work already decided, delete the duplicate translation and rewrite both comments. Verify by: `check_links_diff.mjs` continues to cross-check disk-read vs. memory-read over the one implementation. Size: M. Verified: confirmed (V2); the conflict this raises with L4's "deliberately mirrored" reading is resolved under Where the passes were wrong.
+
+#### A5-2 — R2, conv
+**The documented 0/1/2 exit convention is broken in two scripts that crash instead of exiting 1 for a genuine finding.** Where: `docs/Documentation/Extending.md:640-648` (the convention); `build_dot_metrics.mjs` (no catch around its puppeteer work; its only intentional exit path is `exitCode=1` for STALE); `pick_a11y_sample.mjs`'s `discover()` (`:159-169`, an absent tree throws uncaught, ending at the same exit 1 as a real coverage gap at `:310`); contrast `check_dot_fit.mjs:31-34`, which has the guard and cites the convention by name. Recorded reason: the convention exists; these two don't follow it. Cost: `pick_a11y_sample.mjs` runs live in `check.bat:23` — a crash there reads as a coverage gap to anyone triaging a red `check.bat`. Fix in place: wrap both in the convention's try/catch, exit 2 for a genuine crash. Verify by: a deliberately broken tree exits 1, not crashes, for a real gap; exits 2 for an actual crash. Size: S. Verified: confirmed (V2).
+
+#### A5-3 / L4-9 (part 1) — R2, dup
+**Page discovery, tag counting, and a stub ceiling are each implemented independently in two scripts whose outputs must agree with each other.** Where: `pick_a11y_sample.mjs:122,159-169,184` `STUB_CEILING` vs. `sweep_a11y.mjs:78,129-153` `STUB_TAG_CEILING` (both 100 today); `pick_a11y_sample.mjs`'s `--propose` reads the exact JSONL file `sweep_a11y.mjs`'s `--out` default writes. Recorded reason: none. Cost: the two ceilings must stay in step for the join between them to mean anything, and nothing enforces that today. Fix in place: one shared discovery/ceiling module. Verify by: `--propose` output unchanged. Size: S. Verified: confirmed (V2).
+
+#### A5-5 — R2, dup
+**Two scripts hand-build an identical Inter-webfont host page and an identical puppeteer launch, including a flag neither explains, while a third tool's equivalent launch explains its flags but omits this one.** Where: `check_dot_fit.mjs:76-87` and `build_dot_metrics.mjs:57-68` (both `["--no-sandbox", "--disable-dev-shm-usage", "--allow-file-access-from-files"]`, no comment on the third flag); `axe-scan.mjs:258-264` `LAUNCH_ARGS` (explains the first two, silent on the third for the same class of `file://` load); three different repo-root idioms coexist in scope at the same time. Recorded reason: none. Cost: a maintainer changing one launch has to find and update the other by hand. Fix in place: one shared puppeteer-launch helper with the flag documented once. Verify by: tree comparison (diagram fit, metrics unchanged). Size: S. Verified: confirmed (V2).
+
+#### A5-6 / L2-3 — R2, dup
+**A gate re-scans `.dot` sources by hand instead of importing the shared, already-exported-elsewhere walker.** Where: `check_dot_fit.mjs:49-67` `findDotSvgs` vs. `builder/dot.mjs:135-155` `listDotSources` (never exported; only `regenerateDot` calls it internally); five other gates already import the shared `markdown-files.mjs` chain instead of re-implementing the same traversal. Neither function uses `isOutputTree`; their shared, broader `"_"/"."` skip is safe today, since no `.dot` file sits under `_data`/`_sass`/`_App`/`_Images`. Recorded reason: none. Cost: the mirror-fault shape this review is watching for — a second copy that silently stops matching the first if either changes. Fix in place: export `listDotSources`, or a shared `filesUnder(root, {ext, skip})` helper. Verify by: identical file lists before/after. Size: S. Verified: confirmed (V2).
+
+#### A6-3 — R2, conv
+**A tool about to become a pre-commit check has no path from a crash to a distinct exit code, which will matter once it runs inside the pre-commit hook already planned for it.** Where: `convert_em_dash_separators.mjs` has exactly one `process.exit(...)` (`:214`), fed only 0 or 1 from `main()` (`:210`); no `catch`/`uncaughtException` anywhere. `PLAN-10.md:690-693,795-797` both defer exactly that hook to later. Recorded reason: the hook doesn't exist yet, so the gap is dormant by design, not by oversight. Cost: none today; matters the day the hook ships. Fix in place: the shared crash-handler pattern from **A6-1**. Verify by: a crash now exits 2, a real finding still exits 1. Size: S. Verified: confirmed (V2).
+
+#### A6-4 — R2, struct
+**Nothing reads either CI workflow to confirm it runs the same gates as the wrappers, though today the two workflows' shared steps are byte-identical.** Where: `check_gate_lists.mjs` never touches a workflow file (confirmed: zero matches for `.github`/`workflow`/`.yml`); the two workflows' 12 shared gate steps are identical, in the same order, differing only in three already-recorded ways (`checks.yml`'s extra fixture-built link-checker step, the deploy build's extra `--url`/`--baseurl`, and deploy-only steps). Recorded reason: decision 6 already commits to building this gate; the design (a new `scripts/check_ci_workflows.mjs` plus a shared `scripts/lib/gate-roster.mjs`, generalising `check_gate_lists.mjs`'s `gatesFromBat`) is in the ledger but not yet evaluated by a verifier. Cost: nothing today would catch a workflow silently dropping a gate, reordering one around a critical flag, or losing `--check-audit-index`. Fix in place: as designed — compare wrapper roster vs. each workflow, the two workflows against each other, and critical build flags, against an explicit allowlist of the three recorded deltas. Verify by: probes for a missing gate, an extra step, reordering, a missing flag, and confirmation the three allowlisted deltas do not fire. Size: L. Verified: facts confirmed (V2, which did not evaluate the proposed design, per its own scope).
+
+#### A7-2 — R2, struct (kind arguable — reads closer to an unguarded workaround; left as filed)
+**A timeout return value is silently discarded at every call site, reopening the cursor-reset race the function exists to prevent.** Where: `tb-operate.mjs:421-429` `afterReveal` (returns `false` on timeout) vs. its three call sites, `openFile:453`, `setCursor:461`, `select:475` (each a bare `await afterReveal(c);`). Recorded reason: the race is documented (`:401-409`, the measured `"xyz"→"zy"` cursor corruption); the discard that reopens it is not. Cost: silently proceeding after a timeout risks exactly that corruption again. Fix in place: check the return value at all three call sites; retry or fail loudly on timeout. Verify by: `addin-test.bat` green, same lanes. Size: S. Verified: confirmed (V3).
+
+#### A7-3 — R2, dup
+**Two CDP click primitives exist, one minimal and one with scroll, hit-test and retry, and the minimal one is used for the build icon alone.** Where: `tb-ide.mjs:848-861` `clickCenter` (no scrollIntoView, hit-test or retry; exactly 2 call sites, both the build icon: `tb-ide.mjs:757`, `tbrun.mjs:295`) vs. `tb-operate.mjs:79-134` `click` (scrollIntoView, shadow-root-aware hit test, retry loop; used directly in 4 of the 10 `test/addin/*.test.mjs` files — appdata, panes, sample10, sample15 — plus twice inside `tb-operate.mjs` itself). Recorded reason: none. Cost: the build-icon click path skips the retry and hit-test logic every other click path relies on. Fix in place: route `clickCenter`'s two call sites through `click`, or justify why the build icon is exempt. Verify by: `addin-test.bat` green, same lanes. Size: S. Verified: corrected (V3) — usage overstated as "every scenario"; corrected to the exact 4-of-10 file list.
+
+#### A7-4 — R2, dup
+**Two small helpers are byte-identical between a shared module and its one caller, which separately imports ten names from that module.** Where: `tb-registry.mjs:588-590` `alive` and `:462` `norm` (both private/unexported) vs. `addin_test.mjs:105` and `:230` (identical bodies); `addin_test.mjs:60-61` imports 10 names from `tb-registry.mjs` (corrected from an initial count of 9). Recorded reason: none. Cost: none yet — not diverged. Fix in place: export `alive`/`norm` from `tb-registry.mjs`, remove the private copies. Verify by: `addin-test.bat` green. Size: S. Verified: corrected (V3) — import count corrected to 10. A conflict with L4's "no clone region, below the detector's window" reading is resolved in A7-4's favour: both bodies were read directly and found identical.
+
+#### A7-5 — R2, conv
+**The harness's three CLI tools disagree on defaulting behaviour in independently reproducible ways, one of which produces a misleading diagnostic.** Where: `tbbuild.mjs:61-62,68,70,118-122` (`opt()` returns `undefined` on a trailing flag, so `Number(undefined)` is `NaN` for `--port`/`--timeout`; a `NaN` timeout makes `tb-ide.mjs:436`'s polling loop run zero times, producing "the IDE never reported `` as open" instead of a timeout message); `tbrun.mjs:105-108`/`addin_test.mjs:68-69` (substitute the default for a trailing flag *and* for an explicit `""`); `tbbuild.mjs`'s positional finder skips a token after any `--flag` regardless of whether it takes a value, so `tbbuild --json proj` fails (dormant — `check_examples.mjs:602` always puts the path first); `die()` exists in three different shapes across the three tools. Recorded reason: none. Cost: a misconfigured timeout produces the wrong diagnosis. Fix in place: the shared cli.mjs module — `numberOption()` closes the `NaN` hole, a flag registry replaces the order-dependent positional finder. Verify by: the reproduced cases above become fixture cases. Size: M. Verified: confirmed, every sub-claim independently reproduced by script or trace (V3).
+
+#### A7-8 — R2, dup
+**Every one of the ten add-in test scenarios hand-writes its own lane preamble, and six of the ten hand-write a "console lines since a mark" reader four different ways.** Where: all ten `test/addin/*.test.mjs` files (the `HERE`/`HOST`/lane preamble and the skip object); `appdata.test.mjs:33`/`panes.test.mjs:86` `probeLines` (two hard-coded slice offsets), `arch.test.mjs:31`/`reload.test.mjs:37` (two different regex-capture readers), `keys.test.mjs:28` (plain split+trim), `sample10.test.mjs:81` (bare trim, no split). Recorded reason: none. Cost: a change to how the console mark works must be found and fixed in up to four different shapes. Fix in place: a scenario helper, plus one shared `linesSince` beside `readConsole`. Verify by: `addin-test.bat` green, all ten lanes. Size: M. Verified: confirmed (V3).
+
+#### A8-2 — R2, struct
+**A `builder/` module depends on `scripts/`, against the codebase's own stated rule, and a gate has a carve-out that exists only for this one file.** Where: `census_attributes.mjs:79` imports `../scripts/lib/tb-packages.mjs`; `render.mjs:383`'s comment states, verbatim, "builder/ must not depend on scripts/"; `check_tree_fresh.mjs:57-63` `IGNORED_FILES` exists only for `census_attributes.mjs`. Recorded reason: none for the violation itself. Cost: 11 files cite the `builder/census_attributes.mjs` path directly, and 13 cite the bare filename (corrected — the pass's original ten-item list wrongly included `WIP.Build.md`, which only cites the bare filename, and omitted `scripts/build_package_api.mjs:12` and `scripts/lib/tb-fences.mjs:334,349`, which cite the literal path); every one needs updating on a move. Fix in place: move `census_attributes.mjs` to `scripts/`, beside `build_package_api.mjs` (already scheduled in Phase 1 of the plan). Verify by: tree comparison; all citations updated; `check_tree_fresh.mjs`'s carve-out removed. Size: M. Verified: corrected (V3) — citation count and list corrected.
+
+#### A8-4 — R2, struct
+**Two independent keyword/phrase classifiers, both with a documented history of silent misparses, have no fixture-based regression test.** Where: `census_attributes.mjs` (no `selftest`/`node:test`/assertion anywhere; its own header at `:42-67` names a 14-site Interface-member loss and, at `:150-152`, a 368-Declare misparse) and `gen_attribute_probes.mjs`'s `parseTargets` (`:961-978`, its own comment at `:946-947` names a plural-matching bug it already shipped). Recorded reason: both histories are recorded; neither has a test that would have caught them by construction. Cost: the next silent misparse ships the same way the last ones did. Fix in place: a shared "labelled-keyword-list classifier plus ride-along probes" pattern, covering both (paired with **A8-1**'s fix). Verify by: the probes themselves, run in `test.bat`. Size: M. Verified: confirmed (V3).
+
+#### A9-1 — R2, hack
+**The pdf-lib shims are contained but not guarded: only an exact version pin protects them, which catches accidental drift and nothing else.** Where: all 13 production shim files under `book/lib/` (e.g. `fast-array-onebuf.mjs`, `fast-sync-load.mjs`); each has only an idempotency guard (`__xInstalled`-style), never a check of what it overwrites; `parallel-deflate.mjs:50`'s `PDFStreamWriter` subclass has no guard of any kind; `package.json:20` pins `pdf-lib` at exactly `1.17.1`. Recorded reason: `perf/notes/08-pdf-lib.md:1751-1757` states the pin exists "so a stray `npm update` can't silently swap upstream" — covers accidental drift only, exactly as claimed; pdf-lib's upstream is genuinely abandoned (the `@cantoo` fork was evaluated and rejected, `08-pdf-lib.md:5048-5061`), which is why this is R2 rather than R1. Cost: a deliberate future version bump, or a mistaken hand-edit to a shim, fails silently. Fix in place: load-time shape assertions per shim, modelled on `axe-scan.mjs`'s `SOURCE_PATCHES` counts. Verify by: a deliberately wrong shape throws by name. Size: M. Verified: confirmed — matches the rubric's un-met "guarded" test exactly (V3).
+
+#### A9-2 — R2, struct
+**No test anywhere compares the shimmed output against stock pdf-lib.** Where: no test file exists under `book/` at all; the only comparisons are one-off, manual notes in `perf/notes/08-pdf-lib.md`. Recorded reason: none. Cost: a shim regression would only be caught by visual inspection of the rendered PDF. Fix in place: `check_pdf_shims_equiv.mjs`, modelled on `check_axe_patch_equiv.mjs` (stock pdf-lib run in a child process); would become a new `test.bat` gate. Verify by: the new gate, run against a deliberately broken shim. Size: M. Verified: confirmed, no test file exists anywhere under `book/` (V3). See Decisions for you (c).
+
+#### A9-3 — R2, dup
+**Two of six helpers shared between the array and dict "onebuf" implementations are identical apart from naming; the other four are similarly shaped but not copies.** Where: `fast-array-onebuf.mjs` vs. `fast-dict-onebuf.mjs` — `_registerContext` and `_appendArray` are identical bar internal names; `pack`, `_cow`, `_makeFromRange`, `_makeFromAppend` differ by real bit-packing and subclass-dispatch logic dict needs and array does not. Recorded reason: `fast-array-onebuf.mjs:36-39` names only the singleton-context mechanism, leaving `_appendArray`'s exact duplication unaddressed. Cost: a fix to the two identical helpers must be applied twice. Fix in place: `book/lib/onebuf-range.mjs`, a factory that parametrises over the subclass-dispatch and gap-mask logic the two genuinely need differently — not a naive single-body extraction. Verify by: the book pipeline's oracle (page count, outline, extracted text unchanged). Size: M. Verified: confirmed, with the "identical" claim narrowed to exactly two of the six helpers (V3).
+
+#### A9-6 — R2, struct — Resolved in `90624841`
+**Four A/B-baseline shims lived in `book/lib/` but were loaded only by `perf/`.** Where: `fast-refs`, `fast-dict-array`, `fast-dict-iter`, `fast-parse-dict`; `render-book.mjs:56-58` records them as baselines, not their location. Recorded reason: needed the user's decision, since it is the converse of decision 1 (moving code the other direction, from `book/` toward `perf/`). Cost: production code (`book/lib/`) held code nothing in production loaded. Resolution: the user approved deleting the four rather than moving them, since `perf/`'s own measurements against them are already recorded in `perf/notes/08-pdf-lib.md` and git keeps the code. Verified: confirmed at pass time (V1, via the deletion decision's own cross-check of importers).
+
+#### A9-7 — R2, dup
+**A safety-critical code/pre guard alternation is re-typed by hand four times, including twice in the same file, with no gate tying the copies together.** Where: `book.mjs:210` `CODE_OR_PRE_BOOK` and `:228-229` `IMG_SRC_RE_BOOK` (same file); `pdf.mjs:143-144` `IMG_SRC_RE` (byte-identical to `book.mjs`'s); `offline-rewrite.mjs:299` `HTML_COMBINED_RE` (opens with the identical alternation before diverging). Recorded reason: none. Cost: this is the exact guard shape `WIP.Build.md` prescribes for a whole-page HTML rewrite; a fix to it — say, a missing HTML5 void-element case — must be found and applied in four places. Fix in place: one exported fragment, composed into each regex. Verify by: `book.html`, the PDF and the offline HTML, byte-identical before/after. Size: S. Verified: confirmed (V3).
+
+#### A9-8 / A2-6 — R2, dup
+**A URL-normalising helper is byte-identical in two files, and the comment justifying the private copy describes a plugin architecture this codebase no longer has.** Where: `book.mjs:303-307` `normalizeBaseurl` vs. `offline-rewrite.mjs:232-236` (exported); `book.mjs:300-302`'s comment cites "book-href-rewrite.rb keeps its own copy... so plugins are independent" — the old Ruby/Jekyll architecture. Recorded reason: expired. `book.mjs` already imports two sibling `builder/` modules (`compressHtml` from `compress.mjs`, `loadData` from `data.mjs`); nothing prevents it from importing `normalizeBaseurl` the same way. The comment is also wrong about the function's current location (`offline-rewrite.mjs`, not `offline.mjs`). Cost: a fix to URL normalisation must be applied twice. Fix in place: `book.mjs` imports `normalizeBaseurl` from `offline-rewrite.mjs`. Verify by: byte-identical output before/after. Size: S. Verified: confirmed byte-identical (V1). Conflict resolved in favour of this finding — see Where the passes were wrong.
+
+#### A10-3 — R2, dup
+**Wisdom re-implements the shared markdown file-walker from scratch, and its recorded reason does not actually cover what the shared version requires.** Where: `wisdom/extract/sitemap.mjs:61-67` `walk()` (bare recursive `readdirSync`, no `.md` filter, no output-tree skip). Recorded reason: `PLAN-3.md:404` — "a minimal built-in parser, no dependency on `builder/`" — but `scripts/lib/markdown-files.mjs` imports only `node:fs`/`node:path`, zero dependency on `builder/`, so the stated reason does not justify skipping it. Cost: currently harmless — the call is scoped to `docs/Reference/`, which never contains an output tree. Fix in place: use `markdownFiles` from `scripts/lib/markdown-files.mjs`. Verify by: identical file lists. Size: S. Verified: confirmed (V2).
+
+#### A10-4 — R2, dead
+**A schema module is imported by nothing and has drifted from the inline schemas actually in use.** Where: `wisdom/extract/schemas.mjs` (zero importers repo-wide) vs. `workflow.mjs:49` (`thread_path`, not `schemas.mjs`'s `source_thread`) and `workflow.mjs:77` (`section` as an enum, not `schemas.mjs`'s free-text string). Recorded reason: none. Cost: none live; would mislead if ever revived. Fix in place: delete `schemas.mjs`, or bring it into line with `workflow.mjs` and connect it. Verify by: grep for importers = 0, before deleting. Size: S. Verified: confirmed, both divergences confirmed by direct comparison (V2).
+
+#### A10-5 — R2, conv
+**Two wisdom state files are written non-atomically, with no parse guard on load, next to a sibling that is explicitly commented as doing both correctly.** Where: `wisdom/discord/messages.mjs:10-12` `saveManifest` (plain `writeFileSync`), `wisdom/wisdom.mjs:146` (same); `loadManifest` has no try/catch around its `JSON.parse`. Contrast `wisdom/extract/state.mjs:62-75`, explicitly commented "write state atomically (temp file + rename)" and doing exactly that. Recorded reason: none for the inconsistency. Cost: an interrupted write can corrupt `manifest.json`/`denied.json`, and a corrupt file crashes the next load with no diagnostic. Fix in place: the same temp-and-rename pattern, applied to both. Verify by: an interrupted-write simulation leaves the previous file intact. Size: S. Verified: confirmed (V2).
+
+#### A10-6 — R2, struct
+**Two parallel implementations have 19 byte-identical self-test names in the same order, and nothing runs either self-test suite.** Where: `impexp.mjs` and `impexp.py`, 19 `test(...)` calls each, confirmed byte-identical in name and order. Recorded reason: `Tools.md:958` states, unhedged, "the two editions print the same output and write byte-identical project files," with nothing mechanical behind the claim. Cost: real engineering discipline (keeping the two ports in lock-step) with no automated backstop at all. Fix in place: `check_impexp_parity.mjs`, run from `test.bat`. Verify by: the parity check itself, against a deliberately diverged copy. Size: M. Verified: confirmed (V2). See Decisions for you (b) — this fix makes `test.bat` and CI depend on Python.
+
+#### L1-5 — R2, dead
+**A script still tolerates unknown flags "passed through via `check.bat`'s `%*`," and `check.bat` no longer calls it at all.** Where: `check_links.mjs:307-314,406-412`; `check.bat` (read in full) never calls `check_links.mjs` — link and integrity checking moved into `build.bat`. Recorded reason: stale — corroborated independently by `PLAN-checks.md:14`. Cost: dead tolerance code, easy to mistake for a live contract. Fix in place: delete the stale tolerance and its comment. Verify by: `check_links.mjs`'s own tests unaffected. Size: S. Verified: confirmed (V2).
+
+#### L1-6 — R2, conv
+**`--help` is handled four different ways across 36 entry points, and one of the four gaps has a live filesystem side effect.** Where: stdout/exit 0; stderr/exit 0 (`pick_a11y_sample`, `sweep_a11y`); stderr/exit 2 via the general usage-error path (`tbbuild`, `tbrun`, `addin_test`); unhandled in 12 tools. `gen_attribute_probes.mjs:1147-1158` takes a bare `--help` as its `out_dir` argument and actually creates a `--help/Sources` directory on disk — traced, not just "fails to print help." `render-book.mjs:210-220` rejects `--help` as an unknown argument (exit 2) and never reaches its own usage text. Recorded reason: none. Cost: the first command a confused user types against `gen_attribute_probes.mjs` writes to disk. Fix in place: the shared `printHelpAndExit()` in the cli.mjs module. Verify by: `--help` on every tool now prints usage and exits 0, with no side effect. Size: M. Verified: confirmed, the `gen_attribute_probes.mjs` side effect independently confirmed as a live write (V2).
+
+#### L1-7 — R2, conv
+**Unknown-flag handling is five-way inconsistent.** Where: ignore (11 tools), warn (`check_links`), exit 2 (8 tools), throw → exit 1 or 2, exit 1 (wisdom). Recorded reason: none. Cost: the same mistake produces a different outcome depending on which tool it's made against. Fix in place: the shared cli.mjs module's strict-mode handling, one behaviour for all. Verify by: each tool's current, intentional exceptions preserved explicitly. Size: M. Verified: not re-verified — outside the set of citations the verifiers covered.
+
+#### L1-8 — R2, conv
+**`--json` means a boolean-to-stdout flag in four tools and a value-taking output file in a fifth.** Where: `tbbuild`, `tbrun`, `check_examples`, `census_attributes` (boolean) vs. `check_a11y_fingerprint` (takes a path). Recorded reason: none. Cost: the same flag name means something different depending on the tool. Fix in place: document the two shapes explicitly in the cli.mjs spec, or rename one. Verify by: `Tools.md`'s CLI tables, checked against real behaviour. Size: S. Verified: not re-verified — outside the set of citations the verifiers covered.
+
+#### L1-13 — R2, struct
+**Nothing tests any tool's own argument handling, anywhere in the repository — the missing seam that let L1-1, L1-2, L1-3 and L1-6 ship.** Where: every `--self-test`/`selfTest` in the repo (`check_code_regions.mjs`, `check_gate_lists.mjs`, `check_links.mjs`/`check_links_diff.mjs`, `check_regex_safety.mjs`, `impexp.mjs`) asserts domain logic, never argv parsing; no dedicated CLI test file exists anywhere. Recorded reason: none. Cost: demonstrated positively — four real, currently shipping argv defects a minimal parser test would have caught. Fix in place: the shared cli.mjs module has its own ride-along self-test in `test.bat`. Verify by: the self-test itself, plus regression coverage for L1-1/L1-2/L1-3/L1-6. Size: M. Verified: confirmed (V2).
+
+#### L2-5 — R2, dup
+**A repository-root path is derived inline in 19 files, by roughly five distinct expressions, and the one place that names a convention for it overstates how widely that convention is actually followed.** Where: 19 files confirmed exactly (`census_attributes`, `tbdocs`, `build_corpus`, `nav_hops`, `run_case`, `site_search`, `addin_test`, `build_dot_metrics`, `build_package_api`, `check_code_regions`, `check_dot_fit`, `check_examples`, `check_gate_lists`, `check_links_diff`, `check_tree_fresh`, `convert_em_dash_separators`, `gen_attribute_probes`, `wisdom/extract/prep.mjs`, `builder/write.mjs`); `axe-scan.mjs:29` exports `REPO_ROOT`, imported by only 2 of the 19. Recorded reason: `Extending.md:654` states a convention ("nearly all of them do" a specific expression), but only 3 of the 19 actually use that exact expression — the nested-`dirname` shape is the majority, 10 of 19. Cost: a change to the repository layout must be traced through five different idioms. Fix in place: `scripts/lib/repo-paths.mjs`, re-exported by `axe-scan.mjs`; correct `Extending.md:654`. Verify by: every derived root byte-identical before/after. Size: M. Verified: corrected (V2) — count of distinct expressions raised from four to roughly five.
+
+#### L3-2 — R2, dup
+**One spawn call has no `'error'` handler and awaits `'exit'` rather than `'close'`, and a failure there skips the registry-restore step that exists to guard against exactly this.** Where: `check_examples.mjs:601-612` `buildStaged` (spawn at `:608`, `'exit'` at `:612`); a spawn `'error'` with no listener is an uncaught exception at the emitter's own emit point, not a promise rejection, so it bypasses both `main()`'s local `catch` (`:1678-1680`) and `main().catch` (`:1810`) — neither is in its call stack, and no `uncaughtException` handler exists anywhere in the file. Two siblings (`addin_test.mjs:180,185`, `check_regex_safety.mjs:347-348`) register both events. Recorded reason: none. Cost: a spawn failure (a bad `execPath`, for instance) leaves the tbIDE registry unrestored — the exact hazard `tb-registry.mjs`'s tidy machinery exists to prevent. Fix in place: register both `'error'` and `'close'`, matching the two siblings. Verify by: a deliberately unspawnable command still restores the registry. Size: S. Verified: confirmed (V4).
+
+#### L4-3 — R2, dup
+**Two vendoring functions each independently re-implement the same fetch-with-fallback-and-atomic-write sequence.** Where: `vendor-assets.mjs:222-252` `fetchToFile` and `:279-319` `fetchAttachment`; both wrap `fetch()` in try/catch, check `res.ok`, wrap `arrayBuffer()` in a second try/catch, then use the identical `` `${destPath}.tmp-${pid}-${Date.now()}` `` temp-and-rename idiom (confirmed byte-identical). Recorded reason: none for the shared half; the two functions' validation logic legitimately differs (content-type checking differs between them, tracked separately in the previous review). Cost: a fix to the atomic-write idiom must be applied twice. Fix in place: one shared `fetchWithFallback`/`atomicWrite` pair, called from both. Verify by: tree comparison (vendored assets unchanged). Size: S. Verified: confirmed (V1).
+
+### Tier 3 — local untidiness
+
+| ID(s) | Kind | Statement | Where | Verified |
+|---|---|---|---|---|
+| A1-7 | dead | `cpu-worker.mjs:510` writes the literal `4` (FAILED status); nothing anywhere reads it. | `builder/cpu-worker.mjs:510` | Not re-verified. |
+| A1-8 | struct | `tbdocs.mjs`'s own argument parser is not exported and has no test. | `builder/tbdocs.mjs:91-194` | Not re-verified. |
+| A2-8 | dup | `replaceAll("\\","/")` inline at 10 sites; two more files each keep a private `posix()`. | `offline-rewrite.mjs:34,39,44,219,226`; `offline.mjs:133,311,370,375,380`; `publish-policy.mjs:189`; `check-tree.mjs:46` (recorded) | Not re-verified. |
+| A3-4 / A2-5 / A3-5 / A4-1 (L4-8 cluster) | dup | Four small leaf helpers — `isNonEmpty`, `encodeSpaces`, `splitFragment`, `statSafe` — are each duplicated once, three of them byte-identical, `encodeSpaces` a different idiom with the same behaviour. `makeTimer`, the fifth member of this cluster, is the substantial one and is covered as **A1-4** in Tier 2. | `nav.mjs:338` / `seo.mjs:164`; `search.mjs:204` / `template.mjs:925`; `render.mjs:1593` / `crawl_check.mjs:51`; `link-check.mjs:455-457` / `check_links.mjs:151-153` | Confirmed, all four (V1, V2). |
+| A3-8 | dup | Three near-identical palette-building loops in one file. | `highlight-theme.mjs:336-344,351-359,364-372` | Not re-verified. |
+| A3-9 | dead | `precomputeSeo` has zero callers. | `seo.mjs:90-94` | Not re-verified. |
+| A3-10 | dead | `kramdownSlug` is exported but used only inside its own module. | `render.mjs:1352` | Not re-verified. |
+| A5-4 (= L4-9 part 2) | dup | `pad`/`median` are byte-identical in two scripts. | `pick_a11y_sample.mjs:229-232,259`; `sweep_a11y.mjs:279-283` | Confirmed, byte-identical (V2). |
+| A6-5 | conv | `serve.bat` does not propagate its child process's exit code. | `serve.bat` | Not re-verified. |
+| A7-6 | conv | A comment says `tbrun`'s snapshot uses `-EncodedCommand`; the code uses `-Command` (safe today — a fixed literal). | `tb-registry.mjs:49-50` vs. `tbrun.mjs:376` | Not re-verified. |
+| A7-7 | dup | A 180-second compile timeout is a literal, repeated at 7 sites in 4 files. | across `scripts/lib/tb-*` and the three harness CLIs | Not re-verified. |
+| A7-9 | struct | `tbbuild`'s shutdown skips its tidy step when the IDE handle was never set; safe only by an invariant in `tb-launch.ps1`. | `tbbuild.mjs` shutdown path | Not re-verified. |
+| A8-3 | dup | The fence unit key is computed twice in the same file. | `check_examples.mjs:424` (`makeBatches`), `:675` (`unitsOf`) | Not re-verified. |
+| A9-4 | dup — **Resolved in `90624841`** | `_writeUint`/`_digitCount` duplicated between two shims, both since deleted. | (deleted) `fast-refs.mjs`, `fast-refs-class.mjs` | Moot; not separately re-verified. |
+| A9-5 | dup | The `createRequire` + `require('pdf-lib/cjs/...').default` block is repeated in 12 files (corrected from an estimated ~15); 3 of the 12 were among the shims deleted in `90624841`, leaving 9 in production. | `book/lib/fast-*.mjs` (list in [`REVIEW-TOOLING-fe9ce12b/V3.md`](REVIEW-TOOLING-fe9ce12b/V3.md)) | Corrected (V3). |
+| A9-9 | dead — **Resolved in `90624841`** | A comment cited a file moved in `4c36c8d6`; the citing file has since been deleted entirely. | (deleted) `fast-dict-array.mjs:9` | Moot; not separately re-verified. |
+| A10-1 | conv | Four `eval/` scripts each parse arguments differently; one exits 1 on a bare `--help`. | `eval/*.mjs`, incl. `transcript.mjs:190-198` | Not re-verified. |
+| L1-9 | conv | `--src` means the docs root in some tools and the exported package tree in others. | `tbdocs`, `check_publish_policy` vs. `census_attributes`, `build_package_api` | Not re-verified. |
+| L1-10 | conv | `--name=value` form works only in `tbdocs`, and not for two of its own flags. | `builder/tbdocs.mjs` | Not re-verified. |
+| L1-12 | dup | "Usage text; exit code follows whether help was asked" is repeated six times across `eval/` and wisdom. | `eval/`, `wisdom/` | Not re-verified. |
+| L2-6 | conv | A scratch directory is not removed in a `finally` in two scripts; four others do this correctly (corrected from "three"). | `check_publish_policy.mjs:152-189`, `check_links_diff.mjs:651-750` (wrong); `check_links.mjs`, `impexp.mjs`, `check_page_baseline.mjs`/`check_symbol_index.mjs`'s `withBaseline` (right) | Corrected (V2). |
+| L2-7 | dup | Three small tree walkers, not proposed as a fix. | `offline.mjs:401-416,608-626`; `write.mjs:281-299` | Not re-verified. |
+| L3-6 | conv | Two `spawnSync` calls throw unguarded; the resulting stack trace reaches the terminal at the wrong exit code and without the tool's own `error:` convention. | `check_links_diff.mjs`: `fusedBuild` (`:426,432`), `ensureBasePathTree` (`:518,525`) | Confirmed — fails loudly, just inconsistently (V4). |
+| L4-2 | dup | Two SCSS-compiling functions share the same body. | `scss.mjs:65-76,78-89` (`compileLightScss`/`compileDarkScss`) | Not re-verified. |
+
+---
+
+## Split candidates
+
+Per decision 2, these are candidates for a later, separate pass — not proposed as fixes here.
+
+- **`builder/render.mjs`** — strongest evidence. The image-renderer rule order matters and is silent about it (`svgInlinePlugin` at `:518` must run before `remoteImagePlugin` at `:520`; swapping them reverses the behaviour with no error); the ellipsis plugin assumes the dashes plugin already ran (`:515`/`:516`); shared helpers thread through every plugin registered on the instance; no plugin is tested in isolation; anchors reference third-party rule names directly (`"curly_attributes"`). Separately, its two fence/code-span parsers (**A3-1**) sit 1,550 lines apart with no cross-reference to each other.
+- **`builder/tbdocs.mjs`** — the `TASKS` literal fuses DAG topology with task bodies across 61% of the file (`:260-1257`); `dispatch`'s `submit()` is SAB machinery (`:788-911`); check glue (`:1085-1256`); Gantt/timing logic (`:1268-1403`) belongs beside `gantt.mjs` — **A1-1** is the visible cost of it not being there; the console report (`:1478-1525`); exit policy scattered across the file (**A1-6**).
+- **`scripts/check_examples.mjs`** — eight distinguishable jobs in one file: templates (`:125-193`), selection (`:201-378`), batching (`:380-506`), staging (`:508-568`), orchestration (`:570-655,928-949`), bisection (`:657-927`, 270 lines across 14 functions), diagnostic mapping inside `buildStaged` (`:643-653`), and three report modes threaded through as booleans — plus a 425-line probe suite (`:1157-1581`) that is itself a candidate on its own. `gen_attribute_probes.mjs` is explicitly **not** a candidate: 47% of that file is its own exploratory data table, not logic.
+- **`builder/book.mjs`** — a resolver (§A), `book.html` assembly (§B–F, including `rewriteBookHrefs`), and a coverage checker (§G) are already three separable concerns; `pdf.mjs` already imports all three as if they were separate modules.
+- **`builder/template.mjs`** — weaker evidence. Its `navActivationCss` deferral trigger, named in `PLAN-4.md` §3, is close to being met, which would remove part of the case for splitting it; **A3-7**'s finding (a date formatter, URL helpers, and an escape-helper block, all beyond templating) stands independent of whether the file is ever split.
+- **`scripts/lib/tb-ide.mjs`** — console reading and add-in introspection are separable from the coupled build-state core.
+- **`scripts/lib/axe-scan.mjs`** — **not proposed.** It has a single-source property the review found worth protecting rather than splitting: any split would have to re-export everything through one module anyway, so a split would add a layer without removing one.
+
+---
+
+## Verified sound
+
+**Build orchestration.** Small modules elsewhere in `builder/`; the SAB layout (174,100 bytes) matches `Builder.md`; `HANDLERS` key sets match across the scheduler and the workers; SAB constants are imported by name everywhere except **A1-7**; the barrier invariant holds; `--dry-run` and `--profile-offline` are both live and `Tools.md`'s flag table matches 20 of 20; the stall watchdog matches `WIP.Build.md`.
+
+**Output stages.** The `offline.mjs`/`offline-rewrite.mjs` split is a recorded, deliberate one; `HTML_COMBINED_RE` follows the code-guard shape; `deriveOfflineJtdJs` uses a real parser (`acorn`), not a regex; `vendor-assets.mjs` is contained and guarded; `symbols.mjs` is not a `.twin` scanner (a common misreading); `substitute()`/`decode()` name collisions across files are coincidental, not duplicated logic; the publish-allowlist's own disjointness self-test passes; the gates run clean today (publish 6/6, page-baseline 11/11, symbol-index 46/46).
+
+**The Markdown dialect.** `nav.mjs`; the table and fence fix-ups operate on single, already-known tokens; token-traversing `md.core` plugins are immune to the class of bug the rewrite-site theme describes, because they consult markdown-it's own parsed structure rather than re-deriving it; `html_block`-scoped rewrites are correctly scoped; `template.mjs` reuses `seo.mjs`'s `stripHtml` rather than a third copy; `clampContrast` fails loudly rather than silently; `VOID_TAGS_RE` is properly gated.
+
+**Link checker.** `check.mjs`, `check_links.mjs` and `check_links_diff.mjs` are coherent as three files, with no split candidate among them.
+
+**Accessibility and diagram gates.** The axe patch (`SOURCE_PATCHES`) earns the review's model status for a workaround — see Themes; the dot-metrics WASM patch passes emphatically (signature match, a read-back check, a real layout check, a `WeakSet` guard); `check_tree_fresh.mjs` correctly uses `isOutputTree`; the `FAMILIES` table is data, not logic that needs factoring; `check_a11y.mjs`'s mutual-exclusion handling is correct.
+
+**Gates as a system.** `check_gate_lists.mjs` is exemplary (18 probes); `check_publish_policy.mjs`'s check is bidirectional; `check_regex_safety.mjs` and `scripts/lib/regex-fold.mjs` are the strongest pair in the gate roster; `check_code_regions.mjs` and `scripts/lib/markdown-files.mjs` are strong; `convert_em_dash_separators.mjs` correctly uses the shared walker and preserves line endings; the exit convention is followed everywhere except **A5-2**/**A6-3**; the `.bat` idiom is deliberate and cross-referenced; the workflow files' own comments point at their source rather than restate it.
+
+**The compiler harness.** A strict DAG with no cycles; `tb-registry.mjs` is imported only by the three CLIs that need it; `stageProject` is genuinely shared; the `laneProjectId` allocator is correct; `check_tb_registry.mjs` has its own independent fixtures; every PowerShell payload is a fixed literal, with real data passed through the environment or stdin, never interpolated into the script text; the job-object-plus-suspended-kill path is sound; retry loops are recorded and fail loudly rather than silently.
+
+**Samples and the package API.** The `serialize`/`strip`/`kindOf` name collisions across files are coincidental, not duplicated logic; `symbols.mjs` is not a `.twin` scanner; `tb-fences.mjs` is tested, indirectly, through `check_examples.mjs`'s 74-probe `runProbes` suite.
+
+**The book pipeline.** `book.bat` is correct; `render-book.mjs`'s 13-shim import list is internally consistent; every shim's `__xInstalled` guard works as intended; the paged.js fork's divergence from upstream is fully recorded (20 `[PATCH]` tag families across 52 sites, in `Fixes-PagedJS.md`); `pdf.mjs`'s `IMG_SRC_RE` follows the code-guard rule; `outline.mjs` and `postprocesser.mjs` are attributed, unmodified ports of `pagedjs-cli` and should be treated as vendored code, not first-party tooling.
+
+**Smaller tools.** The site's own client-side JavaScript is a set of self-contained IIFEs, with no shared state between them; `svg-inline.js` does not duplicate `svgInlinePlugin` — one is markup-time, the other runtime; `build_fonts.py` matches what `WIP.Fonts.md` describes; the evaluator's process-isolation design is deliberate and sound; `wisdom/extract/merger.mjs` and the wisdom API's pagination are both fine apart from **L3-3**.
+
+**Paths, configuration, dependencies.** Exactly three real YAML parse sites exist (`tbdocs.mjs:271`, `check_publish_policy.mjs:69`, `data.mjs:12`), each loaded once per build; `check_publish_policy.mjs`'s cwd-relative `SRC` is a recorded, deliberate choice (`Extending.md:654`); atomic temp-and-rename writes are used correctly wherever they are used; `tbrun`'s per-port scratch-directory cleanup at the next start is deliberate; only `picocolors` and `pako` are undeclared dependencies; the lockfile and `npm ci` are used correctly throughout.
+
+**Browsers, processes, text rewrites.** The four puppeteer launches are consistent wherever consistency matters; `--allow-file-access-from-files` is needed wherever a page `fetch()`es a sibling `file://` resource; child processes never use `shell:true`, always pass array arguments, are killed by pid (never by image name), and the one stdin hazard the harness had was fixed once, centrally, in `runCompiler`; PowerShell payloads pass data through the environment or stdin, never interpolated; cleanup is disciplined across `tb-lane`, `tb-addin`, `addin_test.mjs` (on `SIGINT`), the add-in tests' own `after()` hooks, and `check_tb_registry.mjs`; `runShard` is the model worth following elsewhere; `check_links.mjs`'s worker dispatch and `worker-pool.mjs` are both sound. On text: `applyPreRenderRewrites` correctly brackets its four rewrites between mask and restore; token-scoped `md.core` rules are a third sound mechanism that `WIP.Build.md` does not currently name (a documentation gap, not a code one); `HTML_COMBINED_RE` and `IMG_SRC_RE` both follow the documented guard shape; every other unguarded rewrite in the tree targets build-generated markers, not author content.
+
+---
+
+## Where the passes were wrong
+
+The verification step exists because a pass's first read is not always the last word. These are the specific corrections that changed a finding's severity, its impact, or its standing:
+
+- **A4-3's severity.** First filed as R2 with a count of "20" attribute-tag pairs. The verifier found the build checker's table actually has 21 tags and 26 pairs, and — more importantly — that the gap between it and `crawl_check.mjs`'s own five-pair table is not merely a risk: it already causes the only post-release checker to miss seven kinds of link on the live site. Raised to R1.
+- **A8-1's impact.** The pass reported a live misclassification of `Overridable`-attributed members. The orchestrator traced it against the real BETA 983 package cache and found zero attributed `Overridable` members exist there today — the keyword-list divergence is real and already diverged (satisfying R1 on its own terms), but its specific claimed consequence is dormant, not live.
+- **The inventory's staging.md claim.** The markdown inventory reported "live, on-disk section corruption" in the committed `staging.md`, with a specific claimed mis-pairing of metadata between two named sections. Running the real `parseStaging` against the real file, the orchestrator found the named section (`Len.md · after-remarks`) has exactly the metadata beneath it on disk, correctly; the heading's own "see also thread" text, which lists other duplicate sections, is what misled the inventory's read. **Not confirmed.** What is real, and confirmed, is a narrower defect: a two-line content slip at `staging.md:14245-14246`, missing the blockquote continuation marker, which would render as a broken admonition and a runaway code fence if that section is ever grafted into `Len.md`. See Decisions for you (f).
+- **A7-1's count.** The orchestrator, reading the two patterns, first reported that tbrun's regex matches 2 of the 5 documented failure shapes. The verifier ran both regexes against all five shapes and found it actually matches 3 of 5 — `tbrun`'s case-insensitive flag catches both listed casings of the BUILD shape. The two shapes it misses, and the resulting false-success consequence, are unchanged.
+- **Corrected counts**, none changing a verdict's direction but each changing a specific number the finding cites: **A5-1** (7 scripts → 8, `check_tree_fresh.mjs` added); **A7-3** ("used by every scenario" → used directly in 4 of 10 test files); **A7-4** (9 imports → 10); **A8-2** (the ten-item citation list had one wrong entry and two missing; corrected to 11 strict-path / 13 bare-filename citations); **A9-5** (~15 files → 12, 3 of which were later deleted in `90624841`); **L2-5** ("four expressions" → confirmed 19 files, closer to five distinct AST shapes); **L2-6** ("three siblings do it right" → four).
+- **L4's classification of `findingsFor`/`buildFindings`.** L4 read the parallel-shape comments in `check.mjs` and `check_links.mjs` as a deliberate, still-valid design (mirrored on purpose so `check_links_diff.mjs` can cross-check two independent implementations) and declined to flag it. That reading was correct when the comments were written. It is superseded by the user's own 2026-09-25 decision to make `check_links.mjs` a thin wrapper over `builder/check.mjs` (decision 5): after that refactor, one implementation sits behind two front ends, not two independent ones, and both comments need to be rewritten to say so.
+- **L4's acceptance of `book.mjs`'s "plugins are independent."** L4 accepted the recorded reason for `book.mjs`'s private copy of `normalizeBaseurl`/`escapeRegExpBook` at face value. The reason describes the old Ruby/Jekyll plugin architecture, where `book-href-rewrite.rb` and `offlinify.rb` had no shared-code mechanism between them. That premise expired at the Node cutover — `book.mjs` already imports two sibling `builder/` modules for other things — and A9-8's reading is the one this document adopts.
+- **L3's "sound" verdict on the three whole-page HTML rewrites.** L3 called `padEmptyCells`, `normaliseVoidTags` and `injectAnchorHeadings` sound, reasoning that every code path escapes `&<>` before they run. Traced against all four ways text can end up inside ``/`
`, that is true for three of the four and false for the fourth (raw, hand-authored HTML). Both readings are partly right; **A3-6** stands as filed, at R2, because the fourth path is real even though nothing in the current corpus triggers it.
+
+---
+
+## Decisions for you
+
+**Decided on 2026-09-25: all seven as recommended.** The one question left open inside them is (b)'s local behaviour, settled when that gate is written.
+
+**(a) The shared markdown module — home, scope, and migration order.**
+Home: a new top-level `lib/` directory, sibling to `builder/`, `scripts/`, `wisdom/` and `eval/`. `builder/render.mjs:383`'s own comment rules out `scripts/lib/`, since `builder/` is a required consumer and must not depend on `scripts/`; `wisdom/`'s stated reason for its own private parser (`PLAN-3.md:404`, "no dependency on `builder/`") also rules out putting the module inside `builder/`. Scope: markdown-it block regions (fence/`code_block`/`html_block`, with `.map` line ranges) plus one tested inline-code-span splitter plus a frontmatter splitter built on `js-yaml` 4 directly, dropping `gray-matter` (and its bundled `js-yaml` 3) entirely. Migration order: the five rewrite sites first (`render.mjs`'s two, `convert_em_dash_separators.mjs`, `check_examples.mjs`'s marker splice, which is already close, and wisdom's `parseStaging`/`serializeStaging`), since a wrong region there can corrupt committed content; the ten read-only scan sites after, since their failure mode is under- or over-counting, not corruption.
+*Recommendation:* a new top-level `lib/`, with the inventory's API sketch (`blockRegions`, `maskCode`, `splitCodeSpans`, `splitOnMarker`, `parseFrontmatter`) as the starting point. The sketch was built by testing the real markdown-it instance against the real corpus, not from the sites' existing behaviour. The oracle for moving each rewriter onto it is the byte comparison of the built trees; for dropping `gray-matter`, every page's parsed frontmatter compared both ways.
+
+**(b) A parity gate for `impexp.mjs`/`impexp.py`.**
+**A10-6** found the two ports keep 19 byte-identical self-test names with nothing running either suite. Adding `check_impexp_parity.mjs` to `test.bat` closes that, but it is the first gate that would make `test.bat`, and both CI workflows, depend on a Python interpreter being present.
+*Recommendation:* add it, and run it unconditionally in CI, whose runners have Python. The discipline behind the parity is already real (the two test suites are kept in step by hand, and `Tools.md` promises byte-identical output); a gate only formalises a check the project already believes it needs. The part to decide is local: whether `test.bat` should require Python, or report this one gate as skipped, loudly, when no interpreter is found. `build_fonts.py` already requires Python, but only for someone regenerating fonts; nothing in the routine local loop does today.
+
+**(c) The pdf-lib shims — guard, or record the pin as the only guard.**
+**A9-1**/**A9-2**: either add load-time shape assertions per shim plus an equivalence test against stock pdf-lib (modelled on `check_axe_patch_equiv.mjs`), or explicitly record that the exact version pin is the only guard this workaround will ever have, given upstream pdf-lib is abandoned and the `@cantoo` fork was evaluated and rejected.
+*Recommendation:* add the assertions. The pin only protects against an accidental `npm update`; it does nothing for a deliberate, intentional bump to a newer pdf-lib release, or for a future contributor's mistaken hand-edit to a shim file. Both are realistic over the tool's lifetime, and the axe patch shows the cost of the guard is small once one example exists to copy.
+
+**(d) A composite CI action as a follow-on to the roster gate.**
+**A6-4**'s design proposes `.github/actions/run-gates` after the CI-roster gate is in place (decision 6), not as a reusable workflow — a separate job would lose the built trees the deploy job needs.
+*Recommendation:* proceed in that order. Building the roster gate first gives the composite action something to check itself against; building the action first would let it and the gate drift from each other before either is proven.
+
+**(e) The command-line convention Phase 3 should converge on.**
+`impexp.mjs`'s own discipline (a verb `Map`, an `EXIT` object, `usageError()`) is the best-disciplined CLI in the repository today and is the model **L1** names.
+*Recommendation:* for Phase 3, converge on that discipline: `--help` prints usage to stdout and exits 0; an unknown flag or a bad value prints to stderr and exits 2; each tool names its exit codes in one table. For Phase 2, which must change no behaviour, build the shared `scripts/lib/cli.mjs` module L1 specified — `parseCli()` over `node:util` `parseArgs` (strict, `allowPositionals`, tokens) with camelCase aliases and positional-count checks; `withUsageError()` preserving each tool's current message, stream and exit code; `numberOption()` closing the `NaN` hole; `printHelpAndExit()` preserving each tool's current `--help` text until Phase 3 actually changes it. Migrate `tbdocs.mjs` last — its cross-flag ordering (`--no-check` resets other flags) needs a scan over the parsed tokens, not a simple options object. Migrate wisdom last, or not at all — its subcommand shape needs custom code the shared module does not cover.
+
+**(f) The `staging.md` content slip.**
+The one confirmed, real defect in the committed data itself: `wisdom/data/findings/staging.md:14245-14246` is missing a blockquote continuation marker on a fenced sample's content line.
+*Recommendation:* fix the data now, independent of the tooling fix in (a): add the missing `> ` to lines 14245 and 14246. It is a two-line editorial correction to a file reviewers already edit by hand, and there is no reason to wait for the shared parser before correcting it.
+
+**(g) State the dependency-pinning policy, and fix `Builder.md`'s stale Dependencies section.**
+`Builder.md`'s "Dependencies" section omits `recheck` entirely, states `wasm-graphviz` as `^1.21` when it is `^1.29.1`, and says `axe-core` is the only exact pin when four packages are pinned exactly. This is a repeat of a drift the previous review already fixed once, in `74b3395`.
+*Recommendation:* fix the section, and add one sentence stating the policy itself — exact pin when the code patches the dependency's internals or depends on undocumented behaviour, caret otherwise — so a future caret on an output-changing dependency reads as a decision rather than an oversight.
+
+Not re-opened here, because each was already decided by the owner before or during the review: the link-checker architecture (`check_links.mjs` stays a thin wrapper over `builder/check.mjs`, decision 5); `perf/`'s scope (nothing moves out, `detach-pages.js` documented in place, decision 1 as amended); the four superseded pdf-lib shims (deleted in `90624841`); formatting as the last phase (decision 4, Phase 6); and working directly on `staging` with no PR branch.
+
+---
+
+## Appendix
+
+### Baseline survey
+
+Taken at `fe9ce12b` with `scripts/survey_tooling.mjs` (a token-level clone detector — identifiers and literals normalised, 60-token windows — plus an import graph over 185 first-party JavaScript files, `perf/` included, vendored code excluded):
+
+| Measure | At `fe9ce12b` | After the shim deletion (`90624841`) |
+|---|---|---|
+| Clone regions of 60+ tokens | 680, of which 318 outside `perf/` | 571, of which 266 outside `perf/` |
+| Top-level function names defined in 2+ files | 77, of which 57 outside `perf/` | 74, of which 54 outside `perf/` |
+| Command-line tools outside `perf/` reading `process.argv` | 32, none using `node:util` `parseArgs` | (unchanged by the deletion) |
+| Tools with a private copy of `flag`/`opt`/`die` | 6, with `opt` in three different versions | (unchanged by the deletion) |
+| Packages imported but not declared | 2: `picocolors`, `pako` | (unchanged by the deletion) |
+| Clone regions by pair of areas (`builder`/`scripts`, `scripts`/`scripts`, `builder`/`builder`) | 55, 95, 39 | (not re-measured) |
+
+Observed by hand at `fe9ce12b`: 5 places state which gates run (three `.bat` files, two workflows) plus `Tools.md`; no lint or format tooling exists; line endings are LF in the repository but CRLF in 118 of 140 working-tree files under `core.autocrlf=true`, with no `.gitattributes`; quote style is double in `builder/`, `scripts/`, `test/`, `eval/` and single in `book/`, `wisdom/`, `perf/`.
+
+The drop in clone regions and repeated function names between the two columns reflects the four pdf-lib shims deleted during the review. Apart from the three findings that deletion resolved, none of this document's fixes has been made yet.
+
+### Finding counts, merged, by tier and kind
+
+Counts are of merged findings (one row per finding as presented above), not of raw pass-level citations; a finding with a compound kind label (for example "dup/hack") is counted under its first-listed kind.
+
+| Tier | dup | hack | struct | conv | dead | Total |
+|---|---|---|---|---|---|---|
+| Tier 1 (R1) | 16 | 3 | 1 | 0 | 0 | 20 |
+| Tier 2 (R2) | 19 | 2 | 9 | 8 | 5 | 43 |
+| Tier 3 (R3) | 11 | 0 | 2 | 7 | 4 | 24 |
+| **Total** | **46** | **5** | **12** | **15** | **9** | **87** |
+
+Eighty-seven merged findings account for 109 raw finding IDs across the fourteen passes; the difference is the roughly twenty IDs the ledger's explicit MERGE and `=` directives fold into another finding. Three findings (**A9-4**, **A9-6**, **A9-9**) are resolved, in `90624841`, ahead of the fix phases the rest of this document's findings still await.
diff --git a/builder/REVIEW-TOOLING-fe9ce12b/README.md b/builder/REVIEW-TOOLING-fe9ce12b/README.md
new file mode 100644
index 00000000..8f64c28d
--- /dev/null
+++ b/builder/REVIEW-TOOLING-fe9ce12b/README.md
@@ -0,0 +1,18 @@
+# Evidence for REVIEW-TOOLING-fe9ce12b.md
+
+The working records behind [the review](../REVIEW-TOOLING-fe9ce12b.md), kept as they were
+when it was written. Line numbers in them refer to `fe9ce12b`.
+
+| File | What it is |
+|---|---|
+| [ledger.md](ledger.md) | The orchestrator's condensed record of every pass's findings (A1–A10, L1–L4), the owner's decisions as they were taken, and the verification status. Where its top sections correct a pass, the correction wins. |
+| [V1.md](V1.md) | Verifier V1: the `builder/` findings. |
+| [V2.md](V2.md) | Verifier V2: the gates, the link checkers, the accessibility tools, CI and the smaller tools. |
+| [V3.md](V3.md) | Verifier V3: the compiler harness, sample compiling, the package API and the book pipeline. |
+| [V4.md](V4.md) | Verifier V4: pass L3's findings, and the ruling on the A3-6 conflict. |
+| [markdown-inventory.md](markdown-inventory.md) | Every place the tooling processes markdown as text, and what markdown-it can supply. The ledger calls it `INVENTORY.md`. Its claim of section mis-pairing in `staging.md` was **not confirmed**; see the review's "Where the passes were wrong". |
+
+These files cite `scratchpad/...` paths: the session folder where the passes and verifiers
+ran small scripts. The scripts were not kept. Some copied repository functions verbatim for
+comparison, and would distort `scripts/survey_tooling.mjs`'s clone counts if committed; each
+result is described in the file that cites it.
diff --git a/builder/REVIEW-TOOLING-fe9ce12b/V1.md b/builder/REVIEW-TOOLING-fe9ce12b/V1.md
new file mode 100644
index 00000000..9a131a03
--- /dev/null
+++ b/builder/REVIEW-TOOLING-fe9ce12b/V1.md
@@ -0,0 +1,80 @@
+# V1 verification report
+
+Verified against commit `fe9ce12b` (confirmed `HEAD == fe9ce12b` at start; working tree later
+gathered the expected drift described in the task — four `book/lib/fast-*.mjs` deletions and
+comment edits in `book/render-book.mjs`, `perf/measure.mjs`, `perf/phase0-measure.mjs` — none of
+which touches any file cited below). All line numbers are as read via `git show fe9ce12b:`,
+copied into `scratchpad/verify/V1/`.
+
+## Table
+
+| ID | Verdict | Severity note | Correction / evidence (path:line at fe9ce12b) |
+|---|---|---|---|
+| A1-1 | CONFIRMED | R1 struct fits — already silently drops tasks on every `--check` build. | `builder/gantt.mjs:39-49` (`mainSections = [["Seeds",...],["Spine",...],["Write",...]]`, loop at 41-48 only pushes `t.lane!=null` (Workers), `"Seeds"`, `"Spine"/"Render"`, `"Write"` — anything else, incl. `"Check"` and `"Other"`, is silently dropped). `builder/tbdocs.mjs:1270-1282` `GANTT_SECTION`/`GANTT_SECTION_ORDER` include `"Check"` (`linkJoin`/`checkBook`/`checkReport`); `checkBook`(1178-1196)/`checkReport`(1200-…) both `runOnMain:true` so never get a `.lane`. `vendorAssets` (539-562) is `runOnMain:true` and has no `GANTT_SECTION` entry, so falls to `"Other"` (line 1294) — also dropped. `COLORS.Other` (gantt.mjs:12) confirmed unreferenced by any code path that actually runs. `docs/Documentation/Builder.md:410` promises "Tasks without a section fall into a generic 'Other' bucket" — false, they vanish. |
+| A1-2 | CONFIRMED | R1 fits (breaks on the next `puppeteer`/`cosmiconfig` bump). "hack" label is a loose fit for an undeclared transitive dep, but no better bucket exists; not worth reclassifying. | `builder/tbdocs.mjs:35` `import pc from "picocolors";`, `builder/scheduler.mjs:5` same. `package.json` `devDependencies` (fe9ce12b) has no `picocolors` entry. Lockfile chain confirmed exactly as claimed: `puppeteer` (line 1902-1905) → `cosmiconfig` (912-922) → `parse-json` (1801-1811) → `@babel/code-frame` (1795, dep `"picocolors": "^1.1.1"` at line 40). |
+| A1-3 (=L4-1) | CONFIRMED, incl. 4th-path claim | R2 fits — not yet diverged, but a bug fix (e.g. adding the FAILED-status write) must be applied 3x by hand. | Three near-identical "run handler, time it, report `perWorkerTiming`" blocks in `builder/cpu-worker.mjs:359-377` (idle task), `423-440` (nested dep), `469-486` (dep) — same shape (`t0=Date.now()`/try-catch calling `handlerById[...handlerIdx]()`/`t1=Date.now()`/`Atomics.store(perWorkerDone,...)`/`postMessage({perWorkerTiming:true,...})`), differing only in which index/meta/result variable is used. 4th path (`497-540`, "Execute task") is genuinely different: calls `handler(taskIdx)` (with an arg, not `handlerById[meta.handlerIdx]()`), posts a **different message shape** (`{done,output,timing,lane}` not `{perWorkerTiming,...}`), writes `Atomics.store(views.status,taskIdx,4)` on failure (ties to A1-7), and — per its own comment at 515-525 — deliberately posts the message **before** the SAB `onTaskDone` update (opposite order from the other three, which `Atomics.store` before `postMessage`), to close the exact race that "lost pages from the search index." That ordering concern has no counterpart in the other three blocks. |
+| A1-4 | CONFIRMED | R2 fits — two copies, and the stated reason for the private one is gone, so a future editor has no way to know which copy offline.mjs actually uses. | `builder/tbdocs.mjs:196-209` `export function makeTimer()`; repo-wide grep for `makeTimer` outside its own file and `builder/offline.mjs` finds only `builder/PLAN-9.md:68`, a planning doc, never an actual import. `builder/offline.mjs:98-111` private, textually identical copy, called at `offline.mjs:151`. Its comment (95-97) says the private copy avoids a cyclic import because "the verify harnesses and diff tools import offline.mjs without going through tbdocs.mjs's main() side effects" — `git show 644d6bdb --stat` confirms that commit deleted exactly those tools (`_diff.mjs`, `_diff_all.mjs`, `_triage.mjs`, `_sitemap_diff.mjs`, `_audit_accepted.mjs`, `accepted-divergences.mjs`, `verify-phase1..8.mjs`). |
+| A1-6 | CONFIRMED | R2 conv fits (not yet a live defect — see "noticed in passing" #3 for why the ordering is currently accidentally safe). | Exactly 7 sites, all bare `1`/`2` literals, no named constants: `tbdocs.mjs:560` (`process.exitCode = 1`, vendorAssets), `1469` (dot), `1470` (scss), `1570` (`(process.exitCode??0)\|code` inside the 1568-1573 check block), `1573` (recheck-only branch), `1598` (page-baseline drift), `1610` (symbol-baseline drift). Bit 0 (`=1`/`\|1`) is set independently by 5 of those 7: vendorAssets(560), dot(1469), scss(1470), page-baseline(1598), symbol-baseline(1610) — "conflates 5 causes" confirmed exactly. |
+| A2-1 (+L4-4, A1 lead) | CONFIRMED | Lean **R2**, not the pass's own hedged R1 — nothing here has actually diverged (it's unreachable, not two live copies disagreeing); the R1 rubric example is "copies that disagree." Matches the source pass's own "[likely R2]" hedge. | `writeOfflinePages` defined `offline.mjs:244-289`; repo-wide grep for `writeOfflinePages\(` (a call, not the definition) finds **zero** hits anywhere, including inside `offline.mjs` itself — `writeOffline`'s own `Promise.all` (179-195) no longer calls it (comment at 174: "Offline page HTML is already on disk from per-worker flush"). Its body (258-280, the `byDir`/`navCache` pre-pass) is line-for-line the same algorithm as the **live** copy in `cpu-worker.mjs:183-201` (same variable roles: `writable`/`byDir`/`navCache`, same loop shapes), confirming the "clone" half. `writeOffline`'s `precomputed=false` destructured param (`offline.mjs:117`) is never referenced anywhere in `writeOffline`'s body (117-198) — confirmed dead by direct read. `buildOfflineState`'s `sitePaths ?? await buildSitePaths(...)` fallback (207, 213) is unreachable: `tbdocs.mjs:705-706` always computes and sets `state.sitePaths` unconditionally, and the `writeOffline` task (1024-1027) always forwards `sitePaths: state.sitePaths`; `buildSitePaths` itself spans exactly `359-394` as cited. `writeSearchData` (`search.mjs:18-24`) and `extractSitemapUrls` (`sitemap.mjs:70-74`, off by one from the cited 69-74 — the function itself starts at 70, a one-line `LOC_RE` const is at 69) both have zero call sites repo-wide (only their own definitions and doc-table mentions). Re-export block `offline.mjs:55-88` (24 names): confirmed via `import { writeOffline, enumerateVendoredThemeAssets } from "./offline.mjs"` (`tbdocs.mjs:58`, neither name is in the 24) and `import { buildSitePathsSync, deriveOfflineCss, normalizeBaseurl } from "./offline-rewrite.mjs"` (`tbdocs.mjs:59-60`, bypassing offline.mjs's re-export entirely) that **every one of the 24 re-exported names has zero importers anywhere in the repository** — confirmed repo-wide, not just for the names A1's lead singled out. |
+| A2-2 (+A9-10 comment half) | CONFIRMED | R2 is defensible (a stale comment claiming a real external consumer is what lets dead code like A9-10's `extractImagePaths` survive review); could be argued R3, not clearly wrong either way. | All citations verified to name the tools `644d6bdb` deleted: `offline-rewrite.mjs:411` ("`_diff.mjs`'s per-call buildOfflineState"), `search.mjs:65-66` ("`_triage.mjs`, `_diff.mjs`... `accepted-divergences.mjs`"), `sitemap.mjs:67-68` ("`_triage.mjs` and `_sitemap_diff.mjs`") and `:97` ("`_triage.mjs` / `_diff.mjs`"), `redirects.mjs:35-36` ("`_triage.mjs` / `_diff.mjs`"). `pdf.mjs:6-9` ("`_diff.mjs --book` / `_triage.mjs auditBook*`") and `:99-107` ("used by... the diff tools"; "retained below as a fallback/diagnostic export for the bulk-triage tools") — and `extractImagePaths` (`pdf.mjs:146-158`) confirmed zero callers repo-wide (only its own definition and a `docs/Documentation/Pipeline-Stages.md:847` doc-table row referencing it as documentation, not a call). |
+| A2-3 (+L2-4, L4-5) | CONFIRMED | R2 dup fits well. | `readBaseline` byte-identical: `page-baseline.mjs:83-90` vs `symbol-baseline.mjs:45-52`, both exactly `try { return JSON.parse(await readFile(file,"utf8")); } catch(err) { if (err.code==="ENOENT") return null; throw err; }`. Six-branch structure re-typed: `checkPageBaseline` spans exactly `117-176`, `checkSymbolBaseline` spans exactly `75-125` — both: skip-if-wrong-src, force-write, missing-baseline (fail-if-!write / create-if-write), regression (fail), growth (write-if-write), fallthrough no-op. `WIP.Build.md:589` ("The drift guard is the page-count guard's shape applied to URLs") is a real recorded reason but — as the pass argues — explains only why the two look alike by design, not why the actual branch bodies weren't factored into one shared state machine. |
+| A2-4 (+A3-5 escapeRegExp) | CONFIRMED | R2 dup fits; not yet diverged so not R1. | Three byte-identical bodies, `s.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")`: `render.mjs:2235-2237` (`escapeRegExp`), `offline-rewrite.mjs:239-241` (`export function escapeRegExp`), `book.mjs:212-214` (`escapeRegExpBook`). `scripts/lib/regex-fold.mjs:58-63` confirms the cost cited: it documents recognising the escaper "by shape rather than by name, so the three copies in this tree (`escapeRegExp` twice, `escapeRegExpBook`) and any fourth are all covered without a list to maintain" — i.e. the gate had to be specially built to tolerate this exact duplication, which is evidence *for* the finding, not a defeating "recorded reason" (it justifies the gate's design, not the duplication itself). |
+| A2-6 (=A9-8) | CONFIRMED | R2 dup fits (not yet diverged). | `book.mjs:303-307` vs `offline-rewrite.mjs:232-236` (`export`ed): byte-identical bodies. See Conflicts §2 for the "by design" comment ruling. |
+| A2-7 | CONFIRMED | R2 dup fits; connects to a named prior incident, so the "cost" argument is well grounded. | `compress.mjs:73-84` `collapseWhitespace` (regex-based, `/[ \t\n\r\f\v]+/g` + `.trim()`/one-sided regex trims) vs `search.mjs:251-261` `stripAsciiWhitespace`+`isAsciiWs` (charCode-scan based). Both explicitly preserve NBSP (U+00A0) with near-identical comments explaining why. `WIP.Build.md:291-296` "### Whitespace inside inline code is content" confirms a real, named prior shipped defect in exactly this area of `compress.mjs`. |
+| A3-1 | CONFIRMED (directly reproduced) | R1 fits — the divergence is demonstrated, not hypothetical. | **Reproduced** in `scratchpad/verify/V1/test_A3-1_repro.mjs` by importing `builder/render.mjs` (copied at fe9ce12b) directly. Source `` "```abc`def\n> [!NOTE]\n...\n```" ``: `maskCodeRegions` (render.mjs:141, open-fence test at 148 `/^[ \t]{0,3}(`{3,}\|~{3,})/`, no info-string check) treats the whole block as ONE masked region and restores it byte-for-byte unchanged. `rewriteAdmonitions` → `stashCodeFences` (1706, guard at 1713 `if (ticks[0]==="`" && info.includes("`")) { out.push(lines[i]); continue; }`) refuses to recognise the same line as a fence (CommonMark-correct: a backtick fence's info string may not contain a backtick) — so it does **not** protect the "> [!NOTE]" line, and `ADMONITION_RE` converts it into a real `
`. Confirmed both via `rewriteAdmonitions` alone and via the full `applyPreRenderRewrites` pipeline. Control case (fence with no backtick in the info string) confirmed the admonition is correctly left alone, isolating the bug to exactly this edge case. Did not independently re-trace the "check_code_regions structurally blind to it" sub-claim beyond what WIP.Build.md documents about the gate's known "mirror fault" blind spot — plausible, not separately re-verified. | +| A3-2 | CONFIRMED | R2 dup fits — dormant today (as the finding itself says), so not R1. | `highlight.mjs:249-254`: 3-char `escapeHtml` (`&<>` only), with an explicit, specific recorded reason ("Rouge's HTML formatter escapes only `& < >`... Match that"). `render.mjs:2225-2227`: 5-char `escapeHtml` (`&<>"'`). `headingTocHtml` (`render.mjs:1437-1450`) confirmed mixing both **within the same function**: line 1440 uses the 5-char `escapeHtml` for `text` tokens, line 1442 uses the 3-char `escapeHtmlMinimal` (2230-2233, itself a near-duplicate of highlight.mjs's escaper) for `code_inline` tokens. See Conflicts §3. | +| A3-3 (+L4-6) | CONFIRMED | R1 fits — rubric's own R1 example is "copies that disagree," and these genuinely do, in 3 separate documented ways. | `seo.mjs:121-148` (config-arg `absoluteUrl`/`relativeUrl`, `ensureLeadingSlash` **forces** a leading slash via 145-148, no space-encoding) vs `template.mjs:917-936` (baseurl-arg, url left **unchanged** if it doesn't already start with `/` — line 922 `return url;` — and `encodeSpaces` called at 920). L4-6's two extra disagreements both verified: (1) protocol-relative `//host` — `template.mjs:919`'s condition `url.startsWith("/") && !url.startsWith("//")` explicitly excludes `//host` from baseurl-prepending (leaves it alone), while `seo.mjs`'s `isAbsoluteUrl` (160-162, scheme-only regex) returns false for `//host`, so `seo.mjs`'s `relativeUrl` prepends baseurl to it regardless — traced by hand that this is currently masked because this site's `baseurl` is always `""`. (2) non-string/null input — `seo.mjs`'s `absoluteUrl` returns `null` for `input==null` (line 122) vs `template.mjs`'s `absoluteUrl`/`relativeUrl` both return `""` for `typeof x !== "string"` (929-930, 917-918). | +| A3-4 | CONFIRMED | Lean **R3**, not R2 — two 4-line, leaf, no-divergence-risk predicate functions; smaller and lower-cost than the other escape/URL dups grouped nearby. Not a clear-cut call either way, so not insisting on a change. | `nav.mjs:338-342` byte-identical to `seo.mjs:164-168`: `function isNonEmpty(value) { if (value==null) return false; if (typeof value==="string") return value.length>0; return true; }`. | +| A3-6 | CONFIRMED | R2 hack fits — fails the rubric's "guarded" test (nothing fails loudly if the entity-escaping invariant ever breaks). | `render.mjs:74-80` `padEmptyCells` (`html.replace(/<(t[dh])([^>]*)><\/\1>/g,...)`), `render.mjs:351-354` `normaliseVoidTags` (`html.replace(VOID_TAGS_RE,...)`), `template.mjs:714-727` `injectAnchorHeadings` (`html.replace(HEADING_REGEX,...)`, `HEADING_REGEX` itself at 694 has no ``/`
` leading alternative) — all three are bare whole-string `.replace()` calls over **rendered** HTML with no `replaceOutsideCode`-style guard, contrary to WIP.md's own Don't rule ("a rendered-HTML rewrite uses `replaceOutsideCode` or the ``/`
` leading-alternation shape"). Safety today rests on code content already being entity-escaped by the renderer (stated only locally, e.g. template.mjs:711-713, not as a general contract), which is outside anything `check_code_regions.mjs` verifies for the post-render chain. |
+| A3-7 | CONFIRMED | R2 struct fits (textbook "several unrelated jobs"). | `template.mjs` confirmed to hold, beyond its core templating job: a strftime formatter spanning **exactly** `940-989` (`STRFTIME_*` tables 940-945, `formatDate` 947-980, `parseDate` 982-989); URL helpers `relativeUrl`/`absoluteUrl`/`encodeSpaces` (917-936); and — beyond what A3-7's own citation mentions — a fourth escape-helper implementation under an explicit "`---------- §5.15 escape helpers ----------`" section header (991-998, `HTML_ESCAPE`/`escText`), reinforcing the "several unrelated jobs" struct claim. |
+| A9-7 | CONFIRMED | R2 dup fits (not yet diverged; "safety-critical" framing justifies keeping it above A3-4/A2-4's tier despite being nominally the same severity). | `book.mjs:210` `CODE_OR_PRE_BOOK = /]*>[\s\S]*?<\/code>\|]*>[\s\S]*?<\/pre>/` re-typed, **in the same file**, inside `book.mjs:228-229` `IMG_SRC_RE_BOOK` (whose first two alternatives are that exact text). `pdf.mjs:143-144` `IMG_SRC_RE` confirmed byte-identical in full to `book.mjs`'s `IMG_SRC_RE_BOOK`. `offline-rewrite.mjs:299` `HTML_COMBINED_RE` opens with the identical code/pre alternation text before diverging into its own href/src third alternative. Did not independently confirm "no gate ties them" beyond consistency with everything else read (no fragment-identity check found anywhere in the gates I reviewed). |
+| A9-10 | CONFIRMED | Same R2-vs-R3 judgment call as A2-2 — defensible either way. | `extractImagePaths` (`pdf.mjs:146-158`) has zero callers repo-wide: only its own definition and a documentation-table row (`docs/Documentation/Pipeline-Stages.md:847`) reference the name; no `.mjs` file calls it. `deriveBookOutputs` (108-110) confirmed live, called from `writePdf` at line 61. |
+| L2-1 | CONFIRMED, both halves | R1 fits strongly — this is a literal recurrence of a bug class already fixed once elsewhere (`check_tree_fresh.mjs`), with a concrete failure scenario demonstrated for each half. | **serve.mjs half:** `builder/serve.mjs:138` `IGNORED_PREFIXES = ["_site","_site-offline","_site-pdf","_serve","_pdf","node_modules",".git"]`, checked via exact `.includes(segs[0])` (line 144) — no `_site-basepath` entry, so a `--dest docs/_site-basepath` run mid-`serve.bat` is not filtered, causing the claimed spurious rebuild. Git history confirmed exactly as claimed: `scripts/lib/markdown-files.mjs` (with its general `isOutputTree` prefix-matcher, `OUTPUT_TREES=["_site","_serve","_pdf"]`, `.startsWith(prefix)`) was created in `3834815d` (2026-09-23 21:33:29); `60bb6f5` (2026-09-23 21:45:22, 12 minutes later, same day) then edited `IGNORED_PREFIXES` directly (`git show 60bb6f5 -- builder/serve.mjs` shows the literal diff, removing `"_serve-offline","_serve-pdf"`) **without** switching it to the newly-created shared helper. **build_corpus.mjs half:** `eval/build_corpus.mjs:59-71` `EXCLUDED_PATHS` does list `"docs/_site-basepath"` explicitly, but `isExcluded` (109-111) tests `rel===p \|\| rel.startsWith(p+"/")`, which does not match `"docs/_site-basepath-offline"` or `"...-pdf"` (next char after the prefix is `-`, not `/`) — confirmed by hand-tracing the string test. `markdown-files.mjs`'s `isOutputTree` correctly covers all of these via a plain prefix match. |
+| L4-3 | CONFIRMED | R2 dup fits (single-file scope, moderate cost). | `vendor-assets.mjs:222-252` `fetchToFile` and `:279-319` `fetchAttachment` both independently implement: `fetch()` wrapped in try/catch → `{ok:false,status:"network error (...)"}"`, then `if(!res.ok)` check, then a second try/catch around `Buffer.from(await res.arrayBuffer())`, then — after their (legitimately different) validation logic — the **identical** atomic-write idiom: `` const tmpPath = `${destPath}.tmp-${process.pid}-${Date.now()}`; await fs.mkdir(...,{recursive:true}); await fs.writeFile(tmpPath,buf); await fs.rename(tmpPath,destPath); ``. |
+| L4-7 | CONFIRMED equivalent today | R2 dup fits, matching A2-7's reasoning (a real prior bug lends weight even though not yet diverged). | Structural comparison: `render.mjs:180-204` `maskInlineCode` and `convert_em_dash_separators.mjs:74-99` `splitInlineCode` share the identical backtick-run-counting algorithm (same `k`/`n`/`p`/`m`/`found` shape), differing only in output format (masked string+stash vs `{code,text}` segment array). **Empirically verified equivalent** via `scratchpad/verify/V1/test_L4-7_equiv.mjs` (scratch-only `export` added to each private function, not the repo): 16 hand-picked edge cases (empty string, no backticks, single/double/quadruple-backtick runs, unmatched backticks, adjacent spans, line-boundary spans, mismatched-length runs, a doubled-backtick span containing one literal backtick, multiple equal-length runs) — all 16 produced identical masked output and identical extracted code-span lists. `convert_em_dash_separators.mjs:70-73`'s own comment confirms a real prior shipped bug in this exact logic ("The previous regex matched single-backtick spans only, so an em-dash inside a doubled-backtick span was rewritten as prose"). |
+| L4-8 | CONFIRMED, with one exception | Per-component severities as separately recorded (R2 for makeTimer/isNonEmpty, R3 for encodeSpaces/splitFragment/statSafe) all fit; no change suggested. | Of the five: **isNonEmpty** (`nav.mjs:338-342`/`seo.mjs:164-168`) — byte-identical. **makeTimer** (`tbdocs.mjs:196-209`/`offline.mjs:98-111`) — byte-identical (=A1-4). **splitFragment** (`render.mjs:1593-1597`/`crawl_check.mjs:51-55`) — byte-identical modulo the parameter name (`href` vs `url`). **statSafe** (`link-check.mjs:455-457`/`check_links.mjs:151-153`) — byte-identical. **encodeSpaces is the one exception**: `search.mjs:204-206` uses `s.replaceAll(" ","%20")`, `template.mjs:925-927` uses `s.replace(/ /g,"%20")` — same behaviour, genuinely different idiom, exactly as A2-5's own "(different idioms)" note already says. So 4 of 5 are byte-identical; the fifth is functionally-but-not-textually equivalent. |
+
+## Conflicts
+
+### 1. `writeBaseline` in `builder/symbol-baseline.mjs`
+
+**Ruling: sides with L2-4.** It equals `JSON.stringify(x, null, 2) + "\n"` for every list except the empty one.
+
+Code (`symbol-baseline.mjs:55-58`):
+```js
+async function writeBaseline(file, src, urls) {
+  const body = urls.map((u) => `    ${JSON.stringify(u)}`).join(",\n");
+  await writeFile(file, `{\n  "src": ${JSON.stringify(src)},\n  "urls": [\n${body}\n  ]\n}\n`, "utf8");
+}
+```
+
+Empirically verified by calling the real (copied, unmodified) `checkSymbolBaseline` with `force:true` against scratch output files (`scratchpad/verify/V1/test_writeBaseline.mjs`):
+
+- Non-empty (`urls:["a","b"]`): actual output `{\n  "src": "docs",\n  "urls": [\n    "a",\n    "b"\n  ]\n}\n` — **byte-identical** to `JSON.stringify({src:"docs",urls:["a","b"]}, null, 2) + "\n"`.
+- Empty (`urls:[]`): actual output `{\n  "src": "docs",\n  "urls": [\n\n  ]\n}\n` (blank line between `[` and `  ]`) vs `JSON.stringify({src:"docs",urls:[]}, null, 2) + "\n"` = `{\n  "src": "docs",\n  "urls": []\n}\n` — **differs**.
+
+L4's "deliberately different (one URL per line)" framing doesn't hold up: `JSON.stringify(...,null,2)` *also* puts one URL per line for any non-empty array (proven above), so "one URL per line" doesn't actually distinguish the hand-rolled version from `JSON.stringify` in any case that matters — they diverge only in the single degenerate empty-list case, and no comment anywhere claims that specific rendering (`[\n\n  ]` instead of `[]`) is wanted. Reads as an accidental artifact of building the array body by `.map().join(",\n")` instead of delegating to `JSON.stringify`, not a deliberate choice. Harmless in practice — `[\n\n  ]` is still valid JSON and round-trips through `readBaseline`'s `JSON.parse` unchanged — so this is a cosmetic R3, not a correctness bug.
+
+### 2. `normalizeBaseurl` / `escapeRegExpBook` in `builder/book.mjs`
+
+**Ruling: sides with A9 (=A9-8/A2-6).** The recorded reason has expired.
+
+`book.mjs:300-302`'s comment: "PLAN-8 §6.12 / §6.13 (duplicated from offline.mjs by design -- book-href-rewrite.rb keeps its own copy of normalize_baseurl so plugins are independent)."
+
+That reason describes the *old Ruby/Jekyll* architecture, where `docs/_plugins/book-href-rewrite.rb` and `docs/_plugins/offlinify.rb` were separate Ruby plugin files with no shared-code mechanism between them. **What `book.mjs` imports today** (lines 27-28): `import { compressHtml } from "./compress.mjs"; import { loadData } from "./data.mjs";` — `book.mjs` is an ordinary Node ES module that already freely imports from two sibling `builder/` modules. The "plugins are independent" premise that justified a private copy no longer describes the code: nothing about the current module graph prevents `book.mjs` from doing `import { normalizeBaseurl } from "./offline-rewrite.mjs"` the same way it already imports `compressHtml` and `loadData`. The comment is also now factually wrong about the location — the current home of `normalizeBaseurl` is `offline-rewrite.mjs` (`offline-rewrite.mjs:232-236`, exported), not `offline.mjs` (confirmed via A2's "sound" note that the offline/offline-rewrite split happened after this comment was written, and via `offline.mjs`'s own import of `normalizeBaseurl` *from* `offline-rewrite.mjs` at line 42).
+
+### 3. `escapeHtml` 3-char vs 5-char
+
+**Ruling: keep A3-2, narrowed, per the ledger's own proposed resolution — confirmed correct.**
+
+`highlight.mjs:249-251` records a specific, narrow, still-valid reason for *its own* 3-char escaper: "Rouge's HTML formatter escapes only `& < >` -- not quotes. Match that so string literals inside code blocks keep their literal \" character." That fully explains why `highlight.mjs` (syntax-highlighted code blocks) differs from a generic 5-char escaper — it is not an unexplained inconsistency by itself.
+
+It does **not**, however, say anything about `render.mjs`'s own internal consistency. `render.mjs` has both a 5-char `escapeHtml` (2225-2227) and a 3-char `escapeHtmlMinimal` (2230-2233, itself unrelated to Rouge/highlighting), and `headingTocHtml` (1437-1450, unrelated to code highlighting — it renders TOC entries) mixes them for sibling inline-token types within one function: `escapeHtml` for `text` nodes (1440), `escapeHtmlMinimal` for `code_inline` nodes (1442). That specific mixing has no recorded justification anywhere and is a separate, narrower fact from highlight.mjs's Rouge-parity choice. So: don't treat highlight.mjs's 3-char choice as an unexplained global inconsistency (it's explained), but do keep the narrower claim that `render.mjs`'s own TOC rendering inconsistently escapes apostrophes between adjacent token types (dormant today — confirmed no counter-evidence found that any current TOC'd heading contains an apostrophe, but not exhaustively re-swept).
+
+## Noticed in passing
+
+- `offline.mjs:364-365`'s `buildSitePaths` comment also says "the diff tools call this without running templatePhase first" — a fifth live reference to the deleted `_diff`/`_triage` tools beyond A2-2's cited list.
+- `render.mjs` alone now has two near-duplicate HTML escapers (5-char `escapeHtml` 2225-2227, 3-char `escapeHtmlMinimal` 2230-2233), and the latter is itself a third copy of `highlight.mjs`'s escaper (249-254) — a wider "escape helper" cluster than A3-2 alone captures (A2's own "Leads" section already flagged "HTML escape maps x4").
+- A1-6's three plain-`=` bit-0 sites (`tbdocs.mjs:560,1469,1470`) are only safe today because they execute strictly before the three `\|=`'d sites (1570/1573/1598/1610) in source order; nothing enforces that order, so reordering the task graph or the summary code is a silent bit-clobbering hazard beyond what A1-6's "magic numbers" framing alone conveys.
+- `pdf.mjs:115-131` `resolveBookPage` is called at line 59 purely for its throw-on-missing/duplicate side effect; its return value is discarded (the real lookup goes through `site.bookData` elsewhere per its own comment, 112-114) — not a finding, just an easy-to-miss-on-first-read pattern.
+- `render.mjs:19-23` and `counts.mjs:68` form a deliberate, documented circular import (`maskCodeRegions` re-imported back into the module that partly motivates it); confirmed safe as described (both sides are hoisted function declarations), included here only because it's the kind of thing that looks like a bug until you check.
diff --git a/builder/REVIEW-TOOLING-fe9ce12b/V2.md b/builder/REVIEW-TOOLING-fe9ce12b/V2.md
new file mode 100644
index 00000000..cc337dd1
--- /dev/null
+++ b/builder/REVIEW-TOOLING-fe9ce12b/V2.md
@@ -0,0 +1,69 @@
+# Verifier V2 results
+
+Verified against commit `fe9ce12b` (read via `git show fe9ce12b:` and `git archive fe9ce12b`
+throughout; the working branch moved to `86a70107` partway through this session, confirmed not to
+affect any citation below since every read was pinned to the `fe9ce12b` ref).
+
+## Findings table
+
+| ID | Verdict | Severity note | Correction or evidence (path:line at fe9ce12b) |
+|---|---|---|---|
+| A4-1 | CONFIRMED | R3 fits (trivial one-liner, self-contained) | `statSafe` byte-identical: `builder/link-check.mjs:455-457` and `scripts/check_links.mjs:151-153` both read `function statSafe(p) { try { return fs.statSync(p); } catch { return null; } }`. |
+| A4-2 | CONFIRMED | R2 fits | `builder/check.mjs:422-449` `findingsFor` and `scripts/check_links.mjs:602-634` `buildFindings` run the same broken/forbidden/html/a11y/dupIds/remoteAssets translation logic (same message templates, e.g. `` `${s}: a11y-img-missing-alt: src=${e.src}` ``), differing only in the path-relativizer (`posix(src)` vs `rel(p)`) and a null-guard. "Human-readable twin already shared" confirmed: both files import `formatLinkReport, formatIntegrityReport` from `builder/link-check.mjs` (check.mjs:35-40, check_links.mjs:69-75) — only the *structured*-findings half is unshared. |
+| A4-3 | CORRECTED | Substance holds; suggest R1 over R2 (see below) | crawl_check's 5 is exact: `scripts/crawl_check.mjs:98-102`'s if/else chain covers exactly a/href, link/href, img/src, script/src, iframe/src. But `LINK_ATTR_TABLE` (`builder/link-check.mjs:38-60`) is **21 tags / 26 flattened tag-attr pairs / 9 distinct attribute names**, not 20 (counted programmatically in scratch `count_table.mjs`). Likely an off-by-one against the map's 21 entries. No record of the narrower table found in crawl_check.mjs (only import is `htmlparser2`'s `Parser`). Severity: the rubric calls "already diverged" copies the **worst** case for `dup`, and this one already has — crawl_check silently never checks `srcset`, `poster`, `cite`, `formaction`, `action`, `data`, `longdesc` links on the live, post-release site. That reads as R1 ("has already produced a divergence") rather than R2; flagging for the orchestrator's judgment, not insisting. |
+| A5-1 | CORRECTED | R1 dup/conv fits | All three claims confirmed by reading: `check_a11y.mjs` (63-79) has no `--help` case, so `--help` hits `else { console.error("unknown arg..."); process.exit(2); }`; `pick_a11y_sample.mjs:144-147` and `sweep_a11y.mjs:106-110` both print usage via `console.error` (stderr) and `process.exit(0)`. `check_dot_fit.mjs:47` and `build_dot_metrics.mjs:55` each have exactly one argv read, `process.argv.includes(...)`, no validation, so a typo is silently ignored. stdout-vs-stderr also confirmed: `check_a11y_fingerprint.mjs:115-121` and `check_axe_patch_equiv.mjs:43-45` print `--help` via `console.log` (stdout) at exit 0, vs `pick_a11y_sample`/`sweep_a11y`'s stderr. Correction: "7 scripts" underc­ounts by one. `check_tree_fresh.mjs` is explicitly in A5's own named scope (PLAN-TOOLING-REVIEW.md's A5 row) and has the identical hand-rolled `for` loop (check_tree_fresh.mjs:76-90) *and* is itself a third stdout example (`console.log` at line 82) — it directly supplies the "stdout in some" half of this very finding's third claim, so excluding it from the headcount looks like an oversight, not a deliberate scoping choice. Correct count: 8 CLI scripts in A5's scope hand-roll argv (5 loop-based + 2 `.includes()`-based + check_tree_fresh), none via a shared helper. |
+| A5-2 | CONFIRMED | R2 fits | Convention at `docs/Documentation/Extending.md:640-648` (exact: 640 heading, 644-646 the 0/1/2 table, 648 "Separating 1 from 2..."). `build_dot_metrics.mjs` has no `uncaughtException`/try-catch around its puppeteer work; its only exit path is `process.exitCode = 1` for STALE (line 148) — a crash and a real finding both read 1. `pick_a11y_sample.mjs`'s `discover()` (159-169) calls `readdirSync` with no try/catch; an absent tree throws uncaught → Node's default exit 1, same as `if (missingPages.length) process.exit(1)` at line 310. Confirmed this script runs live in `check.bat:23` (`node scripts/pick_a11y_sample.mjs --check`). `check_dot_fit.mjs:31-34` has the guard, with a comment citing "Extending.md's gate conventions" verbatim. |
+| A5-3 | CONFIRMED | R2 fits | `pick_a11y_sample.mjs:122` `STUB_CEILING = 100`; discover fn at 159-169; use at 184. `sweep_a11y.mjs:78` `STUB_TAG_CEILING = 100`; `staticTagCount`+`discoverPages` at 129-153. Both recursive walkers are near-identical, independently written. `pick_a11y_sample.mjs:131,142,217-218` confirm `--propose` reads `perf/results/a11y-sweep.jsonl`, the exact file `sweep_a11y.mjs`'s `--out` default writes — so the two ceilings do have to stay in step for the join to mean anything. |
+| A5-5 | CONFIRMED | R2 fits | `check_dot_fit.mjs:76-87` and `build_dot_metrics.mjs:57-68` each hand-build an Inter-webfont host-page string and call `puppeteer.launch({ headless: true, args: ["--no-sandbox", "--disable-dev-shm-usage", "--allow-file-access-from-files"] })` verbatim, no comment near either call explaining the third flag. `axe-scan.mjs:258-264`'s `LAUNCH_ARGS` (`["--no-sandbox", "--disable-dev-shm-usage"]`, no third flag) has a detailed comment explaining the first two and noting `book/render-book.mjs` uses "the same pair for the same reason" — silent on the third flag because it doesn't carry it, for the same class of file:// load. "Three repo-root idioms in scope" confirmed: check_dot_fit/build_dot_metrics both use `path.dirname(path.dirname(fileURLToPath(import.meta.url)))` (identical to each other), axe-scan.mjs:29 uses `resolve(fileURLToPath(import.meta.url), "../../..")` — a third, different expression. |
+| A5-6 (+L2-3) | CONFIRMED | R2 fits | `builder/dot.mjs:135-155` `listDotSources` is never exported (only `regenerateDot`, line 45, calls it internally) — confirmed via `export` grep. `check_dot_fit.mjs:49-67` `findDotSvgs` independently re-implements the same readdir/ENOENT-catch/underscore-skip/recurse walk, then extra-maps each `.dot` to its `.svg`. "Five gates already import the chain" confirmed as the precedent: `scripts/lib/markdown-files.mjs` has exactly 5 importers (`eval/nav_hops.mjs`, `check_code_regions.mjs`, `check_tree_fresh.mjs`, `convert_em_dash_separators.mjs`, `lib/tb-fences.mjs`) — the established "import the shared walker" convention this instance doesn't follow. L2-3 merge confirmed: neither `listDotSources` nor `findDotSvgs` uses `isOutputTree`; both use the same broad `"_"/"."`-prefix skip (dot.mjs:146, check_dot_fit.mjs:58). |
+| A6-1 | CONFIRMED | R1 fits (concrete, already-diverged copies) | Accumulator+report loop byte-close across all three: `check_page_baseline.mjs:38-44,128-139`, `check_book_coverage.mjs:81-87,158-170`, `check_symbol_index.mjs:40-45,361-370` — line numbers match exactly. The cited divergence is exact: `check_book_coverage.mjs:162` is the **only** one with `` detail.replaceAll("\n", "\n       ") ``; the other two print `` `       ${detail}` `` unindented. The 4-gate crash handler (`process.on("uncaughtException", ...); process.exit(2);`) is byte-identical at `check_page_baseline.mjs:34`, `check_book_coverage.mjs:27`, `check_symbol_index.mjs:38`, `check_publish_policy.mjs:29`. |
+| A6-2 | CONFIRMED | R1 fits (three-way live disagreement, on the page about avoiding exactly this) | Compared all three accounts side by side. `test.bat:34-43` frames round 4 as: round 3's *fix* "had put three more wrong numbers into Building.md and one into README.md, under a gate that read neither file" (a fix-introduces-errors narrative). `check_gate_lists.mjs`'s header (~lines 30-44) and `Tools.md:435,437` instead agree with each other: Building.md and README.md "were already wrong, in the same commit that shipped the gate green" (Tools.md:437: "both wrong on the day the gate shipped green") — a pre-existing-staleness narrative, discovered (not caused) at round 4. test.bat is the odd one out, with specific counts ("three... one") not corroborated by either other account. |
+| A6-3 | CONFIRMED | R2 fits (dormant until the hook exists) | `convert_em_dash_separators.mjs` has exactly one `process.exit(...)` in the whole file (line 214: `process.exit(await main(...))`); `main()` only ever `return`s 0 or 1 (line 210); no `catch`/`uncaughtException` anywhere — confirmed by grep. `PLAN-10.md:690-693` ("pre-commit hook would be the right home, not the integrity checker") and `:797` ("Pre-commit hook for em-dash normalisation" listed as deferred/out-of-scope) both confirmed at those exact lines. |
+| A6-4 | CONFIRMED (facts only, per instructions) | R2 struct fits for the missing seam; did not evaluate the proposed design | Fact 1: `check_gate_lists.mjs` never reads a workflow — confirmed, zero matches for `.github`/`workflow`/`.yml` anywhere in the file. Fact 2: the two workflows' gate steps match except the recorded deltas — confirmed by extracting every `name:`+`run:` pair from both files: the 12 shared "Verify.../Accessibility check" steps are byte-identical between `checks.yml` and `tbdocs-gh-pages.yml`, in the same order. The only differences are exactly the three recorded: `checks.yml:126-127`'s extra "Verify the build's own link checker" (`--case fixture-built --case fixture-built-offline`) step, absent from the deploy workflow; the deploy build step's extra `--url`/`--baseurl` (`tbdocs-gh-pages.yml:96`); and deploy-only steps (Setup Pages, Render book PDF, artifact uploads, deploy/release jobs). |
+| L1-1 | CONFIRMED | R1 fits | `sweep_a11y.mjs:120-121` and `check_a11y_fingerprint.mjs:134-135` are both exactly `themeArg === "both" ? THEMES : [themeArg]`, no validation, vs `check_a11y.mjs`'s `pick()` (82-95) which exists specifically because `--theme drak` once did this. Traced the full consequence: `axe-scan.mjs`'s `buildMatrix` (699-739) builds the report label straight from the string (`` `${filePath} [${theme}, ${viewport}]` ``, line 737) and `gotoPage` (629-634) applies it via `setAttribute("data-theme", t)` with no validation either; the dark CSS is scoped to `[data-theme=dark]` only, so an unrecognized value silently renders light while every report line is mislabelled — exactly reproducing the original bug in two tools that never got the fix. |
+| L1-2 | CONFIRMED | R1 fits | All 7 `opt()`-shaped copies found and classified into 4 variants: (A) unsafe `(n,d)=>{const i=argv.indexOf(...); return i<0?d:argv[i+1];}` — `census_attributes.mjs:86`, `check_examples.mjs:93`, `tbbuild.mjs:61` (3 copies); (B) safe, checks `argv[i+1]` truthiness — `addin_test.mjs:68`, `tbrun.mjs:105-108` (2 copies); (C) 1-arg, no default — `build_package_api.mjs:56` (1); (D) inline/unnamed — `check_publish_policy.mjs:31-33` (1). 3+2+1+1 = 7 in 4 variants, exact. NaN trace confirmed: `tbbuild.mjs:68` (`Number(opt("port", 9333))`) and `:70` (`Number(opt("timeout", 180))`) use variant A, so a trailing `--port`/`--timeout` returns `undefined` not the default, giving `NaN`; `check_examples.mjs:102-104` (jobs/port/batch) same. |
+| L1-3 | CONFIRMED | R1 dup/hack fits | `tbrun.mjs:112-116` `VALUE_FLAGS` names every value-taking flag explicitly. `tbbuild.mjs:62`: `` const proj = args.find((a, i) => !a.startsWith("--") && !args[i - 1]?.startsWith("--")); `` — traced `tbbuild --keep proj`: args=["--keep","proj"]; index 0 fails (starts with "--"); index 1 fails too (`args[0]` = "--keep" starts with "--", so the *previous-token* guard rejects it even though `--keep` is a boolean switch with no value). `proj` ends up `undefined` → line 75's `if (!proj || ...)` fires the usage error. Reproduced by trace, not by running. |
+| L1-4 | CONFIRMED | R1 fits (tbdocs is the flagship, CI-invoked tool) | Full trace confirmed: `tbdocs.mjs:189-191` `else { throw new Error(\`Unknown argument: ${a}\`); }`; `main()` (1616-1624) calls `parseArgs` synchronously as its first statement; `main().catch((err) => { ...; process.exit(1); })` at 1628-1635 catches everything uniformly. Bit 0 (exit 1) is documented elsewhere in the same function (line 1610, `process.exitCode \|= 1` for a link-check failure) as meaning "link check failed" — a parse error reuses that exact code with nothing to distinguish it. |
+| L1-5 | CONFIRMED | R2 fits | `check.bat` (34 lines, read in full) does not call `check_links.mjs` anywhere; its own header (lines 1-5) says link/integrity checking "moved into the build itself." `check_links.mjs:307-308` ("Tolerate unknown flags passed through via check.bat's %*") and `:411` ("check.bat / CI both pass the same string for --root-dir...") are both now stale. Independently corroborated at `builder/PLAN-checks.md:14`: "`check.bat` no longer invokes `check_links.mjs`." |
+| L1-6 | CONFIRMED | R2 conv fits generally; the gen_attribute_probes case is arguably a live R1-class defect in its own right | Traced (not run) both: `gen_attribute_probes.mjs:1147-1158` `main(argv)` only short-circuits when `argv.length < 1`; a lone `--help` has length 1, so `out = argv[0] = "--help"`, and `await fs.mkdir(path.join("--help", "Sources"), { recursive: true })` runs — it really does create a `--help/Sources` directory instead of printing help. `render-book.mjs:210-220`: no `-h`/`--help` case in the loop; `--help` fails every explicit check (starts with `-`, so the bare-positional branch also refuses it) and falls to `else { console.error("unknown arg: --help"); process.exit(2); }` — it never reaches the actual usage text at line 223. |
+| L1-11 | CONFIRMED | R2 fits | The 4-gate handler cross-checked again (see A6-1). `check_tb_registry.mjs`: no `uncaughtException` handler anywhere (grep confirmed); its only error path is a local `try{...}catch(e){...; process.exitCode = 1;}` (266-268) wrapping the whole self-test body, so both a genuine assertion failure and an environment crash land on exit 1. `Tools.md:368` for check_publish_policy.mjs ends "Exits 1 naming each failed assertion." with no mention of exit 2, unlike the neighboring entries for `check_tree_fresh.mjs` (line 375: states all of 0/1/2) and `check_gate_lists.mjs` ("Exits 1 ... 2 if it cannot run.") — confirmed the omission is real and is specific to this one entry. |
+| L1-13 | CONFIRMED | R2 struct fits (matches the rubric's own "missing seam" example) | Read every `--self-test`/`selfTest` implementation in the repo (`check_code_regions.mjs`, `check_gate_lists.mjs`, `check_links.mjs`+`check_links_diff.mjs`, `check_regex_safety.mjs`, `impexp.mjs`): all assert the tool's *domain* logic (gate comparison, mutation detection, regex classification, round-trip fidelity), none assert anything about the tool's own argv/flag handling. No dedicated CLI/argv test file exists anywhere in the tree (checked `test/`, and every `node:test` user, all of which are the add-in harness, unrelated). This absence is also positively demonstrated: L1-1, L1-2, L1-3 and L1-6 are all real, currently-shipping argv-handling defects that a minimal parser test would have caught. |
+| L2-5 | CORRECTED | R2 fits; "four expressions" is a defensible bucketing, not a precise count | 19 inline-deriving files confirmed exactly (census_attributes, tbdocs, build_corpus, nav_hops, run_case, site_search, addin_test, build_dot_metrics, build_package_api, check_code_regions, check_dot_fit, check_examples, check_gate_lists, check_links_diff, check_tree_fresh, convert_em_dash_separators, gen_attribute_probes, wisdom/extract/prep.mjs, builder/write.mjs). At strict AST-shape granularity there are closer to 5 distinct expressions (nested-dirname ×10; URL-relative ×3; `resolve(dirname,"..")` ×4 in eval/, structurally the same as `write.mjs`'s `resolve(__dirname,"..")` once its local `__dirname` shim is inlined [+1]; `join(__dirname,'..','..')` in wisdom/prep.mjs, same strategy but one level deeper [+1]) — "four" holds only if the eval/write.mjs/wisdom trio is counted as one "dirname + relative join" idiom family rather than split by exact depth. `axe-scan.mjs:29`'s exported `REPO_ROOT` confirmed imported by exactly 2 (`pick_a11y_sample.mjs:47`, `sweep_a11y.mjs:51`). `Extending.md:654` confirmed at that exact line — and, notably, its own text ("`resolve(fileURLToPath(new URL("..", import.meta.url)))` is what nearly all of them do") is itself inaccurate: that exact expression is used by only 3 of the 19, not "nearly all" (the nested-dirname shape is the majority, 10 of 19). |
+| L2-6 | CORRECTED | R3 fits (local untidiness, not a correctness defect) | `check_publish_policy.mjs:152-189` confirmed: `mkdtemp` at 152, `fs.rm(tmp, ...)` at 189 is a plain unprotected statement (any throw from the `probe()` helper's uncaught `fs.mkdir`/`fs.writeFile` calls, lines 153-156, skips it). `check_links_diff.mjs:651-750` confirmed: `fixtureDir` declared at 651, `mkdtempSync` at 682, cleanup `if (fixtureDir) fs.rmSync(...)` at 750 sits after two loops (656-728, 734-748) that run the actual checks and can throw, all unprotected by try/finally. "Three siblings do it right" is close but not exact: **four files** use a proper `try{...}finally{...rm...}` — `check_links.mjs` (selfTest, finally at line 755), `impexp.mjs` (selfTest, finally at 762-763), and **both** `check_page_baseline.mjs:46-55` and `check_symbol_index.mjs:314-323`'s `withBaseline`. The "three" reading only works if the latter two (which L4-13 already notes are near-clones of each other) are counted as one pattern rather than two files. |
+| A10-2 | CONFIRMED | R1 as assigned is defensible given how easily a BOM reappears (WIP.md: "editors on Windows add one without being asked"); genuinely dormant today | `wisdom/extract/sitemap.mjs:69-87` `parseFrontmatter`: strips `\r\n`→`\n` (line 70) but never strips a BOM; `content.startsWith('---')` (71) would be false behind a leading ``. `builder/discover.mjs:94-117` has the dedicated `stripBom()` (105-107) called before the frontmatter test (110), with a comment (94-104) narrating the AppGlobalClassObject incident by name. Independently ran a scratch scan (`check_bom.mjs`) over all 912 `docs/**/*.md` files archived at `fe9ce12b` via `git archive`: **0 begin with a UTF-8 BOM** — confirms "dormant" directly rather than by inference. Coercion disagreement with `prep.mjs:343-388` also confirmed: `prep.mjs`'s `parseThreadFrontmatter` additionally handles inline arrays and coerces `'true'`/`'false'`/digit-strings to real booleans/numbers (378-382), while `sitemap.mjs`'s version leaves everything as a string — same input frontmatter parses to different JS types. (This also makes it a *third* independent frontmatter parser alongside discover.mjs and sitemap.mjs, not just a second disagreement.) |
+| A10-3 | CONFIRMED | R2 fits | `sitemap.mjs:61-67` `walk(dir, callback)` is a bare recursive `readdirSync` with no `.md` filter and no output-tree skip (filtering happens in the caller instead, `buildSitemap` line 8). `wisdom/PLAN-3.md:404` confirmed verbatim: "a minimal built-in parser — no dependency on builder/". But `scripts/lib/markdown-files.mjs` (read in full) imports only `node:fs`/`node:path` — zero dependency on `builder/` — so the stated reason doesn't actually justify skipping it. In this specific call (scoped to `docs/Reference/`, which never contains an output tree) the missing `isOutputTree` skip is currently harmless, consistent with R2 rather than a live bug. |
+| A10-4 | CONFIRMED | R2 fits (dead but drifted, so wrong-if-ever-revived) | `wisdom/extract/schemas.mjs` confirmed imported by nothing anywhere in the repo (repo-wide grep for both quote styles, zero hits). Both cited divergences confirmed by direct comparison: `schemas.mjs:12` `source_thread: { type: 'string' }` vs the live `workflow.mjs:49` `thread_path: { type: 'string' }`; `schemas.mjs:24` `section: { type: 'string' }` (free text) vs live `workflow.mjs:77` `section: { enum: ['after-remarks','example','see-also','new-section'] }`. |
+| A10-5 | CONFIRMED | R2 fits | `wisdom/discord/messages.mjs:10-12` `saveManifest` is a plain `writeFileSync`, no temp file. `wisdom/wisdom.mjs:146` (`writeFileSync(deniedPath, ...)`) same. `loadManifest` (4-8) does `JSON.parse(readFileSync(...))` with no try/catch anywhere in the function. Contrast confirmed: `wisdom/extract/state.mjs:62-75` is explicitly commented "Write state atomically (temp file + rename)" and does exactly that (`tmpPath = path + '.tmp'`; `writeFileSync(tmpPath,...)`; `renameSync(tmpPath, path)`). |
+| A10-6 | CONFIRMED | R2 struct fits | Both self-tests counted directly: `impexp.mjs` has 19 `test(...)` calls, `impexp.py` has 19 `test(...)` calls, **and the two lists of test names are byte-identical in the same order**, all 19 (e.g. both open with `'Parse and serialize round-trip in memory'` and close with `'sample.twinpack survives export, import and export again'`) — answers "are the names the same?": yes. Answers "is `--self-test` run by any .bat/workflow/gate?": no — exhaustive repo-wide search of every `.bat` and `.yml` for "impexp" found only `docs/_config.yml:101-104`'s `bundle_extra` download-copy declarations; the two gate scripts that mention "impexp" (`check_gate_lists.mjs:159`, `check_publish_policy.mjs:103`) do so only in unrelated comments, neither invokes it. `Tools.md:958` confirmed verbatim: "the two editions print the same output and write byte-identical project files" — an unhedged claim with nothing mechanical behind it. |
+| L4-9 | CONFIRMED | = A5-3 (R2) + A5-4 (R3); both independently confirmed | A5-3 evidence above. A5-4 (`pad`/`median` identical): confirmed byte-identical — `pick_a11y_sample.mjs:229-232,259` and `sweep_a11y.mjs:279-283` both have `` const pad = (s, w) => String(s).padStart(w); `` and the same 3-line sort-and-midpoint `median`. |
+| L4-13 | CONFIRMED | = A6-1 (R1) + L1-11 (R2) + withBaseline dup; both independently confirmed above | withBaseline pair confirmed byte-close: `check_page_baseline.mjs:46-55` and `check_symbol_index.mjs:314-323` are the same `mkdtemp` → `try{ write initial; return fn(file) }finally{ rm dir }` shape, differing only in the tmp-prefix string, filename, and the exact JSON written. |
+
+## Conflicts
+
+### 1. `findingsFor` (builder/check.mjs) vs `buildFindings` (scripts/check_links.mjs)
+
+**Ruling:** the comments do record the mirroring as deliberate. Quoted exactly:
+
+- `builder/check.mjs:410-414`:
+  > The same conclusions, in the shape scripts/check_links.mjs's `structured` mode emits, so scripts/check_links_diff.mjs can diff the two implementations category by category. This is the whole gate: check.bat going green proves nothing about whether the fused pass still looks at everything the standalone script does.
+
+- `scripts/check_links.mjs:599-601`:
+  > Reduce a pass's conclusions to sorted arrays of tree-relative strings. Category names match the flags that produce them, so a diff says which check drifted rather than only that something did.
+
+Both comments tie the parallel-shape design to `check_links_diff.mjs` cross-checking two *independent* implementations against each other — a legitimate, stated reason under the review's own "recorded decision is not a finding" rule, at the time it was written. The user's option-2 decision (A4, 2026-09-25: `check_links.mjs` becomes a thin wrapper over `builder/check.mjs`) supersedes the premise these comments describe — after that refactor there is one implementation called from two front ends, not two independently-written ones a diff tool cross-checks. Per the task framing, I have not judged whether that decision is correct, only confirmed the comments say what L4 and A4-2 both claim, so the fix that lands with the thin-wrapper refactor needs to rewrite both.
+
+### 2. `extractFromHtml` (link-check.mjs) vs crawl_check.mjs
+
+**Ruling:** A4-3's point holds independently of L4's "different sizes, not a clone" — they answer different questions, not contradictory ones. L4's clone detector correctly reports these as non-clones: `link-check.mjs`'s SAX walker is table-driven off a 21-entry `Map` (`extractFromHtml` proper, plus `LINK_ATTR_TABLE`), while `crawl_check.mjs:91-108`'s `extractFromHtml` is a 5-line hand-written if/else chain — genuinely different code shape and size, so a token-window clone detector is right to stay silent. But A4-3 isn't claiming they're textual clones; it's claiming the two independently-maintained *lists of which HTML attributes carry links* have diverged, with a concrete, live consequence: `crawl_check.mjs` (the one post-release, real-HTTP checker per Building.md:653-655) silently never follows `srcset`, `poster`, `cite`, `formaction`, `action`, `data`, or `longdesc` links, all of which the offline/build-time checker does cover. Both things are true at once: not a clone, and a real coverage gap. Keep A4-3, independent of L4's classification.
+
+### 3. `parseFrontmatter` (wisdom/extract/sitemap.mjs) vs builder/discover.mjs
+
+**Ruling:** same shape of non-conflict as #2. L4 is right that these are "different dialects" — `discover.mjs`'s version wraps the full `gray-matter` library (arbitrary YAML), while `sitemap.mjs`'s is a minimal line-based `key: value` reader with no nesting, no multi-line values, no arrays. A token-clone detector correctly declines to call these clones. A10-2's point is orthogonal: regardless of how sophisticated the rest of the parsing is, *both* implementations start by testing whether the raw text begins with `---`, and only one of the two guards that test against a leading BOM. That gap doesn't depend on dialect sophistication at all — a "richer" or "thinner" YAML dialect is equally blind to a BOM sitting in front of its own opening fence check. Confirmed independently (see A10-2 row: 0/912 files carry a BOM today, so dormant, but the gap is real and unrelated to which dialect is being parsed). Keep A10-2, independent of L4's "different dialects."
+
+## Noticed in passing
+
+- `docs/Documentation/Extending.md:654` itself overstates its own convention: it says `resolve(fileURLToPath(new URL("..", import.meta.url)))` is "what nearly all of them do," but only 3 of the 19 inline repo-root derivations actually use that expression — the nested-`dirname` shape is the majority (10 of 19).
+- `builder/PLAN-10.md:692` cites `scripts/convert_em_dash_separators.py` — stale; the tool has been `.mjs` since the JavaScript port (WIP.md lists only `impexp.py` and `build_fonts.py` as the two remaining Python files).
+- `gen_attribute_probes.mjs --help` doesn't just fail to print help (L1-6's "conv" framing) — traced, it actually creates a `--help/Sources` directory on disk via `fs.mkdir`, a live side effect from the exact command a confused user would type first.
+- wisdom/ has *three* independent hand-rolled-or-library frontmatter readers, not two: `discover.mjs` (gray-matter + BOM strip), `sitemap.mjs` (minimal, no BOM/coercion), and `prep.mjs` (minimal + arrays + boolean/number coercion) — A10-2 covers two of the three pairwise gaps but the triplication itself is broader than either single comparison.
+- impexp.mjs/impexp.py's 19 self-test names are byte-identical in the same order in both editions (real, by-hand engineering discipline to keep the ports in lockstep) — which makes A10-6's "and nothing mechanically checks they still agree" more notable, not less: all that manual care has no automated backstop at all.
diff --git a/builder/REVIEW-TOOLING-fe9ce12b/V3.md b/builder/REVIEW-TOOLING-fe9ce12b/V3.md
new file mode 100644
index 00000000..24f33d23
--- /dev/null
+++ b/builder/REVIEW-TOOLING-fe9ce12b/V3.md
@@ -0,0 +1,100 @@
+# V3 verification -- compiler harness, sample compiling, package API, book pipeline
+
+Verified against commit `fe9ce12b` (all source read via `git show fe9ce12b:`, so later
+commits landing on `staging` during this session did not affect any citation below). Scratch
+scripts under `scratchpad/verify/V3/`: `a7-1-regex.mjs`, `a7-5-opt.mjs`, `l4-10-logicallines.mjs`
+(plus extracted copies of the source files used for line-numbered reading).
+
+## Table
+
+| ID | verdict | severity note | correction or evidence (path:line at fe9ce12b) |
+|---|---|---|---|
+| A7-1 | CORRECTED | R1 dup fits (round-8 bug class cited in-file is a real prior defect) | tbrun.mjs:327 `/^\[(BUILD\]\s+failed\|LINKER\]\s+FAILED)\b/i` vs tb-ide.mjs:734 `BUILD_FAILED`. Ran both against all 5 shapes named in tb-ide.mjs:727-731 (`a7-1-regex.mjs`): tbrun's regex matches **3 of 5** (LINKER FAILED, BUILD FAILED, BUILD failed -- the `/i` flag makes the BUILD-alternative case-insensitive so it catches both listed casings), not "2 of 5". It still misses exactly the two shapes claimed -- "[BUILD] ERROR" and "[LINKER] compilation (codegen) error" -- so the "exit 0 on a failed build" consequence is correct; only the count is off. |
+| A7-2 | CONFIRMED | R2 fits; kind label arguable (reads more like the rubric's "hack" -- an unguarded workaround -- than "struct"), not clearly wrong either way | tb-operate.mjs:421-429 `afterReveal` returns `false` on timeout. All three call sites discard it: openFile tb-operate.mjs:453, setCursor:461, select:475 -- each is a bare `await afterReveal(c);`. Grep confirms no other call site checks the return. The race afterReveal exists to prevent is documented at tb-operate.mjs:401-409 (the measured "xyz"->"zy" cursor-reset corruption); silently proceeding after a timeout reopens exactly that class of corruption. |
+| A7-3 | CORRECTED | R2 dup fits | tb-ide.mjs:848-861 `clickCenter` (no scrollIntoView, no hit-test, no retry) has exactly two call sites, both for `"buildIcon"`: tb-ide.mjs:757 (buildProject) and tbrun.mjs:295 -- confirmed exact. tb-operate.mjs:109-134 `click` has scrollIntoView (115), a shadow-root-aware hit test (119-124) and a retry loop (110-133). "used ... by every scenario" overstated: `click` is called directly in only 4 of the 10 `test/addin/*.test.mjs` files (appdata.test.mjs:83, panes.test.mjs x7, sample10.test.mjs x5, sample15.test.mjs x4), plus twice inside tb-operate.mjs itself (309 msgbox answer, 361 restartIcon). arch/entry/ideserver/keys/reload/symbols never call it. |
+| A7-4 | CORRECTED | R2 dup fits (no divergence yet, so R1 would be wrong) | tb-registry.mjs:588-590 `alive` and :462 `norm`, both unexported. addin_test.mjs:105 `alive` and :230 `norm` are byte-identical in body to their tb-registry.mjs counterparts (only `function`/`const =>` syntax differs). Import count wrong: addin_test.mjs:60-61 imports **10** names from tb-registry.mjs (deleteSettings, finishTidy, ideLists, restoreKeys, SETTINGS_ROOT, settingsKey, snapshotKeys, startTidy, subkeyNames, sweepArchitectureMemory), not 9 -- confirmed by direct count and by `sed -n '60,61p'`. |
+| A7-5 | CONFIRMED | R2 conv fits | All sub-claims reproduced in `a7-5-opt.mjs` plus direct reads. tbbuild.mjs:61 `opt` on a trailing flag returns `undefined` -> `Number()` = `NaN` (script output confirms). tbrun.mjs:105-108 / addin_test.mjs:68 `opt` substitutes the default both for a trailing flag AND for an explicit `""` (script: `optTbrun(["--port",""],"port",9346)` -> `9346`). tbbuild.mjs:62 positional finder: `tbbuild --json proj` finds no positional (script confirms `undefined`) because it only excludes a token whose *predecessor* starts with `--`, with no value/boolean distinction; check_examples.mjs:602 puts `staged.proj` first (confirmed exact line), keeping the bug dormant. tb-ide.mjs:436 `while (Date.now() - t0 < timeout)` with `timeout=NaN` never executes (script: 0 iterations) -> `waitForCompile` returns `{loaded:false,...}` at once -> tb-ide.mjs:484-491 `compileOutcome` reports exit 3 "the IDE never reported `` as open" -- the misleading message, confirmed by tracing tbbuild.mjs:145-146. `die()` "three structures" confirmed: tbbuild.mjs:118-122 (function, calls `shutdown()`), tbrun.mjs:110 / addin_test.mjs:69 (identical one-line arrow, no shutdown call built in), and check_tb_registry.mjs:267-268 (no `die` helper at all -- bare try/catch setting `process.exitCode = 1`). |
+| A7-8 | CONFIRMED | R2 dup fits | All ten `test/addin/*.test.mjs` files hand-roll the HERE/HOST/lane preamble (`fileURLToPath` + `addinLane()` + `path.join(...,"host")`) and the byte-identical `describe(..., { skip: lane ? false : "run it with addin-test.bat" }, ...)` object -- confirmed by grep across all ten. Six touch a "console lines since a mark" reader: appdata.test.mjs:33 `probeLines` and panes.test.mjs:86 `probeLines` (same name, different bodies: `.slice(15)` for `"[AppDataProbe] "` vs `.slice(13)` for `"[PanesProbe] "`); arch.test.mjs:31 `loadsSince` and reload.test.mjs:37 `probeSince` (both regex-capture, different regexes); keys.test.mjs:28 `linesSince` (plain split+trim, no filter); sample10.test.mjs:81 inline (no split at all, bare `.trim()`). That is 4 distinguishable shapes (prefix-slice x2, regex-capture x2, plain-trim, bare-trim) reasonably compressed to "three different ways" plus a degenerate case; not a stretch. |
+| A8-1 | CONFIRMED | R1 dup fits (already-diverged keyword lists; the orchestrator's own "dormant" finding for the main Overridable claim already satisfies R1's "will [misfire] on the next ordinary change") | census_attributes.mjs:170-179 `blankStrings` toggles `inStr` on every literal `"` with no dedicated `""`-pair branch. twin-api.mjs:67-71 explicitly special-cases `c==='"' && s[k+1]==='"'` (blanks both to two spaces, stays in-string, skips the second char). Confirmed divergence exists. Caveat worth recording: because a valid `""` escape is always *two* toggles, census's naive parity-toggle still tracks the true string boundary correctly on syntactically valid source (verified by hand-tracing `"it's ""fine"" isn't it"` and by the L4-10 script's equivalent case) -- so this is a real code-shape/fragility divergence, not a demonstrated misclassification today. BLOCK_COMMENT_RE claim fully confirmed: census_attributes.mjs:164/166 `decomment` is called fresh per individual line (lines array built at :286 via plain `.split(/\r?\n/)`, `decomment` invoked per-`lines[i]`/`lines[j]` throughout, e.g. :308,:316,:349) with no shared state between calls; twin-api.mjs:56 declares `inBlock` outside the per-line loop and threads it across lines (:63-66). A `/* */` spanning a line boundary is not stripped by census_attributes.mjs at all. |
+| A8-2 | CORRECTED | R2 struct fits | Core claim confirmed: census_attributes.mjs:79 imports `../scripts/lib/tb-packages.mjs`; render.mjs's comment (ending line 383) states verbatim "builder/ must not depend on scripts/". check_tree_fresh.mjs:57-63 IGNORED_FILES / its dedicated comment for census_attributes.mjs confirmed. But the "ten path citations" list is wrong: `git grep -n "builder/census_attributes.mjs" fe9ce12b` (excluding the file's own self-reference at builder/census_attributes.mjs:4) finds **11** files, and the broader `git grep -n "census_attributes" fe9ce12b` the task specifies finds **13**. The ledger's ten-item list wrongly includes WIP.Build.md (only cites the bare filename at :581, "shared with `census_attributes.mjs`" -- never the `builder/` path) and omits two files that DO cite the literal path: scripts/build_package_api.mjs:12 and scripts/lib/tb-fences.mjs:334 and :349. |
+| A8-4 | CONFIRMED | R2 struct fits | Neither has self-test scaffolding: `grep -i "selftest\|node:test\|assert"` is empty for census_attributes.mjs; gen_attribute_probes.mjs's only "assert" hits (:261,:591,:1213) are unrelated prose, not a JS assertion. gen_attribute_probes.mjs:961-978 `parseTargets` carries its own historical-misparse comment at :946-947 (the `\b`-after-"const" plural bug). census_attributes.mjs documents its own past silent misparses in its header (:42-67, e.g. point 5's 14 lost Interface-member sites) and at :150-152 (368 Declares swallowed into a phantom Type) -- satisfying "both have shipped silent misparses before" even though census_attributes.mjs has no function literally named `parseTargets` (confirmed: zero matches for that name in the file; the ledger's possessive grammar ties "parseTargets" only to gen_attribute_probes, which matches). |
+| L2-2 | CONFIRMED | R1 dup fits (the two functions can pick different installs off the same Desktop today, not just on a future change) | census_attributes.mjs:99-119 `findInstall`'s Desktop scan validates each BETA folder via `existsSync(path.join(b.dir, "packages"))` (:115). tb-install.mjs:19-35 `findIde`'s scan validates via `existsSync(path.join(desktop, name, "twinBASIC.exe"))` (:28-29) instead -- a genuinely different criterion, confirmed by direct comparison. findInstall's home-dir fallback `process.env.USERPROFILE \|\| os.homedir()` (:108) has no counterpart in findIde, which uses `process.env.USERPROFILE ?? ""` alone (:22) -- an unset USERPROFILE makes findIde build a relative `"Desktop"` path instead of falling back to the real home directory. tb-install.mjs's own header (:1-3) states the shared-module rationale this private copy undercuts. |
+| L4-10 | CONFIRMED | R1 dup fits | tb-fences.mjs:423-445 `logicalLines` (confirmed private -- absent from the file's `^export` list) vs twin-api.mjs:51-86 `logicalLines` (exported). Both claimed differences directly confirmed by copying both functions verbatim into `l4-10-logicallines.mjs` and running them: twin-api strips a leading BOM (its own dedicated test: tb-fences leaves it, twin-api removes it) and threads `/* */` across physical lines (tb-fences does not: a 2-line block comment leaves both comment delimiters embedded as literal text in tb-fences' output, confirmed by the script). See the dedicated section below for the full replacement-question answer the task asked for. |
+| A9-1 | CONFIRMED | R2 hack fits (contained per-file, but not guarded -- matches the rubric's un-met workaround test #2 exactly) | Grepped all 13 production shim files (fast-array-onebuf, fast-decode-name, fast-dict-onebuf, fast-indirect-objects, fast-inflate, fast-number-to-string, fast-parse-name, fast-parse-number, fast-parse-object, fast-pdfnumber-pool, fast-refs-class, fast-size-in-bytes, fast-sync-load) for shape/version assertions: every one has only a `__xInstalled`-style idempotency guard (e.g. fast-array-onebuf.mjs:166, fast-sync-load.mjs:88) that blocks double-installation, never a check of what is being overwritten. parallel-deflate.mjs:50 `ParallelStreamWriter extends PDFStreamWriter` has no guard of any kind. package.json:20 pins `"pdf-lib": "1.17.1"` exactly, confirmed. perf/notes/08-pdf-lib.md:1751-1757 confirms verbatim: "The shim's correctness depends on the upstream pdf-lib source being structurally what the line-by-line port assumed," and the pin "so a stray `npm update` can't silently swap upstream" -- covers accidental drift only, exactly as claimed. perf/notes/08-pdf-lib.md:5048-5061 confirms the @cantoo fork evaluation and rejection verbatim. |
+| A9-2 | CONFIRMED | R2 struct fits | `git grep -l "pdf-lib" fe9ce12b -- test/ scripts/` is empty; no test file exists anywhere under book/ (`git ls-tree` for `book/.*test` is empty). No automated shimmed-vs-stock comparison exists anywhere in the tree; only prose (e.g. the @cantoo A/B numbers in perf/notes/08-pdf-lib.md). scripts/check_axe_patch_equiv.mjs (confirmed real, SOURCE_PATCHES-based, see its header :1-27) is a structurally apt model for the proposed fix. |
+| A9-3 | CONFIRMED | R2 dup fits | Compared all six named helpers between fast-array-onebuf.mjs and fast-dict-onebuf.mjs directly. Exactly two are identical apart from naming: `_registerContext` (array :104-110 vs dict :144-150, differ only in the file-label string inside the thrown Error) and `_appendArray` (array :121-125 vs dict :161-165, differ only in the shared buffer/counter names `arrayMain`/`arrayMainLen` vs `main`/`mainLen`). The other four are NOT identical, only similarly shaped: `pack` uses different bit-packing constants (array :91-95 `POW_24`/24-bit vs dict :127-131 `POW_25`/23-bit -- dict reserves 2 extra gap bits PDFArray does not need); `_cow` (array :129-138 vs dict :175-184) differs by dict's extra `+ (d & GAP_MASK)` term; `_makeFromRange`/`_makeFromAppend` (dict :226,:239) take an extra leading `ProtoClass` parameter array's versions (:155,:160) do not, for PDFCatalog/PDFPageTree/PDFPageLeaf dispatch. The recorded reason at fast-array-onebuf.mjs:36-39 names only the singleton-context mechanism, leaving `_appendArray`'s exact duplication unaddressed by any recorded rationale. |
+| A9-5 | CORRECTED | R3 dup fits regardless of the exact count | The pattern is real but the count is off: exactly **12** files in book/lib/ (not "~15") contain `createRequire` + `require('pdf-lib/cjs/...').default`: fast-array-onebuf, fast-dict-array, fast-dict-iter, fast-dict-onebuf, fast-indirect-objects, fast-number-to-string, fast-parse-dict, fast-parse-name, fast-parse-number, fast-parse-object, fast-size-in-bytes, fast-sync-load -- confirmed exhaustively (`git grep -l "require('pdf-lib" fe9ce12b -- book/lib/`, cross-checked against `createRequire` count, both give 12). Zero matches in book/render-book.mjs, builder/book.mjs, builder/pdf.mjs. Of the 12, three (fast-dict-array, fast-dict-iter, fast-parse-dict) are among the four A/B-baseline shims already approved for deletion elsewhere in the ledger, so the production-only count is 9. The five ESM-style shims (fast-decode-name, fast-pdfnumber-pool, fast-refs-class, fast-refs, outline.mjs) use plain `import {X} from "pdf-lib"` instead and do not participate in this particular duplication. |
+
+## L4-10: could twin-api's logicalLines replace tb-fences' without losing its quote-aware comment stripping?
+
+**Yes on the specific question asked.** Both functions were copied verbatim (fe9ce12b) into
+`scratchpad/verify/V3/l4-10-logicallines.mjs` and run on the requested cases plus a few more.
+Full output is in that file's run log; the relevant lines:
+
+```
+=== apostrophe inside a string ===          Dim s As String = "it's a test"
+tb-fences : "Dim s As String = \"it's a test\""
+twin-api  : "Dim s As String = \"           \""      (content blanked, but NOT truncated)
+
+=== "" escape then an in-string apostrophe ===   "it's ""fine"" isn't it"
+tb-fences : "Dim s As String = \"it's \"\"fine\"\" isn't it\""   (unchanged, no early break)
+twin-api  : "Dim s As String = \"                      \""       (blanked, no early break)
+```
+
+Neither implementation mistakes the apostrophe for a comment start, including in the combined
+case (a `""` escape immediately followed by a real apostrophe, still inside the string) -- so
+swapping in twin-api's version would **not** lose tb-fences' quote-aware comment stripping.
+twin-api's version is quote-aware for the same structural reason tb-fences' is: the `'`-breaks-
+the-line check sits behind an `if (inStr) { ...; continue; }` branch in both, so it is never
+reached while a string is open.
+
+**It is not a drop-in swap, though** -- four other differences surfaced by the same test run,
+none of which the ledger's citation mentions, that a real replacement would have to absorb:
+
+1. **Return shape.** twin-api keeps a blank source line as an empty-text entry; tb-fences drops
+   it. Confirmed on a 3-line source with a blank middle line: tb-fences returns 2 entries
+   (`at:0`, `at:2`), twin-api returns 3 (`line:1`, `line:2` with `text:""`, `line:3`).
+2. **Field naming/numbering.** `line` (1-based) vs `at` (0-based) -- a caller reading `.at`
+   off twin-api's result gets `undefined`.
+3. **No trim.** twin-api never calls `.trim()`; tb-fences trims every returned line. Confirmed
+   on `"Dim x As Long  ' this is a comment"`: tb-fences returns `"Dim x As Long"`, twin-api
+   returns `"Dim x As Long  "` (two trailing spaces kept). Left untrimmed, a line with leading
+   whitespace could fail tb-fences' own `^`-anchored classifier regexes (none of which allow
+   for arbitrary leading space before the first token).
+4. **`Rem` recognition.** twin-api blanks a whole `Rem ...` line to `""`; tb-fences leaves it
+   as ordinary text (confirmed: `"Rem this whole line is a comment"` survives unchanged through
+   tb-fences). Likely dormant either way -- `Rem` is legacy VB6, unlikely to appear in the
+   corpus -- but it is a behaviour change a swap would introduce, not just a bug fix.
+
+One thing the swap would **fix**, not just preserve: tb-fences has **no `/* */` handling at
+all**, not even the single-line case. Run on census_attributes.mjs's own DAO.twin example
+(its header, :58-60), `"/* voffset &H00A8*/ Property Get X()"`, tb-fences returns the comment
+delimiters embedded verbatim in the logical line; twin-api correctly returns `"  Property Get
+X()"`. On a comment spanning two physical lines, tb-fences returns the *second* physical line as
+`"spans two lines */ Dim z As Long"` -- the trailing declaration is buried mid-line, unreachable
+by any of tb-fences' `^`-anchored regexes, which is exactly the "silent misclassification"
+shape census_attributes.mjs's own header (:58-60) says cost it 14 Interface-member sites. twin-api
+correctly recovers `"  Dim z As Long"` as its own line. This is a real latent gap in tb-fences
+today, independent of whatever the corpus currently exercises (see noticed-in-passing #1 below).
+
+## Noticed in passing
+
+1. tb-fences.mjs's `logicalLines` has zero `/* */` handling (not just missing the multi-line
+   case) -- exposed to the exact 14-site failure class census_attributes.mjs's own header
+   documents paying for; worth its own finding if docs/ code samples ever use block comments.
+2. tb-fences.mjs's apparent BOM-safety is accidental, not designed: `"Dim x".trim()`
+   strips the BOM because ECMAScript's `trim()` treats U+FEFF as whitespace (verified directly),
+   not because tb-fences.mjs does anything BOM-aware -- fragile if ever accessed without the
+   trailing `.trim()` call.
+3. A8-1 and A8-4 point at the same underlying gap from two angles: census_attributes.mjs's
+   MODS list and gen_attribute_probes.mjs's parseTargets are both hand-kept keyword/phrase
+   classifiers with a documented history of silent misparses and no fixture-based regression
+   test; a shared "labelled-keyword-list classifier + ride-along probes" fix could cover both,
+   not just census_attributes.mjs alone.
+4. A9-3's proposed `book/lib/onebuf-range.mjs` factory needs to parametrize over the
+   subclass-dispatch (`ProtoClass`) and gap-mask logic dict needs and array does not, not just
+   rename the buffer variable -- `_cow`/`_makeFromRange`/`_makeFromAppend` are genuinely
+   different, not copy-paste twins, so a naive single-body factory would under-serve one side.
+5. tbbuild.mjs, tbrun.mjs and addin_test.mjs each have their own `flag()`/`opt()` pair in
+   addition to the three `die()` shapes A7-5 catalogues -- the same three-way split repeats
+   one level up (not re-verified in detail here; flagged for whoever picks up L1-2/L1-3).
diff --git a/builder/REVIEW-TOOLING-fe9ce12b/V4.md b/builder/REVIEW-TOOLING-fe9ce12b/V4.md
new file mode 100644
index 00000000..01d95bd1
--- /dev/null
+++ b/builder/REVIEW-TOOLING-fe9ce12b/V4.md
@@ -0,0 +1,134 @@
+# V4 verification report -- L3 (browsers, child processes, text rewrites)
+
+Verified against commit `fe9ce12b` (`git show fe9ce12b:`). Source copies read into
+`scratchpad/verify/V4/src/`; reproduction scripts in `scratchpad/verify/V4/*.mjs`.
+
+## Table
+
+| ID | verdict | severity note | correction or evidence (path:line at fe9ce12b) |
+|---|---|---|---|
+| L3-1 | CONFIRMED | R1 dup fits. `main()`'s only exit from `runMatrix` to `browser.close()` is the success path; any exception (the very `PAGE_STATES` throw the file's own comment names) leaks a Chromium child on every platform, not just CI/Linux -- a real, already-wired trigger, not a hypothetical one. | `scripts/check_a11y.mjs:167` `launchBrowser()`, `:218` `browser.close()` (success path only), `:244-247` `main().catch` (exits 2, never closes); check_a11y.mjs:229 comment ("though PAGE_STATES throws before it gets that far") shows the throw path was already known. Siblings guard it: `check_a11y_fingerprint.mjs:234` launch, `:238-248` try/finally; `sweep_a11y.mjs:209` launch, `:214-274` try/finally; `check_axe_patch_equiv.mjs:99` launch, `:104-116` try/finally. `lib/axe-scan.mjs` PAGE_STATES appliers throw on a failed assertion: `section-links-open` (`:122-137`, throws `:125`,`:132`), `content-details-open` (`:147-172`, throws `:151`,`:166`); comment at `:117-120` states the "MUST assert" design rule. No browser was started to verify this. |
+| L3-2 | CONFIRMED | R2 dup is defensible: `spawn(process.execPath, ...)` essentially never fails at the OS level in ordinary operation (it is re-spawning the running Node binary), so "will fail on the next ordinary change" is a stretch -- but the consequence (an un-restored tbIDE registry, the exact hazard `lib/tb-registry.mjs`'s tidy exists to prevent) is severe enough that R2 rather than R3 is right. | `scripts/check_examples.mjs:601-612` `buildStaged`: spawn at `:608`, no `.on("error", ...)`, awaits `"exit"` at `:612` (`child.on("exit", r)`), not `"close"`. Call path: `buildStaged` (601) <- `laneOf(...).build` (`:770-775`, calls it `:772`) <- `runBatch` (`:898-899`) <- `runAll`'s lane loop (`:931-947`, inside `Promise.all` `:947`) <- `main()`'s `try { results = await runAll(...) } catch (e) { ...; finishTidy(tidy); process.exit(2); }` (`:1678-1680`). A spawn `'error'` event with no listener is a Node uncaught exception at the EventEmitter's own emit point (not a promise rejection), so it bypasses that `catch` and bypasses `main().catch(...)` at `:1810` (which also calls `finishTidy`) -- neither is in its call stack. No `process.on("uncaughtException", ...)` exists anywhere in the file (grep confirmed zero matches), so `finishTidy(tidy)` -- the registry restore -- is skipped and the process dies via Node's default uncaught-exception handling. Siblings register both: `test/scripts/addin_test.mjs:180` `child.on("error", ...)` + `:185` `child.on("close", ...)`; `scripts/check_regex_safety.mjs:347` `child.on("error", reject)` + `:348` `child.on("close", ...)`. (Note: `addin_test.mjs` lives at `scripts/addin_test.mjs`, not `test/addin/addin_test.mjs` as the ledger's path implies -- same file, path only.) |
+| L3-3 | CONFIRMED, with one nuance | R1 hack is defensible for an actively-written pipeline even though currently dormant (see evidence) -- the ledger's own "Not live today" qualifier is accurate, so the severity already reflects the dormancy correctly. | `wisdom/extract/merger.mjs:118-129` `parseStaging` splits on lines exactly equal to `'---'`, zero fence-awareness. `parseSection:162,168` returns `null` when `buf[0]` does not start with `"## "`; the caller drops a falsy result silently at `:147-148` and `:152-153`. Comment at `:111-112` reads verbatim: "The parser is forgiving: anything it can't categorise gets preserved as part of a section body so we never silently drop reviewer content" -- contradicted. **Reproduced** in `scratchpad/verify/V4/repro_l3_3.mjs` (imports the real `merger.mjs` by relative path, confirmed side-effect-free at import together with its one local import `state.mjs`): a synthetic staging.md whose 2nd section's fenced ` ```tb ` block contains a bare `---` line does **not** make the section itself disappear -- `parsed.sections.length` is unaffected (3 sections still returned, matching the 3 "## " headings). What silently disappears is the **tail** of that section's own body (everything after the embedded `---`: the rest of the code, the closing fence, trailing prose) **and its entire meta line** (`_Source threads: ... confidence: ..._`, i.e. its `finding_ids`/`confidence`) -- the orphaned tail chunk starts with `"    x = 1"`, not `"## "`, so `parseSection` returns `null` for it and it is dropped exactly as the finding describes. So: not "a section disappears," but "a section survives with its tail and metadata silently amputated," which is arguably worse (a corrupted, mis-attributed section is harder to notice than a missing one). `wisdom/data/findings/staging.md` **is tracked** at `fe9ce12b` (`git ls-files wisdom/data/findings/staging.md` returns it; `git show fe9ce12b:wisdom/data/findings/staging.md` succeeds, 15,553 lines). **Ground truth on the real file** (`scratchpad/verify/V4/check_staging_chunks.mjs`, which mirrors `parseStaging`'s own chunk-split logic exactly rather than guessing at fence boundaries): 1,160 bare `"---"` lines produce 1,160 post-preamble chunks, **all 1,160 start with `"## "`** -- 0 dropped chunks, confirming "not live today" precisely. The 61-line gap between a naive raw `"## "` count (1,221) and the parsed section count (1,160) is fully explained by 61 `"## "`-prefixed lines occurring later *inside* an already-open section's body (folded into prose, not lost) -- a milder, separate, lossless side effect, not the defect under test. |
+| L3-4 | CONFIRMED | R2 dup is reasonable standalone; note the ledger's own A3-1 (not mine to verify) rates the render.mjs half of this identical gap R1 with a live repro, so the eventual merge should likely inherit R1 rather than average down to R2. | `scripts/convert_em_dash_separators.mjs:59`: `const FENCE_OPEN_RE = /^[ \t]{0,3}(\`{3,}|~{3,})/;` -- used at `:144` (`line.match(FENCE_OPEN_RE)`) inside `convertText` (`:135-166`) with no examination of anything after the marker anywhere in that function. This is **byte-for-byte identical** to `builder/render.mjs:148`'s `maskCodeRegions` open pattern (`/^[ \t]{0,3}(\`{3,}|~{3,})/`, confirmed by direct comparison) -- not merely similar. Contrast `render.mjs:1704`'s `stashCodeFences` pattern, `FENCE_OPEN_RE = /^([ \t]*)(\`{3,}|~{3,})(.*)$/`, which captures the info string and explicitly refuses a match at `:1712-1713` when `ticks[0] === "\`" && info.includes("\`")`, per the CommonMark rule stated in the comment at `:1700-1703`. convert_em_dash_separators.mjs has no equivalent guard anywhere. |
+| L3-5 | CONFIRMED | R2 dup fits -- classic same-operation duplication, sharpened by a real same-name/different-behavior collision (a maintainer grepping for "escapeHtml" and picking the wrong module's copy would silently under-escape). | Minimal (`&<>`), 3 copies: `render.mjs:2230-2233` `escapeHtmlMinimal`; `highlight.mjs:251-254` `escapeHtml` (comment `:249-250`: "Rouge's HTML formatter escapes only \`& < >\` -- not quotes"); `gantt.mjs:213` `esc`. Full (`&<>"'`), 4 copies: `render.mjs:2225-2228` `escapeHtml`; `template.mjs:993-998` `escText`; `template.mjs:999-1001` `escAttr`; `sitemap.mjs:106-113` `xmlEscape`. **`template.mjs`'s `escAttr` (999-1001) is byte-identical to `escText` (996-998)**: both bodies are exactly `return String(s).replace(HTML_ESCAPE_RE, (c) => HTML_ESCAPE[c]);` against the same shared `HTML_ESCAPE`/`HTML_ESCAPE_RE` constants at `:993-994` -- two exported names for one implementation, confirmed equal. "escapeHtml names both classes" also confirmed: `render.mjs`'s `escapeHtml` (full, 5-char) and `highlight.mjs`'s `escapeHtml` (minimal, 3-char) are different functions in different files sharing the identical name and doing different things. |
+| L3-6 | CONFIRMED | R3 conv fits -- it fails loudly (a full stack trace reaches the terminal), just inconsistently with the tool's own `error: ...` convention and with a nonstandard exit code (Node's default 1, not this repo's documented 2), rather than silently or by producing a wrong answer. | `scripts/check_links_diff.mjs`, `main()` spans `:598-761`. Its only `try`/`catch` is `:600-601`, scoped to exactly `opts = parseArgs(process.argv.slice(2));`. `fusedBuild` (`:410-438`) calls `spawnSync` at `:426` and `throw new Error(...)` at `:432` when the build produced no findings file, with no local try/catch. `ensureBasePathTree` (`:510-528`) calls `spawnSync` at `:518` and `throw new Error(...)` at `:525` on nonzero exit, likewise unguarded. Both are invoked from `main()`'s case loop, **outside** the `:600-601` try/catch: `fusedBuild(c.fused)` at `:670` and `:676`, `ensureBasePathTree(...)` at `:677`. The module-level call at `:763`, `process.exit(main());`, has no try/catch either, so either throw becomes a raw, unformatted Node stack trace instead of the tool's `error: ${e.message}` pattern used at `:601`. |
+
+## The A3-6 conflict
+
+**Ruling: both sides are partly right, and neither statement stands alone.** A3-6 is right
+that none of the three rewrites has a code guard and that no invariant is stated anywhere
+near them. L3 is right that nothing in the corpus is corrupted by this today. Where L3
+overstates is the *reason* it gives -- "every code-rendering path escapes `& < >` first" is
+true of three paths out of four, and the fourth is a path this site's own configuration
+deliberately keeps open to authors.
+
+### Traced paths
+
+All four token types that can put text inside ``/`
` in the final page, checked
+against `builder/render.mjs`'s `createMarkdownIt` (`:360-379`) and `builder/highlight.mjs`:
+
+1. **Fenced, highlighted** (a recognised Shiki grammar, including the `tb`/`vb`/`vba`/
+   `twinbasic` aliases). `render.mjs`'s `fence` rule (`:399-414`) calls
+   `options.highlight(tok.content, lang)` (`:409`), wired at `:378` to
+   `ctx.highlighter.render`, which is `renderCodeBlock` (`highlight.mjs:96,102-142`). The
+   Shiki branch (`:131-136`) renders through `renderThemedSpans` (`:158-247`), which calls
+   `escapeHtml(ex.content)` / `escapeHtml(tok.content)` on every token segment (`:217`,
+   `:220`) before wrapping it in a ``. **Always escaped.**
+2. **Fenced, plain** (unrecognised language, or explicit `plaintext`/`text`/`txt`/empty).
+   Same `renderCodeBlock`, `else` branch (`:137-138`): `tokenizedHtml = escapeHtml(codeBody)`
+   over the whole block in one call. **Always escaped.**
+3. **Indented (4-space) code blocks.** `render.mjs`'s `code_block` rule (`:420-424`):
+   `const body = escapeHtmlMinimal(tok.content);`. No Shiki involvement at all (no
+   language info exists for this token type). **Always escaped.**
+4. **Inline code spans.** `render.mjs`'s `code_inline` rule (`:431-434`):
+   `escapeHtmlMinimal(tok.content)`. A second copy for TOC-heading generation at
+   `:1441-1442` does the same. **Always escaped** (for `&<>`; `"`/`'` survive by design,
+   irrelevant to this trace since none of the three rewrites key on quotes).
+5. **Code inside a raw, hand-authored HTML block** (e.g. an author types `
...
` + directly in the markdown source instead of using a fence). `render.mjs:364` sets + `html: true` on the `MarkdownIt` instance. Grepping every `md.renderer.rules.*` + assignment in `render.mjs` (fence, code_block, code_inline, table_open/close, th_open, + td_open, ordered_list_open, footnote_*, image x2) finds **no override for `html_block` + or `html_inline`**. Confirmed directly against the vendored package, + `node_modules/markdown-it/lib/renderer.mjs:102-106` (markdown-it 14.2.0, per its + `package.json`): `default_rules.html_block = (tokens, idx) => tokens[idx].content` and + the same for `html_inline` -- verbatim passthrough, **never escaped**, by markdown-it's + own design (contrast `default_rules.text` at `:98-100`, which does call `escapeHtml`). + `template.mjs:81` feeds `injectAnchorHeadings` the page's full `renderedContent`, not a + pre-filtered subset, so this passthrough content is exactly as exposed to + `HEADING_REGEX`/`VOID_TAGS_RE`/the empty-cell pattern as everything else on the page. + +So: **yes, a path exists** (path 5) where a raw, unescaped `<` or `>` can land inside a +`
` or `` element before `padEmptyCells` (`render.mjs:74-80`), `normaliseVoidTags`
+(`render.mjs:341,351-354`) or `injectAnchorHeadings` (`template.mjs:714-727`) run over the
+assembled page HTML (call order confirmed at `render.mjs:60-69`: `applyPreRenderRewrites` →
+`md.render` → `normaliseVoidTags` → `padEmptyCells`; `injectAnchorHeadings` runs later over
+`page.renderedContent`, `template.mjs:81`). `HEADING_REGEX` (`/<(h[1-6])(\s[^>]*?)?>([\s\S]*?)<\/\1>/g`,
+`template.mjs:694`) in particular mutates *any* `

`-`

` pair it finds -- even one with +no `id` attribute, it still pads the body with spaces (`:725`) -- so a hand-written `
`
+block illustrating literal heading or void-tag HTML (very plausible on a page describing this
+site's own rendering pipeline) would have its example text silently altered.
+
+**Is it live?** No. `git grep` across all 912 `docs/*.md` files at `fe9ce12b` finds zero
+block-starting ``-`
` tags anywhere. It does find 42 +lines that are bare void tags (`
` etc.) at column start, but every sampled hit (e.g. +`docs/Features/Packages/Creating a TWINPACK package.md:16,17,22,23,...`) is an ordinary manual +line break used for layout, with no enclosing code context -- exactly the kind of real markup +`normaliseVoidTags` exists to normalise, not example text it corrupts. So L3's practical +conclusion holds: **nothing in the current corpus is being corrupted.** + +### Is the invariant written down? + +**No, not at any of the three call sites.** The surrounding comments explain *what* each +rewrite does and *why* (kramdown byte-parity for `padEmptyCells`/`normaliseVoidTags`, +a11y/anchor rationale for `injectAnchorHeadings`, `template.mjs:709-713`) -- none of them +states or relies on an escaping guarantee. + +**Also not written down in `WIP.Build.md`**, which is the one place in the tree that states +this exact policy. Its section "Never rewrite markdown source without knowing what is code" +(`WIP.Build.md:236-289`) says a rendered-HTML rewrite must use `replaceOutsideCode` +(`builder/book.mjs`) or "the same leading-alternation shape found in `offline-rewrite.mjs:299`, +`pdf.mjs:138` and `book.mjs`'s `IMG_SRC_RE_BOOK`, which consume `` and `
`
+atomically" (`:275-278`) -- and never mentions `padEmptyCells`, `normaliseVoidTags` or
+`injectAnchorHeadings`, either as compliant examples or as a stated exception to the rule.
+Worth noting: the same section carries a directly analogous, already-learned lesson two
+paragraphs above (`:256-267`) -- `book.mjs`'s chapter-transform rewrites (`id="`, `href="#`,
+`src="/`) were assumed safe because "highlighted blocks escape this only by accident: the
+highlighter splits attributes across `` boundaries", until inline code spans (no such
+splitting) let all three patterns match inside a sample and corrupted six code spans in the
+published PDF. The document's own conclusion: "**Do not rely on that accident.**" That is a
+different mechanism (quote-survival in already-escaped inline code, not raw-HTML-block
+passthrough) but the identical shape of mistake -- trusting an unstated, implementation-
+dependent escaping guarantee for a whole-page rewrite -- and it is on record one file away
+from the three functions in question, without being connected to them.
+
+**Conclusion:** A3-6's "no code guard, unstated invariant" holds exactly as written. L3's
+"sound because every code path escapes" is true for three of the four paths that can put text
+in ``/`
`, false for the fourth (raw hand-authored HTML, structurally open via
+`html: true` and never overridden), and not currently triggered by anything in `docs/`. A3-6's
+R2 hack rating is fair: this codebase has already paid for exactly this class of mistake once
+(the book.mjs incident above), the fix is cheap (state the invariant and add a probe, or move
+to the code-guarded pattern already used elsewhere), and the trigger (an author illustrating
+raw HTML output by hand) is an ordinary kind of edit for pages that describe this pipeline's
+own rendering -- not R1 only because nothing is broken yet.
+
+## Noticed in passing
+
+1. `convert_em_dash_separators.mjs:59` and `render.mjs:148`'s fence-open regexes are not just
+   similar, they are byte-for-byte identical literals -- a stronger duplication than "shares
+   the gap" suggests.
+2. `check_examples.mjs` (1,810 lines) registers no `uncaughtException`/`unhandledRejection`
+   handler anywhere; the `buildStaged` spawn (L3-2) is one instance of a wider exposure that
+   would affect any stray async error in the file, not only this one.
+3. The real, committed `staging.md` already has a near-miss of the L3-3 shape at source lines
+   14244-14246: a blockquoted ` ```tb ` fence whose interior lines drop their `"> "`
+   continuation prefix. It doesn't trip the defect (confirmed: 0 dropped chunks file-wide),
+   but shows how easily a slightly different malformed blockquote-fence would.
+4. `escapeHtml` is a three-way name collision, not two: markdown-it's own internal
+   `escapeHtml` (used by its default `text` rule), `render.mjs`'s `escapeHtml` (full 5-char),
+   and `highlight.mjs`'s `escapeHtml` (minimal 3-char) are three unrelated functions sharing
+   one name across the same build.
+5. `check_a11y.mjs:229`'s own comment ("though PAGE_STATES throws before it gets that far")
+   shows the author already knew the exact exception path that leaks the browser in L3-1, one
+   line above code with no try/finally to catch it.
diff --git a/builder/REVIEW-TOOLING-fe9ce12b/ledger.md b/builder/REVIEW-TOOLING-fe9ce12b/ledger.md
new file mode 100644
index 00000000..a37ffd58
--- /dev/null
+++ b/builder/REVIEW-TOOLING-fe9ce12b/ledger.md
@@ -0,0 +1,577 @@
+# Review ledger (condensed pass reports, for synthesis)
+
+## VERIFICATION STATUS (details in scratchpad/verify/V*.md)
+
+- V1 (builder/, 23 findings): all CONFIRMED. Severity notes: A2-1 R2 not R1 (unreachable code, no
+  live divergence); A3-4 R3 not R2 (two 4-line predicates). L4-8: 4 of 5 helpers byte-identical,
+  encodeSpaces same behaviour, different idiom. Conflicts: writeBaseline -> L2-4 right (equals
+  JSON.stringify(x,null,2)+"\n" except an empty list, run); normalizeBaseurl -> A9 right (premise
+  expired: book.mjs imports compress.mjs and data.mjs); escapeHtml -> keep A3-2, narrowed. Five
+  noticed-in-passing items in V1.md (another stale diff-tool reference, a third HTML-escaper copy,
+  an exit-code ordering hazard, ...).
+- V4 (L3, 6 findings): all CONFIRMED. L3-3 nuance: a bare --- inside a fence does not make the
+  section vanish; it cuts off the rest of that section's body AND its meta/finding_ids line (the
+  caller drops the null); replay of the real 15,553-line staging.md drops 0 chunks today. L3-2: an
+  unhandled spawn 'error' is an uncaught exception, bypassing both catches and finishTidy -- no
+  uncaughtException handler in the file. A3-6 CONFLICT RULED: both partly right -- fenced, indented
+  and inline code are always escaped, but hand-authored raw HTML (
, headings; html:true,
+  markdown-it's own html_block/html_inline) passes through unescaped and IS exposed to the three
+  whole-page rewrites. Not live: zero hand-written 
 or heading tags in 912 pages. The invariant
+  is written nowhere near the sites nor in WIP.Build.md, which has an adjacent precedent ("Do not
+  rely on that accident"). -> keep A3-6 as R2 hack: a real, unguarded, undocumented path.
+- V3 (harness, samples, book; 15 findings): 10 CONFIRMED, 5 CORRECTED (details only):
+  A7-1 tbrun matches 3 of the 5 shapes (case-insensitive), still misses "[BUILD] ERROR" and
+  "[LINKER] compilation (codegen) error" (I had told the user "2 of 5" -- corrected to them);
+  A7-3 click() used directly in 4 of 10 scenario files (appdata, panes, sample10, sample15), not all;
+  A7-4 addin_test imports 10 names from tb-registry, not 9; A8-2 path citations are 11 (strict) or 13
+  (bare filename) -- the list wrongly had WIP.Build.md and missed build_package_api.mjs and
+  tb-fences.mjs (V3.md has the exact list); A9-5 12 files repeat the createRequire block, not ~15.
+  L4-10: YES, twin-api's logicalLines keeps tb-fences' quote-aware comment stripping (8+ cases incl.
+  "" then a real apostrophe), but the swap is not mechanical: twin-api keeps blank lines, `line`
+  1-based vs `at` 0-based, no trim, recognises Rem. tb-fences has NO /* */ handling at all.
+  A7-2's kind (struct vs hack) arguable, left.
+- V2 (gates, links, a11y, CI, small tools; 29 findings): 25 CONFIRMED, 4 CORRECTED: A4-3
+  LINK_ATTR_TABLE is 21 tags / 26 flattened pairs (not 20); crawl_check's 5 exact; RAISE TO R1 -- the
+  gap is live (crawl_check misses srcset, poster, cite, formaction, action, data, longdesc today).
+  A5-1: 8 scripts, not 7 (check_tree_fresh.mjs too). L2-5: 19 files exact; ~5 AST shapes rather
+  than 4. L2-6: four files use try/finally cleanup, not three. All three conflicts: keep the
+  original findings alongside L4's classification (different questions).
+- ALL FOUR VERIFIERS DONE.
+- ORCHESTRATOR CHECK of the inventory's "live staging.md corruption" claim: NOT CONFIRMED. Running
+  the real parseStaging on the real file: section 1060 (Len.md, holds the ReDim sample) carries
+  finding_ids ["1314412625714479125"], dates 2024-12-06 -- exactly the meta beneath it on disk;
+  sections 1059 and 1061 keep their own. The heading's "see also thread ..." lists the OTHER
+  duplicates, which misled the agent. What IS real: a content slip -- staging.md:14245-14246 lack
+  the "> " continuation, so if that section is ever grafted into Len.md it renders as a broken
+  admonition and a fence that runs on (markdown-it reads 14246-14297 as one fence). A data defect a
+  markdown-aware check of section bodies would catch; not mis-pairing.
+- ORCHESTRATOR CHECK: docs/Documentation/Wisdom.md:304 opens a fence, :305 is "## docs/..." inside
+  it -> check_gate_lists.mjs's splitSections splits there (dormant: that phantom section states no
+  gate count). CONFIRMED.
+
+## THEME raised by the user (2026-09-25): markdown handled as text without parsing first
+
+The user's observation: markdown is repeatedly treated with regexes directly, without first parsing
+the file to separate code. Instances: render.mjs maskCodeRegions/maskInlineCode and stashCodeFences
+(A3-1), convert_em_dash_separators.mjs (L3-4, L4-7), wisdom merger.mjs '---' split (L3-3), wisdom's two
+frontmatter parsers (A10-2), census_attributes.mjs's Attributes.md line regexes; precedent for doing it
+right: tb-fences.mjs and check_code_regions.mjs (markdown-it tokens), discover.mjs (gray-matter).
+Direction: ONE shared module deriving prose/code/HTML/frontmatter regions from markdown-it's block
+parse (the renderer's own view) + one tested inline-code splitter; every markdown scan/rewrite goes
+through it. This is a headline theme for the review, with a design decision for the user.
+Inventory agent DONE -> scratchpad/md-inventory/INVENTORY.md (15 sites: 5 rewrite/A,
+10 read-only-scan/B, cross-referenced against A3/A10/L3/L4's findings above and confirms
+them; sharpest new result: live, on-disk section corruption in the COMMITTED
+wisdom/data/findings/staging.md, traced with markdown-it against the real 15,553-line file
+-- a blockquote-fence missing its "> " continuation at line 14245 makes markdown-it read
+lines 14246-14297 as one unclosed fence, and parseStaging's bare-"---" splitter attaches
+a DIFFERENT section's _Source threads:_/_Date range:_ meta to the "Len.md · after-remarks"
+heading; re-running extract --merge would serialize the wrong pairing back to the repo.
+Revises L3-3's "not live today" -- it IS live, just not via the bare-dash-in-fence shape
+L3-3 checked; a naive fence-aware toggle also mis-verifies this, worth noting for
+whoever writes the fix. Also confirmed A3-1's repro independently (own read, before
+seeing A3's note). New: B3 check_gate_lists.mjs's splitSections also mis-splits on a
+"## ..." line inside a fenced staging.md-format example at Documentation/Wisdom.md:305
+(dormant, same root cause class as L3-3, different file). Task 2: markdown-it answers
+all confirmed empirically (fixtures + real corpus); frontmatter is ACTIVELY misparsed
+(hr + setext H2), not just ignored, if not stripped first. Timing: 912 files/3.98MiB,
+full parse 234-257ms (0.26-0.28ms/file), block-only 34ms (0.037ms/file, ~7x cheaper,
+verified to skip inline splitting for real). Recommended home: new top-level lib/
+(sibling to builder/scripts/wisdom/eval), since builder/ can be imported FROM scripts/
+already (check_code_regions.mjs does) but not the reverse -- scripts/lib/ is out because
+render.mjs:383 says so verbatim, and putting it inside builder/ would make wisdom/eval/
+depend on the site generator for fence-detection, which is backwards. Also verified the
+js-yaml skew independently: gray-matter@4.0.3 nests js-yaml@3.14.2
+(node_modules/gray-matter/node_modules/js-yaml), while builder/data.mjs, tbdocs.mjs and
+check_publish_policy.mjs import the top-level js-yaml@4.1.1 directly -- confirms the
+follow-up note above from package.json reads, not just cited.
+Follow-up (user asked why two parsers): gray-matter is a frontmatter splitter + YAML, not a markdown
+parser -- but it bundles js-yaml 3.14.2 while the build parses _config.yml/_book.yml with the direct
+js-yaml 4.1.1: page frontmatter and site config go through two YAML major versions. Users:
+builder/discover.mjs:8, eval/nav_hops.mjs:35. discover.mjs strips a BOM itself because matter.test()
+does not. Recommendation (told the user): drop gray-matter; a small frontmatter splitter in the
+shared markdown module + js-yaml 4. Oracle: every page's parsed frontmatter deep-equal both ways
+(906 pages) + identical built trees.
+
+Verifier column: what must be re-read before a finding goes into the review.
+
+## A4 -- link checkers (done)
+
+Two-checker question, evidence:
+- "Tree the build did not produce" (release zip, bisect, foreign artifact) is stated in six places, exercised nowhere: CI runs check_links.mjs only in-process via check_links_diff.mjs on fixtures; the release zips the tree the same job already checked; git history has no such use. crawl_check.mjs (live HTTP, person-run after release, Building.md:653-655) is the only post-release check and shares no code.
+- Shared (link-check.mjs): extraction, resolution, oracle, cross-file checks, human-readable reports. Parallel: tree loading (deliberate, PLAN-checks "What is being moved"); findings translation (clone); statSafe; 1/2/3 exit assembly (deliberate: tbdocs ORs build-failure bits too, tbdocs.mjs:1566-1584).
+- Harness cost: 763 lines, 3 CI steps (~0.3 s, ~1 s, ~0.3 s), not in check/test.bat; fixture counts break on unrelated changes (fonts preload: broken 3 -> 9, test/README.md).
+- Harness catches (all in the shared core or fixtures, none algorithm-vs-algorithm): 39e03ab2 brokenUnique double count; d2c3a743 skipped check printed as clean; Phase 2 coordinate bugs (95,839 and 11,815 phantom links); Phase 3 writeOfflineRedirects dependency gap and resolution-cache bug; the dead --check-html gate; revived scriptSelfTest.
+- Recommendation: option 2 -- check_links.mjs keeps its disk walk and CLI but calls check.mjs's exported checkChunk/joinChunks/findingsFor/formatReport; harness then compares disk-read vs memory-read over one implementation (still catches index/coordinate bugs); fixture counts stay as a regression test. Not option 3: capability cheap to keep; tbdocs would gain an unrelated job; post-release checking wants HTTP anyway.
+
+Findings:
+- A4-1 R3 dup statSafe twice: link-check.mjs:455-457 (FsOracle), check_links.mjs:151-153. VERIFY identical.
+- A4-2 R2 dup findings translation: check.mjs:438-449 findingsFor vs check_links.mjs:621-634 buildFindings; human-readable twin already shared. VERIFY clone.
+- A4-3 R2 dup crawl_check.mjs:91-108 own attribute table, 5 tag/attr pairs vs link-check.mjs:38-60 LINK_ATTR_TABLE's 20; no record. VERIFY counts.
+- Split candidates: none (check.mjs, check_links.mjs, check_links_diff.mjs coherent).
+- Lead for L2: check_links.mjs:276 oracle default depends on platform (index on win32) -- reasoned.
+- DECIDED by the user (2026-09-25): option 2. check_links.mjs stays as a thin wrapper over
+  builder/check.mjs. Record under decision 5 in PLAN-TOOLING-REVIEW.md when the passes are in;
+  the fix goes in Phase 2 (shared code). A4-3 (crawl_check's table) is separate, not decided.
+
+## A9 -- book pipeline (done)
+
+Decision 1 scope: the book build loads exactly ONE file from perf/ at run time: perf/detach-pages.js
+(via render-book's --additional-script). perf/timing-handler.js is only mentioned in a comment
+(progress-handler.js:8). Places naming detach-pages.js: book.bat:13,39; tbdocs-gh-pages.yml:196;
+render-book.mjs:28 (comment); Tools.md:108,975; PDF-Generation.md:44,197,395,396,430; Building.md:86;
+docs/assets/css/print.css:136 (comment); paged.browser.js:30542 (comment, fork). AND five perf scripts
+resolve it next to themselves -- perf/measure.mjs:366, probe-tabs-vs-procs.mjs:48,
+probe-renderer-mem.mjs:71, probe-parallel.mjs:61, probe-memory.mjs:66 -- which break on the move
+unless their paths are updated (mechanical; perf otherwise untouched).
+Original of the measure-pass copy: perf/phase0-measure.mjs (f76701a4); book/lib/measure-pass.mjs was
+extracted from it 40 min later (d963c183). Note only.
+
+Findings:
+- A9-1 R2 hack: pdf-lib shims (fast-*.mjs, parallel-deflate's PDFStreamWriter subclass) unguarded;
+  only the exact pin 1.17.1 (recorded perf/notes/08-pdf-lib.md:1751-1757) -- covers accidental drift,
+  not a deliberate bump; a behavioural upstream change fails silently. Fix: load-time shape
+  assertions per shim, modelled on axe-scan's SOURCE_PATCHES counts. pdf-lib upstream abandoned
+  (@cantoo fork evaluated and rejected, 08-pdf-lib.md:5048-5061) -> R2 not R1.
+- A9-2 R2 struct: no equivalence test of shimmed vs stock pdf-lib output anywhere (one-off manual
+  checks in perf notes). Fix: check_pdf_shims_equiv.mjs like check_axe_patch_equiv (stock pdf-lib in
+  a child process). Would be a new test.bat gate -> gate lists/CI/decision-6 gate.
+- A9-3 R2 dup: onebuf machinery (pack, _cow, _registerContext, _appendArray, _makeFromRange,
+  _makeFromAppend) in both fast-array-onebuf and fast-dict-onebuf; recorded reason
+  (fast-array-onebuf.mjs:36-39) covers only the singleton; _registerContext/_appendArray identical
+  bar names. Fix: book/lib/onebuf-range.mjs factory.
+- A9-4 R3 dup: _writeUint/_digitCount in fast-refs and fast-refs-class. MOOT if A9-6 moves
+  fast-refs.mjs to perf/ (orchestrator confirmed fast-refs.mjs is imported only by perf/measure.mjs:428
+  and perf/phase0-measure.mjs:28; fast-refs-class does not import it).
+- A9-5 R3 dup: ~15 shims repeat createRequire + require('pdf-lib/cjs/...').default blocks.
+  Fix: book/lib/pdf-lib-internals.mjs.
+- A9-6 R2 struct: four A/B-baseline shims live in book/lib but only perf/ loads them: fast-refs,
+  fast-dict-array, fast-dict-iter, fast-parse-dict (render-book.mjs:56-58 records them as baselines,
+  not their location). Fix: move into perf/ and update perf's imports. NEEDS USER OK: the converse of
+  decision 1 (book -> perf), which the user did not decide.
+- A9-7 R2 dup: the code-guard alternation (/
 leading alternative) re-typed by hand:
+  book.mjs:210 CODE_OR_PRE_BOOK vs IMG_SRC_RE_BOOK book.mjs:228-229 (same file!), pdf.mjs:143-144
+  IMG_SRC_RE (identical), offline-rewrite.mjs:299 HTML_COMBINED_RE. Safety-critical, no gate ties
+  them. Fix: one exported fragment. (Expect overlap with L3.)
+- A9-8 R2 dup: normalizeBaseurl book.mjs:303-307 == offline-rewrite.mjs:232-236 (exported); recorded
+  reason (Jekyll plugins independent, book.mjs:300-302) expired at the Node cutover; comment also
+  misnames offline.mjs. Fix: import it. VERIFY byte-identical.
+- A9-9 R3 dead: fast-dict-array.mjs:9 cites docs/lib/fast-parse-dict.mjs (moved in 4c36c8d6).
+- A9-10 R2 dead: pdf.mjs:6-9, 99-107 justify exports by the deleted _diff.mjs/_triage.mjs (644d6bdb);
+  extractImagePaths (pdf.mjs:146-158) has zero callers. deriveBookOutputs is live (writePdf).
+  VERIFY zero callers.
+- Split candidate: builder/book.mjs -- resolver (§A), book.html assembly (§B-F, incl. rewriteBookHrefs),
+  coverage checker (§G); pdf.mjs already imports the three as if separate.
+- Verified sound: book.bat; render-book's 13-shim import list consistent; __xInstalled guards;
+  paged.js fork divergence fully recorded (20 [PATCH] tag families, 52 sites, Fixes-PagedJS.md);
+  pdf.mjs IMG_SRC_RE follows the guard rule; postprocesser/outline are attributed ports.
+- Leads: render-book.mjs:204-225 hand-rolled argv (L1); offline.mjs imports from offline-rewrite
+  (the model A9-8 should follow).
+
+## A8 -- samples and package API (done)
+
+- A8-1 R1 dup: census_attributes.mjs MODS (138-148) lacks Overridable (+Iterator, Dim; tb-fences has
+  Async/PtrSafe/... too) vs twin-api.mjs:123-125 and tb-fences.mjs:340-343. "Public Overridable Sub"
+  -> DECL_RE fails -> VAR_RE matches -> "Variable". ORCHESTRATOR VERIFIED the trace. IMPACT CLAIM FALSE:
+  BETA 983 cache has 31 Overridable Sub/Function/Property lines, 0 carry an attribute (inline or above,
+  past comments/#If), and block tracking uses only type keywords (OPEN_RE/CLOSE_RE, 157-159), so no
+  stack corruption. Dormant divergence. Two more claimed divergences to VERIFY: blankStrings (170-179)
+  no "" escape; BLOCK_COMMENT_RE per line, stateless (twin-api stateful, 56-66).
+- A8-2 R2 struct: census_attributes.mjs in builder/ breaks render.mjs:383's rule "builder/ must not
+  depend on scripts/"; check_tree_fresh.mjs:57-63 IGNORED_FILES exists only for it. Move to scripts/
+  beside build_package_api; ten path citations: WIP.md, WIP.Harness.md, WIP.HelpAddin.md,
+  WIP.ExamplesBuild.md, WIP.Build.md, BUGS-TO-REPORT.md:430, Tools.md #census-attributes,
+  docs/Reference/Attributes.md:588 (HTML comment, published page), tb-packages.mjs:2, twin-api.mjs:17.
+- A8-3 R3 dup: fence unit key twice in check_examples.mjs (:424 makeBatches, :675 unitsOf).
+- A8-4 R2 struct: census_attributes and gen_attribute_probes' parseTargets have no ride-along probes;
+  both have shipped silent misparses before (parseTargets' own comment).
+- Split: check_examples.mjs -- 8 jobs (templates 125-193, selection 201-378, batching 380-506,
+  staging 508-568, orchestration 570-655/928-949, bisection 657-927 [270 lines, 14 fns], diag mapping
+  inside buildStaged 643-653, 3 report modes threaded as booleans) + 425-line probe suite 1157-1581.
+  NOT gen_attribute_probes (47% is the EXPLORATORY data table).
+- Sound: serialize/strip/kindOf name collisions coincidental; symbols.mjs is not a .twin scanner;
+  tb-fences tested only via check_examples' runProbes (74 probes).
+- Leads: L2 -- no gate enforces "builder/ must not depend on scripts/" (import-direction check).
+
+## A1 -- build orchestration (done)
+
+- A1-1 R1 struct: Gantt drops checkBook, checkReport (section "Check") and vendorAssets (no
+  GANTT_SECTION entry) on every build -- gantt.mjs:39-49 renders Seeds/Spine(+Render)/Write only; the
+  "Other" colour (gantt.mjs:12) is unused; Builder.md:410 promises an "Other" bucket. Reaches the
+  published Build Info page; their durations still stretch the axis. ORCHESTRATOR VERIFIED.
+- A1-2 R1 hack: picocolors undeclared (tbdocs.mjs:35, scheduler.mjs:5); via puppeteer -> cosmiconfig
+  -> parse-json -> @babel/code-frame. Fix: declare 1.1.1.
+- A1-3 R2 dup: on-demand task run/time/report block x3 in cpu-worker.mjs (360-377, 423-440, 469-486).
+- A1-4 R2 dead: tbdocs.mjs:196-209 exports makeTimer, zero callers; offline.mjs:98-111 private copy;
+  its reason (offline.mjs:95-97, "verify harnesses and diff tools") expired with 644d6bdb.
+- A1-5 = A2-1 (dead writeOfflinePages branch cloning cpu-worker.mjs:183-201). MERGE.
+- A1-6 R2 conv: exit-code bits set at 7 sites with magic numbers (tbdocs.mjs:560, 1469-1470,
+  1568-1573, 1598, 1610); bit 0 conflates 5 causes. Fix: failBuild(bit) + named constants.
+- A1-7 R3 dead: cpu-worker.mjs:510 writes literal 4 (FAILED) -- and nothing ever reads FAILED.
+- A1-8 R3 struct: tbdocs parseArgs (91-194) not exported, untested.
+- Split: tbdocs.mjs -- TASKS literal fuses DAG topology with task bodies (260-1257, 61%); dispatch's
+  submit() is SAB machinery (788-911); check glue (1085-1256); Gantt/timing (1268-1403) belongs by
+  gantt.mjs (A1-1 is the cost); console report (1478-1525); exit policy scattered.
+- Sound: small modules; SAB layout 174,100 bytes matches Builder.md; HANDLERS key sets match; SAB
+  constants imported by name (except A1-7); barrier invariant; --dry-run and --profile-offline LIVE,
+  Tools.md flag table matches 20/20; stall watchdog matches WIP.Build.md.
+- Leads: offline.mjs's "Re-export surface for diff tools" (55-88, ~25 names) has zero importers (A2).
+
+## A2 -- output stages (done)
+
+- A2-1 [agent R1, likely R2] dead: writeOfflinePages (offline.mjs:244-289, zero callers; 258-280 clone
+  of cpu-worker.mjs:183-201), writeOffline's unread `precomputed` param (117), buildSitePaths +
+  fallback (207, 213, 359-394; tbdocs.mjs:705-706 always sets sitePaths), writeSearchData
+  (search.mjs:18-24), extractSitemapUrls (sitemap.mjs:69-74). Plus A1's lead: offline.mjs 55-88
+  re-export block unused. All from the retired _diff/_triage tools and the SAB split.
+- A2-2 R2 dead: comments naming deleted _diff.mjs/_triage.mjs/_sitemap_diff.mjs/accepted-divergences:
+  offline-rewrite.mjs:411, search.mjs:65-66, sitemap.mjs:67-68,97, redirects.mjs:35-36 -- and
+  pdf.mjs:6-9,99-107 (A9-10). MERGE with A9-10's comment half.
+- A2-3 R2 dup: readBaseline byte-identical (page-baseline.mjs:83-90, symbol-baseline.mjs:45-52); the
+  six-branch drift-guard state machine re-typed (117-176 vs 75-125). Recorded: "the page-count
+  guard's shape applied to URLs" (WIP.Build.md) -- covers look-alike, not the copy.
+- A2-4 R2 dup: escapeRegExp x3 -- render.mjs:2235, offline-rewrite.mjs:239 (exported), book.mjs:212
+  (escapeRegExpBook). regex-fold recognises escape helpers "by body" because of the copies. MERGE A3-5.
+- A2-5 R3 dup: encodeSpaces search.mjs:204 vs template.mjs:925 (different idioms). MERGE A3-3/A3-5.
+- A2-6 R2 dup: normalizeBaseurl = A9-8. MERGE.
+- A2-7 R2 dup: ASCII-only NBSP-preserving trim twice: compress.mjs:73-84 (regex) vs search.mjs:251-261
+  (charCode scan) -- the shipped-defect class of "Whitespace inside inline code is content".
+- A2-8 R3 dup: replaceAll("\\","/") inline x10 (offline-rewrite 34,39,44,219,226; offline 133,311,
+  370,375,380); publish-policy.mjs:189 has a private posix(); check-tree.mjs:46 another (recorded).
+- Split: none (offline.mjs sections; loses ~100 lines with A2-1).
+- Sound: offline/offline-rewrite split recorded; HTML_COMBINED_RE follows the guard rule;
+  deriveOfflineJtdJs uses acorn; vendor-assets contained+guarded; symbols.mjs not a .twin scanner;
+  substitute() and decode() collisions coincidental; publish-policy disjointness self-test; gates run
+  clean (publish 6/6, page-baseline 11/11, symbol-index 46/46).
+- Leads: HTML escape maps x4 (highlight.mjs:251, render.mjs:2225/2230, template.mjs:993, gantt esc
+  213); atomic-write idiom; write.mjs:173-178 vs offline-rewrite 442-465 url() regexes (different jobs).
+
+## A3 -- Markdown dialect (done)
+
+- A3-1 R1 dup: fence-opener predicates differ -- maskCodeRegions (render.mjs:148,
+  /^[ \t]{0,3}(`{3,}|~{3,})/) accepts a backtick fence whose info string holds a backtick;
+  stashCodeFences (1713) refuses it (CommonMark-correct). Agent REPRODUCED: "```abc`def" masked by one,
+  unprotected by the other (a "> [!NOTE]" inside became an admonition). No live trigger in docs/;
+  check_code_regions structurally blind to it. Fix: one predicate + a probe. VERIFY repro.
+- A3-2 R2 dup: escapeHtml = 3 chars in highlight.mjs:252, 5 chars in render.mjs:2226; headingTocHtml
+  (1437-1450) uses 5-char for text, 3-char for code_inline; markdown-it escapes &<>" -> a TOC'd heading
+  with an apostrophe renders ' in the TOC, ' in the heading. Not live (no TOC'd page has one).
+- A3-3 R1 dup: absoluteUrl/relativeUrl ported twice and diverged: seo.mjs:121-148 (config arg, forces
+  leading slash, no space encoding) vs template.mjs:917-936 (baseurl arg, leaves bare values
+  unchanged, encodes spaces). No call site hits the gap today. Fix: builder/url.mjs.
+- A3-4 R2 dup: isNonEmpty nav.mjs:338 == seo.mjs:164.
+- A3-5 R3 dup: escapeRegExp (MERGE A2-4), splitFragment render.mjs:1593 vs crawl_check.mjs:51,
+  encodeSpaces (MERGE A2-5).
+- A3-6 R2 hack: three post-render whole-page regex rewrites with no code guard -- padEmptyCells
+  (render.mjs:74-80), normaliseVoidTags (351-354), injectAnchorHeadings (template.mjs:714-727). Safe
+  only by an unstated invariant (code content is entity-escaped); outside check_code_regions (which
+  covers only the pre-render chain). Fix: state the invariant; extend the gate. (Expect L3 overlap.)
+- A3-7 R2 struct: template.mjs holds a strftime formatter (940-989), URL helpers, escape helpers.
+- A3-8 R3 dup: highlight-theme.mjs three palette loops (336-344, 351-359, 364-372).
+- A3-9 R3 dead: precomputeSeo (seo.mjs:90-94) zero callers.
+- A3-10 R3 dead: kramdownSlug exported (render.mjs:1352), used only inside.
+- Split: render.mjs -- STRONG evidence: image renderer rule chained by md.use order (svgInline 518
+  before remoteImage 520; swapping reverses silently); anchors on third-party rule names
+  ("curly_attributes"); ellipsis plugin assumes dashes plugin ran first (515/516); shared helpers
+  thread through all plugins; no plugin tested in isolation. Seams listed. template.mjs weaker:
+  navActivationCss (446-567) deferral trigger in PLAN-4 §3 close to met.
+- Sound: nav.mjs; table/fence fixups on single known tokens; token-walking plugins immune to code;
+  html_block-scoped rewrites; template reuses seo's stripHtml; clampContrast loud; VOID_TAGS_RE gated.
+
+## A5 -- a11y and diagram gates (done)
+
+- A5-1 R1 dup/conv: argv loops copied in 7 scripts, diverged: check_a11y.mjs --help -> "unknown arg"
+  exit 2 while siblings print usage exit 0; check_dot_fit/build_dot_metrics use argv.includes and
+  ignore typos; usage printed to stdout in some, stderr in others. (Feeds L1.)
+- A5-2 R2 conv: 0/1/2 convention (Extending.md:640-648) broken in build_dot_metrics.mjs (no catch;
+  crash exits 1 = STALE) and pick_a11y_sample.mjs discover() (ENOENT exits 1 = coverage gap; live in
+  check.bat). check_dot_fit.mjs:31-34 has the guard with a comment citing the convention.
+- A5-3 R2 dup: page discovery + tag count + stub ceiling in pick_a11y_sample (122, 159-171, 184) and
+  sweep_a11y (78, 129-153); ceiling named STUB_CEILING vs STUB_TAG_CEILING (both 100); the two must
+  agree (--propose reads sweep's JSONL).
+- A5-4 R3 dup: pad/median identical in the same two files.
+- A5-5 R2 dup: check_dot_fit.mjs and build_dot_metrics.mjs duplicate the Inter host page + puppeteer
+  scaffold (font filenames, launch args incl. --allow-file-access-from-files unexplained there;
+  axe-scan's LAUNCH_ARGS omits it for the same kind of load). Three repo-root idioms in scope.
+- A5-6 R2 dup: check_dot_fit.mjs:49-67 re-walks .dot sources instead of importing dot.mjs's
+  listDotSources (135-155, private) -- the mirror-fault shape; five gates already import the chain.
+- Split: axe-scan.mjs NOT proposed (single-source property; any split must re-export).
+- Sound: axe SOURCE_PATCHES earns model status (exit implicit: "until axe ships a modern build");
+  dot-metrics WASM patch passes emphatically (signature match, read-back, real layout check, WeakSet);
+  check_tree_fresh uses isOutputTree; FAMILIES table is data; check_a11y mutual exclusion correct.
+
+## A6 -- gates as a system (done)
+
+- A6-1 R1 dup: self-test accumulator + report loop in check_page_baseline (38-44, 128-139),
+  check_book_coverage (81-87, 158-170), check_symbol_index (40-45, 361-370), + the crash handler in 4
+  gates. Diverged: only check_book_coverage.mjs:162 re-indents multi-line detail. Shared helper must
+  (a) keep probes unconditional, (b) take the probe-failure exit code as a parameter (1 for these
+  three; 2 for gates whose probes guard a separate sweep: check_gate_lists, check_regex_safety).
+- A6-2 R1 dup: test.bat:34-43's history of check_gate_lists contradicts the script's header (9-44)
+  and Tools.md:435-437. Fix: trim to a citation, like the other seven comments.
+- A6-3 R2 conv: convert_em_dash_separators.mjs:213-215 has no exit-2 path; matters once it runs in
+  the pre-commit hook (PLAN-10.md:690-693,795-797 deferred exactly that hook).
+- A6-4 R2 struct: nothing reads the workflows (decision 6). DESIGN: new standalone
+  scripts/check_ci_workflows.mjs + shared scripts/lib/gate-roster.mjs (gatesFromBat generalised,
+  check_gate_lists.mjs:111-118). Compares (1) wrapper roster vs each workflow (canonical: test.bat
+  U check.bat - check_tree_fresh U recorded CI-only check_links_diff), (2) the two workflows with each
+  other, (3) load-bearing build flags (--check-audit-index, --no-fetch-assets). Allowlist of recorded
+  deltas: checks.yml's fixture-built link-diff step (checks.yml:114-126); deploy's --url/--baseurl;
+  deploy-only steps. Probes: missing gate, extra step, reordered, missing --check-audit-index, and the
+  allowlisted deltas must NOT fire. Registration -> test.bat + both workflows + Tools.md list grows.
+- A6-5 R3 conv: serve.bat does not propagate the exit code.
+- Q4: composite action (.github/actions/run-gates) as a follow-on AFTER the gate -- not a reusable
+  workflow (separate job loses the built trees the deploy job needs). Would also fold checks.yml's
+  three install steps vs deploy's one.
+- Sound: check_gate_lists exemplary (18 probes); check_publish_policy bidirectional; regex_safety +
+  regex-fold strongest; code_regions + markdown-files strong; convert_em_dash uses markdownFiles and
+  preserves line endings; exit convention followed except A6-3/A5-2; .bat idiom deliberate and
+  cross-referenced; workflow comments point rather than restate. Probe counts: 18, 6, 11, 12, 46, 22, 12.
+- Leads: convert_em_dash --check as a pre-commit entry (closes the PLAN-10 deferral); A4's two
+  selfTest shapes and impexp's test trio are further probe-harness shapes.
+
+## A7 -- compiler harness (done)
+
+- A7-1 R1 dup: tbrun.mjs:327's failed-build test covers 2 of the 5 shapes tb-ide.mjs:730-734
+  BUILD_FAILED lists; misses "[BUILD] ERROR" and "[LINKER] compilation (codegen) error" -> exit 0 on a
+  failed build (the round-8 bug class). ORCHESTRATOR VERIFIED the regexes. Fix: export BUILD_FAILED.
+- A7-2 R2 struct: afterReveal's false (timeout) discarded by openFile/setCursor/select
+  (tb-operate.mjs:421-429, 453, 461, 475) -- reopens the "MsgBox( -> gBox(s" race silently.
+- A7-3 R2 dup: two CDP click primitives: tb-ide.mjs:848-861 clickCenter (no scroll/hit-test/retry,
+  used for buildIcon by buildProject and tbrun.mjs:295) vs tb-operate.mjs:79-134 click (used for
+  restartIcon and by every scenario).
+- A7-4 R2 dup: alive() and norm() in tb-registry.mjs (588-590, 462, private) and addin_test.mjs
+  (105, 230); addin_test already imports 9 names from tb-registry.
+- A7-5 R2 conv: CLI divergences REPRODUCED: tbbuild opt trailing -> undefined -> port NaN, timeout NaN
+  -> waitForCompile exits at once with a misleading message; tbrun/addin_test substitute defaults
+  (also for an explicit ""); tbbuild's positional finder skips a token after ANY --flag, so
+  `tbbuild --json proj` fails (dormant: check_examples.mjs:602 puts the path first); die() three
+  structures. (Feeds L1.)
+- A7-6 R3 conv: tb-registry.mjs:49-50 says tbrun's snapshot uses -EncodedCommand; tbrun.mjs:376 uses
+  -Command (safe today: fixed literal).
+- A7-7 R3 dup: 180 s compile timeout literal at 7 sites in 4 files.
+- A7-8 R2 dup: all ten test/addin scenarios hand-roll the HERE/HOST/lane preamble and skip object;
+  six hand-roll a "console lines since a mark" reader three different ways. Fix: a scenario helper +
+  linesSince beside readConsole.
+- A7-9 R3 struct: tbbuild's shutdown skips finishTidy when ide was never set; safe only by an
+  invariant in tb-launch.ps1 (suspended start); tbrun always tidies.
+- Split: tb-ide.mjs (console reading and add-in introspection separable; build-state core coupled);
+  tb-registry.mjs (file-boundary only); tbrun's reap tail -> tb-reap.mjs (would host A7-6's fix).
+- Sound: strict DAG, no cycles; tb-registry imported only by the three CLIs; stageProject shared;
+  laneProjectId allocator; check_tb_registry independent fixtures; PowerShell payloads are fixed
+  literals with data via env/stdin; job object + suspended-kill path; retry loops recorded + loud.
+
+## A10 -- smaller tools (done)
+
+- A10-1 R3 conv: eval's four parseArgs differ on unknown args and exit codes; transcript.mjs:190-198
+  exits 1 on bare --help.
+- A10-2 [agent R1] dup: wisdom sitemap.mjs:69-87 parseFrontmatter has no BOM strip (discover.mjs
+  94-117 does, after the AppGlobalClassObject incident) and disagrees with prep.mjs:343-388 on
+  coercion. ORCHESTRATOR: no markdown under docs/ has a BOM today -> dormant.
+- A10-3 R2 dup: wisdom sitemap.mjs:61-67 walk() re-implements markdownFiles; its reason ("no
+  dependency on builder/", PLAN-3.md:404) does not reach scripts/lib.
+- A10-4 R2 dead: wisdom/extract/schemas.mjs imported by nothing and drifted from workflow.mjs's inline
+  schemas (source_thread vs thread_path; section string vs enum).
+- A10-5 R2 conv: wisdom manifest.json/denied.json written non-atomically (messages.mjs:10-12,
+  wisdom.mjs:146); loadManifest has no parse guard; Phase 3 uses temp+rename.
+- A10-6 R2 struct: impexp.mjs/impexp.py self-tests (19 each) run by nothing; parity claimed in
+  Tools.md:958, unchecked. Fix proposed: check_impexp_parity.mjs in test.bat -- NEEDS DECISION: it
+  makes test.bat (and CI) depend on Python.
+- Split: none (impexp single-file downloads).
+- Sound: site JS clean IIFEs, svg-inline.js does not duplicate svgInlinePlugin (markup vs runtime);
+  build_fonts.py matches WIP.Fonts.md; eval isolation stack deliberate; merger.mjs; wisdom api paging.
+- Leads: atomic write x4 with inconsistent cleanup (impexp x2, wisdom state/merger); wisdom.mjs
+  switch-shaped parseArgs (another CLI variant).
+
+## DECISIONS since the plan was written
+
+- Decision 1 AMENDED by the user: detach-pages.js STAYS in perf/ (moving it breaks five perf scripts
+  and touches many docs). Instead: two header lines in perf/detach-pages.js (+ one in perf/README.md)
+  saying book.bat and the deploy workflow load it. Nothing moves out of perf/.
+- APPROVED by the user: DELETE the four superseded shims (fast-refs, fast-dict-array, fast-dict-iter,
+  fast-parse-dict). Do it AFTER L3/L4 finish (L4 reads their clone pairs), as its own commit.
+- USER: NO BRANCH. staging is the working branch ("whatever I'm working on right now"), merged to
+  upstream/main by the user when a chunk of work is done. Commit on staging. Revise the plan's
+  "Each phase lands as one pull request" to match: commits on staging, merging upstream is the user's. Touches: the four files; perf/measure.mjs's four
+  flags (+ exclusivity checks, imports); perf/phase0-measure.mjs:28 import -> fast-refs-class (its
+  "production-equivalent" claim is currently false); comments pointing at the files:
+  render-book.mjs:46,57, fast-indirect-objects.mjs:21, fast-parse-object.mjs:39,
+  fast-refs-class.mjs:3,37, fast-sync-load.mjs:80. No published page names them. Closes A9-4, A9-6.
+  Fallback if the user prefers: move into perf/ + path updates in the same two perf scripts.
+
+## L1 -- CLI conventions (done)
+
+36 entry points inventoried (32 argv readers + 4 argument-less gates). Appendix has the table and the
+cli.mjs spec.
+- L1-1 R1 dup: --theme/--viewport validation (check_a11y.mjs:82-95 pick(), fixed after "--theme drak"
+  labelled a light run "drak") missing from sweep_a11y.mjs:120-121 and check_a11y_fingerprint.mjs:
+  134-135 (the axe-upgrade gate). Fix: hoist pick() into axe-scan.mjs or validate in buildMatrix.
+  VERIFY.
+- L1-2 R1 dup: opt() x7 in 4 variants -- adds check_publish_policy.mjs:31-33 (inline, unnamed). NaN
+  from a trailing value flag: tbbuild.mjs:68,70 (port, timeout), check_examples.mjs:102-104 (jobs,
+  port, batch).
+- L1-3 R1 dup/hack: positional detection -- tbrun VALUE_FLAGS (112-116); tbbuild order-dependent
+  find (62): `tbbuild --keep proj` -> usage error.
+- L1-4 R1 hack: tbdocs.mjs:189-191 throws on an unknown argument -> main().catch -> exit 1, the "link
+  check failed" code. Fix: exit 2 for parse errors. VERIFY.
+- L1-5 R2 dead: check_links.mjs:307-314,406-412 tolerate unknown flags "passed through via
+  check.bat's %*" -- check.bat no longer calls it (PLAN-checks.md:14).
+- L1-6 R2 conv: --help in four shapes (stdout/0; stderr/0 in pick_a11y_sample, sweep_a11y; stderr/2
+  via the usage error in tbbuild, tbrun, addin_test; none in 12 tools). gen_attribute_probes takes
+  --help as out_dir and mkdirs "--help/Sources"; render-book rejects --help as unknown (exit 2).
+- L1-7 R2 conv: unknown flag -> ignore (11 tools) / warn (check_links) / err 2 (8) / throw -> 1 or 2
+  / err 1 (wisdom).
+- L1-8 R2 conv: --json boolean-to-stdout (tbbuild, tbrun, check_examples, census) vs value-taking file
+  (check_a11y_fingerprint).
+- L1-9 R3 conv: --src = docs root (tbdocs, check_publish_policy) vs exported package tree (census,
+  build_package_api).
+- L1-10 R3 conv: --name=value only in tbdocs, and not for --check-findings/--symbol-gaps.
+- L1-11 R2 dup: uncaughtException->exit 2 one-liner in check_page_baseline:34, check_book_coverage:27,
+  check_symbol_index:38, check_publish_policy:29; missing in check_tb_registry.mjs (crash exits 1).
+  Tools.md's check_publish_policy entry omits its exit 2. (Overlaps A6-1.)
+- L1-12 R3 dup: "usage; exit by whether help was asked" x6 in eval/ and wisdom.
+- L1-13 R2 struct: nothing tests any tool's argv handling -- the seam that let L1-1/L1-2 ship. The
+  shared module should carry its own ride-along self-test in test.bat.
+- Spec (scripts/lib/cli.mjs): parseCli(argv, {options, positionals}) over node:util parseArgs
+  (strict, allowPositionals, tokens) + camelCase aliases + positional-count checks + tokens pass-
+  through; withUsageError(fn, {onError}) keeps each tool's own message/stream/code; numberOption()
+  closes the NaN hole; printHelpAndExit(text, {stream, exitCode}) keeps each tool's current --help
+  shape until Phase 3. Care: multiple:true for --forbid (check_links), --source (check_tree_fresh),
+  --case (check_links_diff), --additional-script (render-book), --channel (wisdom); tbdocs's
+  cross-flag order (--no-check resets others) needs a walk over tokens -> MIGRATE TBDOCS LAST;
+  strict:false still parses undeclared options; check_links's warn-collect and wisdom's subcommands
+  need custom code (wisdom last or never).
+- Exit schemes: link bitmask (tbdocs, check_links: deliberate, documented); tbbuild 0-4, tbrun 0-3,
+  addin_test/check_examples 0-2; gates 0/1/2; impexp 0-6. Misleading: L1-4; addin_test's 2 means
+  either "harness failed" or "registry not restored".
+- Sound: .bat capture-before-popd x7; check_examples --json keeps stdout clean; impexp.mjs is the
+  best-disciplined CLI (verb Map, EXIT object, usageError) -- the model for Phase 3; census --help
+  prints its own header.
+- Leads: check_regex_safety has an internal --shard re-exec flag (A6); buildMatrix validation (A5);
+  serve.mjs exit codes vs tbdocs (A1).
+
+## L2 -- paths, config, dependencies (done)
+
+- L2-1 R1 dup: two more hand-kept lists of output trees, the exact defect check_tree_fresh was fixed
+  for (WIP.Build.md:376-388): builder/serve.mjs:138 IGNORED_PREFIXES has no _site-basepath* (a
+  check_links_diff --b fused run while serve.bat runs -> spurious rebuild); eval/build_corpus.mjs:59-71
+  EXCLUDED_PATHS has docs/_site-basepath but its prefix test misses _site-basepath-offline/-pdf.
+  60bb6f5 edited serve.mjs's list the same day markdown-files.mjs landed. Fix: isOutputTree. VERIFY.
+- L2-2 R1 dup: census_attributes.mjs:99-119 findInstall re-implements tb-install.mjs:19-35 findIde
+  (whose header exists to prevent a private copy) and diverged: validates packages/ vs twinBASIC.exe;
+  os.homedir() fallback only in the private copy. Fix: use findIde, keep its packages/ check, move the
+  homedir fallback into tb-install.
+- L2-3 R2 dup: = A5-6 (dot.mjs listDotSources vs check_dot_fit findDotSvgs); neither uses
+  isOutputTree -- their broader "_"/"." skip is safe today (no .dot under _data/_sass/_App/_Images).
+  Fix: one filesUnder(root, {ext, skip}) helper; keep the broad skip. MERGE with A5-6.
+- L2-4 R2 dup: = A2-3 readBaseline; plus symbol-baseline.mjs:55-58's hand-rolled writeBaseline emits
+  `"urls": [\n\n  ]` for an empty list where JSON.stringify gives `[]` (agent verified). Four
+  generated JSON files, four serializers (inter-metrics 1-space and package-api hand-built have
+  reasons). MERGE with A2-3.
+- L2-5 R2 dup: repo root derived inline in ~19 files, four expressions; axe-scan.mjs:29 exports
+  REPO_ROOT, imported by 2. Extending.md:654 names the convention. Fix: scripts/lib/repo-paths.mjs
+  (re-exported by axe-scan).
+- L2-6 R3 conv: mkdtemp scratch not removed in a finally: check_publish_policy.mjs:152-189,
+  check_links_diff.mjs:651-750 (three siblings do it right).
+- L2-7 R3: three small tree walkers in offline.mjs (401-416, 608-626) and write.mjs (281-299) -- not
+  proposed.
+- Sound: exactly three real YAML parse sites (tbdocs.mjs:271, check_publish_policy.mjs:69,
+  data.mjs:12), bare yaml.load, each loaded once per build; check_publish_policy's cwd-relative SRC
+  is recorded (Extending.md:654); atomic temp+rename writes where used; tbrun cleans its per-port work
+  dir at the next start on purpose; only picocolors and pako are undeclared; lockfile + npm ci.
+- Leads: Builder.md:412-440 "Dependencies" is stale -- misses recheck, shows wasm-graphviz ^1.21
+  (really ^1.29.1), and says axe-core is the only exact pin (four are: axe-core, pdf-lib, puppeteer,
+  recheck). A REPEAT of a drift the last review fixed (74b3395). Each exact pin has a recorded
+  technical reason; nothing states the policy (exact = we patch or depend on internals; caret =
+  the rest), so carets on output-changing deps look like an accident when they are not.
+
+## L4 -- clone leads classified (done)
+
+37 regions >= 100 tokens + 57 same-named functions, both sides opened.
+NEW:
+- L4-2 R3 dup: scss.mjs compileLightScss/compileDarkScss (65-76, 78-89) same body.
+- L4-3 R2 dup: vendor-assets.mjs fetchToFile (222-252) / fetchAttachment (279-319) each re-implement
+  fetch-with-two-tiers + atomic temp+rename write.
+- L4-7 R2 dup: CommonMark code-span splitting twice -- render.mjs:180-204 maskInlineCode vs
+  convert_em_dash_separators.mjs:74-~100 splitInlineCode; the em-dash tool already shipped a bug in
+  this exact logic (its comment 70-73). Equivalent today. Fix: shared splitOnCodeSpans.
+- L4-10 R1 dup: logicalLines -- tb-fences.mjs:423-445 (private) vs twin-api.mjs:51-~90 (exported);
+  twin-api handles a BOM and multi-line /* */ block comments, tb-fences neither. VERIFY that swapping
+  in twin-api's version keeps tb-fences' quote-aware comment stripping (its reason, :423).
+MERGES: L4-1 = A1-3 (and the 4th, "claimed task" path, 497-540, is legitimately different: its own
+ordering trap 515-525). L4-4 -> A2-1 (buildSitePaths is dead: delete, don't delegate). L4-5 = A2-3.
+L4-6 = A3-3, with two more disagreements: protocol-relative //host (template.mjs skips it; seo.mjs's
+isAbsoluteUrl misses it and prepends baseurl -- masked while baseurl is ""), non-string input (null
+vs ""). L4-8 = isNonEmpty/encodeSpaces/splitFragment/statSafe/makeTimer cluster (A3-4, A2-5, A3-5,
+A4-1, A1-4) -> a low-level util module; route makeTimer so offline.mjs never imports tbdocs.mjs.
+L4-9 = A5-3/A5-4. L4-11/12 = L1-2/L1 (CLI). L4-13 = A6-1/L1-11 (+ withBaseline tmpdir fixture twice:
+check_page_baseline.mjs:46-55, check_symbol_index.mjs:314-323). L4-14 -> A7-8 (probeLines with
+hard-coded .slice(15)/.slice(13)).
+CONFLICTS to settle in verification:
+- findingsFor/buildFindings: L4 says (c) recorded -- mirrored so check_links_diff can cross-check
+  (check.mjs:410-414, check_links.mjs:599-601). SUPERSEDED by the user's option-2 decision (A4); the
+  fix must rewrite those two comments.
+- normalizeBaseurl/escapeRegExpBook: L4 accepts book.mjs:298-300's "by design ... plugins are
+  independent"; A9-8 says that premise (Ruby plugins) expired at the Node cutover -- book.mjs already
+  imports siblings. Side with A9, cite both.
+- escapeHtml 3 vs 5 chars: L4 (c) via highlight.mjs:249-250 (Rouge). Covers highlight's choice, not
+  render.mjs's TOC mixing both -> keep A3-2, narrowed.
+- writeBaseline: L4 says deliberately different (one URL per line); L2-4 says it equals
+  JSON.stringify(...,2) except an empty list. READ the code.
+- alive/norm: L4 (b) "no clone region" (below the window); A7-4 read both: identical. Keep A7-4 (R3).
+- parseFrontmatter: L4 (b) different dialects; A10-2's BOM gap is independent of dialect. Keep,
+  dormant.
+- extractFromHtml: L4 (b) different sizes; A4-3 is a coverage gap (5 vs 20 tag/attr pairs), not a
+  clone. Keep A4-3.
+Sound (b): data tables (GANTT_SECTION~HUMAN, SCOPE_TO_SYMBOL~MUST_REFUSE, FAMILIES, fixtures), main,
+the five selfTest shapes, classify, page, kindOf, zero-pad pad, decode, resolve, strip, serialize,
+select, runAll, printHelp, discover, walk, fmtMs, log, count, substitute, node:test boilerplate.
+(c): check-tree.mjs's inlined fnmatch/excluded (import cost on the dispatch path).
+
+## L3 -- browsers, processes, text rewrites (done)
+
+- L3-1 R1 dup: check_a11y.mjs:167-218 launches the browser with no try/finally (browser.close at 218
+  only on success; main().catch at 244-247 exits 2 without closing); siblings on the same library
+  guard it: check_a11y_fingerprint.mjs:234-248, sweep_a11y.mjs:209-273, check_axe_patch_equiv.mjs:
+  99-115. PAGE_STATES appliers (axe-scan.mjs:117-172) throw by design. Leaks Chromium in CI (Linux);
+  Windows unverified. Fix: withBrowser(fn) in axe-scan.mjs for all four.
+- L3-2 R2 dup: check_examples.mjs:601-612 (buildStaged) spawn has no 'error' handler and awaits
+  'exit' not 'close'; addin_test.mjs:180 and check_regex_safety.mjs:347-348 do both. A spawn failure
+  crashes past finishTidy (registry restore).
+- L3-3 R1 hack: wisdom/extract/merger.mjs:114-129 parseStaging splits on bare '---' with no fence
+  awareness; parseSection (162-168) returns null for a chunk not starting "## " and the caller drops
+  it -- contradicting 111-112 ("never silently drop reviewer content"). staging.md is committed and
+  PLAN-3.md:181-183 puts fenced tb code in every section. Not live today (section count = '---' count).
+- L3-4 R2 dup: three fence/code-span parsers -- render.mjs maskCodeRegions (141-175) + maskInlineCode
+  (180-204), stashCodeFences (1704-1730), convert_em_dash_separators.mjs FENCE_OPEN_RE/CLOSE_RE
+  (59-60) + splitInlineCode (74-99) + convertText (141-157). The info-string guard is only in
+  stashCodeFences; the em-dash tool shares maskCodeRegions' gap. MERGE with A3-1 and L4-7.
+- L3-5 R2 dup: HTML escapers -- minimal (&<>) x3: render escapeHtmlMinimal 2230, highlight escapeHtml
+  251, gantt esc 213; full (&<>"') x4: render escapeHtml 2225, template escText 993-998, template
+  escAttr 999-1001 (== escText), sitemap xmlEscape 106-113. escapeHtml names both classes.
+  html-entities used only to decode (outline.mjs:22). MERGE with A3-2 and A2's lead.
+- L3-6 R3 conv: check_links_diff.mjs:598-601 main()'s try/catch covers only argument parsing;
+  fusedBuild (426-432) and ensureBasePathTree (518-525) spawnSync unguarded -> raw stack trace.
+- Split: render.mjs -- the two fence parsers sit 1,550 lines apart with no cross-reference (evidence
+  for A3's candidate).
+- CONFLICT with A3-6: L3 calls the unguarded whole-page HTML rewrites SOUND -- padEmptyCells,
+  normaliseVoidTags, injectAnchorHeadings, search sanitiseContent/extractSections, seo stripHtml,
+  book.mjs bookChapterTransform steps 2/2b/4 -- because every code path escapes & < > before they
+  run (highlight.mjs:252-253, render.mjs:2231-2233); book.mjs steps 1 and 5 (quote-anchored) use
+  replaceOutsideCode. A3-6 says the same invariant is unstated at the sites and outside the gate.
+  Likely resolution: the invariant holds (so not a live hazard); the finding becomes "state it and
+  probe it", R3 or low R2.
+- Sound: four puppeteer launches consistent where it matters; --allow-file-access-from-files needed
+  where the page fetch()es a sibling file:// resource (A5-5 still right that the dot tools don't say
+  so); share the lifecycle, not the args. Child processes: no shell:true, array args everywhere,
+  killTree by pid, stdin hazard fixed once in runCompiler, PowerShell -EncodedCommand with data via
+  env/stdin, tbrun's -Command a fixed literal; cleanup disciplined across tb-lane, tb-addin,
+  addin_test (SIGINT), panes.test after(), check_tb_registry; runShard is the model; check_links'
+  Worker dispatch; worker-pool. Text: applyPreRenderRewrites brackets its four rewrites; token-scoped
+  md.core rules are a third sound mechanism WIP.Build.md does not name; HTML_COMBINED_RE and
+  IMG_SRC_RE follow the documented shape; other unguarded rewrites target build-generated markers.
+- Leads: document the token-scoped mechanism in WIP.Build.md; book/lib/outline.mjs and
+  postprocesser.mjs are pagedjs-cli ports -- treat as vendored (A9 found them attributed, unmodified);
+  L3-1 needs an empirical browser test.
+
+
diff --git a/builder/REVIEW-TOOLING-fe9ce12b/markdown-inventory.md b/builder/REVIEW-TOOLING-fe9ce12b/markdown-inventory.md
new file mode 100644
index 00000000..9e1fe95b
--- /dev/null
+++ b/builder/REVIEW-TOOLING-fe9ce12b/markdown-inventory.md
@@ -0,0 +1,273 @@
+# Markdown-as-text inventory
+
+Read-only investigation. Nothing in the repository was changed; verification used
+small Node scripts in this folder (`task2_fixtures.mjs`, `task2_timing.mjs`,
+`probe_staging.mjs`, `probe_staging2.mjs`, `verify_navhops.mjs`,
+`verify_navhops2.mjs`, `verify_staging_rigorous.mjs`) run against the repo's own
+`node_modules` (resolved through the scratchpad's junction) and, for two probes,
+against the real committed `wisdom/data/findings/staging.md`.
+
+---
+
+## Task 1 — Inventory
+
+### Legend for "code awareness"
+
+CommonMark rules a fence/inline parser can know about, abbreviated in the table:
+
+- **F** — backtick *and* tilde fences recognised
+- **I3** — up to 3 leading spaces on the opener allowed (a 4-space opener is an indented block, not a fence)
+- **CL** — closing-fence rule: same character as the opener, run length ≥ opener's, line otherwise blank
+- **IS** — info-string rule: a *backtick* fence's info string may not itself contain a backtick
+- **IC** — indented (4-space) code blocks recognised as code
+- **NEST** — a fence nested inside a blockquote (`> `) or list item is still recognised, with the prefix stripped
+- **CS** — inline code spans, multi-backtick runs (`` ``a`b`` ``) matched correctly
+- **HTML** — raw HTML blocks recognised as opaque (not prose)
+- **FM** — YAML frontmatter recognised/stripped before anything else runs
+
+"none" = no rule from this list; "hand-rolled (X, Y)" = only the listed rules, by hand, no markdown-it.
+
+### A — Rewrite sites (mutate source or a file)
+
+| # | Site | Matches | Awareness | Misfires on |
+|---|---|---|---|---|
+| A1 | `builder/render.mjs:141-169` `maskCodeRegions` + `:180-204` `maskInlineCode` | fenced blocks (mask), then per-line backtick/tilde runs (mask) | hand-rolled (F, I3, CL, CS). **No IS** — open regex `/^[ \t]{0,3}(\`{3,}\|~{3,})/` never checks the info string for a backtick. No IC (deliberate, documented gap). No NEST test (works structurally by line-scan, so blockquote/list prefixes pass through as ordinary text — it happens to work for masking purposes but the closing-fence scan doesn't know it's "inside" anything). | A backtick-fenced opener whose info string itself holds a backtick (e.g. `` ```abc`def ``) is accepted as an opener here but refused by A2's `stashCodeFences` (CommonMark-correct) 1550 lines below in the *same file* — the two disagree on the same input. Reproduced directly: `` ```abc`def `` masked by A1, left unprotected by A2, and a `> [!NOTE]` inside it was then rewritten into a live admonition. **Not observed in `docs/` today** — `check_code_regions.mjs` is structurally blind to this class (see its "KNOWN GAP" comments), so a future instance would ship silently. |
+| A2 | `builder/render.mjs:1670` `ADMONITION_RE`, `:1704-1730` `stashCodeFences`, `:1732-1772` `rewriteAdmonitions` | admonition blockquotes (`> [!NOTE]` etc.) outside the A1 mask (deliberately — a fence inside an admonition still carries `> ` markers when A1 has already run and been restored) | hand-rolled (F, CL, IS). No I3 (opener anchored with `[ \t]*`, same as A1). No IC, no NEST tracking beyond the line-scan `stashCodeFences` already does for the reason above. | **This is the site the repo's own comments cite as having already shipped broken**: `docs/Reference/Attributes.md`'s `Description` attribute example (a `[Description("...")]` string built from twinBASIC literals `"\`\`\`basic"` and `"\`\`\`"` used as *sample text*, not real fences — see lines 379-393 of that file) used to close the surrounding fence on the embedded literal instead of the real closer, corrupting every admonition on the rest of the page. **Fixed** by the current line-based `stashCodeFences` (it requires a *standalone* fence-only closing line, which the embedded literals never are, since they sit inside a longer source line). Read today: confirmed the current code handles this specific page correctly. The class of bug is real and documented (`check_code_regions.mjs`'s own `ADMONITION_PROBES` reproduce five variants), just not currently tripped by this exact page. |
+| A3 | `scripts/convert_em_dash_separators.mjs:59-60` `FENCE_OPEN_RE`/`FENCE_CLOSE_RE`, `:74-99` `splitInlineCode`, `:135-166` `convertText` | same fence-boundary shape as A1 | hand-rolled (F, I3, CL, CS). **Same missing-IS gap as A1** — literally the same regex shape, independently written. No IC (the file's own comment states this as a known, accepted gap, citing `Reference/Core/Get.md`/`Option.md`'s 2-space list-item fences by name). | Verified against the real corpus: `Get.md` and `Option.md` do carry exactly the cited 2-space fences (confirmed at `docs/Reference/Core/Get.md:49-51,69-73`), each nested under a bullet item. This tool's own header comment records that its *inline* splitter (`splitInlineCode`) already shipped one dash-inside-a-doubled-backtick-span bug and was fixed — the defect class has a track record in this exact file, not just a theoretical one. |
+| A4 | `scripts/check_examples.mjs:1123-1154` `applyMarkers` | one exact fence-opener *line*, located by an already-markdown-it-derived line number (`fence.line`, from `tb-fences.mjs`'s `collectFences`, which itself uses `md.parse`) | **hybrid**: the *location* is markdown-it-correct; the *edit* is a guarded plain-text splice — reads the line, requires it to match `/^[ \t]*(?:>[ \t]*)*(\`{3,}\|~{3,})tb[ \t]*$/` (NEST via blockquote-prefix tolerance built in) before touching it, else refuses with a named error. CRLF-preserving by construction (captures/reattaches a trailing `\r` per modified line). | Low risk by design — it only ever touches a line it can re-verify, and refuses rather than guessing. Included because it duplicates "split file into lines, edit one, rejoin, preserving CRLF" logic that a shared module should provide once rather than leave as a one-off. |
+| A5 | `wisdom/extract/merger.mjs:114-160` `parseStaging`, `:412-435` `serializeStaging` | splits `staging.md` into chunks on any line that is *exactly* `---`, then requires each chunk to start with `## ` | **none** — no fence rule at all, of any kind. `parseSection` (`:162-168`) `return`s `null` for a chunk not starting `## `, and the caller (`:150-154`) silently drops it — which directly contradicts the file's own header comment ("the parser is forgiving... we never silently drop reviewer content", `:106-113`). | **Confirmed live in the real, committed `wisdom/data/findings/staging.md`** — see the dedicated section below. This is the most severe finding in the inventory: real, on-disk section corruption today, not a hypothetical. |
+
+### B — Read-only scan sites
+
+| # | Site | Matches | Awareness | Misfires on |
+|---|---|---|---|---|
+| B1 | `builder/census_attributes.mjs:377-392` `documentedAttributes` | `docs/Reference/Attributes.md`, per-line `/^Syntax:\s*\*\*\[(\w+)/` and `/^Applicable to:\s*(.*)$/` | none | No fence exclusion at all. Dormant today: the page's own `Description`-attribute example (lines 366-406) demonstrates the *convention* using `"### Syntax"` (a heading, inside a string literal) rather than the literal string `Syntax:`, so it does not trip this scanner as written. A future example demonstrating the `Attributes.md` page format itself (plausible — it is meta-documentation about attributes) would. |
+| B2 | `scripts/gen_attribute_probes.mjs:66-90` `parseAttributes` | same file, near-identical `/^Syntax:\s*(.*)$/` + `/^Syntax:\s*\*\*\[(\w+)/` + `/^Applicable to:\s*(.*)$/`, line-by-line | none | Same exposure as B1 — this is an independent, near-duplicate implementation of the same scan (**a third hand-written `Attributes.md` parser**, after B1 and A2's shared subject matter), with its own CRLF-normalisation comment ("Python reads in text mode... Match that, or every line carries a trailing CR") showing it was ported from a different predecessor than B1. |
+| B3 | `scripts/check_gate_lists.mjs` (whole file: `gatesFromBat` `:111-118`, `sectionBody` `:121-128`, `gatesFromDoc` `:138-145`, `commandRuns` `:164-180`, `splitSections` `:251-264`, `subjectWrapper` `:273-289`, `proseClaims` `:297-351`) | `README.md` + every `docs/Documentation/*.md` page: batch-file step lists, and five regex shapes over prose (possessive/verbal/line-initial/section-total/back-reference gate-count claims) | **none** — `splitSections` treats *any* line matching `/^#{1,6}\s/` as a new section, full stop; nothing excludes a fenced code block first. | **Confirmed live**: `docs/Documentation/Wisdom.md:305` is a fenced example of the `staging.md` format (see A5) whose first line, `## docs/Reference/VB/Form/index.md · after-remarks`, matches the heading regex. `splitSections` genuinely splits `### staging.md format` into two phantom sections at that point — verified with a direct `awk` sweep of every fenced region in `README.md` and `docs/Documentation/*.md` for a line matching `^#{1,6} `; this is the only hit. It does not currently flip a verdict (the phantom section's body happens to contain no `check.bat`/`test.bat` count phrase), but the tool's internal model of "what section am I in" is provably wrong at that point today, for a reason that has nothing to do with gate counts. |
+| B4 | `builder/counts.mjs:112-116` `countAttributeAnchors` | `Reference/Attributes.md`'s **raw** `page.rawContent`, `/^\{: #[a-z0-9]+ \}/gm` | none (regex over raw source, not the rendered/masked form used elsewhere in the same file) | Narrow pattern, low realistic collision surface; no fence in `Attributes.md` currently contains a line of that exact shape. |
+| B5 | `builder/counts.mjs:130-140` `countEnumerations` | `Reference/Enumerations.md`'s raw content: split on `/^## Alphabetical index\s*$/m`, then on `/^#{1,6} /m`, then counts `/^- \[/gm` | none — the function's own comment admits it: "it scans one page's raw markdown, so a change to that page's list formatting moves the number." | No fence currently sits inside that page's "Alphabetical index" section, so dormant. |
+| B6 | `wisdom/extract/sitemap.mjs:69-87` `parseFrontmatter` | any `docs/Reference/**/*.md`: `content.startsWith('---')`, `indexOf('\n---', 3)`, then per-line `/^(\w[\w_]*)\s*:\s*(.+)$/` | hand-rolled (**FM**, partial — no BOM strip, unlike `discover.mjs`; no YAML lists/multi-line values, only quoted scalars) | Dormant today — "no markdown under `docs/` has a BOM" was checked (matches the AppGlobalClassObject incident's fix). Live the moment an editor reintroduces one, which `WIP.md` documents as something Windows editors do "without being asked." Also, `sitemap.mjs:61-67`'s `walk()` re-implements `scripts/lib/markdown-files.mjs`'s directory walk from scratch (does not skip `docs/_site*` etc. by the shared rule) rather than importing it — a second, independent copy of "which folders are output trees." |
+| B7 | `wisdom/extract/prep.mjs:343-388` `parseThreadFrontmatter` | Discord-thread `.md` frontmatter under `wisdom/data/threads/`: same `---`-delimited-block approach as B6 plus inline-array and boolean/number coercion | hand-rolled (**FM**, partial — no BOM strip; a *different* dialect from B6: arrays and type coercion B6 doesn't do) | Two independently hand-written, behaviourally-diverging frontmatter parsers (B6, B7) for what is conceptually one problem. A thread title or tag value containing an unquoted `:` (plausible in harvested Discord content) would misparse under the same `/^(\w[\w_]*)\s*:\s*(.+)$/` per-line rule both share. |
+| B8 | `eval/run_case.mjs:99-110` `evaluatorProtocol` | `eval/protocol.md`: `lines.indexOf("---")` for the start marker, `lines.findIndex(l => l.startsWith("## For the orchestrator"))` for the end, slices between | none | Checked the real file: exactly one `---` line (line 10) and zero fences in `eval/protocol.md` today, so dormant — but the mechanism is the same "first bare marker line, no code awareness" shape as A5/B3, applied to the document that becomes the evaluator's own system prompt. A fenced example added before the `---` or containing one would silently move the boundary and could leak orchestrator-only instructions into what the evaluator reads (or vice versa). |
+| B9 | `eval/nav_hops.mjs:86-94` `hrefs` | every page's link targets, after stripping fenced blocks with `/^(\`\`\`\|~~~)[^\n]*\n[\s\S]*?^\1[ \t]*$/gm` | **hand-rolled but deliberately fence-aware** (F, CL) — the *only* site in the inventory outside the markdown-it-based ones that tries. Verified it correctly handles CRLF (JS regex `$` in multiline mode treats a bare `\r` as its own line terminator, so `[ \t]*$` matches before it — tested directly). **No I3** (opener anchored at column 0, no indent tolerance) and **fixed-length-3 only** (`` ``` `` / `~~~` exactly, not "3 or more" — CommonMark's actual rule), so it cannot recognise a 4+-backtick fence or one indented for a list/blockquote. | Measured precisely against every page under `docs/`: markdown-it finds 1,371 real fences; this regex recognises 1,353 of them (≈98.8%), missing 11 indented and 3 four-backtick fences. Of the 3 real fences anywhere in `docs/` whose content contains `](` (link-shaped text), **zero** are among the missed ones — so no live misfire today, but the gap is real and of the same shape already known to bite this corpus (the same 2-space list-item fences A3 documents by name). |
+| B10 | `test/addin/symbols.test.mjs:57-58` `declaredIn`/`firstLine` | markdown returned live by the twinBASIC compiler's LSP hover (rendered from a `[Description(...)]` attribute string, see A2/`Attributes.md`), `/^## \*\*\w+\*\*[^\`\r\n]*\`in ([\w.]+)\`/m` | none | Low risk in practice — the regex only needs the *first* heading line, and every convention-following `Description` puts the heading first (`## **Name** ... \`in Module\``), before any fenced example could appear. Included for completeness since it is literally "markdown held as a string, regexed with no fence awareness," per the task's own phrasing, sourced from a live compiler response rather than a file. |
+
+### C — Checked, out of scope (not markdown-as-text, or not markdown at all)
+
+For completeness, sites the same searches surfaced and were read, then excluded:
+
+- `eval/build_corpus.mjs` — copies `.md` files byte-for-byte (CRLF→LF only) into the evaluator's corpus; no structural parsing.
+- `eval/site_search.mjs`, `eval/transcript.mjs` — read the built `search-data.json` / stream-JSON transcripts; no markdown involved.
+- `scripts/check_links.mjs`, `check_links_diff.mjs` — operate on the *built* `docs/_site` HTML tree via `builder/link-check.mjs`, never on markdown source.
+- `scripts/pick_a11y_sample.mjs`, `sweep_a11y.mjs` — `split("\n")` over their own JSONL result logs, not markdown.
+- `scripts/check_regex_safety.mjs` — scans regex *literals inside `.mjs` source files* for ReDoS safety; mentions `Attributes.md`/`Tools.md` only in a comment.
+- `builder/highlight-theme.mjs:196-208` `parseTheme` — parses a TextMate-style `Name: value;` theme file, a different text format entirely, not markdown.
+- `builder/symbols.mjs:110-124` `headingsOf` — scans **rendered HTML** (`` tags) with a hand-written linear scan rather than regex (explicitly to avoid `O(n^2)`); real, but not markdown source, so out of the task's stated scope. Same for `render.mjs`'s post-render whole-page HTML rewrites (`padEmptyCells`, `normaliseVoidTags`) and `template.mjs`'s `injectAnchorHeadings` — HTML text, not markdown.
+- `wisdom/process/render.mjs:67-70` `truncate` — takes the first line of a Discord message for a reply-quote preview (`.split('\n')[0]`); not structural, no heading/fence interpretation attempted.
+- `wisdom/process/thread.mjs`, `wisdom/discord/api.mjs` — read/write JSON and generate `.md`, never scan existing markdown for structure.
+- `wisdom/extract/state.mjs` — pure JSON state, no markdown.
+
+### D — Positive precedent (already token/parser-based)
+
+- `scripts/lib/tb-fences.mjs:295-328` `collectFences` — `const md = new MarkdownIt({ html: true })`; walks `md.parse(src, {})`'s tokens for `type === "fence"`, using `t.map` for page line numbers.
+- `scripts/check_code_regions.mjs:75-90` `codeRegions` — same bare-instance pattern; walks `fence`/`code_block`/`code_inline` tokens for its before/after diff gate.
+- `builder/discover.mjs:94-117` `parseFrontmatter`/`stripBom` — `gray-matter`, with an explicit BOM-strip in front of it because `matter.test()` doesn't do that itself.
+- `builder/counts.mjs:242-249` `findCountRefs` (+ `:287-303` `validateCountNames`) — reuses `render.mjs`'s `maskCodeRegions` rather than re-deriving "what is code."
+- `builder/counts.mjs:322-327` `findSurvivingPlaceholder` — scans **rendered HTML** with the ``/`
` leading-alternation guard (`WIP.Build.md`'s documented safe shape for an HTML-level rewrite).
+- `builder/render.mjs:860-1173` (`standaloneIalForwardPlugin`, `tightLooseListPlugin`, `looseDeflistPlugin`) — these register as `md.core.ruler` passes that run *after* block parsing and consult `token.map` to index back into `state.src.split("\n")` only for already-parsed, already-located tokens. Not a "site" in the same sense (no independent fence-detection logic — they consume markdown-it's own structural output), included as the architecture the rest of the file's two fence parsers (A1/A2) do not follow.
+
+### The `staging.md` finding, in detail
+
+This is the strongest concrete result in the inventory: real, on-disk corruption in the
+**already-committed** `wisdom/data/findings/staging.md` (15,553 lines), caused exactly
+by A5's lack of fence/blockquote awareness. Traced with markdown-it against the real file
+(`verify_staging_rigorous.mjs`):
+
+Real source, `wisdom/data/findings/staging.md:14238-14251` (unedited excerpt):
+
+```
+14238: ## docs/Reference/VBA/Strings/Len.md · after-remarks [DUPLICATE? -- see also thread ...]
+14239:
+14240: > [!NOTE]
+14241: >
+14242: > In twinBASIC x64 builds, `Len()` and `LenB()` return different values...
+14243: >
+14244: > ```tb
+14245: ReDim arr(LenB(udt) - 1)
+14246: ```
+14247:
+14248: _Source threads: 1314412625714479125 · confidence: high_
+14249: _Date range: 2024-12-06_
+14250:
+14251: ---
+```
+
+Line 14245 is missing its blockquote continuation marker (`> `) — a real authoring slip in
+a fenced sample nested inside a `> [!NOTE]` admonition. Per CommonMark, an unprefixed line
+cannot lazily continue a blockquote-wrapped fence, so markdown-it does **not** close the
+fence at line 14246 (which itself has an empty info string and is not part of the same
+fence at all): it opens a *new*, unclosed fence there that runs for 52 lines, to line 14297,
+swallowing the real meta lines (14248-14249), the real section-separator `---` (14251), and
+the entire next heading and its body as literal fence content.
+
+`parseStaging` knows nothing of this. Its splitter still cuts a chunk at line 14251 (a
+literal `---`, which is all it checks), producing a section headed `## docs/.../Len.md ·
+after-remarks [DUPLICATE? ...]` whose parsed body is truncated to 7 lines ending mid-fence
+(`"\`\`\`"`, confirmed) — and whose `_Source threads:_`/`_Date range:_` metadata
+(`1314412625714479125` / `2024-12-06`) belongs to the **earlier, unrelated** "ReDim arr"
+section, not to any of the six thread IDs the heading itself names. Re-running
+`extract --merge` today would serialize this wrong pairing back to the committed file.
+`raw bare --- lines: 1160` and `parsed.sections.length: 1160` agree only because the
+splitter always produces exactly one section per `---` it sees, whether or not that `---`
+was a real boundary — count-matching is not evidence of correctness here, and a naive
+same-shape check (a manual fence-open/close toggle with no CommonMark closing-length rule)
+independently produced a *misleading* "100 bare dashes inside a fence" figure before this
+markdown-it-based re-check narrowed it to the one genuine, confirmed case above — which is
+itself a small demonstration of the task's point: even a "fence-aware" hand check can get
+the wrong answer.
+
+---
+
+## Task 2 — Can markdown-it supply the regions?
+
+Tested against the repo's own `markdown-it` (from `node_modules`, both a bare
+`new MarkdownIt({ html: true })` instance — matching `tb-fences.mjs`/`check_code_regions.mjs`
+— and the site's configured `createMarkdownIt` from `builder/render.mjs`, imported by file
+URL; importing it has no observable side effect, its top-level code is declarations only).
+
+**(a) Fence/`code_block`/`html_block` tokens with `.map`, top level.** Yes. A top-level fence
+`` ```tb\nDim x\n``` `` parses to a `fence` token with `map=[2,5]` (half-open line range,
+0-indexed) — confirmed the range covers exactly the opener, content and closer lines.
+
+**(b) Same, inside a blockquote and inside a list item — does `.map` cover the *prefixed*
+source lines?** Yes to both, tested separately and combined. A fence inside a plain list
+item (`- item\n\n  \`\`\`tb\n  Dim x\n  \`\`\`\n`) gets its own correctly-nested `fence`
+token (`map=[2,5]`) under `bullet_list_open` / `list_item_open`, same shape as top level.
+For the prefix question specifically: `` > ```tb\n> Dim x\n> y = 1\n> ``` `` gives
+`map=[2,6]`, and indexing the *original, prefixed* source at that range yields
+`["> \`\`\`tb", "> Dim x", "> y = 1", "> \`\`\`"]` verbatim — the map is in terms of raw
+source lines, prefix included, while `token.content` is separately given already unwrapped
+(`"Dim x\ny = 1\n"`, `>` markers stripped). Both are available from one parse. Same result
+one level deeper still (fence inside a blockquote inside a list item): `map` correctly
+spans just the fenced lines with their combined `"   > "` prefix intact in the raw slice.
+
+**(c) Inline code span positions.** Confirmed absent, exactly as the task assumed:
+`code_inline` tokens carry `map=null` always (only block-level tokens get a `.map`), plus
+`.content` (the text between the backtick delimiters) and `.markup` (the delimiter run
+itself, e.g. `` ` `` vs `` `` ``) — no character offset into the source line. A correct
+inline splitter therefore cannot be "ask markdown-it"; it has to be the same manual
+backtick/tilde-run scan `maskInlineCode` (A1) and `splitInlineCode` (A3) already implement
+independently — markdown-it can supply *which source lines* an inline run occupies (via the
+parent `inline` token's own `.map`), but not offsets within them.
+
+**(d) Frontmatter.** Nothing in either instance (bare or site-configured — identical
+output) handles it, and it is actively **misparsed**, not just ignored: given
+`"---\ntitle: X\npermalink: /y\n---\n\n# Heading\n"`, both instances produce `hr` for the
+first `---`, then treat `"title: X\npermalink: /y"` followed by the second `---` as a
+**setext H2 heading** (`heading_open map=[1,4]`, inline text `"title: X"` + softbreak +
+`"permalink: /y"`), then the real `# Heading` as a second, separate H1. This confirms
+`discover.mjs`'s approach is necessary, not optional: strip a BOM, then hand the *whole*
+source to `gray-matter` (or an equivalent frontmatter splitter) **before** any markdown-it
+call, exactly as it does — never lean on markdown-it to recognise frontmatter itself.
+
+**Timing** (`task2_timing.mjs`, single-threaded, this machine): 912 markdown files under
+`docs/` (via `markdownFiles`, output trees already excluded), 3.98 MiB total source.
+
+| parse | total | per-file mean |
+|---|---|---|
+| full `md.parse` (block + inline + core rules) | 234-257 ms | 0.26-0.28 ms |
+| block-only (`md.block.parse` called directly, bypassing the `inline`/`linkify`/`replacements`/`smartquotes` core rules) | 33.7 ms | 0.037 ms |
+
+Block-only is genuinely block-only, not merely faster for some other reason — verified
+directly: after calling `md.block.parse()` alone, every `inline`-type token's `.children`
+is still the empty array `[]` (markdown-it's `Token` constructor default) and its
+`.content` is the raw, unsplit source text; the "inline" *core rule* — the thing that
+actually walks a line for `` `code` ``/`**bold**`/etc. and would double as A1/A3's job if it
+exposed offsets, which per (c) it does not — never ran. Full-parse per-file distribution:
+min 0.012 ms, median 0.092 ms, p95 1.10 ms, max 5.50 ms. Block parsing alone is ~7x cheaper
+and is sufficient for every region-finding need in this inventory (fences, code blocks,
+html blocks, headings for section-splitting) — only inline code-span detection needs
+anything more, and that "more" is the hand-written splitter from (c), not a deeper
+markdown-it call.
+
+---
+
+## Task 3 — The union of needs, and where the module should live
+
+### What every site in Task 1 is actually asking for
+
+| Need | Sites that need it |
+|---|---|
+| Block regions (fence / code_block / html_block / prose) with line ranges, from one real block parse | A1, A2, A3, B3, B4, B5, B9, D (already has it, twice, independently) |
+| A "mask code, rewrite prose, restore" helper built on the above, correctly covering the info-string rule the current one misses | A1, A2, A3 |
+| One tested inline code-span splitter (backtick/tilde-run aware, offsets into the raw line) | A1, A3 |
+| "Split into sections on a marker line, but only outside a code region" | A5 (bare `---`), B3 (`#{1,6}` headings), B8 (`---` twice) |
+| Frontmatter: BOM-strip + `---`-delimited block + YAML parse, one implementation | B6, B7 (currently two, diverging), plus `discover.mjs`/`nav_hops.mjs`'s `gray-matter` calls, which could use it instead |
+| CRLF / byte-exact round-trip (no forced LF normalisation of the caller's *output*) | A3 (has this today, carefully) — the module must not regress it |
+| The existing directory walker, reused rather than re-implemented | B6 (`sitemap.mjs`'s private `walk()` duplicates `scripts/lib/markdown-files.mjs`) |
+
+A fact worth folding into the frontmatter design specifically: `gray-matter@4.0.3` bundles
+its **own** nested `js-yaml@3.14.2` (`node_modules/gray-matter/node_modules/js-yaml`,
+confirmed by reading its `package.json`), while `builder/data.mjs`, `builder/tbdocs.mjs` and
+`scripts/check_publish_policy.mjs` import the top-level `js-yaml@4.1.1` directly for
+`_config.yml`/`_book.yml`. Page frontmatter and site configuration are parsed by two major
+versions of the same YAML library today. A shared frontmatter helper should parse the
+`---`-delimited block with the direct `js-yaml@4` dependency instead of pulling in
+`gray-matter` (and its private v3) at all — one YAML implementation for the whole build.
+
+### API sketch
+
+```js
+// a new shared module, e.g. lib/markdown-regions.mjs + lib/frontmatter.mjs
+
+/** Every fence / code_block / html_block region, in document order, from one
+ *  bare (site-plugin-free) block parse. map is [startLine, endLine) 0-indexed,
+ *  against the raw source exactly as split("\n") would give it — blockquote/
+ *  list prefixes included, per Task 2(b). */
+export function blockRegions(src) // -> {type, map:[a,b], content, info?}[]
+
+/** Generalizes maskCodeRegions (A1) and stashCodeFences (A2) into one,
+ *  CommonMark-correct (fixes the missing info-string rule both A1 and A3
+ *  share) implementation. indented:false matches today's pre-render-rewrite
+ *  choice; true is what check_code_regions.mjs's KNOWN GAP 1 wants instead. */
+export function maskCode(src, { indented = false } = {}) // -> {masked, restore(masked) => src}
+
+/** The one tested backtick/tilde-run scanner inline positions actually need
+ *  (Task 2(c)): splits ONE line into alternating segments. */
+export function splitCodeSpans(line) // -> {code: boolean, text: string}[]
+
+/** Chunks src on lines matching isMarker, using blockRegions to skip any
+ *  marker-shaped line that sits inside a fence/code_block/html_block. Both
+ *  A5's bare "---" and B3's "^#{1,6} " are isMarker predicates over this. */
+export function splitOnMarker(src, isMarker) // -> {heading: string|null, lines: string[]}[]
+
+/** BOM-strip + "---"-delimited block + direct js-yaml v4 (no gray-matter).
+ *  Byte-preserving on `content` (CRLF untouched) per the A3 constraint. */
+export function parseFrontmatter(raw) // -> {data: object, content: string} | null
+```
+
+### Where it should live
+
+The constraint stated in the task is real and current: `builder/render.mjs:383`'s own
+comment — *"Matched against the raw info string rather than parsed with
+scripts/lib/tb-fences.mjs's parseInfo: builder/ must not depend on scripts/"* — rules out
+`scripts/lib/` as the home, because `builder/` is one of the four required consumers. The
+reverse direction is already precedented and fine (`scripts/check_code_regions.mjs` imports
+`../builder/render.mjs` directly), so the module *could* physically live under `builder/`
+— but `wisdom/` and `eval/` importing from the site generator for something as basic as
+"find the fences in a markdown string" is a layering smell in the other direction: neither
+tool has anything else to do with `builder/`, and `builder/` is the actively-churning,
+large module in this repo (per `PLAN-scheduler*.md`) — not somewhere a Discord-harvesting
+tool's frontmatter parsing should have to track.
+
+**Recommended: a new top-level directory, sibling to all four** — e.g. `lib/` at the repo
+root (`lib/markdown-regions.mjs`, `lib/frontmatter.mjs`), imported by relative path from
+`builder/`, `scripts/` (and `scripts/lib/`), `wisdom/` (and `wisdom/extract/`), and `eval/`
+alike. This is the only placement where none of the four is made to depend on another
+subsystem's tree, it matches the existing `scripts/lib/` precedent of "a lib/ folder holds
+cross-cutting non-CLI modules" one level up, and `eval/nav_hops.mjs` already reaches across
+tree boundaries for exactly this kind of shared helper (it imports
+`scripts/lib/markdown-files.mjs` today, by absolute file URL, with a comment explaining why
+— the same import shape this new module would use from `wisdom/`, `eval/`, and `builder/`).
+The one naming risk is visual adjacency to `scripts/lib/`; `mdlib/` is a fine alternative if
+that confusion matters more than the short name.

From 671e6f19d5e3bb08ad82cdc8d5d33fb18d1ac424 Mon Sep 17 00:00:00 2001
From: Kuba Sunderland-Ober 
Date: Fri, 25 Sep 2026 05:09:43 +0200
Subject: [PATCH 07/53] builder: record the review's decisions in
 PLAN-TOOLING-REVIEW.md

---
 builder/PLAN-TOOLING-REVIEW.md | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/builder/PLAN-TOOLING-REVIEW.md b/builder/PLAN-TOOLING-REVIEW.md
index 85d2b335..d54f787e 100644
--- a/builder/PLAN-TOOLING-REVIEW.md
+++ b/builder/PLAN-TOOLING-REVIEW.md
@@ -16,6 +16,10 @@ this file once they are in.
   settled further while the passes ran (below), and the four superseded pdf-lib shims were
   deleted. The passes also raised a theme the review will lead with: markdown is repeatedly
   processed as text, each tool deciding privately what counts as code.
+- 2026-09-25: the review is written, [REVIEW-TOOLING-fe9ce12b.md](REVIEW-TOOLING-fe9ce12b.md),
+  with its evidence beside it in [REVIEW-TOOLING-fe9ce12b/](REVIEW-TOOLING-fe9ce12b/README.md).
+  The owner accepted all seven of its decisions as recommended (listed below). Decision (f),
+  the `staging.md` content slip, is already fixed. Next: the commit plan, added to this file.
 
 ## Decisions
 
@@ -52,6 +56,26 @@ Made on 2026-09-25, before the review started.
 6. **A gate checks that both CI workflows run the same gates as the wrappers**, rather than
    generating the workflows and wrappers from one list.
 
+Taken on the review's recommendations, 2026-09-25; the letters are the review's:
+
+- **(a) One shared markdown module, in a new top-level `lib/`**: markdown-it block regions,
+  one tested inline-code splitter, and a frontmatter splitter on js-yaml 4. gray-matter, and
+  the js-yaml 3 it bundles, are dropped. The five rewriting sites move onto it before the ten
+  scanning sites.
+- **(b) A parity gate for `impexp.mjs` and `impexp.py`**, run unconditionally in CI. Whether
+  `test.bat` then requires Python or reports the gate as skipped, loudly, is settled when the
+  gate is written.
+- **(c) Load-time checks in each pdf-lib shim, and an equivalence test against stock
+  pdf-lib**, modelled on the axe patch and `check_axe_patch_equiv.mjs`.
+- **(d) A composite CI action for the two workflows' shared steps**, after the roster gate of
+  decision 6.
+- **(e) Command lines:** Phase 2 builds `scripts/lib/cli.mjs` on `node:util` `parseArgs` with no
+  change in behaviour; Phase 3 converges on `impexp.mjs`'s discipline (`--help` to stdout and
+  exit 0, errors to stderr and exit 2, one table of exit codes per tool).
+- **(f) The `staging.md` content slip is fixed now**, as data, ahead of the tooling.
+- **(g) The pinning policy is stated** (exact where the code patches a dependency or relies on
+  its internals, caret otherwise), and Builder.md's stale Dependencies section is fixed.
+
 ## Scope
 
 | Area | Size | Treatment |

From d66aaa81bebbbbd966ac45cd12967dc21cc345db Mon Sep 17 00:00:00 2001
From: Kuba Sunderland-Ober 
Date: Fri, 25 Sep 2026 06:15:51 +0200
Subject: [PATCH 08/53] builder: add the tooling review's commit plan to
 PLAN-TOOLING-REVIEW.md

---
 builder/PLAN-TOOLING-REVIEW.md | 1755 ++++++++++++++++++++++++++++++--
 1 file changed, 1689 insertions(+), 66 deletions(-)

diff --git a/builder/PLAN-TOOLING-REVIEW.md b/builder/PLAN-TOOLING-REVIEW.md
index d54f787e..b579e6b6 100644
--- a/builder/PLAN-TOOLING-REVIEW.md
+++ b/builder/PLAN-TOOLING-REVIEW.md
@@ -6,8 +6,9 @@ repetition, and about sound design against hacks. That makes it a different kind
 from [REVIEW-c9f2dfe0-1b6922b.md](REVIEW-c9f2dfe0-1b6922b.md), which asked whether one
 range of commits was correct.
 
-The findings will be written to `REVIEW-TOOLING-fe9ce12b.md`. The commit plan is added to
-this file once they are in.
+The findings are in [REVIEW-TOOLING-fe9ce12b.md](REVIEW-TOOLING-fe9ce12b.md). The commit
+plan that addresses them is under [Execution](#execution), with a coverage table that maps
+every finding to its commit.
 
 ## Status
 
@@ -20,6 +21,13 @@ this file once they are in.
   with its evidence beside it in [REVIEW-TOOLING-fe9ce12b/](REVIEW-TOOLING-fe9ce12b/README.md).
   The owner accepted all seven of its decisions as recommended (listed below). Decision (f),
   the `staging.md` content slip, is already fixed. Next: the commit plan, added to this file.
+- 2026-09-25: the commit plan is written, under [Execution](#execution): 87 commits in seven
+  phases. Planning against the tree turned up ten places where a finding's fix, one of its
+  facts, or a detail of this charter had to change; see
+  [Where this plan departs from the review](#where-this-plan-departs-from-the-review).
+  No commit of it has landed yet; C01 is next. The owner confirmed the four commands that
+  needed it (C05, C08, C09 and C40) the same day, C08 on condition that the hook runs Biome
+  and nothing else.
 
 ## Decisions
 
@@ -234,80 +242,1695 @@ compared before and after:
 
 | Tool | Comparison |
 |---|---|
-| `tbdocs` | Build before and after into scratch `--dest` folders with `--no-fetch-assets`; the three trees must match byte for byte. The first step is to show that two builds of one commit already match. Known differences to exclude: `assets/images/gantt.svg`, which holds the build's own timings, and the commit on the PDF title page. This becomes a committed tool (decision 3). |
+| `tbdocs` | `scripts/compare_trees.mjs` (C02, decision 3): build before and after into scratch `--dest` folders with `--no-fetch-assets`; the three trees must match byte for byte. The first step is to show that two builds of one commit already match. Known differences are normalised, each for a stated reason, rather than excluded: the build's own timings in `assets/images/gantt.svg` and in the copy of that chart inlined into `Documentation/Development/BuildInfo.html` in the online and offline trees, and the commit and date on the PDF title page, which differ only when the two sides are different commits. |
 | gates | The same output on the real tree; and a changed gate must still fail when its original defect is put back. |
 | harness | The `examples.bat` summary unchanged (1,119 samples) and `addin-test.bat` green, against the local BETA 983. Never two harness runs at once. |
-| book | Page count, outline and extracted text unchanged; the PDF's bytes include timestamps. |
-| CI | A workflow dispatch on the fork, for any commit that changes a workflow. |
+| book | Page count, outline and extracted text unchanged; the PDF's bytes include timestamps. From C66, `check_pdf_shims_equiv.mjs`, which compares the shimmed pdf-lib with stock, is the oracle for the shims. |
+| command lines | From C47, `check_cli.mjs`'s recorded cases: each migrated tool's exit code, stream and message for the invocations that stop during argument parsing. |
+| CI | For a commit that changes a workflow or the composite action: a dispatch of `checks.yml` on `origin`, and the deploy workflow's own run on the next push of `staging`. A dispatch of the deploy workflow cuts a GitHub release, so it is not a test. Pushing is the owner's call, so these commits wait for it. |
 
 ## Execution
 
 Each phase lands as commits on `staging`, the working branch, which is merged upstream when a
-chunk of work is done.
-
-**Phase 0: process and oracles**, before any finding is fixed.
-
-- The tree comparison tool (decision 3).
-- The CI-roster gate (decision 6), with probes that make it fail on a deliberately
-  mismatched workflow.
-- The linter, lint rules only (decision 4):
-  - Biome, pinned to an exact version, is the candidate: one package, `npm install` still
-    enough to run everything. Confirm it on the tree before adopting it, counting findings
-    and false positives on the mixed Node and browser code. ESLint is the fallback.
-  - Scope: `builder/`, `scripts/`, `book/`, `eval/`, `wisdom/`, `test/`,
-    `docs/assets/js/`. Excluded: `perf/`, the vendored code, generated JSON (the two
-    baselines, `package-api.json`, `inter-metrics.json`), `package-lock.json`, and every
-    Markdown, SCSS, YAML and `.bat` file.
-  - Correctness rules only. No style rules until Phase 6.
-  - Findings with a mechanical fix are fixed here. A rule whose findings need design work
-    starts disabled, and the phase that fixes them enables it.
-  - A gate in `test.bat` and both workflows; a `pre-commit` hook in a committed
-    `.githooks/`, checking the staged files; the rule in WIP.md and Tools.md. Enabling the
-    hook in a clone sets `core.hooksPath`, and that change to git configuration is
-    confirmed before it is made.
-
-**Phase 1: remove and relocate.** Move `census_attributes.mjs` out of `builder/`; declare
-the two packages; delete the dead code the review identifies. Done ahead of it, during the
-review: the four superseded pdf-lib shims deleted, and `perf/detach-pages.js` marked as
-code the book build loads (decision 1).
-
-**Phase 2: shared code, in place, with no change in behaviour.** Shared modules for what
-the review finds repeated, adopted one tool at a time: argument parsing on `node:util`
-`parseArgs` that preserves every tool's current flags and exit codes, paths and
-configuration, the gates' self-test scaffolding, browser launching, and the helpers
-`builder/` defines twice.
-
-**Phase 3: conventions users see.** Flags, exit codes and help text brought into line,
-with Tools.md updated in the same commit.
-
-**Phase 4: splits.** A separate pass over the split candidates, taken only where the
-evidence supports it (decision 2).
-
-**Phase 5: documentation and measurement.** The Builder.md module map, Tools.md and
-WIP.Build.md brought up to date; the survey re-run and compared with the baseline above.
-
-**Phase 6: formatting** (decision 4), once every fix from the review is in.
-
-- The formatter configured to the majority style and pinned to an exact version, with any
-  style lint rules alongside it.
-- Line endings settled so the check means the same on a CRLF Windows checkout and an LF CI
-  checkout; otherwise every local run flags 118 files. Either a `.gitattributes` for the
-  formatted file types or the formatter's own setting, shown to agree on both platforms.
-- One mechanical commit, listed in `.git-blame-ignore-revs`. The tree comparison shows
-  what it changed in the output; only the two published site scripts should differ.
-- The gate and the hook extended to check formatting; WIP.md and Tools.md updated.
+chunk of work is done. The commits below are numbered in the order they are meant to land. A
+commit that needs a follow-up takes a letter (C07a) rather than renumbering the rest; when one
+lands, its heading gains the hash, and a **Landed** note records anything that differed from
+the entry, as in [PLAN-REVIEW-c9f2dfe0-1b6922b.md](PLAN-REVIEW-c9f2dfe0-1b6922b.md). Line
+numbers are the review's, at `fe9ce12b`, and move as the commits land.
+
+### The organising idea
+
+Three things decide the order.
+
+**Nothing is refactored before the oracles exist.** Most commits below claim to change no
+behaviour, and such a claim is only as good as the comparison behind it. Phase 0 builds the
+tree comparison and shows that two builds of one commit already agree, then the roster gate
+and the linter, before any finding is touched.
+
+**Defects are fixed in place before code is shared.** Phase 2 promises no change in
+behaviour. A defect still present when its tool moves onto a shared module is either
+preserved by the move, where the comparison approves it, or fixed inside the move, where the
+comparison reports a difference that is not a regression. So Phase 1 fixes, in place and
+each against its own oracle, every R1 defect whose fix does not wait on a Phase 2 module, and
+Phase 2's comparisons have one right answer: identical. Twelve of the twenty R1 findings are
+closed by the end of Phase 1, and L3-3's silent drop is made loud there. The other seven are
+duplications whose fix is the shared module itself. Five of them change nothing on today's
+content (A3-1, A3-3, A8-1, A10-2, L4-10), and the other two are conventions rather than wrong
+results: A5-1's ignored typo and A6-1's indentation.
+
+**Rewriters before scanners, `tbdocs` last, the roster gate before the shared action.**
+Decision (a) moves the five markdown rewriters before the ten scanners, because a wrong region
+in a rewriter corrupts committed content. Decision (e) migrates `tbdocs`'s command line last.
+Decision (d) builds the composite action only once the roster gate can check it, and building
+both before the first new gate means every later gate is registered once and checked from
+then on.
+
+### Phase order
+
+| Phase | Commits | Why here |
+|---|---|---|
+| 0: process and oracles | C01–C08 | Every later commit is judged by them. |
+| 1: remove, relocate, fix in place | C09–C30 | Dead code goes before anyone factors it; two moves the later modules need; the defects that need no Phase 2 module, so that Phase 2's comparisons have one right answer. |
+| 2: shared code, no behaviour change | C31–C70 | The review's duplications, one shared module at a time, each adopted under the tree comparison or the tool's own oracle. |
+| 3: conventions users see | C71–C75 | Behaviour changes on purpose, once every tool parses its command line through one module. |
+| 4: splits | C76–C81 | Only where the evidence holds (decision 2). |
+| 5: documentation and measurement | C82–C83 | Written against the code as it then is. |
+| 6: formatting | C84–C87 | Last, so the one mechanical commit touches only code that survived. |
+
+Four commits wait for the owner before a command runs: C05, C09 and C40 change installed
+packages, and C08 changes this clone's git configuration. All four were confirmed on
+2026-09-25, C08 on condition that the hook runs Biome and nothing else. The commits that change a workflow
+or add a gate to one (C03, C04, C06, C47, C62, C66, C70, C87) wait for the owner's next push
+before CI can confirm them.
+
+### Where this plan departs from the review
+
+Planning against the tree turned up places where a finding's stated fix does not fit the code,
+or where the charter's own details were short. None changes a decision; each changes how one
+is implemented.
+
+1. **A command-line error in `tbdocs` cannot exit 2 (L1-4).** `tbdocs` reports link failures
+   as 1, integrity failures as 2 and both as 3 (`tbdocs.mjs:1563-1570`), so a usage error at
+   2 reads as an integrity failure. `check_links.mjs` has the same scheme and already returns
+   2 for its three argument errors (`:384,399,403`), which the review did not list. C18 gives
+   a command-line error one value outside the bitmask, the same in both tools.
+2. **`builder/` cannot import `isOutputTree` (L2-1).** `serve.mjs` is in `builder/`, which
+   must not import `scripts/` (`render.mjs:383`), and `isOutputTree` is in
+   `scripts/lib/markdown-files.mjs`. C12 moves that module into the new top-level `lib/`
+   first. For the same reason `cli.mjs`, onto which decision (e) migrates `tbdocs`, and the
+   repository-root helper, whose nineteen callers (L2-5) include two in `builder/`, go in
+   `lib/` rather than `scripts/lib/`. `lib/` holds modules that any part of the tree may
+   import, and it imports none of them.
+3. **`withBrowser` goes in `scripts/lib/browser.mjs`, not `axe-scan.mjs` (L3-1).** The two
+   diagram tools need the same browser lifecycle (A5-5) and have no other reason to load the
+   accessibility module.
+4. **The tree comparison normalises three regions, not two.** `injectGanttChart`
+   (`tbdocs.mjs:1312`) also inlines the chart into `Documentation/Development/BuildInfo.html`
+   in the online and offline trees, so excluding `assets/images/gantt.svg` alone would fail
+   every comparison. The PDF title page's commit comes from `git rev-parse --short HEAD`
+   (`build-info.mjs`), so it differs only when the two sides are different commits.
+5. **The pinning policy has to cover the linter (decision (g)).** Worded as the review words
+   it (exact where the code patches a dependency or relies on its internals), it does not
+   explain the exact pin decision 4 gives Biome, which patches nothing: the reason there is
+   that a new version changes the gate's verdict on unchanged code. C01 words the policy to
+   include that.
+6. **A dispatch of the deploy workflow is not a test.** It cuts a GitHub release
+   (`tbdocs-gh-pages.yml`'s `release` job). A commit that changes the workflows is checked by
+   a dispatch of `checks.yml` on `origin` and by the deploy workflow's run on the next push of
+   `staging`, which is the owner's to make. The Oracles table now says so.
+7. **The command-line defects are fixed before `cli.mjs` exists (L1-2, L1-3, A7-5).** The
+   review's fix for each is the shared module, but decision (e) makes Phase 2 change no
+   behaviour, so C17 fixes them in place first. L3-3 likewise gets a loud failure in Phase 1
+   (C26), ahead of the fence-aware split (C36).
+8. **`check_tb_registry.mjs`'s missing crash handler changes an exit code** (a crash goes
+   from 1 to 2; L1-11), so it is fixed with A5-2's two tools in Phase 1 (C28), not in the
+   helper commit that must change nothing (C43).
+9. **L1-10 closes as a side effect.** `node:util` `parseArgs` accepts `--name=value` for
+   every option and cannot be told not to, so this is the one behaviour change Phase 2's
+   migrations make, and it only adds a form.
+10. **Three of the review's facts were wrong**, found by proofreading this plan against the
+    source. `check_tree_fresh.mjs`'s `IGNORED_FILES` has three entries, not one only for
+    `census_attributes.mjs`, so C10 removes that entry and keeps the other two (A8-2);
+    `offline.mjs`'s unused re-export block has 32 names, not 24 (A2-1); and
+    `check_examples.mjs`'s probe suite has 119 probes, of which 74 are the inline ones the
+    review counted (L4-10).
+
+## Phase 0: process and oracles
+
+Before any finding is fixed.
+
+### C01 — `docs: state the dependency pinning policy, and correct Builder.md's list`
+
+**Decision (g).** `docs/Documentation/Builder.md`'s Dependencies section omits `recheck`,
+gives `@hpcc-js/wasm-graphviz` as `^1.21` where `package.json` has `^1.29.1`, and names
+`axe-core` as the only exact pin where four are exact (`axe-core`, `pdf-lib`, `puppeteer`,
+`recheck`). The last review fixed the same drift once, in `74b3395`.
+
+**Change.** Correct the section against `package.json`. Give each exact pin its recorded
+reason, cited where it is recorded rather than restated. State the policy in one sentence:
+exact where the code patches the dependency or relies on its internals, or where a new
+version would change a gate's verdict on unchanged code; caret otherwise. From here on, a
+commit that changes `package.json` updates this section in the same commit (see the bar).
+
+**Verify.** Every row re-read against `package.json` and `package-lock.json`; `build.bat`
+for the page.
+
+### C02 — `scripts: compare_trees.mjs, the built trees before and after a change`
+
+**Decision 3.** The oracle for every `builder/` commit below.
+
+**Change.** A tool, not a gate.
+
+- The *before* side is built from a temporary `git worktree` at `--before ` (default
+  `HEAD`), with `node_modules` linked to this checkout's; the *after* side is the working
+  tree. Both run `node builder/tbdocs.mjs --src docs --dest docs/_site-cmp-
+  --no-fetch-assets` with `CI=1` in the environment, so the page and symbol baselines are
+  read and never written (`tbdocs.mjs:1589`); the only other reader of `CI` is the asset
+  fetch, which the flag already turns off. Arguments after `--` go to both builds; C53
+  needs a `--baseurl` build.
+- All three tree pairs are compared file by file: missing, extra and differing files, with
+  the first differing lines of a text file. Exit 0 identical, 1 different, 2 the tool failed.
+- Known differences are normalised, each with a stated reason, never excluded wholesale: the
+  timings in `assets/images/gantt.svg` and in the copy of that chart inlined into
+  `Documentation/Development/BuildInfo.html` (both trees), and the commit and date on the PDF
+  title page.
+- `--keep` leaves the trees and the worktree for inspection; otherwise both are removed.
+  `docs/.gitignore`, which names each output tree, gains the `_site-cmp*` trees.
+- A Tools.md entry; WIP.Build.md names it as the oracle for a `builder/` change.
+
+**Verify.** First the A/A run the charter asks for: `HEAD` against a clean working tree must
+be identical after normalisation. Anything else it finds is either given a reason and a
+normaliser, or fixed as nondeterminism. Then a one-character change to a template must show
+in all three trees. Record how long a comparison takes.
+
+### C03 — `scripts: check_ci_workflows.mjs, the workflows against the wrappers' gates`
+
+**Decision 6, A6-4 (R2).** Nothing reads either workflow to confirm it runs the gates the
+wrappers run. Today the two workflows' twelve shared gate steps are identical and in the same
+order, and the workflows differ only in three recorded ways.
+
+**Change.** As A6-4's design in the ledger:
+
+- `scripts/lib/gate-roster.mjs` generalises `check_gate_lists.mjs`'s `gatesFromBat`
+  (`:111-118`) to read a wrapper or a workflow's `run:` steps. `check_gate_lists.mjs` moves
+  onto it unchanged.
+- `scripts/check_ci_workflows.mjs` compares (1) each workflow with the wrappers' roster:
+  `test.bat` and `check.bat`, less `check_tree_fresh.mjs`, which CI does not need because it
+  builds in the same job, plus the recorded CI-only `check_links_diff.mjs` steps; (2) the two
+  workflows with each other; and (3) the build step's critical flags, `--check-audit-index`
+  and `--no-fetch-assets`. An allowlist holds the recorded deltas, each with where it is
+  recorded: `checks.yml`'s fixture-built link-checker step, the deploy build's `--url` and
+  `--baseurl`, and the deploy-only steps. Order is compared within each wrapper's own gates.
+  CI already interleaves the two wrappers' gates, running `check_axe_patch_equiv.mjs` among
+  `check.bat`'s, and that stays allowed.
+- Probes ride along: a missing gate, an extra step, two gates reordered, a missing
+  `--check-audit-index`, and the allowlisted deltas, which must not fire.
+- Registered in `test.bat`, both workflows, Tools.md's numbered list, which
+  `check_gate_lists.mjs` requires, and WIP.md's gate table.
+
+**Verify.** Clean on the real tree; each probe fails as intended; a scratch copy of
+`checks.yml` with one gate step deleted fails. `check_gate_lists.mjs`'s 18 probes unchanged.
+CI waits for the owner's push.
+
+### C04 — `ci: one composite action for the gates both workflows run`
+
+**Decision (d).** After the roster gate, so the action is checked from its first commit.
+
+**Change.** `.github/actions/run-gates/action.yml` holds the steps both workflows share, in
+their current order, each with its comment, and both workflows call it. `checks.yml` keeps
+its fixture-built step; the deploy workflow keeps its build flags and its deploy steps.
+Whether the setup steps join the action (`checks.yml` installs in three steps, the deploy
+workflow in one) is decided here. `check_ci_workflows.mjs` reads through
+`uses: ./.github/actions/run-gates` and gains two probes: a gate missing from the action, and
+a workflow that stops calling it.
+
+**Verify.** The roster gate and its probes. CI: a dispatch of `checks.yml` on `origin`, and
+the deploy workflow's next run.
+
+### C05 — `lint: Biome, correctness rules only, and the fixes it finds`
+
+**Decision 4**, first half. The linter comes first because moved and deleted code leaves
+unused imports and undeclared names behind, and Phases 1 and 2 move and delete a lot.
+
+**Change.**
+
+- Evaluate first, and record the result in the Landed note. Run Biome's defect-finding rules
+  (its correctness and suspicious groups) over the scope, counting findings and false
+  positives in three kinds of code: Node modules; the two browser scripts in
+  `docs/assets/js/`; and the functions passed to `page.evaluate`, which sit in Node files
+  but run in the browser. If Biome cannot tell these apart without blanket suppressions, use
+  ESLint (`eslint`, `@eslint/js`, `globals`) under the same rules.
+- Install it pinned to an exact version, after the **owner's confirmation**, and add its row
+  to Builder.md's Dependencies.
+- One configuration at the root. Scope: `builder/`, `scripts/`, `book/`, `eval/`, `wisdom/`,
+  `test/`, `docs/assets/js/`. Excluded: `perf/`; the vendored code
+  (`book/lib/paged.browser.js`, `builder/vendor/`); `book/lib/outline.mjs` and
+  `postprocesser.mjs`, which the review found to be attributed, unmodified ports of
+  `pagedjs-cli`, to be treated as vendored; the generated JSON (the two baselines,
+  `package-api.json`, `inter-metrics.json`); `package-lock.json`; every Markdown, SCSS, YAML
+  and `.bat` file. The formatter stays off until Phase 6.
+- Findings with a mechanical fix are fixed here. A rule whose findings need design work
+  starts disabled, with a comment naming the phase that enables it.
+
+**Verify.** Lint clean over the scope. The tree comparison identical, since the fixes touch
+`builder/`. `test.bat` and `check.bat` clean.
+
+### C06 — `scripts: check_lint.mjs, a lint gate in test.bat and CI`
+
+**Decision 4.** The backstop for C08's hook.
+
+**Change.** `scripts/check_lint.mjs` runs the pinned linter over the configured scope and
+follows the gate convention: 0 clean, 1 findings, 2 the linter failed. It sits early in
+`test.bat` (no tree, no browser), and goes in the composite action, Tools.md's numbered list
+and WIP.md's gate table. WIP.md gains the rule: lint before every commit.
+
+**Verify.** Clean on the tree; an unused import exits 1; a broken configuration exits 2. The
+roster gate passes. CI waits for the owner's push.
+
+### C07 — `scripts: convert_em_dash_separators exits 2 on a crash`
+
+**A6-3 (R2).** Its one exit is `process.exit(main())`, with 0 or 1 (`:210,214`), and a crash
+also exits 1, which reads as a finding. The review expected it to run from the pre-commit
+hook; the owner approved that hook for Biome only (C08), so the fix stands on its own: a crash
+should not read as a finding wherever the tool runs.
+
+**Change.** The one-line `uncaughtException` handler four gates already have
+(`check_page_baseline.mjs:34` and its siblings), exiting 2. C43 later folds every copy into
+one helper.
+
+**Verify.** A forced throw exits 2; `--check` over `docs/` exits 0; a planted literal dash
+exits 1.
+
+### C08 — `githooks: a pre-commit hook that runs Biome on the staged files`
+
+**Decision 4.** The owner approved the hook on condition that it runs Biome and nothing else.
+The dash check `PLAN-10.md:690-693,795-797` deferred to a hook, and that A6-3 assumed, stays
+out of it unless the owner asks for it.
+
+**Change.** `.githooks/pre-commit`, run by Git for Windows's own `sh` and by `sh` elsewhere,
+runs the pinned Biome, through `check_lint.mjs`, on the staged JavaScript. Enabling it in a
+clone is `git config core.hooksPath .githooks`. This clone's `.git/config` already sets
+`core.hooksPath`, to the default `.git\hooks`, so enabling it here means changing that value,
+which the owner has confirmed. WIP.md and Tools.md say how to enable it, and that CI runs the
+same check for a clone without it.
+
+**Verify.** A staged file with an unused import is refused; a clean commit passes; a commit
+that stages no JavaScript is not slowed. Time the hook on a typical commit.
+
+## Phase 1: remove, relocate, and fix in place
+
+Done ahead of this phase, during the review: the four superseded pdf-lib shims deleted
+(`90624841`), `perf/detach-pages.js` marked as code the book build loads (`26eefeeb`), and the
+`staging.md` content slip fixed (`e0d127f0`, decision (f)).
+
+### C09 — `deps: declare picocolors and pako, which the code imports directly`
+
+**A1-2 (R1).** `picocolors` is imported by `tbdocs.mjs:35` and `scheduler.mjs:5` and installed
+only through `puppeteer → cosmiconfig → parse-json → @babel/code-frame`; `pako` is imported
+by `book/lib/fast-inflate.mjs` and installed only through `pdf-lib`. The day either chain
+changes, the build stops at an import.
+
+**Change.** Declare both at the versions installed today, under C01's policy: `picocolors`
+with a caret (`^1.1.1`), and `pako` exact (`1.0.11`), because `fast-inflate.mjs` replaces its
+`inflate` at run time. The **owner's confirmation** before `npm install`. Builder.md's
+Dependencies gains both rows.
+
+**Verify.** `npm ls picocolors pako` shows both as direct dependencies at the same versions,
+and `package-lock.json` should change only in its root entry. The tree comparison identical.
+
+### C10 — `scripts: move census_attributes.mjs out of builder/`
+
+**A8-2 (R2).** It imports `../scripts/lib/tb-packages.mjs` (`:79`) against `builder/`'s rule
+that it must not depend on `scripts/` (`render.mjs:383`), and `check_tree_fresh.mjs`'s
+`IGNORED_FILES` (`:57-63`) holds an entry only for it, beside the two baselines.
+
+**Change.** `git mv` it to `scripts/`, beside `build_package_api.mjs`, and fix its imports.
+Update every citation of the `builder/` path: `V3.md` lists eleven, the HTML comment in the
+published `docs/Reference/Attributes.md:588` among them; re-run `git grep census_attributes`.
+Remove its entry from `IGNORED_FILES`, whose other two entries, the two baselines, stay. If
+the linter can express it, a restricted-imports rule for `builder/` turns the rule into a
+guard rather than a comment.
+
+**Verify.** The tree comparison: identical, or differing only in that HTML comment if it
+reaches the page. `check_tree_fresh.mjs` clean. `census_attributes.mjs --json` byte-identical
+before and after (a harness run).
+
+### C11 — `scripts: census_attributes finds the install through tb-install`
+
+**L2-2 (R1).** `findInstall` (`census_attributes.mjs:99-119`) recognises an install by its
+`packages/` folder and `tb-install.mjs`'s `findIde` (`:19-35`) by `twinBASIC.exe`, and only
+the private copy falls back to `os.homedir()` when `USERPROFILE` is unset. `tb-install.mjs`'s
+header exists to prevent exactly this copy.
+
+**Change.** Use `findIde`, keep the `packages/` check as census's own validation of what it
+found, and move the home-folder fallback into `tb-install.mjs`, where every harness tool gets
+it.
+
+**Verify.** A scratch script: `findIde` and census resolve the same install with
+`USERPROFILE` set and unset. `census_attributes.mjs --json` unchanged (a harness run).
+
+### C12 — `lib: move markdown-files.mjs to a top-level lib/`
+
+**Decision (a)'s home**, and the prerequisite for C13: `builder/serve.mjs` needs
+`isOutputTree` and may not import `scripts/`.
+
+**Change.**
+
+- `git mv scripts/lib/markdown-files.mjs lib/markdown-files.mjs`, and update its importers
+  (`check_code_regions.mjs`, `check_tree_fresh.mjs`, `convert_em_dash_separators.mjs`,
+  `scripts/lib/tb-fences.mjs`, and `eval/nav_hops.mjs`'s file-URL import) and every citation,
+  WIP.md's Don't rule among them.
+- A header, or `lib/README.md`, says what `lib/` is for: modules that `builder/`,
+  `scripts/`, `book/`, `eval/` and `wisdom/` may all import, and that import none of them.
+- `lib/` joins every list of the tooling's folders: `check_tree_fresh.mjs`'s
+  `DEFAULT_SOURCES` (`docs` and `builder` today; the build imports `lib/` from C13 on, and an
+  edit there must mark the tree stale), `check_regex_safety.mjs`'s globs,
+  `survey_tooling.mjs`'s `TOOLING_DIRS`, the lint scope, and the prose that names the folders
+  (WIP.md, `test.bat`'s header, Tools.md).
+
+**Verify.** `markdownFiles` returns the same list before and after (a scratch comparison).
+`test.bat` clean, since `check_code_regions.mjs` and the dash tool use it; `check.bat` clean,
+since `check_tree_fresh.mjs` uses `isOutputTree`; an edit under `lib/` makes
+`check_tree_fresh.mjs` refuse the tree.
+
+### C13 — `builder, eval: decide what is an output tree with isOutputTree`
+
+**L2-1 (R1).** `serve.mjs:138`'s `IGNORED_PREFIXES` has no `_site-basepath*`, so a
+`--dest docs/_site-basepath` build while `serve.bat` runs triggers a rebuild;
+`eval/build_corpus.mjs:59-71` lists `docs/_site-basepath` but its prefix test misses the
+`-offline` and `-pdf` trees. `check_tree_fresh.mjs` was fixed for this class once already.
+
+**Change.** Both decide output trees with `lib/markdown-files.mjs`'s `isOutputTree`.
+`serve.mjs` keeps `node_modules` and `.git` in its own list, since those are not output
+trees.
+
+**Verify.** With `serve.bat` running, a build to `--dest docs/_site-basepath` causes no
+rebuild, and a page edit still does. A scratch run of `build_corpus.mjs`'s exclusion test
+excludes all three basepath trees.
+
+### C14 — `builder: delete what the retired diff tools left behind`
+
+**A2-1 / A1-5 / L4-4, A1-4, A2-2 / A9-10 (all R2).** `644d6bdb` deleted `_diff.mjs`,
+`_triage.mjs`, `_sitemap_diff.mjs` and their siblings; code only they called, and comments
+naming them, remain.
+
+**Change.**
+
+- Delete `offline.mjs`'s `writeOfflinePages` (`:244-289`), `writeOffline`'s unread
+  `precomputed` parameter (`:117`), `buildSitePaths` and the fallback that calls it
+  (`:207,213,359-394`; `tbdocs.mjs:705-706` always sets `sitePaths`), and the 32-name
+  re-export block that nothing imports (`:55-88`); `search.mjs`'s `writeSearchData`
+  (`:18-24`); `sitemap.mjs`'s `extractSitemapUrls` (`:70-74`); `pdf.mjs`'s
+  `extractImagePaths` (`:146-158`).
+- Delete `tbdocs.mjs`'s unused `makeTimer` export (`:196-209`). `offline.mjs` keeps its
+  private copy (`:98-111`), with a comment that no longer cites the deleted tools.
+- Delete or correct the comments that name them: `offline-rewrite.mjs:411`,
+  `search.mjs:65-66`, `sitemap.mjs:67-68,97`, `redirects.mjs:35-36`, `pdf.mjs:6-9,99-107`,
+  and `offline.mjs:200-201`, which the review missed; and those that refer to them as "the
+  diff tools" without naming them, `offline.mjs:95-97` and `:364-365`.
+- The documentation rows that name a deleted function (`Pipeline-Stages.md:847` for
+  `extractImagePaths`, and any for the others) change in the same commit.
+
+**Verify.** `git grep` finds no caller of any deleted name, and no mention of the deleted
+tools outside the PLAN and REVIEW records. The tree comparison: identical apart from the
+documentation pages this commit edits.
+
+### C15 — `builder, wisdom: delete precomputeSeo and schemas.mjs; unexport kramdownSlug`
+
+**A3-9, A3-10 (R3), A10-4 (R2).** `seo.mjs`'s `precomputeSeo` (`:90-94`) has no caller;
+`render.mjs`'s `kramdownSlug` (`:1352`) is exported and used only inside its module;
+`wisdom/extract/schemas.mjs` is imported by nothing and has drifted from the inline schemas
+`workflow.mjs` uses (`source_thread` where `:49` has `thread_path`, free text where `:77` has
+an enum).
+
+**Change.** Delete `precomputeSeo` and `schemas.mjs`; drop `kramdownSlug`'s `export`.
+Reviving `schemas.mjs` would mean reconciling it with `workflow.mjs` for no caller.
+
+**Verify.** `git grep` finds no importer before each deletion; the tree comparison identical.
+
+### C16 — `scripts: tbrun recognises all five failed-build shapes`
+
+**A7-1 (R1).** `tbrun.mjs:327`'s pattern matches three of the five shapes that
+`tb-ide.mjs:730-734`'s `BUILD_FAILED` lists. A build that fails with `[BUILD] ERROR` or
+`[LINKER] compilation (codegen) error` is reported as a success.
+
+**Change.** Export `BUILD_FAILED` from `tb-ide.mjs`, and use it in `tbrun.mjs`.
+
+**Verify.** A scratch script: all five shapes match the exported list, and the two missed
+ones fail the old pattern. A probe project with a compile error makes `tbrun` exit non-zero,
+and a clean one still prints its output and exits 0 (harness runs).
+
+### C17 — `scripts: harness CLIs reject a missing value; tbbuild finds its project`
+
+**L1-2, L1-3 (R1), A7-5 (R2).** Three defects in hand-written argument parsing, fixed in
+place so that C49 can migrate these tools without changing what they do:
+
+- `opt()` returns `undefined` for a value flag given last, in `census_attributes.mjs:86`,
+  `check_examples.mjs:93` and `tbbuild.mjs:61`, and `build_package_api.mjs:56`'s one-argument
+  `opt()` has the same gap. `Number(undefined)` then makes `tbbuild`'s `--port` and
+  `--timeout` (`:68,70`) and `check_examples`'s `--jobs`, `--port` and `--batch`
+  (`:102-104`) `NaN`. A `NaN` timeout makes `tb-ide.mjs:436`'s poll run zero times and
+  report that the IDE never opened the project, instead of a timeout.
+- `tbbuild.mjs:62` skips any token after a `--flag`, whether or not the flag takes a value,
+  so `tbbuild --keep proj` and `tbbuild --json proj` report a usage error.
+
+**Change.** A value flag with no value, or a numeric flag whose value is not a positive
+number, is a usage error through each tool's existing usage path. `tbbuild` finds its
+positional argument with a table of the flags that take values, as `tbrun.mjs:112-116` does.
+
+**Verify.** Each malformed invocation exits with the tool's usage code before any IDE starts.
+`tbbuild --keep proj` and `tbbuild --json proj` build (harness runs; the kept IDE is ended by
+its pid). The `examples.bat` summary unchanged: 1,119 samples.
+
+### C18 — `builder, scripts: a command-line error exits outside the link bitmask`
+
+**L1-4 (R1)**, and the same fault in `check_links.mjs`, which the review did not list (see
+departure 1). `tbdocs.mjs` throws on an unknown argument (`:189-191`) and `main()`'s catch
+exits 1 (`:1616-1635`), the "link check failed" bit. `check_links.mjs` exits 2 for its
+argument errors (`:384,399,403`), its integrity bit.
+
+**Change.** In both tools a command-line error exits with one value outside the 1/2/3
+bitmask. 4 is recommended: no run that reaches a check can produce it. Both usage texts and
+Tools.md's exit-code rows say so. Phase 3's convention, where an argument error exits 2,
+records these two tools as the exception, and C60 names the value beside the two bits.
+
+**Verify.** `tbdocs --bogus`, and `check_links.mjs` with no input, both exit 4. The fixture
+build (`test/fixtures/check-src`) still exits with its link and integrity bits, and
+`check_links_diff.mjs --self-test` passes.
+
+### C19 — `scripts: close the browser on every exit path, through lib/browser.mjs`
+
+**L3-1 (R1).** `check_a11y.mjs` launches Chromium (`:167`) and closes it only on success
+(`:218`), and `main().catch` exits 2 without closing it (`:244-247`). The comment at `:229`
+names the path that throws: a `PAGE_STATES` applier's failed assertion, which throws by
+design. On Linux, where CI runs this gate, Chromium stays up for the rest of the job. The
+three sibling tools guard their launch.
+
+**Change.** `scripts/lib/browser.mjs` takes `axe-scan.mjs`'s `launchBrowser` and
+`LAUNCH_ARGS` (`:258-264`), with their reasons, and adds `withBrowser(fn, options)`, which
+closes the browser in a `finally`. `axe-scan.mjs` re-exports `launchBrowser`, so its
+importers need not change. `check_a11y.mjs`, `check_a11y_fingerprint.mjs`, `sweep_a11y.mjs`
+and `check_axe_patch_equiv.mjs` use `withBrowser`. C44 moves the two diagram tools onto the
+same module.
+
+**Verify.** With an applier made to fail, a Chromium started by the run is left over before
+the change and none after. Count by the Puppeteer cache path in each process's command
+line, never by image name, since the owner's own Chrome has the same one. `check_a11y.mjs`'s
+findings unchanged; `check_axe_patch_equiv.mjs` passes.
+
+### C20 — `a11y: validate --theme and --viewport wherever a matrix is built`
+
+**L1-1 (R1).** `check_a11y.mjs:82-95`'s `pick()` was written after `--theme drak` labelled a
+light run "drak"; `sweep_a11y.mjs:120-121` and `check_a11y_fingerprint.mjs:134-135`, the gate
+for an axe upgrade, never received it. `buildMatrix` labels the report from the unvalidated
+string and `gotoPage` applies it; the dark CSS matches only `[data-theme=dark]`, so an
+unknown value renders light under a report that says otherwise.
+
+**Change.** `pick()` moves into `axe-scan.mjs`, beside `THEMES` and `VIEWPORTS`, and all three
+tools use it. `buildMatrix` also refuses an unknown value, as the backstop for a future
+caller.
+
+**Verify.** `--theme drak` and `--viewport tiny` fail with a usage error in all three tools;
+`check_a11y.mjs`'s findings unchanged.
+
+### C21 — `builder: the Gantt chart draws Check, vendorAssets and Other`
+
+**A1-1 (R1).** `gantt.mjs:39-49` draws only `Seeds`, `Spine` (which also takes `Render`)
+and `Write`, so
+`checkBook` and `checkReport` (section `Check`) and `vendorAssets` (no `GANTT_SECTION` entry;
+`tbdocs.mjs:539-562`) are drawn on no build, while their durations still stretch the time
+axis. `COLORS.Other` (`:12`) is never used, and `Builder.md:410` says a task with no section
+falls into an "Other" bucket.
+
+**Change.** `mainSections` gains `Check` and `Other`, and `vendorAssets` gets a
+`GANTT_SECTION` entry.
+
+**Verify.** The built `gantt.svg` names `checkBook`, `checkReport` and `vendorAssets` after
+the change and not before. The tree comparison identical apart from its normalised Gantt
+regions. `Builder.md:410` re-read against the result.
+
+### C22 — `scripts: crawl_check follows every link attribute the build checks`
+
+**A4-3 (R1).** `crawl_check.mjs:91-108`, the only checker that runs against the deployed
+site, handles five tag and attribute pairs, where `link-check.mjs:38-60`'s `LINK_ATTR_TABLE`
+has 21 tags and 26 pairs. It never follows `srcset`, `poster`, `cite`, `formaction`,
+`action`, `data` or `longdesc`.
+
+**Change.** Export `LINK_ATTR_TABLE` and `splitSrcset` from `link-check.mjs`, and let the
+table decide `crawl_check.mjs`'s tag handling; its HTTP concerns (concurrency, redirects,
+HEAD then GET) stay its own. This is separate from decision 5, which covers the two
+filesystem checkers only.
+
+**Verify.** Run it before and after against `serve.bat`'s local server if it takes a base
+URL, and otherwise ask before crawling the live site. The after run requests the `srcset` and
+`poster` targets, and reports nothing the before run did not, apart from any link in those
+attributes that is actually broken.
+
+### C23 — `scripts: check_examples restores the registry after a spawn failure`
+
+**L3-2 (R2)**, with V4's note that `check_examples.mjs` has no process-level handler at all.
+`buildStaged`'s spawn (`:601-612`) has no `'error'` listener and waits for `'exit'`. A spawn
+failure is then an uncaught exception at the emitter, outside both `main()`'s catch and
+`main().catch`, so the step that restores the tbIDE registry never runs.
+
+**Change.** Listen for `'error'` and wait for `'close'`, as `addin_test.mjs:180,185` and
+`check_regex_safety.mjs:347-348` do. An `uncaughtException` and `unhandledRejection` handler
+restores the registry and exits 2.
+
+**Verify.** With the spawn pointed at a missing executable (a scratch edit), the run exits 2
+and the registry is as it was found. The `examples.bat` summary unchanged (a harness run).
+
+### C24 — `scripts: tb-operate stops on an afterReveal timeout`
+
+**A7-2 (R2).** `afterReveal` (`tb-operate.mjs:421-429`) returns `false` on a timeout, and
+`openFile` (`:453`), `setCursor` (`:461`) and `select` (`:475`) discard the result. That
+reopens the cursor-reset race the function exists to prevent: `:401-409` records the measured
+`"xyz"` to `"zy"` corruption.
+
+**Change.** Each call site checks the result, retries once, then throws naming the file and
+position.
+
+**Verify.** `addin-test.bat` green, all ten lanes (a harness run).
+
+### C25 — `scripts: tbbuild always tidies; correct tb-registry's -Command note`
+
+**A7-9, A7-6 (R3).** `tbbuild`'s shutdown skips its tidy step when the IDE handle was never
+set, which is safe only by an invariant inside `tb-launch.ps1`; `tbrun` always tidies. And
+`tb-registry.mjs:49-50` says `tbrun`'s snapshot uses `-EncodedCommand`, where `tbrun.mjs:376`
+uses `-Command` with a fixed literal.
+
+**Change.** `tbbuild` tidies on every shutdown path, as `tbrun` does. The comment is
+corrected; the code is safe as it stands.
+
+**Verify.** `tbbuild` on a probe, and `tbbuild` given a missing `--ide`, both leave the
+registry as found (harness runs).
+
+### C26 — `wisdom: parseStaging refuses a chunk it cannot place`
+
+**L3-3 (R1)**, the half that needs no shared module. `parseStaging` (`merger.mjs:114-129`)
+splits on any line equal to `---`, and `parseSection` returns `null` for a chunk that does
+not start with `## ` (`:162-168`), which the caller drops (`:147-148,152-153`). A bare `---`
+inside a fenced sample therefore drops the rest of its section and the section's
+`finding_ids` line without a word, against the header's promise never to drop reviewer
+content (`:111-112`). Today's file produces no such chunk.
+
+**Change.** A chunk that does not start with `## ` is an error naming its line. C36 then
+stops a fenced `---` from making such a chunk; this check stays as the guard for any other
+malformed one.
+
+**Verify.** A synthetic file with a `---` inside a fence: before, the tail disappears; after,
+the run fails naming the line. The real `staging.md` parses to the same 1,160 sections and
+serialises to the same bytes as before.
+
+### C27 — `wisdom: write manifest.json and denied.json atomically`
+
+**A10-5 (R2).** `saveManifest` (`wisdom/discord/messages.mjs:10-12`) and `wisdom.mjs:146`
+write with a plain `writeFileSync`, and `loadManifest` parses without a guard, beside
+`wisdom/extract/state.mjs:62-75`, which writes to a temp file and renames it.
+
+**Change.** Both writes use the temp-and-rename write `state.mjs` already has, shared within
+`wisdom/`; a file that does not parse is reported by name.
+
+**Verify.** With the rename made to throw (a scratch edit), the previous file survives
+intact; a truncated manifest gives an error that names it.
+
+### C28 — `scripts: exit 2 on a crash in three tools that exit 1`
+
+**A5-2 (R2), and the `check_tb_registry.mjs` half of L1-11.** The convention
+(`Extending.md:640-648`) keeps 1 for a finding and 2 for a crash. `pick_a11y_sample.mjs`'s
+`discover()` (`:159-169`) throws uncaught on a missing tree and exits 1, the same as a
+coverage gap (`:310`), and it runs in `check.bat`. `build_dot_metrics.mjs` has no catch
+around its browser work, so a crash exits 1, which is also its STALE result.
+`check_tb_registry.mjs` exits 1 for a crash and for a failure alike.
+
+**Change.** The convention's crash handler, as `check_dot_fit.mjs:31-34` has it, in all
+three, exiting 2. C43 folds the handlers into one helper.
+
+**Verify.** In each, a forced crash exits 2 and a real finding still exits 1.
+`check_tb_registry.mjs`'s fixtures pass (a harness run).
+
+### C29 — `test.bat: cite check_gate_lists for the gate's history`
+
+**A6-2 (R1).** `test.bat:34-43` tells the gate's history in a way that neither
+`check_gate_lists.mjs`'s header (about `:30-44`) nor `Tools.md:435-437` supports, and those
+two agree with each other.
+
+**Change.** Trim the comment to a citation of the header, matching the file's other seven
+comments.
+
+**Verify.** Re-read against both accounts; `check_gate_lists.mjs` and
+`check_ci_workflows.mjs` pass.
+
+### C30 — `scripts: tidy check_links_diff's and check_publish_policy's failures`
+
+**L2-6, L3-6 (R3).** `check_publish_policy.mjs:152-189` and `check_links_diff.mjs:651-750`
+remove their scratch folders only on success, where four other tools do it in a `finally`.
+And `check_links_diff.mjs`'s `fusedBuild` (`:426,432`) and `ensureBasePathTree`
+(`:518,525`) call `spawnSync` unguarded, so a failure reaches the terminal as a stack trace
+at exit 1 rather than as the tool's `error:` line at 2.
+
+**Change.** A `finally` for both scratch folders; both spawns report through the tool's error
+path.
+
+**Verify.** A forced failure in each leaves no scratch folder and prints the tool's `error:`
+line with exit 2. `check_links_diff.mjs --self-test` and CI's fixture cases unchanged.
+
+## Phase 2: shared code, in place, with no change in behaviour
+
+Each commit's oracle must show no difference: the tree comparison for `builder/`, a gate's own
+output and probes for a gate, the harness summaries for the harness, and from C47
+`check_cli.mjs`'s recorded cases for a command line. A difference is a regression unless the
+entry names it as intended.
+
+The groups run in the order below. Within the markdown group the rewriters (C32–C36) come
+before the scanners (C37–C41), as decision (a) requires. The command-line migrations end with
+`tbdocs` (C52), as decision (e) requires. C55 comes before C56, which uses it, and C66 before
+the shim refactors it checks.
+
+*The markdown module (decision (a)): C31–C41.*
+
+### C31 — `lib: markdown.mjs and frontmatter.mjs, with their probes`
+
+**Decision (a).** The module the inventory sketched (`markdown-inventory.md`, Task 3), built
+on what it measured: block tokens have `.map` line ranges, blockquote and list prefixes
+included; inline tokens have none; frontmatter must be split off before markdown-it sees a
+page, which otherwise reads it as a rule and a setext heading; a block-only parse of the
+whole corpus takes about 34 ms.
+
+**Change.**
+
+- `lib/markdown.mjs`: `blockRegions(src, {md})`, every fence, code block and HTML block with
+  its line range, from a block-only parse; `maskCode(src, {indented})`, the mask, rewrite and
+  restore helper, with markdown-it's CommonMark opener rules, the info-string rule that
+  `maskCodeRegions` lacks included; `splitCodeSpans(line)`, the one tested backtick and tilde
+  run scanner, since markdown-it gives inline spans no offsets; `splitOnMarker(src,
+  isMarker)`, sections split on a marker line outside any code region; and a line-splice
+  helper that keeps each line's ending, which `convert_em_dash_separators.mjs` and
+  `check_examples.mjs` both do by hand.
+- The regions come from the caller's markdown-it instance when it passes one. `render.mjs`
+  passes the site's, because its plugins (the definition-list one among them) change what
+  counts as a block; other callers get a bare `html: true` instance.
+- `lib/frontmatter.mjs`: `parseFrontmatter(raw)` strips a BOM, splits off the `---` block,
+  parses it with `js-yaml` 4, and leaves the content's bytes untouched.
+- The probes ride along in `check_code_regions.mjs`, which becomes the gate on the module as
+  well as on the rewrites: A3-1's shape (a ```` ```abc`def ```` fence holding a
+  `> [!NOTE]`), fences inside blockquotes and list items, the 16 code-span cases V1 used for
+  L4-7, CRLF input, a BOM, a fenced `---`, and frontmatter that markdown-it would read as a
+  heading.
+
+**Verify.** The probes. Over every page under `docs/`, `blockRegions` finds the same 1,371
+fences as `check_code_regions.mjs`'s own pass over the tokens.
+
+### C32 — `render: the pre-render rewrites ask lib/markdown what is code`
+
+**A3-1 (R1), first half.** `maskCodeRegions` (`render.mjs:141-175`) accepts a backtick fence
+whose info string holds a backtick, and `stashCodeFences` (`:1704-1730`) refuses it, as
+CommonMark does. Reproduced: ```` ```abc`def ```` is protected by one and exposed to the
+other, and a `> [!NOTE]` inside it becomes a live admonition. `check_code_regions.mjs` cannot
+see this class of fault.
+
+**Change.** `applyPreRenderRewrites` masks through `maskCode` and `splitCodeSpans`, with the
+site's instance and `indented: false` as today. `maskCodeRegions` and `maskInlineCode` go;
+`counts.mjs`'s `findCountRefs`, which reuses the mask, follows.
+
+**Verify.** The tree comparison identical, since no page holds the shape.
+`check_code_regions.mjs` clean, its mirror-fault probes included.
+
+### C33 — `render: admonitions find their fences through lib/markdown`
+
+**A3-1, second half.** `rewriteAdmonitions` protects fences with its own `stashCodeFences`.
+It runs after the mask is restored, so it must recognise a fence inside an admonition with
+its `> ` markers still on; `blockRegions`' ranges include those prefixes.
+
+**Change.** `stashCodeFences` goes, and `rewriteAdmonitions` skips the ranges `blockRegions`
+reports. A3-1's reproduction becomes a probe in `check_code_regions.mjs` against the whole
+pre-render chain.
+
+**Verify.** The tree comparison identical. `check_code_regions.mjs`'s `ADMONITION_PROBES`
+(five variants, `Attributes.md`'s literal fence strings among them) and the new probe pass;
+putting back a private opener test fails the new one.
+
+### C34 — `scripts: convert_em_dash_separators reads code regions from lib/`
+
+**L3-4, L4-7, merged into A3-1.** Its `FENCE_OPEN_RE` (`:59-60`) is byte for byte
+`maskCodeRegions`' pattern, with the same gap, and `splitInlineCode` (`:74-99`) is a second
+copy of the code-span scan, equivalent today; the tool's own comment (`:70-73`) records a bug
+it already shipped in that scan.
+
+**Change.** Code regions from `blockRegions`, code spans from `splitCodeSpans`, line endings
+kept by the module's splice. The file's accepted gap for two-space list-item fences
+(`Reference/Core/Get.md`, `Option.md`) closes, because markdown-it recognises those fences.
+
+**Verify.** `--check` over `docs/` exits 0 before and after. A seeded probe file with CRLF
+endings, a doubled-backtick span and a list-item fence has only its prose converted, and
+keeps its endings.
+
+### C35 — `scripts: check_examples' marker splice uses lib/markdown's line splice`
+
+**Inventory site A4.** `applyMarkers` (`check_examples.mjs:1123-1154`) already finds its line
+through markdown-it and re-verifies it before editing; only the split, edit and rejoin that
+keeps CRLF is private.
+
+**Change.** The splice comes from `lib/markdown.mjs`; the re-verification stays.
+
+**Verify.** An equivalence run in a scratch script: the old and new splice give identical
+files for every `tb` fence opener in `docs/`.
+
+### C36 — `wisdom: parseStaging splits only on real section boundaries`
+
+**L3-3 (R1), second half.** After C26 a fenced `---` makes the run fail instead of dropping
+content; this makes it not a boundary at all.
+
+**Change.** `parseStaging` splits with `splitOnMarker(src, line => line === "---")`, and
+`serializeStaging` writes back through the same module.
+
+**Verify.** The real `staging.md` parses to the same 1,160 sections, with the same headings
+and metadata, and serialises to the same bytes as before. C26's synthetic file now keeps its
+section's tail and metadata.
+
+### C37 — `scripts: check_gate_lists' sections ignore fenced headings`
+
+**Inventory site B3.** `splitSections` (`:251-264`) starts a section at any line matching
+`/^#{1,6}\s/`, so the fenced `staging.md` example at `docs/Documentation/Wisdom.md:304-305`
+splits `### staging.md format` in two. No verdict changes today only because the phantom
+section states no gate count.
+
+**Change.** Sections through `splitOnMarker`, so a heading-shaped line inside a fence starts
+none. A probe: a fenced `## ` line inside a section that states a count.
+
+**Verify.** The gate's verdict and its 18 probes unchanged; the new probe fails with the old
+splitter.
+
+### C38 — `scripts: one Attributes.md reader for census and the probe generator`
+
+**Inventory sites B1, B2.** `census_attributes.mjs`'s `documentedAttributes` (`:377-392`) and
+`gen_attribute_probes.mjs`'s `parseAttributes` (`:66-90`) read `docs/Reference/Attributes.md`
+line by line with near-identical patterns and no fence exclusion, so a future example showing
+the page's own `Syntax:` format inside a fence would be read as an attribute. Both are in
+`scripts/` since C10.
+
+**Change.** One reader in `scripts/lib/`, over `blockRegions`, that skips fenced lines; both
+tools use it.
+
+**Verify.** A scratch script: both tools' parsed attribute lists identical before and after on
+the real page, and a fenced `Syntax:` line ignored.
+
+### C39 — `builder, eval: counts, run_case and nav_hops skip code`
+
+**Inventory sites B4, B5, B8, B9.** `counts.mjs`'s `countAttributeAnchors` (`:112-116`) and
+`countEnumerations` (`:130-140`) apply regexes to raw source. `eval/run_case.mjs`'s
+`evaluatorProtocol` (`:99-110`) cuts `eval/protocol.md` at the first bare `---` and a
+heading, and the text it cuts becomes the evaluator's system prompt. `eval/nav_hops.mjs`'s
+`hrefs` (`:86-94`) strips fences with a regex that recognises 1,353 of the corpus's 1,371;
+the ones it misses are indented or use four backticks. None misfires today.
+
+**Change.** Each finds its lines through `blockRegions`.
+
+**Verify.** The tree comparison identical (the counts); `evaluatorProtocol` returns the same
+text; `hrefs` returns the same links for every page.
+
+### C40 — `builder, eval: frontmatter through lib/frontmatter; drop gray-matter`
+
+**Decision (a)'s frontmatter half.** `gray-matter@4.0.3` bundles its own `js-yaml@3.14.2`,
+while `data.mjs`, `tbdocs.mjs` and `check_publish_policy.mjs` parse the configuration with
+`js-yaml@4.1.1`: page frontmatter and site configuration go through two major versions of one
+library.
+
+**Change.** `discover.mjs` (`:94-117`, which strips the BOM itself because `matter.test()`
+does not) and `eval/nav_hops.mjs` use `parseFrontmatter`. `gray-matter` leaves
+`package.json` and Builder.md's Dependencies, after the **owner's confirmation** for
+`npm uninstall`.
+
+**Verify.** Every page's parsed frontmatter deep-equal under both parsers, and the tree
+comparison identical. Compare before uninstalling, since the before side needs `gray-matter`;
+rebuild once after.
+
+### C41 — `wisdom: read pages and threads through lib/`
+
+**A10-2 (R1), A10-3 (R2).** Three frontmatter readers disagree: `wisdom/extract/sitemap.mjs:69-87`
+has no BOM strip and no type coercion, `prep.mjs:343-388` coerces arrays, booleans and
+numbers, and `discover.mjs` strips a BOM, after the AppGlobalClassObject incident. No page
+has a BOM today. `sitemap.mjs:61-67`'s `walk()` repeats the markdown walker, and its recorded
+reason, "no dependency on `builder/`" (`wisdom/PLAN-3.md:404`), never applied to a walker
+that depends on nothing.
+
+**Change.** Both readers use `parseFrontmatter`, and `sitemap.mjs` lists its pages with
+`lib/markdown-files.mjs`.
+
+**Verify.** Every page's and every harvested thread's parsed frontmatter deep-equal before and
+after, with each difference resolved on purpose. Two are likely: YAML gives numbers and dates
+where `sitemap.mjs` gave strings, and it refuses an unquoted `: ` inside a value, which a
+harvested Discord title may hold. If a thread file has one, the harvester that writes these
+files quotes its values, and the existing files are fixed in the same commit. The walker
+returns the same files.
+
+*The link checker, the gates' scaffolding, the browser tools and the repository root:
+C42–C46.*
+
+### C42 — `scripts: check_links.mjs becomes a thin wrapper over builder/check.mjs`
+
+**Decision 5, A4-2 (R2), A4-1 (R3).** `check_links.mjs:602-634`'s `buildFindings` repeats
+`check.mjs:422-449`'s `findingsFor`, and `statSafe` is in both (`link-check.mjs:455-457`,
+`check_links.mjs:151-153`). The comments that justify the pair (`check.mjs:410-414`, and
+`check_links.mjs`'s header, `:6-14`) describe two independent implementations for
+`check_links_diff.mjs` to compare, which decision 5 replaces with one implementation read two
+ways.
+
+**Change.** `check_links.mjs` keeps its command line and its own reading of a tree from disk,
+and calls `check.mjs`'s `checkChunk`, `joinChunks`, `findingsFor` and `formatReport` for the
+rest. `buildFindings` and its `statSafe` go. Both comments are rewritten to say what
+`check_links_diff.mjs` now compares: a tree read from disk against the same tree held in
+memory, through one implementation.
+
+**Verify.** `check_links_diff.mjs --a script --b fused` agrees on every case, and its
+`--self-test` passes. `check_links.mjs` prints the same report on `docs/_site` and
+`docs/_site-offline` before and after. CI's fixture cases unchanged.
+
+### C43 — `scripts: lib/gate-probes.mjs for the gates' probes and crash handler`
+
+**A6-1 / L1-11 / L4-13 (R1).** A probe accumulator and report loop in
+`check_page_baseline.mjs` (`:38-44,128-139`), `check_book_coverage.mjs` (`:81-87,158-170`) and
+`check_symbol_index.mjs` (`:40-45,361-370`); a crash handler in those three and in
+`check_publish_policy.mjs:29`; `withBaseline` in `check_page_baseline.mjs:46-55` and
+`check_symbol_index.mjs:314-323`. Only `check_book_coverage.mjs:162` re-indents a multi-line
+detail.
+
+**Change.** `scripts/lib/gate-probes.mjs` holds the accumulator, the report, the crash handler
+and `withBaseline`. Probes stay unconditional, and the exit code for a failed probe is a
+parameter: 1 for these gates, 2 where probes guard a separate sweep. The three gates and
+`check_publish_policy.mjs` adopt it, and so do the handlers C07 and C28 added.
+`check_gate_lists.mjs` and `check_regex_safety.mjs` adopt it only if the fit is exact. The
+re-indenting becomes the shared behaviour: the one change in output, and only in gate text.
+
+**Verify.** Each adopting gate's probe count and verdict unchanged; a broken probe still fails
+it; a forced crash exits 2.
+
+### C44 — `scripts: the dot tools share one launch, host page and source list`
+
+**A5-5, A5-6 / L2-3 (R2).** `check_dot_fit.mjs:76-87` and `build_dot_metrics.mjs:57-68`
+build the same Inter host page and the same launch, and neither explains
+`--allow-file-access-from-files`. `check_dot_fit.mjs:49-67`'s `findDotSvgs` repeats
+`builder/dot.mjs:135-155`'s `listDotSources`, which is not exported.
+
+**Change.** Both launch through `scripts/lib/browser.mjs` (C19), whose file-access option
+adds the flag and states its reason once; the host page is built in one place;
+`listDotSources` is exported and `check_dot_fit.mjs` uses it, keeping the broad `_` and `.`
+skip, which is safe today.
+
+**Verify.** `check_dot_fit.mjs`'s output unchanged; `build_dot_metrics.mjs` regenerates
+`builder/inter-metrics.json` byte for byte; both listings name the same files.
+
+### C45 — `a11y: one page discovery and stub ceiling for the sampler and the sweep`
+
+**A5-3 / L4-9, A5-4 (R2, R3).** `pick_a11y_sample.mjs` (`:122,159-169,184`, `STUB_CEILING`)
+and `sweep_a11y.mjs` (`:78,129-153`, `STUB_TAG_CEILING`) each discover pages, count tags and
+apply a ceiling of 100, though `--propose` reads the JSONL the sweep writes, so the two must
+agree. `pad` and `median` are identical in both.
+
+**Change.** One discovery, ceiling, `pad` and `median`, in `axe-scan.mjs`, which both already
+import.
+
+**Verify.** `pick_a11y_sample.mjs --census` and `--propose` output unchanged; a sweep over two
+pages writes records of the same shape.
+
+### C46 — `lib: one repository root for every tool`
+
+**L2-5 (R2).** Nineteen files derive the repository root inline, in about five different
+expressions; `axe-scan.mjs:29` exports `REPO_ROOT`, and two files import it. `Extending.md:654`
+says nearly all use one expression, and three do.
+
+**Change.** `lib/repo-paths.mjs` exports the root, and the few paths several tools build from
+it. The nineteen use it, `builder/`'s two included, which a `scripts/lib/` module could not
+serve; `axe-scan.mjs` re-exports it for its two importers. `Extending.md:654` states the
+convention as it now is.
+
+**Verify.** The tree comparison identical; `test.bat` and `check.bat` clean;
+`check_examples.mjs --census`, and each `eval/` and `wisdom/` tool's cheapest mode, unchanged.
+
+*Command lines (decision (e)): C47–C52.*
+
+### C47 — `lib: cli.mjs on node:util parseArgs, and check_cli.mjs`
+
+**Decision (e), L1-13 (R2).** Nothing tests any tool's argument handling, which is how L1-1,
+L1-2, L1-3 and L1-6 shipped.
+
+**Change.**
+
+- `lib/cli.mjs`, in `lib/` because `tbdocs` migrates onto it (departure 2), to L1's
+  specification in the ledger: `parseCli(argv, {options, positionals})` over `parseArgs`
+  with `strict`, `allowPositionals` and `tokens`, camelCase names and positional-count
+  checks; `withUsageError(fn, {stream, exitCode})`, which keeps each tool's message, stream
+  and code; `numberOption()`; and `printHelpAndExit(text, {stream, exitCode})`, which keeps
+  each tool's current help behaviour until Phase 3. The options that repeat (`--forbid`,
+  `--source`, `--case`, `--additional-script`, `--channel`) are `multiple`.
+- `scripts/check_cli.mjs`, a `test.bat` gate: the module's own probes, and a table of cases
+  per migrated tool (arguments, exit code, stream, and a pattern for the message), recorded
+  from each tool's behaviour before it migrates. Only invocations that stop during argument
+  parsing qualify. They run as child processes, in parallel, each with a timeout, and with the
+  IDE and browser locations pointed at nothing, so a case that got past parsing fails instead
+  of starting either.
+- Registered in `test.bat`, the composite action, Tools.md's list and WIP.md.
+
+**Verify.** The probes; changing one recorded expectation fails the gate; the roster gate
+passes. CI waits for the owner's push.
+
+### C48 — `scripts: the a11y and diagram tools parse through lib/cli.mjs`
+
+**A5-1 (R1).** Eight hand-written loops, diverged three ways. `check_a11y.mjs:63-79` answers
+`--help` with "unknown arg" and exit 2, where `pick_a11y_sample.mjs:144-147` and
+`sweep_a11y.mjs:106-110` print usage to stderr and exit 0; `check_dot_fit.mjs:47` and
+`build_dot_metrics.mjs:55` test with `.includes()` and ignore a mistyped flag;
+`check_a11y_fingerprint.mjs`, `check_axe_patch_equiv.mjs` and `check_tree_fresh.mjs` print
+usage to stdout.
+
+**Change.** All eight parse through `parseCli`, each keeping today's behaviour, the ignored
+typo included, which C72 removes. Their cases go into `check_cli.mjs` first.
+
+**Verify.** `check_cli.mjs`'s cases for the eight, recorded before and passing after;
+`check.bat` unchanged.
+
+### C49 — `scripts: the harness tools parse through lib/cli.mjs`
+
+**A7-5 (R2)'s `die()` half, and L1-2's copies.** `tbbuild`, `tbrun` and `addin_test` each
+have a `flag()` and `opt()` pair and a `die()`, in three shapes (V3's fifth note);
+`check_examples`, `census_attributes`, `build_package_api`, `gen_attribute_probes` and
+`check_tb_registry` parse by hand as well.
+
+**Change.** All eight through `parseCli`, keeping today's behaviour, C17's fixes included;
+`tbrun` and `addin_test` still substitute their default for an empty value until C72.
+
+**Verify.** `check_cli.mjs`'s cases; the `examples.bat` summary and `addin-test.bat`
+unchanged (harness runs, one at a time).
+
+### C50 — `scripts: the gates and link tools parse through lib/cli.mjs`
+
+**Decision (e); L1-2's fourth variant.** The rest of `scripts/`.
+
+**Change.** `check_links.mjs`, whose collect-and-warn handling of unknown flags stays custom
+code until C72; `check_links_diff.mjs`; `crawl_check.mjs`; `check_publish_policy.mjs`, whose
+inline `opt` is L1-2's fourth variant; `check_regex_safety.mjs`, with its internal `--shard`
+flag; `check_code_regions.mjs`; `check_gate_lists.mjs`; `convert_em_dash_separators.mjs`; and
+`survey_tooling.mjs`. Each keeps today's behaviour.
+
+**Verify.** `check_cli.mjs`'s cases; `test.bat` and `check.bat` unchanged;
+`check_links_diff.mjs --a script --b fused` agrees.
+
+### C51 — `book, eval, wisdom: parse through lib/cli.mjs`
+
+**A10-1, L1-12 (R3).** `render-book.mjs:204-225` parses by hand and rejects `--help`;
+`eval/`'s four parsers differ on unknown arguments and exit codes, and `transcript.mjs:190-198`
+exits 1 on a bare `--help`; "usage, then an exit code chosen by whether help was asked" is
+repeated six times across `eval/` and `wisdom/`.
+
+**Change.** `render-book.mjs` and the four `eval/` scripts through `parseCli`, with
+`printHelpAndExit` replacing the six copies, each keeping today's behaviour. `wisdom.mjs`'s
+subcommands need code the module does not have, so it migrates only if the fit is clean, and
+otherwise stays as it is with a comment saying why.
+
+**Verify.** `check_cli.mjs`'s cases; `book.bat` renders; each `eval/` script's cheapest mode
+unchanged.
+
+### C52 — `builder: tbdocs parses through lib/cli.mjs`
+
+**A1-8 (R3), last, as decision (e) says.** `tbdocs.mjs`'s parser (`:91-194`) is neither
+exported nor tested, and `--no-check` resets other flags, so order matters.
+
+**Change.** The option table moves out of `tbdocs.mjs` into a module that exports it. The
+order-dependent resets read `parseCli`'s tokens in order. `--name=value`, which today works
+for only some of its flags (L1-10), then works for all of them.
+
+**Verify.** `check_cli.mjs`'s cases for `tbdocs`, C18's exit value and the `--no-check`
+ordering included; the tree comparison identical; `build.bat`, `serve.bat` and the CI build
+steps behave as before.
+
+*`builder/`'s helpers, defined twice: C53–C60.*
+
+### C53 — `builder: one URL module`
+
+**A3-3 / L4-6 (R1), A9-8 / A2-6 (R2), A2-5, and A3-5's `splitFragment` (R3).** `seo.mjs:121-148`
+and `template.mjs:917-936` each define `absoluteUrl` and `relativeUrl`, and they disagree
+three ways: a forced leading slash; protocol-relative `//host`, which `seo.mjs`'s scheme-only
+`isAbsoluteUrl` misses and prefixes; and `null` against `""` for a non-string.
+`normalizeBaseurl` is byte-identical in `book.mjs:303-307` and `offline-rewrite.mjs:232-236`,
+and `book.mjs:300-302` justifies its copy by the retired Jekyll plugin layout. `encodeSpaces`
+(`search.mjs:204`, `template.mjs:925`) and `splitFragment` (`render.mjs:1593`,
+`crawl_check.mjs:51`) are each written twice.
+
+**Change.** `builder/url.mjs` with one of each. The two URL helpers take the semantics that is
+right for `//host` and for a non-string, keeping a forced leading slash only where a call site
+needs it. `crawl_check.mjs` imports `splitFragment` from it.
+
+**Verify.** The tree comparison identical, and again with `--baseurl /docs`: a non-empty base
+URL is where the two helpers disagree, and this site's is empty.
+
+### C54 — `builder: one module for the HTML, XML and RegExp escapers`
+
+**A3-2 / L3-5, A2-4 (R2).** Seven HTML escapers of two kinds: `&<>` in `render.mjs:2230-2233`,
+`highlight.mjs:251-254` (matching Rouge, a recorded reason) and `gantt.mjs:213`; `&<>"'` in
+`render.mjs:2225-2228`, `template.mjs:993-998` and `:999-1001` (identical bodies under two
+names), and `sitemap.mjs:106-113`. `escapeHtml` names both kinds, and markdown-it has a third
+function of that name. `render.mjs:1437-1450`'s `headingTocHtml` escapes `text` tokens with
+one kind and `code_inline` tokens with the other. `escapeRegExp` is identical in
+`render.mjs:2235-2237`, `offline-rewrite.mjs:239-241` and `book.mjs:212-214`.
+
+**Change.** `builder/escape.mjs` holds one escaper of each kind, named for what it escapes
+(the three-character one for Rouge parity), and one `escapeRegExp`; every copy imports from
+it. `headingTocHtml` escapes both token kinds alike.
+
+**Verify.** The tree comparison identical, since no heading in a table of contents holds an
+apostrophe or quote today; a scratch page with one renders the same text in the heading and
+in the table of contents. `check_regex_safety.mjs` still recognises the escaper, which it does
+by shape.
+
+### C55 — `builder: one code/pre guard and replaceOutsideCode`
+
+**A9-7 (R2).** The ``/`
` leading alternative that WIP.Build.md prescribes for a
+whole-page rewrite is typed four times: `book.mjs:210` and again inside `:228-229`,
+`pdf.mjs:143-144` (identical to `book.mjs`'s), and at the head of `offline-rewrite.mjs:299`.
+
+**Change.** A `builder/` module exports the fragment and `replaceOutsideCode`, which is
+private in `book.mjs:216` today; the four patterns are composed from the fragment.
+
+**Verify.** The tree comparison identical, covering `book.html` in the PDF tree and every
+offline page. `check_regex_safety.mjs` clean on the composed patterns.
+
+### C56 — `builder: guard code in the three whole-page HTML rewrites`
+
+**A3-6 (R2).** `padEmptyCells` (`render.mjs:74-80`), `normaliseVoidTags` (`:351-354`) and
+`injectAnchorHeadings` (`template.mjs:714-727`, whose `HEADING_REGEX` at `:694` has no code
+alternative) rewrite whole pages with no guard. They are safe only because code is
+entity-escaped before they run, which holds for fenced, indented and inline code and fails
+for hand-written raw HTML, which markdown-it passes through; no page has any today. The
+invariant is stated at none of the three, and `check_code_regions.mjs` checks only the
+pre-render chain.
+
+**Change.** Each rewrite goes through C55's `replaceOutsideCode`, or, if a guard would change
+what it does, states the invariant where it runs. `check_code_regions.mjs` gains post-render
+probes: a raw `
` holding an empty cell, a void tag and a heading-shaped line comes
+through all three rewrites unchanged. WIP.Build.md's section on rewrites names these three,
+and also the token-scoped `md.core` rules, the third sound mechanism it does not yet name (a
+lead from L3).
+
+**Verify.** The tree comparison identical; each probe fails with its guard removed.
+
+### C57 — `builder: one drift guard for the page and symbol baselines`
+
+**A2-3 / L2-4 / L4-5 (R2).** `readBaseline` is byte-identical in `page-baseline.mjs:83-90` and
+`symbol-baseline.mjs:45-52`, and the six-branch drift logic is typed twice
+(`checkPageBaseline`, `:117-176`; `checkSymbolBaseline`, `:75-125`). `symbol-baseline.mjs:55-58`'s
+hand-written `writeBaseline` equals `JSON.stringify(x, null, 2) + "\n"` except for an empty
+list.
+
+**Change.** One module for the read, the drift logic and the write, parametrised by what is
+compared, counts or a set of URLs; the write is `JSON.stringify`.
+
+**Verify.** After a build, and after each `--update-*-baseline`, both committed baselines are
+byte-identical. `check_page_baseline.mjs` (11 probes) and `check_symbol_index.mjs` (46) pass,
+and a reintroduced drift fails each.
+
+### C58 — `builder: fold six small duplicates`
+
+Each written twice, with no recorded reason for the copy:
+
+- **A2-7 (R2):** the ASCII-only whitespace collapse that keeps NBSP, by regex in
+  `compress.mjs:73-84` and by char code in `search.mjs:251-261`. WIP.Build.md records a
+  shipped defect in exactly this area.
+- **L4-3 (R2):** `vendor-assets.mjs`'s `fetchToFile` (`:222-252`) and `fetchAttachment`
+  (`:279-319`) each implement the guarded fetch and the temp-and-rename write. Their
+  validation, which differs for good reason, stays apart.
+- **L4-2 (R3):** `scss.mjs`'s `compileLightScss` and `compileDarkScss` (`:65-76,78-89`).
+- **A3-8 (R3):** three palette loops in `highlight-theme.mjs` (`:336-344,351-359,364-372`).
+- **A2-8 (R3):** `replaceAll("\\", "/")` at ten sites in `offline-rewrite.mjs` and
+  `offline.mjs`, and a private `posix()` in `publish-policy.mjs:189`, become one helper.
+  `check-tree.mjs:46`'s copy stays, for its recorded reason: import cost on the dispatch
+  path.
+- **A3-4 (R3):** `isNonEmpty` in `nav.mjs:338` and `seo.mjs:164`.
+
+**Verify.** The tree comparison identical, the search index, compressed pages and both
+stylesheets included. For the fetch, the stubbed-fetch script from the last review
+(`PLAN-REVIEW-c9f2dfe0-1b6922b.md`, C09) still rejects an HTML body and survives a network
+failure, and every committed thumbnail still validates.
+
+### C59 — `builder: cpu-worker's timed task paths share one runner`
+
+**A1-3 / L4-1 (R2), A1-7 (R3).** The same run, time and report block appears three times in
+`cpu-worker.mjs` (`:360-377,423-440,469-486`). The fourth path (`:497-540`) is genuinely
+different, with an ordering that closes a race, and stays as it is. `:510` writes the literal
+`4` for FAILED, the one SAB constant not used by name.
+
+**Change.** One `runTimed()` for the three paths; `:510` uses the named constant.
+
+**Verify.** The tree comparison identical; the build reports its task timings as before.
+
+### C60 — `builder: name tbdocs's exit bits and set them in one place`
+
+**A1-6 (R2).** Seven sites set exit bits with bare literals
+(`tbdocs.mjs:560,1469,1470,1568-1573,1598,1610`), five of them bit 0. The three plain
+assignments are safe only because they run before the ones that OR (V1's third note).
+
+**Change.** Named constants for the two bits and for C18's command-line value, and one
+`failBuild(bit)` that ORs.
+
+**Verify.** Each provoked failure exits as before: a broken link 1, an integrity failure 2,
+both 3, a command-line error 4, a baseline drift 1. A scratch copy that moves an assignment
+after an OR still exits with both bits.
+
+*The harness: C61–C65.*
+
+### C61 — `scripts: one logicalLines for twinBASIC source`
+
+**L4-10 (R1).** `tb-fences.mjs:423-445` and `twin-api.mjs:51-86` each split source into
+logical lines, and differ on a BOM and on block comments: `tb-fences.mjs` has no `/* */`
+handling at all, and survives a BOM only because `trim()` strips U+FEFF (V3). Both strip
+comments with quotes in mind, for the same reason. Swapping one for the other is not
+mechanical: `twin-api.mjs` keeps blank lines, numbers lines from 1 where `tb-fences.mjs`
+counts from 0, never trims, and recognises `Rem`.
+
+**Change.** One exported `logicalLines` in `twin-api.mjs`; `tb-fences.mjs` uses it and
+absorbs each of the four differences on purpose. Probes for a `/* */` spanning two lines and
+for a BOM join `check_examples.mjs`'s `runProbes`.
+
+**Verify.** `check_examples.mjs --census`, which runs its 119 probes and needs no compiler,
+unchanged apart from the new probes. The `examples.bat` summary unchanged, 1,119 samples (a
+harness run).
+
+### C62 — `scripts: one twinBASIC keyword classifier, with probes in test.bat`
+
+**A8-1 (R1), A8-4 (R2).** `census_attributes.mjs`'s `MODS` (`:138-148`) lacks `Overridable`,
+`Iterator` and `Dim`, which `twin-api.mjs:123-125` and `tb-fences.mjs:340-343` have, so
+`Public Overridable Sub` falls through to the variable path. The BETA 983 packages hold 31
+such lines and none has an attribute, so no census result changes today. Two more
+divergences are structural: `blankStrings` has no `""` escape, and the block-comment state is
+kept per line. Neither this classifier nor `gen_attribute_probes.mjs`'s `parseTargets`
+(`:961-978`) has a test, and both have shipped silent misparses
+(`census_attributes.mjs:42-67,150-152`; `gen_attribute_probes.mjs:946-947`).
+
+**Change.** One classifier module in `scripts/lib/` for the modifier keywords and declaration
+shapes, used by all three. Ride-along probes for it, for census's classification and for
+`parseTargets`, in a new `test.bat` gate, `check_twin_parsers.mjs`, registered in the
+composite action, Tools.md and WIP.md.
+
+**Verify.** `census_attributes.mjs --json` unchanged (a harness run);
+`gen_attribute_probes.mjs`'s output unchanged; removing `Overridable` from the list fails a
+probe. CI waits for the owner's push.
+
+### C63 — `scripts: click the build icon like every other control`
+
+**A7-3 (R2).** `tb-ide.mjs:848-861`'s `clickCenter` has no scroll into view, hit test or
+retry, and its two callers are both the build icon (`tb-ide.mjs:757`, `tbrun.mjs:295`).
+`tb-operate.mjs:79-134`'s `click` has all three and serves four of the ten add-in tests.
+
+**Change.** Both callers use the click with the hit test and retry. The harness modules stay a
+DAG: if `tb-operate.mjs` imports `tb-ide.mjs`, the click moves down to where both can reach
+it. If the build icon turns out to need the plain click, it keeps it, with a comment saying
+why.
+
+**Verify.** `addin-test.bat` green, all ten lanes; the `examples.bat` summary unchanged
+(harness runs, one at a time).
+
+### C64 — `scripts: three small harness duplicates`
+
+- **A7-4 (R2):** `alive` and `norm`, private in `tb-registry.mjs` (`:588-590`, `:462`) and
+  repeated in `addin_test.mjs` (`:105,230`), which imports ten other names from it. Export
+  them.
+- **A7-7 (R3):** the 180-second compile timeout, a literal at seven sites in four files. One
+  named constant.
+- **A8-3 (R3):** `check_examples.mjs` computes a fence's unit key in `makeBatches` (`:424`)
+  and again in `unitsOf` (`:675`). One function.
+
+**Verify.** The `examples.bat` summary and `addin-test.bat` unchanged (harness runs, one at a
+time).
+
+### C65 — `test: one scenario preamble and one linesSince for the add-in tests`
+
+**A7-8 / L4-14 (R2).** All ten `test/addin/*.test.mjs` files write their own lane preamble
+and skip object, and six read "console lines since a mark" in four ways: `appdata.test.mjs:33`
+and `panes.test.mjs:86` with hard-coded slice offsets, `arch.test.mjs:31` and
+`reload.test.mjs:37` with two different regex captures, `keys.test.mjs:28` with a split and a
+trim, and `sample10.test.mjs:81` with a bare trim.
+
+**Change.** A scenario helper for the preamble and the skip object, and one `linesSince`
+beside `readConsole`.
+
+**Verify.** `addin-test.bat` green, all ten lanes (a harness run).
+
+*The book's pdf-lib shims (decision (c)): C66–C69.*
+
+### C66 — `book: check_pdf_shims_equiv.mjs, the shims against stock pdf-lib`
+
+**A9-2 (R2).** No test compares shimmed pdf-lib output with stock; the only comparisons are
+one-off notes in `perf/notes/08-pdf-lib.md`.
+
+**Change.** A `test.bat` gate modelled on `check_axe_patch_equiv.mjs`. The same document is
+loaded, changed and saved by stock pdf-lib in a child process and by pdf-lib with the thirteen
+shims installed here, and the two results are compared object by object, with streams
+decompressed, since a different deflate can give different bytes for the same content. The
+document is generated in the gate to reach every shimmed path (parsing, arrays and
+dictionaries, the parallel deflate, the inflate replacement), so the gate needs no built tree.
+Registered in the composite action, Tools.md and WIP.md.
+
+**Verify.** Passes; a deliberately broken shim fails it, and the report names the shim. CI
+waits for the owner's push.
+
+### C67 — `book: one module for pdf-lib's internal requires`
+
+**A9-5 (R3).** The `createRequire` and `require('pdf-lib/cjs/...').default` block is repeated
+in nine production shims.
+
+**Change.** `book/lib/pdf-lib-internals.mjs`, which the nine import.
+
+**Verify.** `check_pdf_shims_equiv.mjs`; `book.bat` renders with the same page count and
+outline.
+
+### C68 — `book: the two onebuf shims share their range machinery`
+
+**A9-3 (R2).** `_registerContext` and `_appendArray` are identical apart from names in
+`fast-array-onebuf.mjs` and `fast-dict-onebuf.mjs`, while `pack`, `_cow`, `_makeFromRange`
+and `_makeFromAppend` differ by real bit-packing and subclass dispatch.
+
+**Change.** `book/lib/onebuf-range.mjs`, a factory parametrised over the subclass dispatch and
+the gap mask that the two genuinely need differently (V3's fourth note), not one body with
+renamed variables.
+
+**Verify.** `check_pdf_shims_equiv.mjs`; the book's page count and outline unchanged; render
+time within noise of before, since these shims exist for speed.
+
+### C69 — `book: each pdf-lib shim checks what it overwrites`
+
+**A9-1 (R2).** The thirteen production shims each guard against being installed twice and
+never check what they replace, and `parallel-deflate.mjs:50`'s `PDFStreamWriter` subclass has
+no guard at all. Only the exact pin protects them (recorded in `08-pdf-lib.md:1751-1757`),
+and it catches an accidental `npm update`, not a deliberate upgrade or a mistaken edit.
+Upstream is abandoned, and the `@cantoo` fork was evaluated and rejected
+(`08-pdf-lib.md:5048-5061`).
+
+**Change.** At load, each shim asserts the shape of what it overwrites (the member exists,
+its arity, and a fingerprint of its source where one is stable), modelled on `axe-scan.mjs`'s
+`SOURCE_PATCHES` counts, and throws naming itself otherwise. Each header states the exit: the
+shim goes when pdf-lib is replaced, or when a release changes what it patches.
+
+**Verify.** With a target altered in a scratch copy of pdf-lib, the named shim throws at load;
+`check_pdf_shims_equiv.mjs` passes; `book.bat` renders.
+
+*impexp (decision (b)): C70.*
+
+### C70 — `scripts: check_impexp_parity.mjs, the two impexp editions compared`
+
+**A10-6 (R2).** `impexp.mjs` and `impexp.py` each have 19 self-tests, with identical names in
+the same order, run by nothing, and Tools.md promises that the two print the same output and
+write byte-identical files.
+
+**Change.** A `test.bat` gate that runs both `--self-test` suites (the same names, all
+passing) and runs the same commands through both editions on the repository's fixtures
+(`indexer/sample.twinpack`, and a project under `test/example-projects/`), comparing printed
+output and written files byte for byte. Decision (b)'s open question is settled here: whether
+`test.bat` without Python fails, or reports the gate skipped, loudly. In CI a missing
+interpreter fails the gate and never skips it. Registered in the composite action, Tools.md
+and WIP.md.
+
+**Verify.** Passes; a copy of one edition with one output line changed fails it. CI waits for
+the owner's push.
+
+## Phase 3: conventions users see
+
+Decision (e): converge on `impexp.mjs`'s discipline. `--help` prints usage to stdout and exits
+0; an unknown flag or a bad value prints to stderr and exits 2, or C18's value in the two link
+tools; each tool has one table of exit codes. Each commit changes `check_cli.mjs`'s
+expectations, the usage texts and Tools.md together.
+
+### C71 — `scripts, book, eval, wisdom: --help prints usage to stdout and exits 0`
+
+**L1-6 (R2), A5-1's help half, A10-1.** `--help` is handled four ways, and twelve tools ignore
+it. `gen_attribute_probes.mjs:1147-1158` takes a bare `--help` as its output folder and creates
+`--help/Sources`; `render-book.mjs:210-220` rejects it as unknown; `transcript.mjs` exits 1.
+
+**Change.** Every tool prints its usage to stdout and exits 0, with no side effect.
+
+**Verify.** `check_cli.mjs` gains a `--help` case for every tool, safe to run for all of them
+once this lands; no file or folder appears.
+
+### C72 — `scripts, book, eval, wisdom: an unknown flag or a bad value exits 2`
+
+**L1-7, L1-5 (R2), A5-1's typo half, A10-1.** Eleven tools ignore an unknown flag,
+`check_links.mjs` warns, eight refuse it with 2, some throw to 1 or 2, and `wisdom` refuses it
+with 1. `check_links.mjs:307-314` still tolerates flags "passed through via check.bat's %*",
+and the comment at `:406-412` still says `check.bat` passes it arguments; `check.bat` no
+longer calls it (`PLAN-checks.md:14`). `tbrun` and `addin_test`
+substitute their default for an explicit empty value.
+
+**Change.** Strict parsing everywhere: an unknown flag or a bad value is an error on stderr,
+with exit 2, or C18's value in the two link tools. `check_links.mjs`'s tolerance goes, and
+both comments are corrected. Any exception a tool keeps is stated in its usage text and in Tools.md.
+
+**Verify.** `check_cli.mjs` gains an unknown-flag case and an empty-value case for every tool.
+
+### C73 — `scripts: one meaning each for --json and --src`
+
+**L1-8 (R2), L1-9 (R3).** `--json` prints to stdout in `tbbuild`, `tbrun`, `check_examples`
+and `census_attributes`, and takes a file in `check_a11y_fingerprint.mjs`. `--src` is the
+documentation root in `tbdocs` and `check_publish_policy`, and the exported package tree in
+`census_attributes` and `build_package_api`.
+
+**Change.** The odd ones out are renamed: `check_a11y_fingerprint.mjs`'s file option and the
+two package-tree options get names of their own. WIP.A11y.md, WIP.Harness.md and Tools.md
+follow.
+
+**Verify.** `check_cli.mjs`; `git grep` finds no old spelling in the documents or scripts.
+
+### C74 — `scripts: one exit-code table per tool, and no code with two meanings`
+
+**Decision (e).** Each tool's usage text and its Tools.md entry get one table of exit codes.
+A code that means two things is split: `addin_test.mjs`'s 2 covers both a harness that failed
+and a registry it could not restore (L1's notes in the ledger), and the second is the one the
+user must act on.
+
+**Verify.** Each table checked against the code; `check_cli.mjs` for the codes it can reach;
+`addin-test.bat` green (a harness run).
+
+### C75 — `serve.bat: return tbdocs's exit code`
+
+**A6-5 (R3).** `serve.bat` does not pass its child's exit code back, unlike the other
+wrappers, which capture it before `popd`.
+
+**Change.** The same idiom.
+
+**Verify.** A `serve.bat` that cannot start, because its port is taken, exits non-zero.
+
+## Phase 4: splits
+
+Decision 2: a split is taken only where the evidence says good practice calls for it, and
+size alone does not qualify. The test for each candidate is whether the split removes a hidden
+dependency, lets a part be tested on its own, or separates parts that Phases 1 to 3 had to
+change for unrelated reasons. Each is decided at the start of the phase against what the
+earlier phases found, and each is a pure move, checked by the tree comparison or the owning
+tool's oracle. `axe-scan.mjs` is not a candidate: the review found its single-source property
+worth keeping.
+
+### C76 — `render: split along its plugin seams`
+
+The strongest evidence. The image rule order matters and nothing says so (`svgInlinePlugin`
+at `:518` must run before `remoteImagePlugin` at `:520`); the ellipsis plugin assumes the
+dashes plugin has run (`:515-516`); the plugins anchor on a third-party rule name
+(`"curly_attributes"`); no plugin is tested alone; the two fence parsers sat 1,550 lines apart
+until C32 and C33. First each ordering becomes explicit and tested, with a probe that
+registers the plugins in the wrong order and fails; then the plugins move into modules along
+those seams.
+
+**Verify.** The tree comparison identical; each ordering probe fails with its order reversed.
+
+### C77 — `builder: tbdocs's Gantt and timing code moves beside gantt.mjs`
+
+`tbdocs.mjs:1268-1403` computes what `gantt.mjs` draws, and A1-1 is what the separation cost.
+Further candidates, taken on evidence: the `TASKS` literal, which mixes the graph's shape with
+the task bodies over 61 % of the file (`:260-1257`); `dispatch`'s `submit()`, which is SAB
+machinery (`:788-911`); the check glue (`:1085-1256`); the console report (`:1478-1525`).
+
+**Verify.** The tree comparison identical; the chart names the same tasks in the same rows.
+
+### C78 — `builder: template.mjs's date formatter moves to its own module`
+
+**A3-7 (R2).** After C53 and C54 take the URL and escape helpers, the strftime tables and
+`formatDate`/`parseDate` (`:940-989`) are the one job left in the file that is not
+templating. Its `navActivationCss` deferral (`PLAN-4.md` §3) is close to its trigger, and may
+remove more.
+
+**Verify.** The tree comparison identical, every formatted date included.
+
+### C79 — `builder: book.mjs as a resolver, an assembler and a coverage check`
+
+Its §A resolver, its §B–F assembly of `book.html` (with `rewriteBookHrefs`) and its §G
+coverage check are separate concerns, and `pdf.mjs` already imports them as though they were
+separate modules.
+
+**Verify.** The tree comparison identical, `book.html` above all; `check_book_coverage.mjs`
+passes.
+
+### C80 — `scripts: check_examples' bisection and probe suite in their own modules`
+
+Eight jobs share one file; bisection (`:657-927`, 270 lines in 14 functions) and the 425-line
+probe suite (`:1157-1581`) are the clearest to separate. `gen_attribute_probes.mjs` is not a
+candidate: half of it is its own data table.
+
+**Verify.** `check_examples.mjs --census` and the `examples.bat` summary unchanged (a harness
+run).
+
+### C81 — `scripts: tb-ide's console reading and add-in introspection move out`
+
+Both separate cleanly from the build-state core, which stays together.
+
+**Verify.** `addin-test.bat` green and the `examples.bat` summary unchanged (harness runs, one
+at a time).
+
+## Phase 5: documentation and measurement
+
+Written last, against the code as it then is, with every claim re-read against the file it
+describes: the last review found its own figures stale by the time its documentation commit
+ran.
+
+### C82 — `docs: the module map, Tools.md, WIP.Build.md and Extending.md, as they now are`
+
+Builder.md's module map (the new `builder/` modules, and `lib/`); Tools.md's entries for the
+new tools and gates, added in their commits and re-read here; WIP.Build.md, with the markdown
+module as the one answer to what is code and the rewrite rules restated against it;
+Extending.md's conventions (`lib/cli.mjs`, `gate-probes.mjs`, `lib/repo-paths.mjs`); WIP.md's
+gate table and wrapper bullets; and `PLAN-10.md:692`, which still cites
+`convert_em_dash_separators.py` (V2's second note).
+
+**Verify.** Every changed claim re-read against its file; `build.bat` and `check.bat` for the
+anchors.
+
+### C83 — `builder: the tooling survey re-run against its baseline`
+
+`node scripts/survey_tooling.mjs --summary`, recorded as an *after* column in this file's
+baseline survey table, with the reason for each measure's movement. Expected: private
+`flag`/`opt`/`die` copies from 6 to 0, `parseArgs` users from none to every migrated tool,
+undeclared packages from 2 to 0, and clone regions and repeated names well below 571 (266
+outside `perf/`) and 74 (54).
+
+**Verify.** The recorded column re-read against the survey's own output; each measure that did
+not move as expected has its reason written down.
+
+## Phase 6: formatting
+
+Decision 4's second half, once every fix is in, so that the review's citations stayed valid
+while the fixes were made and the formatter touches only code that survived them.
+
+### C84 — `repo: settle line endings for the formatted file types`
+
+The repository stores LF, 118 of 140 working-tree files are CRLF under `core.autocrlf=true`,
+and there is no `.gitattributes`. Either a `.gitattributes` for the formatted types or the
+formatter's own line-ending setting, shown to give the same verdict on a CRLF Windows checkout
+and an LF CI checkout; otherwise every local run flags 118 files.
+
+**Verify.** The formatter's check gives the same verdict in this working tree and in a
+worktree checked out with `core.autocrlf=false`.
+
+### C85 — `lint: the formatter and its style rules, set to the majority style`
+
+The formatter, the linter's own from C05, pinned exactly and configured to the majority style:
+double quotes, as `builder/`, `scripts/`, `test/` and `eval/` use, where `book/` and
+`wisdom/` use single. Any style lint rules go beside it. Nothing enforces it yet.
+
+**Verify.** The formatter's check runs clean on its own configuration and lists the files C86
+will change.
+
+### C86 — `format: apply the formatter`
+
+Mechanical, and nothing else.
+
+**Verify.** The tree comparison shows what it changed in the output: only
+`docs/assets/js/svg-inline.js` and `theme-toggle.js`, which the site ships as written, should
+differ. `test.bat`, `check.bat`, the `examples.bat` summary and `addin-test.bat` unchanged.
+
+### C87 — `lint: check formatting in the gate and the hook; blame ignores C86`
+
+`check_lint.mjs` and the pre-commit hook check formatting too; `.git-blame-ignore-revs` lists
+C86; WIP.md and Tools.md say so.
+
+**Verify.** A misformatted staged file is refused by the hook and fails the gate with 1;
+`git blame` on a file C86 touched skips it. CI waits for the owner's push.
+
+## Coverage
+
+Every finding in the review, mapped to the commit that addresses it. A finding closed by more
+than one commit lists each.
+
+### Findings
+
+| Finding | Tier | Commit |
+|---|---|---|
+| A1-1: the Gantt chart drops Check and `vendorAssets` | R1 | C21 |
+| A1-2: `picocolors` undeclared (and `pako`) | R1 | C09; policy in C01 |
+| A3-1 / L3-4 / L4-7: two fence-opener predicates disagree | R1 | C31–C34 |
+| A3-3 / L4-6: two URL helpers diverged | R1 | C53 |
+| A4-3: `crawl_check` misses seven link attributes | R1 | C22 |
+| A5-1: argument loops in eight scripts | R1 | C48; C71, C72 |
+| A6-1 / L1-11 / L4-13: the gates' scaffolding copied | R1 | C43; handlers first in C07, C28 |
+| A6-2: `test.bat`'s account of the gate's history | R1 | C29 |
+| A7-1: `tbrun` misses two failed-build shapes | R1 | C16 |
+| A8-1: census's keyword list lacks `Overridable` | R1 | C62 |
+| A10-2: three frontmatter readers disagree | R1 | C41; module in C31 |
+| L1-1: theme and viewport checked in one tool of three | R1 | C20 |
+| L1-2: `opt()` returns `undefined`, then `NaN` | R1 | C17; C49, C50 |
+| L1-3: `tbbuild --keep proj` fails | R1 | C17 |
+| L1-4: a `tbdocs` usage error exits as a link failure | R1 | C18 |
+| L2-1: two output-tree lists miss `_site-basepath*` | R1 | C13, after C12 |
+| L2-2: census's private install finder | R1 | C11 |
+| L3-1: `check_a11y` leaves Chromium running | R1 | C19 |
+| L3-3: `parseStaging` drops content after a fenced `---` | R1 | C26 (loud), C36 (fence-aware) |
+| L4-10: two `logicalLines` | R1 | C61 |
+| A1-3 / L4-1: a run, time and report block three times | R2 | C59 |
+| A1-4: `makeTimer`'s export unused | R2 | C14 |
+| A1-6: exit bits as bare literals | R2 | C60 |
+| A2-1 / A1-5 / L4-4: dead paths left by the diff tools | R2 | C14 |
+| A2-2 / A9-10: comments naming deleted tools; `extractImagePaths` | R2 | C14 |
+| A2-3 / L2-4 / L4-5: the baseline reader and drift guard | R2 | C57 |
+| A2-4 / A3-5's `escapeRegExp` part: `escapeRegExp` three times | R2 | C54 |
+| A2-7: the whitespace collapse twice | R2 | C58 |
+| A3-2 / L3-5: seven HTML escapers | R2 | C54 |
+| A3-6: three unguarded whole-page rewrites | R2 | C56, after C55 |
+| A3-7: `template.mjs` does four jobs | R2 | C53, C54, C78 |
+| A4-2: the findings translation twice | R2 | C42 |
+| A5-2: two tools crash to exit 1 | R2 | C28 |
+| A5-3 / L4-9: page discovery and stub ceiling twice | R2 | C45 |
+| A5-5: the diagram tools' launch and host page twice | R2 | C44; lifecycle from C19 |
+| A5-6 / L2-3: the `.dot` walker twice | R2 | C44 |
+| A6-3: the dash tool has no exit for a crash | R2 | C07; C43 |
+| A6-4: nothing reads the workflows | R2 | C03, C04 |
+| A7-2: `afterReveal`'s timeout discarded | R2 | C24 |
+| A7-3: two click primitives | R2 | C63 |
+| A7-4: `alive` and `norm` twice | R2 | C64 |
+| A7-5: the harness CLIs' defaults and `die()` | R2 | C17; C49 |
+| A7-8: the add-in scenario preamble ten times | R2 | C65 |
+| A8-2: census in `builder/` | R2 | C10 |
+| A8-4: two classifiers without probes | R2 | C62 |
+| A9-1: the shims are not guarded | R2 | C69 |
+| A9-2: no shim equivalence test | R2 | C66 |
+| A9-3: the onebuf helpers twice | R2 | C68 |
+| A9-6: shims only `perf/` loaded, in `book/lib/` | R2 | resolved in `90624841` |
+| A9-7: the code and pre guard four times | R2 | C55 |
+| A9-8 / A2-6: `normalizeBaseurl` twice | R2 | C53 |
+| A10-3: wisdom's own walker | R2 | C41 |
+| A10-4: `schemas.mjs` dead | R2 | C15 |
+| A10-5: wisdom's state files not written atomically | R2 | C27 |
+| A10-6: impexp parity unchecked | R2 | C70 |
+| L1-5: `check_links`' stale pass-through tolerance | R2 | C72 |
+| L1-6: `--help` handled four ways | R2 | C71 |
+| L1-7: unknown flags handled five ways | R2 | C72 |
+| L1-8: `--json` with two meanings | R2 | C73 |
+| L1-13: no tests of argument handling | R2 | C47 |
+| L2-5: the repository root derived in nineteen files | R2 | C46 |
+| L3-2: `check_examples`' spawn has no `'error'` listener | R2 | C23 |
+| L4-3: the vendoring fetch and write twice | R2 | C58 |
+| A1-7: FAILED written as a literal | R3 | C59 |
+| A1-8: `tbdocs`' parser untested | R3 | C52 |
+| A2-8: path separators flipped at ten sites, `posix()` twice | R3 | C58 |
+| A3-4 / A2-5 / A3-5 / A4-1, the L4-8 cluster: four leaf helpers written twice (its fifth, `makeTimer`, is A1-4) | R3 | C58 (`isNonEmpty`), C53 (`encodeSpaces`, `splitFragment`), C42 (`statSafe`) |
+| A3-8: three palette loops | R3 | C58 |
+| A3-9: `precomputeSeo` dead | R3 | C15 |
+| A3-10: `kramdownSlug` exported | R3 | C15 |
+| A5-4 / L4-9: `pad` and `median` twice | R3 | C45 |
+| A6-5: `serve.bat` drops the exit code | R3 | C75 |
+| A7-6: the `-EncodedCommand` comment | R3 | C25 |
+| A7-7: the 180-second literal at seven sites | R3 | C64 |
+| A7-9: `tbbuild`'s shutdown can skip its tidy step | R3 | C25 |
+| A8-3: the fence unit key twice | R3 | C64 |
+| A9-4: `_writeUint` and `_digitCount` twice | R3 | resolved in `90624841` |
+| A9-5: the `createRequire` block in nine shims | R3 | C67 |
+| A9-9: a comment citing a moved file | R3 | resolved in `90624841` |
+| A10-1: `eval/`'s four parsers | R3 | C51; C71, C72 |
+| L1-9: `--src` with two meanings | R3 | C73 |
+| L1-10: `--name=value` only in `tbdocs` | R3 | C47–C52, as a side effect of `parseArgs` |
+| L1-12: usage-then-exit six times | R3 | C51 |
+| L2-6: scratch folders not removed in a `finally` | R3 | C30 |
+| L2-7: three small tree walkers | R3 | none: the review proposes no fix |
+| L3-6: unguarded `spawnSync` | R3 | C30 |
+| L4-2: the two SCSS compiles | R3 | C58 |
+
+### Decisions
+
+| Decision | Commit |
+|---|---|
+| 1: `perf/` stays a lab notebook | done in `26eefeeb` and `90624841` |
+| 2: fix in place first, split later | Phase 4, C76–C81 |
+| 3: the tree comparison as a committed tool | C02 |
+| 4: the linter first, formatting last | C05, C06, C08; C84–C87 |
+| 5: both impexp editions, `build_fonts.py`, the link checker as a thin wrapper | C42; C70 |
+| 6: a gate on the workflows | C03 |
+| (a): the markdown module in `lib/`, and no `gray-matter` | C12, C31–C41 |
+| (b): the impexp parity gate | C70 |
+| (c): shim checks and an equivalence test | C66, C69; C67 and C68 under it |
+| (d): the composite action | C04 |
+| (e): command lines | C17 first, C47–C52, C71–C74 |
+| (f): the `staging.md` slip | done in `e0d127f0` |
+| (g): the pinning policy and Builder.md's list | C01, then the bar's rule |
+
+### The inventory's fifteen sites
+
+| Site | Commit |
+|---|---|
+| A1: `render.mjs`'s code mask | C32 |
+| A2: `render.mjs`'s admonition rewrite | C33 |
+| A3: `convert_em_dash_separators.mjs` | C34 |
+| A4: `check_examples.mjs`'s marker splice | C35 |
+| A5: wisdom's `parseStaging` and `serializeStaging` | C26, C36 |
+| B1: `census_attributes.mjs`'s `documentedAttributes` | C38 |
+| B2: `gen_attribute_probes.mjs`'s `parseAttributes` | C38 |
+| B3: `check_gate_lists.mjs`'s `splitSections` | C37 |
+| B4, B5: `counts.mjs`'s raw-source counts | C39 |
+| B6: `wisdom/extract/sitemap.mjs`'s frontmatter | C41 |
+| B7: `wisdom/extract/prep.mjs`'s frontmatter | C41 |
+| B8: `eval/run_case.mjs`'s protocol cut | C39 |
+| B9: `eval/nav_hops.mjs`'s links | C39; its frontmatter in C40 |
+| B10: `test/addin/symbols.test.mjs` | none: it reads the first line of a compiler hover reply, which no fence can come before |
+
+`discover.mjs`'s `gray-matter` call, a positive precedent in the inventory, moves in C40.
+
+### Verifier notes folded in
+
+| Note | Commit |
+|---|---|
+| V1: a fifth comment naming the diff tools, `offline.mjs:364-365` | C14 |
+| V1: `render.mjs`'s two escapers, one a copy of `highlight.mjs`'s | C54 |
+| V1: the bit-0 assignments safe only by source order | C60 |
+| V2: `Extending.md:654` overstates its own convention | C46 |
+| V2: `PLAN-10.md:692` cites the `.py` dash tool | C82 |
+| V2: `gen_attribute_probes.mjs --help` creates a folder | C71 |
+| V2: three frontmatter readers, not two | C41 |
+| V3: `tb-fences.mjs` has no `/* */` handling; its BOM safety comes from `trim()` | C61 |
+| V3: A8-1 and A8-4 are one gap | C62 |
+| V3: the onebuf factory must parametrise the dispatch and the gap mask | C68 |
+| V3: a `flag()` and `opt()` pair in each harness CLI | C49 |
+| V4: `render.mjs`'s and the dash tool's fence patterns are identical literals | C34 |
+| V4: `check_examples.mjs` has no process-level handler | C23 |
+| V4: `escapeHtml` names three unrelated functions | C54 |
+| V4: `check_a11y.mjs:229` names the path that leaks the browser | C19 |
 
 ## The bar for each commit
 
-- `build.bat && check.bat && test.bat` clean; after Phase 0, lint clean; after Phase 6,
-  format clean.
-- The commit's oracle from the table above, with every difference it shows explained in
-  the commit message.
+- `build.bat && check.bat && test.bat` clean; from C06, lint clean; from C87, format clean.
+- The commit's oracle from the table above, with every difference it shows explained. The
+  explanation goes in the commit's Landed note in this file, since commit messages stay one
+  line.
 - A commit that changes a gate shows the gate still failing on its reintroduced defect.
-- One concern per commit, with a concise one-line message.
+- One concern per commit, with a concise one-line message under 80 characters and no
+  trailers.
+- **A new gate is registered in the commit that adds it**: in `test.bat` or `check.bat`, in
+  the composite action (in both workflows before C04), in Tools.md's numbered list, and in
+  WIP.md's gate table and wrapper bullet. `check_gate_lists.mjs` and `check_ci_workflows.mjs`
+  catch a miss in the first three; WIP.md is checked by hand.
+- **A commit that changes `package.json`** follows C01's policy and updates Builder.md's
+  Dependencies in the same commit. `npm install` and `npm uninstall` wait for the owner's
+  confirmation, and so does any change to git configuration.
+- **A move or a rename** updates every citation `git grep` finds, in published pages and the
+  WIP files included.
+- **Harness runs** (`examples.bat`, `addin-test.bat`, `tbbuild`, `tbrun`, `census_attributes`,
+  `build_package_api`, `gen_attribute_probes`, `check_tb_registry`) go one at a time, never two
+  at once, and each run's summary line goes in the Landed note.
+- **A commit that changes a workflow or the composite action** says what CI must show, and
+  waits for the owner's push to show it (see Oracles).
+
+## Where the plan was wrong
+
+Nothing yet. When a commit lands and its code turned out different from its entry, the entry
+keeps its text, gains a Landed note, and the correction is listed here, as in the last
+review's plan.
+
+## Found while implementing
+
+Nothing yet: defects the review did not have, found by building something this plan asks for.
 
 ## Open questions
 
-None of the plan's own. Both that it started with are settled: the two link checkers
-(decision 5), and the survey script, which is committed. The decisions the review raises are
-listed in `REVIEW-TOOLING-fe9ce12b.md`.
+Each is settled in the commit named, on the recommendation given there, unless the owner
+decides otherwise:
+
+- the exit value for a command-line error in `tbdocs` and `check_links.mjs`: C18 recommends 4;
+- Biome or ESLint: C05's evaluation decides;
+- whether `test.bat` without Python fails or skips `check_impexp_parity.mjs` loudly: C70,
+  decision (b)'s open question;
+- whether the pre-commit hook should also run the dash check, which A6-3 assumed: the owner
+  approved a hook that runs Biome only (C08), so it stays out unless the owner asks for it.
+
+The two questions this plan started with are settled: the two link checkers (decision 5), and
+the survey script, which is committed.

From 6d85e4077ac1fef821d35c4031b697f3bb05123f Mon Sep 17 00:00:00 2001
From: Kuba Sunderland-Ober 
Date: Fri, 25 Sep 2026 06:16:13 +0200
Subject: [PATCH 09/53] docs: state the dependency pinning policy, and correct
 Builder.md's list

---
 builder/PLAN-TOOLING-REVIEW.md | 10 ++++++++++
 docs/Documentation/Builder.md  | 16 +++++++++++++---
 2 files changed, 23 insertions(+), 3 deletions(-)

diff --git a/builder/PLAN-TOOLING-REVIEW.md b/builder/PLAN-TOOLING-REVIEW.md
index b579e6b6..dd3c4ab8 100644
--- a/builder/PLAN-TOOLING-REVIEW.md
+++ b/builder/PLAN-TOOLING-REVIEW.md
@@ -375,6 +375,16 @@ commit that changes `package.json` updates this section in the same commit (see
 **Verify.** Every row re-read against `package.json` and `package-lock.json`; `build.bat`
 for the page.
 
+**Landed**, saying two things the entry does not. The paragraph under the package list also
+credited `htmlparser2` to the PDF renderer, where only the link checker and `crawl_check.mjs`
+import it; the rewrite says what each package is for, the four it never mentioned (`gray-matter`,
+`js-yaml`, `fast-glob`, `recheck`) included. And the policy had to explain why
+`@hpcc-js/wasm-graphviz` floats although `dot-metrics.mjs` patches it: the patch finds Graphviz's
+width table by an exact signature and fails the build when a release moves it, so the caret is
+a decision, and the section says so. The exact pins cite `PLAN-axe-perf.md`, `08-pdf-lib.md`,
+the change that pinned `puppeteer` with `pdf-lib`, and WIP.Build.md's note on `recheck`'s
+Windows backend. The JSON block now matches `package.json`'s `devDependencies` exactly.
+
 ### C02 — `scripts: compare_trees.mjs, the built trees before and after a change`
 
 **Decision 3.** The oracle for every `builder/` commit below.
diff --git a/docs/Documentation/Builder.md b/docs/Documentation/Builder.md
index 099558e6..720ab877 100644
--- a/docs/Documentation/Builder.md
+++ b/docs/Documentation/Builder.md
@@ -411,12 +411,12 @@ When adding a new task to `TASKS`, give it a `ganttSection` key matching one of
 
 ## Dependencies
 
-A single `package.json` at the repo root contains everything --- the static site generator's deps, the PDF renderer's deps, and the few packages both consume:
+A single `package.json` at the repo root contains everything --- the static site generator's deps, the PDF renderer's deps, the gates' deps, and the few packages several of them consume:
 
 ```json
 {
   "devDependencies": {
-    "@hpcc-js/wasm-graphviz": "^1.21",
+    "@hpcc-js/wasm-graphviz": "^1.29.1",
     "acorn": "^8.0",
     "acorn-walk": "^8.0",
     "axe-core": "4.13.0",
@@ -431,13 +431,23 @@ A single `package.json` at the repo root contains everything --- the static site
     "markdown-it-footnote": "^4.0",
     "pdf-lib": "1.17.1",
     "puppeteer": "25.0.4",
+    "recheck": "4.5.0",
     "sass": "^1.0",
     "shiki": "^1.0"
   }
 }
 ```
 
-No template engine, no framework, no bundler, no postinstall hooks. `acorn` + `acorn-walk` parse the upstream `just-the-docs.js` for the AST-based offline patcher; the `markdown-it-*` packages cover the dialect extensions the legacy parser supported; `shiki` is the syntax highlighter; `@hpcc-js/wasm-graphviz` is the WASM build of Graphviz that renders `.dot` diagram sources; `sass` is Dart Sass for the SCSS compile. `pdf-lib` + `html-entities` + `htmlparser2` + `puppeteer` are the PDF renderer's toolchain (puppeteer controls headless Chromium for the paged.js layout pass). `axe-core` + `puppeteer` also back the standalone accessibility checker ([`scripts/check_a11y.mjs`](https://github.com/twinbasic/documentation/blob/main/scripts/check_a11y.mjs)), which runs the same headless Chromium over the built pages --- neither the checker nor `axe-core` is used by `tbdocs` itself. `axe-core` is the one dependency pinned to an exact version rather than a caret range: the scan injects a patched copy of its bundle, and the patch asserts an exact occurrence count at each substitution point, so a minor bump would fail loudly rather than silently reverting to the slow path.
+No template engine, no framework, no bundler, no postinstall hooks. For the site generator, the `markdown-it-*` packages cover the dialect extensions the legacy parser supported; `gray-matter` splits off page frontmatter and `js-yaml` parses `_config.yml` and `_book.yml`; `fast-glob` finds the source files; `shiki` is the syntax highlighter; `@hpcc-js/wasm-graphviz` is the WASM build of Graphviz that renders `.dot` diagram sources; `sass` is Dart Sass for the SCSS compile; `acorn` + `acorn-walk` parse the upstream `just-the-docs.js` for the AST-based offline patcher; and `htmlparser2` is the SAX parser under the link and integrity check. `puppeteer` + `pdf-lib` + `html-entities` are the PDF renderer's toolchain: puppeteer controls headless Chromium for the paged.js layout pass, and `html-entities` decodes the entities in the PDF outline's entries. `axe-core` + `puppeteer` also back the standalone accessibility checker ([`scripts/check_a11y.mjs`](https://github.com/twinbasic/documentation/blob/main/scripts/check_a11y.mjs)), which runs the same headless Chromium over the built pages, and `recheck` + `acorn` back the regex-safety gate ([`scripts/check_regex_safety.mjs`](https://github.com/twinbasic/documentation/blob/main/scripts/check_regex_safety.mjs)). Neither `axe-core` nor `recheck` is used by `tbdocs` itself.
+
+**Which packages are pinned.** A package is pinned to an exact version where a new release could change what the build produces or what a gate reports without anything failing to say so: where the code patches the package or relies on its internals with no guard that fails when they change, or where the package's own results are what a gate reports. Everything else takes a caret range. Four packages are exact:
+
+- `axe-core` --- the scan injects a copy of its bundle patched at source level, and its rules decide the accessibility gate's verdict. [PLAN-axe-perf.md](https://github.com/twinbasic/documentation/blob/main/builder/PLAN-axe-perf.md) records why the pin is exact.
+- `pdf-lib` --- the shims under `book/lib/` are line-by-line ports of this release's source, and pdf-lib is no longer maintained; [08-pdf-lib.md](https://github.com/twinbasic/documentation/blob/main/perf/notes/08-pdf-lib.md) records the pin.
+- `puppeteer` --- the book renderer and the accessibility gate measure what its Chromium renders, and the performance notes reason about that version at source level. It was pinned in the same change as `pdf-lib`.
+- `recheck` --- the regex-safety gate reports its analysis, and finds its native backend itself, because this release cannot find it on Windows; [WIP.Build.md](https://github.com/twinbasic/documentation/blob/main/WIP.Build.md) records the workaround.
+
+`@hpcc-js/wasm-graphviz` is patched too, and takes a caret range on purpose: [`dot-metrics.mjs`](#diagram-geometry) finds Graphviz's width table by an exact signature match and fails the build when a new release moves it, so an upgrade cannot change the diagrams silently. A change to `package.json` updates this section in the same commit.
 
 Node 22+ is required: the SAB scheduler uses `Atomics.wait`, `Atomics.notify`, and `SharedArrayBuffer` --- all baseline in Node 22 without flags.
 

From b0612a46cd16c45a53be32d6d445b1b0ce5c2a10 Mon Sep 17 00:00:00 2001
From: Kuba Sunderland-Ober 
Date: Fri, 25 Sep 2026 06:20:13 +0200
Subject: [PATCH 10/53] scripts: compare_trees.mjs, the built trees before and
 after a change

---
 .gitignore                     |   4 +
 WIP.Build.md                   |  12 ++
 WIP.md                         |   6 +
 builder/PLAN-TOOLING-REVIEW.md |  16 ++
 docs/Documentation/Tools.md    |  14 ++
 scripts/compare_trees.mjs      | 337 +++++++++++++++++++++++++++++++++
 6 files changed, 389 insertions(+)
 create mode 100644 scripts/compare_trees.mjs

diff --git a/.gitignore b/.gitignore
index ec33af39..6f3687b4 100644
--- a/.gitignore
+++ b/.gitignore
@@ -12,6 +12,10 @@ indexer/.packages/
 # Temporary session handoff notes. Root-anchored: only the repo-root file.
 /HANDOFF.md
 
+# scripts/compare_trees.mjs: the before side's git worktree, both built
+# trees and both build logs. Removed after a clean run unless --keep.
+/.compare-trees/
+
 # scripts/check_links_diff.mjs builds test/fixtures/check-src into these.
 /test/fixtures/_out/
 /test/fixtures/_out-offline/
diff --git a/WIP.Build.md b/WIP.Build.md
index 6741f86d..c7f13013 100644
--- a/WIP.Build.md
+++ b/WIP.Build.md
@@ -25,6 +25,18 @@ every time. `byName` breaks ties on `srcRel` now. Two builds of a commit are
 byte-identical except for `BuildInfo.html` and `gantt.svg`, which record build
 timings and cannot be.
 
+**`scripts/compare_trees.mjs` is the check that relies on it.** It builds a
+commit and the working tree from two git worktrees and compares all three trees
+byte for byte, replacing those two regions and the PDF title page's build line,
+which also differs when the sides are different commits or were built on
+different days. Run it after any change to `builder/` that should leave the
+output alone; a change meant to alter the output is checked the same way, and
+what it reports should be the intended differences and nothing else. Build the
+working tree from a checkout, never in place: under `core.autocrlf` a file a
+tool has rewritten holds LF where a fresh checkout writes CRLF, and every file
+the build copies verbatim then differs, which is what the tool's first version
+found.
+
 ### A hung build times out and says where it hung
 
 Readers get this at [When a build stops instead of
diff --git a/WIP.md b/WIP.md
index 678cd8e0..2fea8c38 100644
--- a/WIP.md
+++ b/WIP.md
@@ -476,6 +476,12 @@ On the dev box that is ~4 s of build against ~37 s of check, of which the axe sc
 build.bat && check.bat && test.bat
 ```
 
+**If the change touched `builder/`, compare the output as well.** `node
+scripts/compare_trees.mjs` builds `HEAD` and the working tree from two git
+worktrees and compares the three trees byte for byte, in about ten seconds. A
+refactor must come out identical; any other change should differ exactly where
+it meant to and nowhere else. See [WIP.Build.md](WIP.Build.md#the-pipeline).
+
 ### The gates, and where their internals are
 
 [Tools and Scripts](docs/Documentation/Tools.md) owns the authoritative lists,
diff --git a/builder/PLAN-TOOLING-REVIEW.md b/builder/PLAN-TOOLING-REVIEW.md
index dd3c4ab8..1674a3f2 100644
--- a/builder/PLAN-TOOLING-REVIEW.md
+++ b/builder/PLAN-TOOLING-REVIEW.md
@@ -413,6 +413,22 @@ be identical after normalisation. Anything else it finds is either given a reaso
 normaliser, or fixed as nondeterminism. Then a one-character change to a template must show
 in all three trees. Record how long a comparison takes.
 
+**Landed, with the after side built from a checkout too.** The entry's design, the working
+tree built in place against a worktree at `HEAD`, failed its first run on eleven files that
+were not differences. Under `core.autocrlf` a fresh checkout writes CRLF, while files a tool
+has rewritten in this working tree hold LF, and everything the build copies verbatim (the
+impexp downloads, the font licences, `theme-toggle.js`, a committed diagram `.svg`) differed by
+line endings alone. So the after side is a second worktree, at a commit object made from the
+working tree through a copy of the index: `git add -A` into the copy, `write-tree`,
+`commit-tree`, with fixed identities. Neither the real index nor any file changes, untracked
+files that are not ignored are included, and both sides get the same line endings.
+
+Both worktrees live under `.compare-trees/` at the repository root, which the root `.gitignore`
+names, rather than `docs/_site-cmp*`; Node finds `node_modules` by walking up from them, so
+nothing is linked. The PDF title page's normaliser covers the whole build line, which holds the
+build's wall-clock date as well as the commit. A comparison takes about ten seconds. The A/A run
+and the template change are recorded in this entry's commit.
+
 ### C03 — `scripts: check_ci_workflows.mjs, the workflows against the wrappers' gates`
 
 **Decision 6, A6-4 (R2).** Nothing reads either workflow to confirm it runs the gates the
diff --git a/docs/Documentation/Tools.md b/docs/Documentation/Tools.md
index 745f01fb..4c553733 100644
--- a/docs/Documentation/Tools.md
+++ b/docs/Documentation/Tools.md
@@ -569,6 +569,20 @@ Measures the repository's own tooling for repetition and structure: code duplica
 
 It is not a gate, and nothing runs it: take a measurement before and after a piece of refactoring. It reads only the files git tracks, so a scratch file never changes a number. `--root` measures another checkout, such as a worktree at an older commit that does not contain the script. `perf/` is measured, but it is counted separately in the summary and left out of the listings unless `--include-perf` is given. Exits 0, or 2 on a bad argument or a folder that is not a git checkout.
 
+### compare_trees.mjs
+{: #compare-trees }
+
+    node scripts/compare_trees.mjs                      # HEAD against the working tree
+    node scripts/compare_trees.mjs --before        # any commit against the working tree
+    node scripts/compare_trees.mjs --keep               # leave both trees and both build logs
+    node scripts/compare_trees.mjs -- --baseurl /docs   # extra tbdocs arguments, for both builds
+
+Builds the site twice and compares the online, offline and PDF trees file by file, byte for byte: once at a commit, `HEAD` unless `--before` names another, and once from the working tree as a commit would hold it, untracked files included. It is the check for a change to `builder/` that should leave the output alone, and for one that should not, whose differences ought to be the intended ones and no others.
+
+Both builds run from git worktrees under `.compare-trees/` at the repository root, which is gitignored, and neither touches the index or the working tree. Building the working tree in place would not do: under `core.autocrlf` a fresh checkout writes CRLF where files a tool has rewritten hold LF, and every file the build copies verbatim would then differ. Both builds run `tbdocs --no-fetch-assets` with `CI=1`, so the committed baselines are read and never written.
+
+Three regions differ between any two builds and are replaced before the comparison: the build's own timings in `assets/images/gantt.svg`, the same chart inlined into the [Build Info](BuildInfo) page, and the PDF title page's build line, which holds the build date and the commit. Everything else must match. A run takes about ten seconds on the development box. It is not a gate, and nothing runs it. Exits 0 when the trees match, 1 when they differ, and 2 when the tool failed; a failed run leaves `.compare-trees/` for inspection, and the next run removes it.
+
 ### tbbuild.mjs
 {: #tbbuild }
 
diff --git a/scripts/compare_trees.mjs b/scripts/compare_trees.mjs
new file mode 100644
index 00000000..e315d939
--- /dev/null
+++ b/scripts/compare_trees.mjs
@@ -0,0 +1,337 @@
+#!/usr/bin/env node
+// Build the site before and after a change, and compare the three trees
+// byte for byte.
+//
+//   node scripts/compare_trees.mjs                      # HEAD against the working tree
+//   node scripts/compare_trees.mjs --before        # any commit against the working tree
+//   node scripts/compare_trees.mjs --keep               # leave the trees and logs for inspection
+//   node scripts/compare_trees.mjs -- --baseurl /docs   # extra tbdocs arguments, for both builds
+//
+// The oracle for a builder/ change that claims to change no output
+// (builder/PLAN-TOOLING-REVIEW.md, decision 3). A change that is meant to
+// alter the output is checked the same way: the differences it reports should
+// be the intended ones and no others.
+//
+// Both sides are built from git worktrees checked out under .compare-trees/
+// at the repository root (gitignored): the before side at --before, the after
+// side at a commit object made from the working tree through a temporary
+// index, so neither the real index nor any file in the working tree changes.
+// Building the working tree in place does not work as an oracle: under
+// core.autocrlf a fresh checkout writes CRLF where files a tool has rewritten
+// hold LF, and every file the build copies verbatim then differs. The after
+// side includes untracked files that are not ignored, which is what a commit
+// of the working tree would hold.
+//
+// The worktrees need no node_modules of their own: Node resolves packages by
+// walking up from the importing file, and the walk reaches this checkout's.
+// Both builds run tbdocs with --no-fetch-assets and CI=1 in the environment,
+// so the page and symbol baselines are read and never written (tbdocs.mjs's
+// mayWrite); the only other reader of CI is the asset fetch, which the flag
+// already turns off.
+//
+// A few regions differ between any two builds, and are normalised rather than
+// excluded, each for the reason given in NORMALISERS below. A normaliser that
+// finds nothing to replace says so, instead of letting the file through.
+//
+// A run that fails keeps .compare-trees/ so its logs can be read; the next
+// run removes it before starting.
+//
+// Exit codes: 0 the trees match, 1 they differ, 2 the tool failed.
+
+import { spawnSync } from "node:child_process";
+import { closeSync, existsSync, openSync } from "node:fs";
+import fs from "node:fs/promises";
+import path from "node:path";
+import { fileURLToPath } from "node:url";
+
+const ROOT = path.resolve(fileURLToPath(new URL("..", import.meta.url)));
+const WORK = path.join(ROOT, ".compare-trees");
+const SIDES = ["before", "after"];
+
+// The three trees one build writes, by the suffix tbdocs adds to --dest.
+const TREES = [["online", ""], ["offline", "-offline"], ["pdf", "-pdf"]];
+
+// Files whose bytes differ between any two builds of the same source. Each
+// normalise() returns the text with the varying region replaced, or null when
+// it cannot find the region -- which is itself worth reporting.
+const GANTT_MARKER = 'data-svg-src="assets/images/gantt.svg"';
+const NORMALISERS = [
+  {
+    trees: ["online", "offline"],
+    path: "assets/images/gantt.svg",
+    reason: "the build's own task timings (gantt.mjs)",
+    normalise: () => "",
+  },
+  {
+    trees: ["online", "offline"],
+    path: "Documentation/Development/BuildInfo.html",
+    reason: "the same chart, inlined by tbdocs.mjs's injectGanttChart",
+    normalise: (s) => {
+      const at = s.indexOf(GANTT_MARKER);
+      const start = at < 0 ? -1 : s.indexOf("", start);
+      if (end < 0) return null;
+      return s.slice(0, start) + "" + s.slice(end + "".length);
+    },
+  },
+  {
+    trees: ["pdf"],
+    path: "book.html",
+    reason: "the title page's build line: the wall-clock date and the commit (book.mjs's renderTitlePage)",
+    normalise: (s) => {
+      const re = /(

)[^<]*(<\/p>)/; + return re.test(s) ? s.replace(re, "$1(build line)$2") : null; + }, + }, +]; + +const TEXT_EXT = /\.(?:html?|css|js|mjs|json|xml|svg|txt|md|map|py|yml)$/i; + +const USAGE = `usage: node scripts/compare_trees.mjs [--before ] [--keep] [--max ] [-- ] + +Builds (default HEAD) and the working tree, each from a git worktree +under .compare-trees/ with tbdocs --no-fetch-assets and CI=1, and compares the +online, offline and PDF trees byte for byte. + + --before the commit to build as the before side (default HEAD) + --keep leave .compare-trees/ in place: both worktrees, their trees + and both build logs + --max list at most n differences per tree (default 20) + -- everything after it is passed to both tbdocs builds + +Exit codes: 0 the trees match, 1 they differ, 2 the tool failed. +`; + +function usageError(message) { + process.stderr.write(`compare_trees: ${message}\n\n${USAGE}`); + process.exit(2); +} + +function parseArgs(argv) { + const opts = { before: "HEAD", keep: false, max: 20, tbdocs: [] }; + for (let i = 0; i < argv.length; i++) { + const a = argv[i]; + if (a === "--") { opts.tbdocs = argv.slice(i + 1); break; } + if (a === "--help" || a === "-h") { process.stdout.write(USAGE); process.exit(0); } + if (a === "--keep") { opts.keep = true; continue; } + if (a === "--before" || a === "--max") { + const v = argv[i + 1]; + if (v === undefined || v.startsWith("--")) usageError(`${a} needs a value`); + i++; + if (a === "--before") opts.before = v; + else { + opts.max = Number(v); + if (!Number.isInteger(opts.max) || opts.max < 0) usageError(`--max takes a whole number, not "${v}"`); + } + continue; + } + usageError(`unknown argument "${a}"`); + } + return opts; +} + +class ToolError extends Error {} + +function git(args, { env, allowFail = false } = {}) { + const r = spawnSync("git", args, { cwd: ROOT, encoding: "utf8", env: env ? { ...process.env, ...env } : process.env }); + if (r.error) throw new ToolError(`git ${args.join(" ")}: ${r.error.message}`); + if (r.status !== 0 && !allowFail) { + throw new ToolError(`git ${args.join(" ")} exited ${r.status}: ${(r.stderr || "").trim()}`); + } + return r; +} + +// Worktrees left by a run that failed, or kept by --keep. +async function removeWorktrees() { + for (const side of SIDES) { + const dir = path.join(WORK, side); + if (!existsSync(dir)) continue; + git(["worktree", "remove", "--force", dir], { allowFail: true }); + await fs.rm(dir, { recursive: true, force: true }); + } + git(["worktree", "prune"], { allowFail: true }); +} + +// A commit object holding the working tree as `git add -A` would stage it, +// made in a copy of the index so that the real one is untouched. Fixed +// identities keep it independent of the user's configuration; nothing refers +// to it afterwards, so git's garbage collection removes it in time. +function snapshotWorkingTree() { + const gitDir = path.resolve(ROOT, git(["rev-parse", "--git-dir"]).stdout.trim()); + const index = path.join(WORK, "index"); + const env = { + GIT_INDEX_FILE: index, + GIT_AUTHOR_NAME: "compare_trees", GIT_AUTHOR_EMAIL: "compare_trees@localhost", + GIT_COMMITTER_NAME: "compare_trees", GIT_COMMITTER_EMAIL: "compare_trees@localhost", + }; + return fs.copyFile(path.join(gitDir, "index"), index).then(() => { + git(["add", "-A"], { env }); + const tree = git(["write-tree"], { env }).stdout.trim(); + const commit = git(["commit-tree", tree, "-p", "HEAD", "-m", "compare_trees: the working tree"], { env }).stdout.trim(); + return { tree, commit }; + }); +} + +function build(label, root, dest, extra) { + const log = path.join(WORK, `${label}.log`); + const fd = openSync(log, "w"); + const started = Date.now(); + const r = spawnSync( + process.execPath, + [path.join(root, "builder", "tbdocs.mjs"), "--src", "docs", "--dest", dest, "--no-fetch-assets", ...extra], + { cwd: root, env: { ...process.env, CI: "1" }, stdio: ["ignore", fd, fd] }, + ); + closeSync(fd); + if (r.error) throw new ToolError(`${label} build: ${r.error.message}`); + const seconds = ((Date.now() - started) / 1000).toFixed(1); + if (!existsSync(dest)) throw new ToolError(`${label} build wrote no tree (exit ${r.status}); see ${log}`); + return { status: r.status, seconds }; +} + +async function listFiles(root) { + const out = []; + async function visit(dir, rel) { + for (const entry of await fs.readdir(dir, { withFileTypes: true })) { + const r = rel ? `${rel}/${entry.name}` : entry.name; + if (entry.isDirectory()) await visit(path.join(dir, entry.name), r); + else if (entry.isFile()) out.push(r); + } + } + if (existsSync(root)) await visit(root, ""); + return out; +} + +function byCodePoint(a, b) { + return a < b ? -1 : a > b ? 1 : 0; +} + +// Where two versions of a file first differ, in a form a person can read: a +// line number and a JSON-quoted excerpt from each side, so whitespace shows. +function firstDifference(rel, x, y) { + if (typeof x !== "string" && !TEXT_EXT.test(rel)) { + let k = 0; + const n = Math.min(x.length, y.length); + while (k < n && x[k] === y[k]) k++; + return `binary, first difference at byte ${k} (sizes ${x.length} and ${y.length})`; + } + const s = typeof x === "string" ? x : x.toString("utf8"); + const t = typeof y === "string" ? y : y.toString("utf8"); + let k = 0; + const n = Math.min(s.length, t.length); + while (k < n && s[k] === t[k]) k++; + const line = s.slice(0, k).split("\n").length; + const from = Math.max(0, k - 40); + return `line ${line}\n before: ${JSON.stringify(s.slice(from, k + 60))}\n after: ${JSON.stringify(t.slice(from, k + 60))}`; +} + +async function compareTree(name, before, after, applied) { + const [fb, fa] = await Promise.all([listFiles(before), listFiles(after)]); + const inBefore = new Set(fb); + const inAfter = new Set(fa); + const onlyBefore = fb.filter((f) => !inAfter.has(f)).sort(byCodePoint); + const onlyAfter = fa.filter((f) => !inBefore.has(f)).sort(byCodePoint); + const common = fb.filter((f) => inAfter.has(f)); + const differ = []; + let next = 0; + async function worker() { + while (next < common.length) { + const rel = common[next++]; + const [x, y] = await Promise.all([fs.readFile(path.join(before, rel)), fs.readFile(path.join(after, rel))]); + const n = NORMALISERS.find((m) => m.path === rel && m.trees.includes(name)); + if (!n) { + if (!x.equals(y)) differ.push({ rel, detail: firstDifference(rel, x, y) }); + continue; + } + const nx = n.normalise(x.toString("utf8")); + const ny = n.normalise(y.toString("utf8")); + if (nx === null || ny === null) { + if (!x.equals(y)) { + differ.push({ rel, detail: `the normaliser for ${n.reason} found nothing to replace\n ${firstDifference(rel, x, y)}` }); + } + continue; + } + applied.push(`${name}: ${rel} (${n.reason})`); + if (nx !== ny) differ.push({ rel, detail: firstDifference(rel, nx, ny) }); + } + } + await Promise.all(Array.from({ length: 16 }, worker)); + differ.sort((p, q) => byCodePoint(p.rel, q.rel)); + return { name, total: new Set([...fb, ...fa]).size, differ, onlyBefore, onlyAfter }; +} + +function printList(title, items, max, render) { + if (!items.length) return; + console.log(` ${title}:`); + for (const item of items.slice(0, max)) console.log(` ${render(item)}`); + if (items.length > max) console.log(` ... and ${items.length - max} more`); +} + +async function main() { + const opts = parseArgs(process.argv.slice(2)); + const before = git(["rev-parse", "--verify", `${opts.before}^{commit}`]).stdout.trim(); + + await removeWorktrees(); + await fs.rm(WORK, { recursive: true, force: true }); + await fs.mkdir(WORK, { recursive: true }); + + let failed = true; + try { + const after = await snapshotWorkingTree(); + const unchanged = after.tree === git(["rev-parse", `${before}^{tree}`]).stdout.trim(); + const commits = { before, after: after.commit }; + const dest = {}; + for (const side of SIDES) { + git(["worktree", "add", "--detach", path.join(WORK, side), commits[side]]); + dest[side] = path.join(WORK, side, ".compare-out", "site"); + } + console.log(`compare_trees: before = ${opts.before} (${before.slice(0, 9)}), after = the working tree${unchanged ? " (the same files)" : ""}`); + const b = build("before", path.join(WORK, "before"), dest.before, opts.tbdocs); + const a = build("after", path.join(WORK, "after"), dest.after, opts.tbdocs); + console.log(` builds: before ${b.seconds} s (exit ${b.status}), after ${a.seconds} s (exit ${a.status})`); + + const applied = []; + const results = []; + for (const [name, suffix] of TREES) { + results.push(await compareTree(name, dest.before + suffix, dest.after + suffix, applied)); + } + + let differences = b.status === a.status ? 0 : 1; + for (const r of results) { + const same = r.total - r.differ.length - r.onlyBefore.length - r.onlyAfter.length; + console.log(` ${r.name.padEnd(8)} ${String(r.total).padStart(5)} files: ${same} identical, ${r.differ.length} differ, ${r.onlyBefore.length} only before, ${r.onlyAfter.length} only after`); + differences += r.differ.length + r.onlyBefore.length + r.onlyAfter.length; + } + if (applied.length) { + console.log(" normalised:"); + for (const line of applied.sort(byCodePoint)) console.log(` ${line}`); + } + if (b.status !== a.status) console.log(` the builds exited differently: before ${b.status}, after ${a.status}`); + for (const r of results) { + if (!r.differ.length && !r.onlyBefore.length && !r.onlyAfter.length) continue; + console.log(` ${r.name}:`); + printList("differ", r.differ, opts.max, (d) => `${d.rel} ${d.detail}`); + printList("only before", r.onlyBefore, opts.max, (f) => f); + printList("only after", r.onlyAfter, opts.max, (f) => f); + } + console.log(differences ? "compare_trees: the trees differ" : "compare_trees: the trees match"); + if (opts.keep) console.log(` kept: ${WORK}`); + failed = false; + return differences ? 1 : 0; + } finally { + // A failed run keeps everything, so the log its message names is still + // there; the next run removes it before starting. + if (failed) process.stderr.write(`compare_trees: left ${WORK} for inspection\n`); + else if (!opts.keep) { + await removeWorktrees(); + await fs.rm(WORK, { recursive: true, force: true }); + } + } +} + +main().then( + (code) => process.exit(code), + (err) => { + process.stderr.write(`compare_trees: ${err instanceof ToolError ? err.message : err.stack}\n`); + process.exit(2); + }, +); From 767f72c69b978d7041ae7105509ff4df5f4f742d Mon Sep 17 00:00:00 2001 From: Kuba Sunderland-Ober Date: Fri, 25 Sep 2026 06:22:31 +0200 Subject: [PATCH 11/53] builder: record compare_trees' A/A and detection runs in the plan --- builder/PLAN-TOOLING-REVIEW.md | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/builder/PLAN-TOOLING-REVIEW.md b/builder/PLAN-TOOLING-REVIEW.md index 1674a3f2..25818bf6 100644 --- a/builder/PLAN-TOOLING-REVIEW.md +++ b/builder/PLAN-TOOLING-REVIEW.md @@ -426,8 +426,17 @@ files that are not ignored are included, and both sides get the same line ending Both worktrees live under `.compare-trees/` at the repository root, which the root `.gitignore` names, rather than `docs/_site-cmp*`; Node finds `node_modules` by walking up from them, so nothing is linked. The PDF title page's normaliser covers the whole build line, which holds the -build's wall-clock date as well as the commit. A comparison takes about ten seconds. The A/A run -and the template change are recorded in this entry's commit. +build's wall-clock date as well as the commit. A comparison takes about ten seconds. + +Verified at `b0612a46`. The A/A run, `HEAD` against a clean tree, found all 3,055 files +identical across the three trees, with the three regions normalised and nothing else. **The +entry's test, that a one-character template change shows in all three trees, was wrong about +the trees.** A change to the generator tag in the page head reached all 913 online pages and no +offline one, because the offline pass removes the whole SEO block (`offline-rewrite.mjs`'s +`stripSeo`); a change to the skip link's text reached 913 pages in each of the online and +offline trees; neither touched the PDF tree, because `book.html` is assembled from each page's +rendered content, not from the page template. A page edit reaches all three: C01's change to +`Builder.md` showed in both trees' `Builder.html`, both search indexes and `book.html`. ### C03 — `scripts: check_ci_workflows.mjs, the workflows against the wrappers' gates` From 186f0ecac1fafce75f7bb7c4f45fc23ee8f0e437 Mon Sep 17 00:00:00 2001 From: Kuba Sunderland-Ober Date: Fri, 25 Sep 2026 06:29:35 +0200 Subject: [PATCH 12/53] scripts: check_ci_workflows.mjs, the workflows against the wrappers' gates --- .github/workflows/checks.yml | 6 + .github/workflows/tbdocs-gh-pages.yml | 6 + WIP.md | 5 +- builder/PLAN-TOOLING-REVIEW.md | 15 ++ docs/Documentation/Building.md | 1 + docs/Documentation/Tools.md | 29 ++- scripts/check_ci_workflows.mjs | 293 ++++++++++++++++++++++++++ scripts/check_gate_lists.mjs | 18 +- scripts/lib/gate-roster.mjs | 62 ++++++ test.bat | 14 +- 10 files changed, 420 insertions(+), 29 deletions(-) create mode 100644 scripts/check_ci_workflows.mjs create mode 100644 scripts/lib/gate-roster.mjs diff --git a/.github/workflows/checks.yml b/.github/workflows/checks.yml index 69e8f802..a93e4609 100644 --- a/.github/workflows/checks.yml +++ b/.github/workflows/checks.yml @@ -142,6 +142,12 @@ jobs: # ~50 ms. - name: Verify the documented gate lists (check_gate_lists.mjs) run: node scripts/check_gate_lists.mjs + # This workflow and tbdocs-gh-pages.yml against the wrappers: every + # gate test.bat and check.bat run, with the same arguments and in the + # same order, and a build with build.bat's flags. See the script's + # header for the differences it allows. No browser, no built tree. + - name: Verify the workflows run the wrappers' gates (check_ci_workflows.mjs) + run: node scripts/check_ci_workflows.mjs # A regex that backtracks exponentially does not fail a build, it # stops one: the corpus passes until a page happens to contain the # trigger, and then a render worker sits inside String.replace diff --git a/.github/workflows/tbdocs-gh-pages.yml b/.github/workflows/tbdocs-gh-pages.yml index 687a5c85..a3736a44 100644 --- a/.github/workflows/tbdocs-gh-pages.yml +++ b/.github/workflows/tbdocs-gh-pages.yml @@ -118,6 +118,12 @@ jobs: # ~50 ms. - name: Verify the documented gate lists (check_gate_lists.mjs) run: node scripts/check_gate_lists.mjs + # This workflow and checks.yml against the wrappers: every gate + # test.bat and check.bat run, with the same arguments and in the same + # order, and a build with build.bat's flags. See the script's header + # for the differences it allows. No browser, no built tree. + - name: Verify the workflows run the wrappers' gates (check_ci_workflows.mjs) + run: node scripts/check_ci_workflows.mjs # An exponentially backtracking regex does not fail a build, it # stops one -- a render worker sits inside String.replace forever # the first time a page contains the trigger. VOID_TAGS_RE shipped diff --git a/WIP.md b/WIP.md index 2fea8c38..35c3d794 100644 --- a/WIP.md +++ b/WIP.md @@ -452,7 +452,7 @@ Why the report separates the wedged task from the merely blocked ones, and why - `build.bat` — runs `node builder\tbdocs.mjs --src docs --check-audit-index` (which implies `--check`) and produces three trees in one pass: the online copy at `_site/`, a `file://`-browsable copy at `_site-offline/`, and the sparse pagedjs source at `_site-pdf/`. The offline pass adds ~700 ms and the PDF pass adds ~150 ms on top of the ~2 s online build. Toggle `also_build_offline` / `also_build_pdf` in `_config.yml` (or pass `--no-offline` / `--no-pdf`) to skip a sibling output. `--check` adds ~1.7 s and runs the link + integrity check over the HTML while it is still in worker memory; `build.bat --no-check` gets a plain build. - `serve.bat` — runs `tbdocs --serve`: initial build, then a long-lived process with watcher, debounced rebuilds, and SSE-driven browser auto-reload. Writes to `docs/_serve/` (disjoint from `build.bat`'s `_site*/`) and skips the offline + PDF passes — so a one-off `build.bat` for the PDF or offline mirror doesn't disturb the live preview. Ctrl+C to stop. - `check.bat` — the gates that read the built site: a freshness check that refuses a stale tree (`scripts/check_tree_fresh.mjs`), the DOT diagram fit check (`scripts/check_dot_fit.mjs`), the a11y sample-coverage check (`scripts/pick_a11y_sample.mjs --check`), then the accessibility check (`scripts/check_a11y.mjs`). The link + integrity check moved into `build.bat`. ~37 s. -- `test.bat` — the tests the *toolchain* has to pass: the publish-allowlist self-test (`scripts/check_publish_policy.mjs`), the gate-list check (`scripts/check_gate_lists.mjs`), the regex-safety gate (`scripts/check_regex_safety.mjs`), the code-region gate (`scripts/check_code_regions.mjs`), the page-count drift-guard probes (`scripts/check_page_baseline.mjs`), the book-coverage probes (`scripts/check_book_coverage.mjs`), the symbol-index probes (`scripts/check_symbol_index.mjs`), and the axe source-patch verification (`scripts/check_axe_patch_equiv.mjs`). ~8 s. See [What belongs in test.bat rather than check.bat](WIP.Build.md#what-belongs-in-testbat-rather-than-checkbat). +- `test.bat` — the tests the *toolchain* has to pass: the publish-allowlist self-test (`scripts/check_publish_policy.mjs`), the gate-list check (`scripts/check_gate_lists.mjs`), the CI-workflow roster check (`scripts/check_ci_workflows.mjs`), the regex-safety gate (`scripts/check_regex_safety.mjs`), the code-region gate (`scripts/check_code_regions.mjs`), the page-count drift-guard probes (`scripts/check_page_baseline.mjs`), the book-coverage probes (`scripts/check_book_coverage.mjs`), the symbol-index probes (`scripts/check_symbol_index.mjs`), and the axe source-patch verification (`scripts/check_axe_patch_equiv.mjs`). ~8 s. See [What belongs in test.bat rather than check.bat](WIP.Build.md#what-belongs-in-testbat-rather-than-checkbat). - `book.bat` — renders the PDF from `docs\_site-pdf\book.html` via `node book\render-book.mjs` into `docs\_pdf\twinBASIC Book.pdf`. Run `build.bat` first to populate `_site-pdf/`; `book.bat` refuses a tree older than its sources rather than rendering the previous book (see [The book refuses a stale source tree](WIP.Build.md#the-book-refuses-a-stale-source-tree)). - `examples.bat` — compiles the documentation's own twinBASIC code samples, every `tb` fence marked `check_build`, and reports the ones the compiler refuses against the line in the page they came from. Needs a twinBASIC install and Windows, so it is outside every gate and outside CI; ~110 s over the 1,119 samples marked today. Two modes need no compiler at all: `--census` classifies every fence and says how many classifiable ones are still unmarked, and `--report ` groups a saved `--propose --json` survey by diagnostic, section and unresolved name. `--propose` itself does compile. See [Compiling the reference's own code samples](#compiling-the-references-own-code-samples) and [WIP.ExamplesBuild.md](WIP.ExamplesBuild.md). @@ -470,7 +470,7 @@ build.bat && check.bat On the dev box that is ~4 s of build against ~37 s of check, of which the axe scan is ~20 s. [builder/PLAN-checks.md](builder/PLAN-checks.md) records how the link checker got folded into the build's task graph, what it cost and what it saved; the axe follow-ons are designed there but not implemented. -**If the change touched `builder/`, `scripts/`, `book/`, `eval/` or `wisdom/`, run `test.bat` as well** --- another ~8 s. Six of its eight gates cannot be affected by a content edit at all. **Two can.** `check_gate_lists.mjs` is the easy one to predict: it reads `README.md` and every page under `docs/Documentation/`, so an edit to any developer page that states a gate count can fail it. **`check_code_regions.mjs` is the one worth understanding**, and which half of it a content edit reaches is worth keeping straight. Its corpus sweep has `ROOT = /docs` and tokenises all 906 markdown files, so a page that provokes a rewrite into *altering* a code region fails it --- that half is content-dependent. Its fixed probes are not: they run against their own sources whatever the tree holds, and they cover the **mirror** fault, where a rewrite silently stops firing. The sweep structurally cannot see that one, because text the rewrite skipped is stashed and restored unchanged and every region still matches. So run `test.bat` after adding an unusual code construct --- a fence whose contents include a fence marker, a 4-space indented block, an admonition wrapping a fence --- and read the built page as well, because for the mirror fault the gate is asserting that the stasher still works rather than checking your page: +**If the change touched `builder/`, `scripts/`, `book/`, `eval/`, `wisdom/`, a wrapper or a workflow, run `test.bat` as well** --- another ~8 s. Seven of its nine gates cannot be affected by a content edit at all. **Two can.** `check_gate_lists.mjs` is the easy one to predict: it reads `README.md` and every page under `docs/Documentation/`, so an edit to any developer page that states a gate count can fail it. **`check_code_regions.mjs` is the one worth understanding**, and which half of it a content edit reaches is worth keeping straight. Its corpus sweep has `ROOT = /docs` and tokenises all 906 markdown files, so a page that provokes a rewrite into *altering* a code region fails it --- that half is content-dependent. Its fixed probes are not: they run against their own sources whatever the tree holds, and they cover the **mirror** fault, where a rewrite silently stops firing. The sweep structurally cannot see that one, because text the rewrite skipped is stashed and restored unchanged and every region still matches. So run `test.bat` after adding an unusual code construct --- a fence whose contents include a fence marker, a 4-space indented block, an admonition wrapping a fence --- and read the built page as well, because for the mirror fault the gate is asserting that the stasher still works rather than checking your page: ```sh build.bat && check.bat && test.bat @@ -503,6 +503,7 @@ wrapper: | `test.bat` | `check_code_regions` | no source or HTML rewrite altered a code region | | `test.bat` | `check_regex_safety` | no regex in the tree can backtrack exponentially | | `test.bat` | `check_symbol_index` | the symbol index still places each kind of symbol, from fixtures | +| `test.bat` | `check_ci_workflows` | both CI workflows run every wrapper gate, with the same arguments and order, and build with `build.bat`'s flags | | `test.bat` | `check_publish_policy`, `check_gate_lists`, `check_page_baseline`, `check_book_coverage`, `check_axe_patch_equiv` | the gates on the gates | **A gate belongs in `test.bat` rather than `check.bat` if it would still mean diff --git a/builder/PLAN-TOOLING-REVIEW.md b/builder/PLAN-TOOLING-REVIEW.md index 25818bf6..70cdbcea 100644 --- a/builder/PLAN-TOOLING-REVIEW.md +++ b/builder/PLAN-TOOLING-REVIEW.md @@ -467,6 +467,21 @@ order, and the workflows differ only in three recorded ways. `checks.yml` with one gate step deleted fails. `check_gate_lists.mjs`'s 18 probes unchanged. CI waits for the owner's push. +**Landed**, somewhat wider than the entry. The gate compares each gate's arguments as well as +its name, since `pick_a11y_sample.mjs` without `--check` is a different gate; it reports an +allowance that no longer matches anything, so the allowlist cannot quietly outlive its reason; +and it reads the build flags as quoted tokens, because the deploy build's `--url` value is +`'${{ steps.pages.outputs.origin }}'`, spaces included. Its probes number 13, on a synthetic set +of wrappers and workflows rather than copies of the real files, so that they mean the same +whatever state the real ones are in; two of them assert that CI may interleave the two +wrappers' gates alike and may not interleave them differently. + +With `check_code_regions.mjs`'s step deleted from the real `checks.yml` (restored from git +afterwards), it reported the missing gate and the two workflows parting at step 5, and exited +1. Registering it found one more place restating `test.bat`: `check_gate_lists.mjs` failed on +`Building.md`'s POSIX command block until the new gate was added there too. `test.bat`'s header +and WIP.md now name a wrapper or a workflow among the changes that call for `test.bat`. + ### C04 — `ci: one composite action for the gates both workflows run` **Decision (d).** After the roster gate, so the action is checked from its first commit. diff --git a/docs/Documentation/Building.md b/docs/Documentation/Building.md index ac151691..71e60153 100644 --- a/docs/Documentation/Building.md +++ b/docs/Documentation/Building.md @@ -70,6 +70,7 @@ Each `.bat` opens with `@pushd "%~dp0"`, which is what lets it be invoked from a node scripts/check_publish_policy.mjs \ && node scripts/check_gate_lists.mjs \ + && node scripts/check_ci_workflows.mjs \ && node scripts/check_regex_safety.mjs \ && node scripts/check_code_regions.mjs \ && node scripts/check_page_baseline.mjs \ diff --git a/docs/Documentation/Tools.md b/docs/Documentation/Tools.md index 4c553733..3c86dc88 100644 --- a/docs/Documentation/Tools.md +++ b/docs/Documentation/Tools.md @@ -65,21 +65,23 @@ One of the four does not mean the same thing locally as it does in CI, on any pl test.bat -The tests the toolchain has to pass. Eight steps, each stopping the run if it fails: +The tests the toolchain has to pass. Nine steps, each stopping the run if it fails: 1. [`scripts/check_publish_policy.mjs`](#check-publish-policy) --- verifies the publish allowlist still refuses the types it is meant to. Needs neither a browser nor a built tree, so it goes first. 2. [`scripts/check_gate_lists.mjs`](#check-gate-lists) --- verifies the two gate lists on this page still match the wrappers that run them. -3. [`scripts/check_regex_safety.mjs`](#check-regex-safety) --- refuses a regex that can backtrack exponentially, written as a literal or built from constants. -4. [`scripts/check_code_regions.mjs`](#check-code-regions) --- verifies no pre-render rewrite alters the contents of a code fence or code span. -5. [`scripts/check_page_baseline.mjs`](#check-page-baseline) --- verifies the page-count drift guard still refuses a fall. -6. [`scripts/check_book_coverage.mjs`](#check-book-coverage) --- verifies the build still warns about a page `docs/_book.yml` does not mention. -7. [`scripts/check_symbol_index.mjs`](#check-symbol-index) --- verifies the symbol index still places each kind of symbol, and its drift guard still refuses a lost URL. -8. [`scripts/check_axe_patch_equiv.mjs`](#check-axe-patch-equiv) --- verifies the vendored axe source patch still produces identical colour values. +3. [`scripts/check_ci_workflows.mjs`](#check-ci-workflows) --- verifies both CI workflows run the gates the wrappers run, and build as `build.bat` does. +4. [`scripts/check_regex_safety.mjs`](#check-regex-safety) --- refuses a regex that can backtrack exponentially, written as a literal or built from constants. +5. [`scripts/check_code_regions.mjs`](#check-code-regions) --- verifies no pre-render rewrite alters the contents of a code fence or code span. +6. [`scripts/check_page_baseline.mjs`](#check-page-baseline) --- verifies the page-count drift guard still refuses a fall. +7. [`scripts/check_book_coverage.mjs`](#check-book-coverage) --- verifies the build still warns about a page `docs/_book.yml` does not mention. +8. [`scripts/check_symbol_index.mjs`](#check-symbol-index) --- verifies the symbol index still places each kind of symbol, and its drift guard still refuses a lost URL. +9. [`scripts/check_axe_patch_equiv.mjs`](#check-axe-patch-equiv) --- verifies the vendored axe source patch still produces identical colour values. POSIX: node scripts/check_publish_policy.mjs \ && node scripts/check_gate_lists.mjs \ + && node scripts/check_ci_workflows.mjs \ && node scripts/check_regex_safety.mjs \ && node scripts/check_code_regions.mjs \ && node scripts/check_page_baseline.mjs \ @@ -87,7 +89,7 @@ POSIX: && node scripts/check_symbol_index.mjs \ && node scripts/check_axe_patch_equiv.mjs -**Six of the eight cannot be affected by an edit confined to `docs/`**, which is why they are separate from `check.bat`. Run this one when the change touches `builder/`, `scripts/`, `book/`, `eval/` or `wisdom/`. Both CI workflows run all eight unconditionally, as they always did, so skipping it locally cannot let a tooling regression reach `staging`. +**Seven of the nine cannot be affected by an edit confined to `docs/`**, which is why they are separate from `check.bat`. Run this one when the change touches `builder/`, `scripts/`, `book/`, `eval/` or `wisdom/`, a wrapper, or a workflow. Both CI workflows run all nine unconditionally, as they always did, so skipping it locally cannot let a tooling regression reach `staging`. The two exceptions are [`check_code_regions.mjs`](#check-code-regions) and [`check_gate_lists.mjs`](#check-gate-lists), which reads this page. The first is worth knowing in detail. Its corpus sweep tokenises every markdown file under `docs/`, so a page that provokes a rewrite into altering a code region fails it. Its fixed probes are a different matter: they run against their own sources whatever the tree holds, and they cover the *mirror* fault, where a rewrite silently stops firing. The sweep cannot see that one --- text the rewrite skipped is stashed and restored unchanged, so every region still matches. Add a page with an unusual code construct and run `test.bat`, but read the built page too. @@ -442,6 +444,17 @@ When it fires on a count that is merely a subset --- *three cheaper gates run fi Its probes ride along in the ordinary run rather than hiding behind `--self-test`, because a green line from a gate that has stopped detecting looks exactly like a green line from a working one. Twelve of the eighteen cover the sweep, each a sentence that was published at the commit round 4 reviewed. Exits 1 on a disagreement or a failed probe, 2 if it cannot run. +### check_ci_workflows.mjs +{: #check-ci-workflows } + + node scripts/check_ci_workflows.mjs + +The same question as [`check_gate_lists.mjs`](#check-gate-lists), asked of the two CI workflows, which nothing else reads. It requires that `checks.yml` and `tbdocs-gh-pages.yml` each run every gate [`test.bat`](#testbat) and [`check.bat`](#checkbat) run, with the same arguments and in each wrapper's own order; that the two workflows run the same gate steps in the same order; and that each workflow's build passes every argument [`build.bat`](#buildbat) passes, plus `--no-fetch-assets`. A step dropped from a workflow, a gate added to a wrapper and never to CI, or a lost `--check-audit-index` would otherwise leave CI green over a check it had stopped making. + +The differences that are meant are listed in the script, each with where it is recorded: `check_tree_fresh.mjs` runs only locally, because CI builds the tree in the same job; the two `check_links_diff.mjs` fixture steps run only in CI, one of them only in `checks.yml`; and the deploy build adds `--url` and `--baseurl`. CI may also interleave the two wrappers' gates, as long as each wrapper's own order holds. Anything else is a finding, and so is an allowance that no longer matches anything. + +Its probes ride along in every run: each plants one defect in a small synthetic set of wrappers and workflows --- a missing gate, a step no wrapper runs, two gates swapped, changed arguments, a build flag lost or added --- and requires exactly the findings it should produce. Pure text: no browser, no built tree. Exits 0 clean, 1 on a finding, 2 when a probe fails or the gate cannot run. + ### check_page_baseline.mjs {: #check-page-baseline } diff --git a/scripts/check_ci_workflows.mjs b/scripts/check_ci_workflows.mjs new file mode 100644 index 00000000..74f4bc83 --- /dev/null +++ b/scripts/check_ci_workflows.mjs @@ -0,0 +1,293 @@ +#!/usr/bin/env node +// The gate that both CI workflows run the gates the wrappers run. +// +// Nothing else reads a workflow. A step dropped from one, a gate added to +// test.bat and never to CI, or a lost --check-audit-index would leave CI green +// over a check it had stopped making. This reads test.bat, check.bat and +// build.bat, and the build job of each workflow, and requires: +// +// 1. each workflow runs every gate the two wrappers run, with the same +// arguments, and in each wrapper's own order. CI interleaves the two +// wrappers' gates -- check_axe_patch_equiv.mjs, a test.bat gate, runs +// among check.bat's -- and that is allowed; +// 2. the two workflows run the same gate steps, in the same order; +// 3. each workflow's build passes every argument build.bat passes, and +// --no-fetch-assets, and nothing else unless ALLOWED records it. +// +// ALLOWED lists the recorded differences, each with where it is recorded. A +// difference not on it is a finding, and so is an allowance that matches +// nothing: one nobody needs is one waiting to hide a real gap. +// +// The probes ride along in every run. Each plants one defect in a small +// synthetic set of wrappers and workflows and requires exactly the findings it +// should produce, so a green run cannot be a gate that has stopped looking. +// +// node scripts/check_ci_workflows.mjs +// +// Exit codes: 0 clean, 1 a finding, 2 a probe failed or the gate crashed. + +import { readFileSync } from "node:fs"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; +import yaml from "js-yaml"; +import { buildArgs, gateSteps, workflowSteps } from "./lib/gate-roster.mjs"; + +process.on("uncaughtException", (err) => { console.error(err); process.exit(2); }); + +const ROOT = path.resolve(fileURLToPath(new URL("..", import.meta.url))); +const JOB = "build"; +const WORKFLOWS = ["checks.yml", "tbdocs-gh-pages.yml"]; + +const ALLOWED = { + localOnly: { + "check_tree_fresh.mjs": "CI builds the tree in the same job, so it cannot be stale (check.bat)", + }, + ciOnly: [ + { + script: "check_links_diff.mjs", + args: "--case fixture --a script --b index", + workflows: ["checks.yml", "tbdocs-gh-pages.yml"], + why: "the standalone link checker against its fixture; no wrapper runs it (the step's comment in each workflow)", + }, + { + script: "check_links_diff.mjs", + args: "--case fixture-built --case fixture-built-offline --a script --b fused", + workflows: ["checks.yml"], + why: "costs two builds, so it runs before a merge and not on every deploy (PLAN-REVIEW-c9f2dfe0-1b6922b.md, decision 1)", + }, + ], + buildFlags: { + "tbdocs-gh-pages.yml": { + "--url": "the Pages origin the deploy publishes to", + "--baseurl": "the Pages base path the deploy publishes under", + }, + }, + requiredBuildFlags: { + "--no-fetch-assets": "CI must never download an asset (vendor-assets.mjs)", + }, +}; + +const keyOf = (s) => (s.args ? `${s.script} ${s.args}` : s.script); + +// Remove one occurrence of each of `drop` from `list`; return the rest and +// whatever in `drop` was not found. +function removeEach(list, drop) { + const rest = [...list]; + const absent = []; + for (const k of drop) { + const at = rest.indexOf(k); + if (at < 0) absent.push(k); + else rest.splice(at, 1); + } + return { rest, absent }; +} + +/** + * The findings for one set of texts, as {kind, where, text}. Pure, so that the + * probes can run it on synthetic sets. `workflows` maps a file name to its + * parsed YAML. + */ +function findings({ testBat, checkBat, buildBat, workflows, allowed }) { + const out = []; + const add = (kind, where, text) => out.push({ kind, where, text }); + + const wrappers = [["test.bat", testBat], ["check.bat", checkBat]].map(([name, text]) => ({ + name, + keys: gateSteps(text).filter((s) => !(s.script in allowed.localOnly)).map(keyOf), + })); + const roster = wrappers.flatMap((w) => w.keys); + const wantBuild = buildArgs(buildBat) ?? []; + const names = Object.keys(workflows); + const shared = new Set( + allowed.ciOnly.filter((e) => names.every((n) => e.workflows.includes(n))).map(keyOf), + ); + const compared = {}; + + for (const wf of names) { + const steps = workflowSteps(workflows[wf], JOB); + const all = steps.flatMap((s) => s.gates.map(keyOf)); + const mine = allowed.ciOnly.filter((e) => e.workflows.includes(wf)); + const { rest: core, absent } = removeEach(all, mine.map(keyOf)); + for (const k of absent) { + add("stale", wf, `the allowance for \`${k}\` matches no step`); + } + + // 1. The wrappers' gates, each once, with its arguments, in its wrapper's order. + const missing = removeEach(roster, core).rest; + const extra = removeEach(core, roster).rest; + for (const k of missing) add("missing", wf, `\`${k}\` is in a wrapper and not in this workflow`); + for (const k of extra) add("extra", wf, `\`${k}\` is in this workflow and in neither wrapper, nor allowed`); + for (const w of wrappers) { + const inWrapper = new Set(w.keys); + const inCore = new Set(core); + const want = w.keys.filter((k) => inCore.has(k)); + const got = core.filter((k) => inWrapper.has(k)); + if (want.join("\n") !== got.join("\n")) { + add("order", wf, `runs ${w.name}'s gates in another order: ${got.join(", ")}`); + } + } + + // 2. What the workflows must have in common: everything but an allowance + // that only some of them have. + compared[wf] = all.filter((k) => roster.includes(k) || shared.has(k)); + + // 3. The build. + const build = steps.find((s) => s.build)?.build; + if (!build) { + add("build", wf, "runs no `node builder/tbdocs.mjs` build"); + continue; + } + for (const t of removeEach(wantBuild, build).rest) { + add("build", wf, `the build does not pass \`${t}\`, which build.bat does`); + } + for (const flag of Object.keys(allowed.requiredBuildFlags)) { + if (!build.includes(flag)) add("build", wf, `the build does not pass \`${flag}\` (${allowed.requiredBuildFlags[flag]})`); + } + const flagsHere = allowed.buildFlags[wf] ?? {}; + const expected = removeEach(build, [...wantBuild, ...Object.keys(allowed.requiredBuildFlags)]).rest; + const used = new Set(); + for (let i = 0; i < expected.length; i++) { + const t = expected[i]; + if (t in flagsHere) { + used.add(t); + if (i + 1 < expected.length && !expected[i + 1].startsWith("--")) i++; + continue; + } + add("build", wf, `the build passes \`${t}\`, which build.bat does not and nothing allows`); + } + for (const flag of Object.keys(flagsHere)) { + if (!used.has(flag)) add("stale", wf, `the allowance for the build's \`${flag}\` matches nothing`); + } + } + + for (let i = 1; i < names.length; i++) { + const a = compared[names[0]]; + const b = compared[names[i]]; + if (a.join("\n") === b.join("\n")) continue; + let at = 0; + while (at < a.length && a[at] === b[at]) at++; + add("differ", `${names[0]} / ${names[i]}`, + `the gate steps part at step ${at + 1}: \`${a[at] ?? "(none)"}\` against \`${b[at] ?? "(none)"}\``); + } + return out; +} + +// ---------------------------------------------------------------- probes + +const P_TEST = [ + '@pushd "%~dp0"', + "node scripts/a.mjs", + "@if errorlevel 1 goto :fail", + "@rem node scripts/commented.mjs", + "node scripts/b.mjs", +].join("\r\n"); +const P_CHECK = "node scripts/fresh.mjs\nnode scripts/c.mjs --check\n"; +const P_BUILD = "node builder\\tbdocs.mjs --src docs --check-audit-index %*\n"; +const P_ALLOWED = { + localOnly: { "fresh.mjs": "probe" }, + ciOnly: [{ script: "ci.mjs", args: "", workflows: ["one.yml"], why: "probe" }], + buildFlags: { "two.yml": { "--url": "probe" } }, + requiredBuildFlags: { "--no-fetch-assets": "probe" }, +}; +const BUILD_ONE = "node builder/tbdocs.mjs --src docs --no-fetch-assets --check-audit-index"; +const BUILD_TWO = "node builder/tbdocs.mjs --src docs --url '${{ steps.pages.outputs.origin }}' --no-fetch-assets --check-audit-index"; +const GOOD = ["a.mjs", "b.mjs", "c.mjs --check"]; + +function wf(gates, build) { + const steps = [{ name: "Checkout", uses: "actions/checkout@v5" }]; + if (build) steps.push({ name: "Build", run: build }); + for (const g of gates) steps.push({ name: g, run: `node scripts/${g}` }); + return { jobs: { [JOB]: { steps } } }; +} + +function pair(one, two) { + return { "one.yml": one, "two.yml": two }; +} + +const PROBES = [ + ["the recorded differences alone", {}, []], + ["a gate missing from one workflow", + { workflows: pair(wf(["a.mjs", "c.mjs --check", "ci.mjs"], BUILD_ONE), wf(GOOD, BUILD_TWO)) }, + ["missing", "differ"]], + ["a step neither wrapper runs", + { workflows: pair(wf([...GOOD, "ci.mjs", "x.mjs"], BUILD_ONE), wf([...GOOD, "x.mjs"], BUILD_TWO)) }, + ["extra"]], + ["two gates reordered in both workflows", + { workflows: pair(wf(["b.mjs", "a.mjs", "c.mjs --check", "ci.mjs"], BUILD_ONE), wf(["b.mjs", "a.mjs", "c.mjs --check"], BUILD_TWO)) }, + ["order"]], + ["the two wrappers interleaved alike", + { workflows: pair(wf(["a.mjs", "c.mjs --check", "b.mjs", "ci.mjs"], BUILD_ONE), wf(["a.mjs", "c.mjs --check", "b.mjs"], BUILD_TWO)) }, + []], + ["the two workflows interleaved differently", + { workflows: pair(wf(["a.mjs", "c.mjs --check", "b.mjs", "ci.mjs"], BUILD_ONE), wf(GOOD, BUILD_TWO)) }, + ["differ"]], + ["a gate's arguments changed", + { workflows: pair(wf(["a.mjs", "b.mjs", "c.mjs", "ci.mjs"], BUILD_ONE), wf(["a.mjs", "b.mjs", "c.mjs"], BUILD_TWO)) }, + ["missing", "extra"]], + ["a gate added to test.bat and not to CI", + { testBat: `${P_TEST}\r\nnode scripts/d.mjs\r\n` }, + ["missing"]], + ["a build without --check-audit-index", + { workflows: pair(wf([...GOOD, "ci.mjs"], BUILD_ONE), wf(GOOD, BUILD_TWO.replace(" --check-audit-index", ""))) }, + ["build"]], + ["a build without --no-fetch-assets", + { workflows: pair(wf([...GOOD, "ci.mjs"], BUILD_ONE.replace(" --no-fetch-assets", "")), wf(GOOD, BUILD_TWO)) }, + ["build"]], + ["a build argument nothing allows", + { workflows: pair(wf([...GOOD, "ci.mjs"], `${BUILD_ONE} --no-offline`), wf(GOOD, BUILD_TWO)) }, + ["build"]], + ["a workflow with no build", + { workflows: pair(wf([...GOOD, "ci.mjs"]), wf(GOOD, BUILD_TWO)) }, + ["build"]], + ["allowances that match nothing", + { workflows: pair(wf(GOOD, BUILD_ONE), wf(GOOD, BUILD_ONE)) }, + ["stale"]], +]; + +function runProbes() { + const failures = []; + for (const [name, override, want] of PROBES) { + const input = { + testBat: P_TEST, checkBat: P_CHECK, buildBat: P_BUILD, + workflows: pair(wf([...GOOD, "ci.mjs"], BUILD_ONE), wf(GOOD, BUILD_TWO)), + allowed: P_ALLOWED, + ...override, + }; + const got = [...new Set(findings(input).map((f) => f.kind))].sort(); + const expected = [...want].sort(); + if (got.join(",") !== expected.join(",")) { + failures.push(`${name}: expected [${expected.join(", ")}], got [${got.join(", ")}]`); + } + } + return failures; +} + +// ------------------------------------------------------------------ main + +const probeFailures = runProbes(); +if (probeFailures.length) { + console.error(`check_ci_workflows: ${probeFailures.length} of ${PROBES.length} probes failed:`); + for (const f of probeFailures) console.error(` ${f}`); + process.exit(2); +} +console.log(`check_ci_workflows: ${PROBES.length} probes, all pass`); + +const read = (rel) => readFileSync(path.join(ROOT, rel), "utf8"); +const workflows = {}; +for (const name of WORKFLOWS) workflows[name] = yaml.load(read(`.github/workflows/${name}`)); +const found = findings({ + testBat: read("test.bat"), + checkBat: read("check.bat"), + buildBat: read("build.bat"), + workflows, + allowed: ALLOWED, +}); + +if (found.length) { + console.error(`check_ci_workflows: ${found.length} finding(s):`); + for (const f of found) console.error(` ${f.where}: ${f.kind}: ${f.text}`); + process.exit(1); +} +const gateCount = gateSteps(read("test.bat")).length + + gateSteps(read("check.bat")).filter((s) => !(s.script in ALLOWED.localOnly)).length; +console.log(`check_ci_workflows: both workflows run the wrappers' ${gateCount} gates, and build as build.bat does`); diff --git a/scripts/check_gate_lists.mjs b/scripts/check_gate_lists.mjs index 8a778a60..454e107a 100644 --- a/scripts/check_gate_lists.mjs +++ b/scripts/check_gate_lists.mjs @@ -81,6 +81,7 @@ import { readFile, readdir } from "node:fs/promises"; import path from "node:path"; import { fileURLToPath } from "node:url"; +import { gatesFromBat } from "./lib/gate-roster.mjs"; const REPO = path.dirname(path.dirname(fileURLToPath(import.meta.url))); const TOOLS_MD = "docs/Documentation/Tools.md"; @@ -100,23 +101,6 @@ const NUMBER_WORDS = [ "six", "seven", "eight", "nine", "ten", "eleven", "twelve", ]; -/** - * The gate scripts a batch file invokes, in order. - * - * Matches a `node scripts/.mjs` invocation at the start of a line. The - * wrappers chain with `@if errorlevel`, not `&&`, so one invocation per line - * holds -- and a continuation or a commented line (`@rem`, `rem`) must not - * count, which anchoring at the line start gives for free. - */ -function gatesFromBat(src) { - const out = []; - for (const line of src.split(/\r?\n/)) { - const m = /^\s*(?:@)?node\s+scripts[\\/]([A-Za-z0-9_]+\.mjs)/.exec(line); - if (m) out.push(m[1]); - } - return out; -} - /** The body of a `### ` section: up to the next heading of any level. */ function sectionBody(md, heading) { const lines = md.split(/\r?\n/); diff --git a/scripts/lib/gate-roster.mjs b/scripts/lib/gate-roster.mjs new file mode 100644 index 00000000..d9c06c95 --- /dev/null +++ b/scripts/lib/gate-roster.mjs @@ -0,0 +1,62 @@ +// The gates a wrapper or a CI workflow runs, read from its own text. +// +// A wrapper runs one `node scripts/.mjs` per line, chained with +// `@if errorlevel` rather than `&&`; a workflow runs one per step's `run:`. +// Both are read the same way: an invocation at the start of a line, so a +// commented line (`@rem`, `rem`, `#`) or a continuation never counts. +// +// check_gate_lists.mjs compares the wrappers with Tools.md's numbered lists, +// and check_ci_workflows.mjs compares them with the two workflows. + +const GATE_LINE = /^\s*@?node\s+scripts[\\/]([A-Za-z0-9_]+\.mjs)[ \t]*(.*?)\s*$/; +const BUILD_LINE = /^\s*@?node\s+builder[\\/]tbdocs\.mjs[ \t]*(.*?)\s*$/; + +function matchLines(text, re) { + const out = []; + for (const line of String(text).split(/\r?\n/)) { + const m = re.exec(line); + if (m) out.push(m); + } + return out; +} + +/** The gate scripts a text invokes, in order, as `{script, args}`. */ +export function gateSteps(text) { + return matchLines(text, GATE_LINE).map((m) => ({ script: m[1], args: m[2] })); +} + +/** The gate scripts a batch file invokes, in order: the names alone. */ +export function gatesFromBat(src) { + return gateSteps(src).map((s) => s.script); +} + +/** + * The arguments of the first tbdocs build a text runs, split into tokens, or + * null if it runs none. A quoted token keeps its spaces, which a workflow's + * `'${{ steps.pages.outputs.origin }}'` needs; a batch file's `%*` is dropped. + */ +export function buildArgs(text) { + const m = matchLines(text, BUILD_LINE)[0]; + if (!m) return null; + const out = []; + const re = /'([^']*)'|"([^"]*)"|(\S+)/g; + let t; + while ((t = re.exec(m[1]))) { + const token = t[1] ?? t[2] ?? t[3]; + if (token !== "%*") out.push(token); + } + return out; +} + +/** + * One job's steps from a parsed workflow, each with the gates and the build it + * runs: `{name, gates, build}`. + */ +export function workflowSteps(workflow, job) { + const steps = workflow?.jobs?.[job]?.steps ?? []; + return steps.map((s) => ({ + name: s.name ?? s.uses ?? "", + gates: gateSteps(s.run ?? ""), + build: buildArgs(s.run ?? ""), + })); +} diff --git a/test.bat b/test.bat index 31afd9e4..f5e5de90 100644 --- a/test.bat +++ b/test.bat @@ -14,8 +14,9 @@ @rem developer page that says how many gates a wrapper runs. @rem @rem Otherwise run it when the change touches builder/, scripts/, book/, -@rem eval/ or wisdom/. Both CI workflows run it unconditionally, so a -@rem tooling regression cannot reach staging by someone skipping it. +@rem eval/, wisdom/, a wrapper or a workflow. Both CI workflows run it +@rem unconditionally, so a tooling regression cannot reach staging by +@rem someone skipping it. @rem @rem The split is by what a gate INTERROGATES, not by what it happens to @rem open: check_axe_patch_equiv.mjs loads a built page, but only because @@ -43,6 +44,15 @@ node scripts/check_publish_policy.mjs @rem failed a gate. Pure text, no tree, no browser, ~50 ms. node scripts/check_gate_lists.mjs @if errorlevel 1 goto :fail +@rem The same question asked of the two CI workflows, which nothing else +@rem reads: do they run every gate this file and check.bat run, with the +@rem same arguments and in the same order, and build as build.bat does? +@rem A step dropped from one would leave CI green over a check it had +@rem stopped making. The differences that are meant are listed, each with +@rem where it is recorded. Its probes ride along. No tree, no browser, +@rem ~100 ms. +node scripts/check_ci_workflows.mjs +@if errorlevel 1 goto :fail @rem A regex that backtracks exponentially is a hang waiting for the @rem right input, and nothing that reads the site can see it: the corpus @rem passes until some page happens to contain the trigger, and then the From 582ab0127b806ddf83edd93fc578780c74c4b814 Mon Sep 17 00:00:00 2001 From: Kuba Sunderland-Ober Date: Fri, 25 Sep 2026 06:35:50 +0200 Subject: [PATCH 13/53] ci: one composite action for the gates both workflows run --- .github/actions/run-gates/action.yml | 170 ++++++++++++++++++++++++++ .github/workflows/checks.yml | 166 +++---------------------- .github/workflows/tbdocs-gh-pages.yml | 110 ++--------------- WIP.md | 5 +- builder/PLAN-TOOLING-REVIEW.md | 17 +++ docs/Documentation/Extending.md | 6 +- docs/Documentation/Tools.md | 4 +- scripts/check_ci_workflows.mjs | 51 +++++++- scripts/lib/gate-roster.mjs | 28 +++-- 9 files changed, 288 insertions(+), 269 deletions(-) create mode 100644 .github/actions/run-gates/action.yml diff --git a/.github/actions/run-gates/action.yml b/.github/actions/run-gates/action.yml new file mode 100644 index 00000000..bcdcb0b9 --- /dev/null +++ b/.github/actions/run-gates/action.yml @@ -0,0 +1,170 @@ +# The gate steps both workflows run, in their order, after the build. +# +# checks.yml runs them on every pull request, before a merge; tbdocs-gh-pages.yml +# runs them on every push to `staging`, before it deploys -- which makes that +# the one place a change pushed straight to `staging`, or a manual dispatch, +# meets them. One list serves both, so a gate cannot be added to one workflow +# and forgotten in the other; scripts/check_ci_workflows.mjs checks this list +# against test.bat and check.bat. Each gate is its own step, shown as its own +# group in the job's log. +# +# Needs the repository checked out, dependencies installed, Chromium +# installed, and the site built into docs/_site*, as both workflows do before +# calling it. +name: Run the gates +description: The gate steps both workflows share, checked against test.bat and check.bat by check_ci_workflows.mjs. +runs: + using: composite + steps: + # The build is the only pass over the site's HTML. This step is NOT a + # second one: it runs the differential harness against a nine-file + # synthetic tree that carries one fault of every kind, so + # scripts/check_links.mjs -- still the tool for a tree the build did not + # produce, and the oracle the fused path is defined against -- cannot rot + # unnoticed. ~0.3 s, no site files touched. + # + # The full script-vs-fused comparison over the real trees stays a manual + # gate (`check_links_diff.mjs --a script --b fused`): running it here would + # mean checking every page twice, which is exactly what folding the check + # into the build removed. + - name: Verify the standalone link checker (check_links_diff.mjs) + shell: bash + run: node scripts/check_links_diff.mjs --case fixture --a script --b index + # The publish allowlist (builder/publish-policy.mjs) is enforced inside + # the build, so a green build already says nothing unpublishable is in + # docs/. It does NOT say the allowlist still refuses anything: one widened + # until it refuses nothing reports the same clean pass. This asserts the + # refusals against named probes -- a stray .bak, a .pem, a scratch .md + # with no frontmatter. No browser, no built tree, ~40 ms. + - name: Verify the publish allowlist (check_publish_policy.mjs) + shell: bash + run: node scripts/check_publish_policy.mjs + # The two gate lists on Tools.md against the wrappers that run them, plus + # every gate count stated in prose in README.md or under + # docs/Documentation/. Documented gate counts have rotted three times; the + # third was round 3's own fix pass, on the two pages the first version of + # this gate did not read. Pure text, ~50 ms. + - name: Verify the documented gate lists (check_gate_lists.mjs) + shell: bash + run: node scripts/check_gate_lists.mjs + # Both workflows, and this action, against the wrappers: every gate + # test.bat and check.bat run, with the same arguments and in the same + # order, and a build with build.bat's flags. See the script's header for + # the differences it allows. No browser, no built tree. + - name: Verify the workflows run the wrappers' gates (check_ci_workflows.mjs) + shell: bash + run: node scripts/check_ci_workflows.mjs + # A regex that backtracks exponentially does not fail a build, it stops + # one: the corpus passes until a page happens to contain the trigger, and + # then a render worker sits inside String.replace forever. VOID_TAGS_RE + # shipped that way for as long as no alt text contained a slash, and its + # first fix was still exponential on a subtler witness. Nothing else here + # can see it, because there is nothing to see until the content changes. + # + # Reads regex literals AND every `new RegExp(...)` the source decides the + # arguments of. That second half is not a refinement: assembling a pattern + # from shared string constants is ordinary JavaScript, and for as long as + # this read literals only it was also a way out of the gate. Six such + # regexes went into one gate unseen, one of them polynomial. + # + # Gates on exponential only -- about a fifth of the patterns here are + # polynomial, nearly all the ordinary `]*>` shape on bounded input, + # and a gate that fails on day one gets switched off. The self-test probes + # run inside the same pass, classification and folding both, so a green + # line here cannot be a gate that has stopped detecting. No browser, no + # built tree, a few seconds sharded across the runner's CPUs. + # + # recheck's native backend is an optional dependency resolved per + # platform; if it is missing the script says so and falls back to the + # pure-JS implementation, which is slower but reaches the same verdict on + # every probe. + - name: Verify regex safety (check_regex_safety.mjs) + shell: bash + run: node scripts/check_regex_safety.mjs + # render.mjs applies several kramdown-parity rewrites to RAW markdown, + # before anything has been parsed, so none of them can tell prose from + # code -- on a site whose subject matter is code. Four defects of that + # shape shipped: `{% raw %}` stripped inside fences, admonition bodies + # losing the indentation of the code they contain, `Items[1](a, b)` + # percent-encoded inside a fence, and a YAML sample's closing `---` + # deleted with the line above it promoted to a heading. + # + # No other gate can see any of it: the corruption is inside , and + # the link, integrity, publish and axe checks all pass over it. Tokenise + # the source, apply the real rewrite chain, re-tokenise, and compare the + # literal regions. Probes ride along in the same run so a clean corpus + # cannot be mistaken for a working gate. No browser, no built tree. + - name: Verify code regions survive the pre-render rewrites (check_code_regions.mjs) + shell: bash + run: node scripts/check_code_regions.mjs + # The page-count drift guard reports nothing on a healthy tree, so a green + # build says exactly what a guard that had stopped working says. These + # probes make the other assertion, against a scratch baseline rather than + # the committed one. The first replays the defect that motivated it: 37 + # pages of AppGlobalClassObject lost to a blanket exclude rule, under a + # guard that knew only a floor of 836 against a real 908. No browser, no + # built tree. + - name: Verify the page-count drift guard (check_page_baseline.mjs) + shell: bash + run: node scripts/check_page_baseline.mjs + # The book-coverage warnings say nothing when every page has an entry in + # docs/_book.yml, which is also all a check that had stopped working would + # say. Before they existed, whole sections dropped out of the PDF without a + # word. These probes give each of the five findings a fault to report, on + # a manifest and pages built in memory. No browser, no built tree. + - name: Verify the book-coverage warnings (check_book_coverage.mjs) + shell: bash + run: node scripts/check_book_coverage.mjs + # The symbol index (tB/symbols.json) is read by the IDE help add-in. A + # build that indexes the reference cleanly says nothing about the rules + # that did not fire on it, so these probes assert each against the case + # that made it necessary -- the .twin scanner's traps, the rules placing a + # symbol on a page or heading, and the drift guard refusing a URL the index + # stopped publishing. Fixtures only: no tree, no install. + - name: Verify the symbol index and its drift guard (check_symbol_index.mjs) + shell: bash + run: node scripts/check_symbol_index.mjs + # Graphviz sizes each node box from a width table; the browser paints the + # label with a real font. Nothing in the build compares the two, so a + # mismatch ships as text hanging outside its box on a green build -- which + # is how 27 labels across three diagrams went out, through a full + # accessibility sweep, unreported. axe does not evaluate SVG + # geometry either. + # + # After the build on purpose: builder/dot.mjs rewrites a stale .svg from + # its .dot in place, so this measures the bytes about to be deployed, not + # the ones that were committed. It loads Inter from the committed .woff2 by + # @font-face rather than relying on an installed face, so unlike the + # target-size rules it measures the same here as on a dev box. ~1 s. + - name: Verify DOT diagram fit (check_dot_fit.mjs) + shell: bash + run: node scripts/check_dot_fit.mjs + # check_a11y.mjs injects a PATCHED axe bundle (plain-color-fields, -26 % on + # a realistic page set -- see builder/PLAN-axe-perf.md), so the patch has + # to be verified before its results are trusted. The patch asserts its + # substitution targets and so fails loudly if an axe-core bump moves the + # code; this catches the other case, where the text still matches but the + # colour maths has changed. The fingerprint gate cannot see that -- it + # compares `incomplete` as a rule-id set. + # + # Cheap (one page, two bundles), and Chromium is already installed. The + # change that bumps axe-core is exactly when it earns its place. + - name: Verify axe source patch (check_axe_patch_equiv.mjs) + shell: bash + run: node scripts/check_axe_patch_equiv.mjs + # The scan is thirteen pages of ~1,160, so its page list decides what it + # can report at all. This fails when the site grows a markup construct no + # sample page carries -- the drift that let the previous six-page sample + # report a clean pass while 54 pages had violations in constructs it never + # saw. No browser, ~1 s. + - name: Verify a11y sample coverage (pick_a11y_sample.mjs) + shell: bash + run: node scripts/pick_a11y_sample.mjs --check + # Matches check.bat: the build's own link check gates this, and a link + # failure short-circuits before the (slower) browser scan runs. Scans + # _site-offline/, whose relative asset paths actually resolve -- _site/ + # uses root-absolute URLs that render unstyled here, making every + # colour-contrast result meaningless. + - name: Accessibility check (check_a11y.mjs) + shell: bash + run: node scripts/check_a11y.mjs diff --git a/.github/workflows/checks.yml b/.github/workflows/checks.yml index a93e4609..6ca64c04 100644 --- a/.github/workflows/checks.yml +++ b/.github/workflows/checks.yml @@ -98,161 +98,23 @@ jobs: # step fails on the exit code: 1 for link failures, 2 for # integrity failures, 3 for both. run: node builder/tbdocs.mjs --src docs --no-fetch-assets --check-audit-index - # The build above is the only pass over the site's HTML. This step is - # NOT a second one: it runs the differential harness against a - # nine-file synthetic tree that carries one fault of every kind, so - # scripts/check_links.mjs -- still the tool for a tree the build did - # not produce, and the oracle the fused path is defined against -- - # cannot rot unnoticed. ~0.3 s, no site files touched. - # - # The full script-vs-fused comparison over the real trees stays a - # manual gate (`check_links_diff.mjs --a script --b fused`): running - # it here would mean checking every page twice, which is exactly what - # folding the check into the build removed. - - name: Verify the standalone link checker (check_links_diff.mjs) - run: node scripts/check_links_diff.mjs --case fixture --a script --b index - # The step above compares two front ends over a hand-written tree. - # This one compares the script against the BUILD's own checker, over - # a three-page tree the build produces from test/fixtures/check-src. - # That is the pass the harness exists for and the one nothing - # exercised: `--b fused` skipped the synthetic `fixture` case every - # time, because the build cannot check a tree it did not write. A - # regression in builder/check.mjs that stopped REPORTING a category - # would have left every gate green. + # The gates both workflows run, in one list: see + # .github/actions/run-gates/action.yml, which check_ci_workflows.mjs + # checks against test.bat and check.bat. Each gate is its own group in + # this step's log. + - name: Run the gates + uses: ./.github/actions/run-gates + # The action's first step compares the standalone link checker's two + # front ends over a hand-written tree. This one compares the script + # against the BUILD's own checker, over a three-page tree the build + # produces from test/fixtures/check-src. That is the pass the harness + # exists for and the one nothing exercised: `--b fused` skipped the + # synthetic `fixture` case every time, because the build cannot check a + # tree it did not write. A regression in builder/check.mjs that stopped + # REPORTING a category would have left every gate green. # # One extra three-page build, ~1 s. It is here and not in the deploy # workflow because this is the PR gate: catching it before a merge is # the point, and the deploy workflow has a site to ship. - name: Verify the build's own link checker (check_links_diff.mjs) run: node scripts/check_links_diff.mjs --case fixture-built --case fixture-built-offline --a script --b fused - # The publish allowlist (builder/publish-policy.mjs) is enforced - # inside the build above, so a green build already says nothing - # unpublishable is in docs/. It does NOT say the allowlist still - # refuses anything: one widened until it refuses nothing reports the - # same clean pass. This asserts the refusals against named probes -- - # a stray .bak, a .pem, a scratch .md with no frontmatter. No - # browser, no built tree, ~40 ms. - - name: Verify the publish allowlist (check_publish_policy.mjs) - run: node scripts/check_publish_policy.mjs - # The two gate lists on Tools.md against the wrappers that run - # them, plus every gate count stated in prose in README.md or - # under docs/Documentation/. Documented gate counts have rotted - # three times; the third was round 3's own fix pass, on the two - # pages the first version of this gate did not read. Pure text, - # ~50 ms. - - name: Verify the documented gate lists (check_gate_lists.mjs) - run: node scripts/check_gate_lists.mjs - # This workflow and tbdocs-gh-pages.yml against the wrappers: every - # gate test.bat and check.bat run, with the same arguments and in the - # same order, and a build with build.bat's flags. See the script's - # header for the differences it allows. No browser, no built tree. - - name: Verify the workflows run the wrappers' gates (check_ci_workflows.mjs) - run: node scripts/check_ci_workflows.mjs - # A regex that backtracks exponentially does not fail a build, it - # stops one: the corpus passes until a page happens to contain the - # trigger, and then a render worker sits inside String.replace - # forever. VOID_TAGS_RE shipped that way for as long as no alt text - # contained a slash, and its first fix was still exponential on a - # subtler witness. Nothing else here can see it, because there is - # nothing to see until the content changes. - # - # Reads regex literals AND every `new RegExp(...)` the source - # decides the arguments of. That second half is not a refinement: - # assembling a pattern from shared string constants is ordinary - # JavaScript, and for as long as this read literals only it was - # also a way out of the gate. Six such regexes went into one gate - # unseen, one of them polynomial. - # - # Gates on exponential only -- about a fifth of the patterns here - # are polynomial, nearly all the ordinary `]*>` shape on - # bounded input, and a gate that fails on day one gets switched - # off. The self-test probes run inside the same pass, classification - # and folding both, so a green line here cannot be a gate that has - # stopped detecting. No browser, no built tree, a few seconds - # sharded across the runner's CPUs. - # - # recheck's native backend is an optional dependency resolved per - # platform; if it is missing the script says so and falls back to - # the pure-JS implementation, which is slower but reaches the same - # verdict on every probe. - - name: Verify regex safety (check_regex_safety.mjs) - run: node scripts/check_regex_safety.mjs - # render.mjs applies several kramdown-parity rewrites to RAW markdown, - # before anything has been parsed, so none of them can tell prose from - # code -- on a site whose subject matter is code. Four defects of that - # shape shipped: `{% raw %}` stripped inside fences, admonition bodies - # losing the indentation of the code they contain, `Items[1](a, b)` - # percent-encoded inside a fence, and a YAML sample's closing `---` - # deleted with the line above it promoted to a heading. - # - # No other gate can see any of it: the corruption is inside , - # and the link, integrity, publish and axe checks all pass over it. - # Tokenise the source, apply the real rewrite chain, re-tokenise, and - # compare the literal regions. Probes ride along in the same run so a - # clean corpus cannot be mistaken for a working gate. No browser, no - # built tree. - - name: Verify code regions survive the pre-render rewrites (check_code_regions.mjs) - run: node scripts/check_code_regions.mjs - # The page-count drift guard reports nothing on a healthy tree, so a - # green build says exactly what a guard that had stopped working says. - # These probes make the other assertion, against a scratch baseline - # rather than the committed one. The first replays the defect that - # motivated it: 37 pages of AppGlobalClassObject lost to a blanket - # exclude rule, under a guard that knew only a floor of 836 against a - # real 908. No browser, no built tree. - - name: Verify the page-count drift guard (check_page_baseline.mjs) - run: node scripts/check_page_baseline.mjs - # The book-coverage warnings say nothing when every page has an entry - # in docs/_book.yml, which is also all a check that had stopped working - # would say. Before they existed, whole sections dropped out of the PDF - # without a word. These probes give each of the five findings a fault - # to report, on a manifest and pages built in memory. No browser, no - # built tree. - - name: Verify the book-coverage warnings (check_book_coverage.mjs) - run: node scripts/check_book_coverage.mjs - # The symbol index (tB/symbols.json) is read by the IDE help add-in. A - # build that indexes the reference cleanly says nothing about the rules - # that did not fire on it, so these probes assert each against the case - # that made it necessary -- the .twin scanner's traps, the rules placing - # a symbol on a page or heading, and the drift guard refusing a URL the - # index stopped publishing. Fixtures only: no tree, no install. - - name: Verify the symbol index and its drift guard (check_symbol_index.mjs) - run: node scripts/check_symbol_index.mjs - # Graphviz sizes each node box from a width table; the browser paints - # the label with a real font. Nothing in the build compares the two, so - # a mismatch ships as text hanging outside its box on a green build -- - # which is how 27 labels across three diagrams went out, through a full - # accessibility sweep, unreported. axe does not evaluate SVG - # geometry either. - # - # Safe on CI's font set: it loads Inter from the committed .woff2 by - # @font-face rather than relying on an installed face, so unlike the - # target-size rules it measures the same here as on a dev box. - - name: Verify DOT diagram fit (check_dot_fit.mjs) - run: node scripts/check_dot_fit.mjs - # check_a11y.mjs injects a PATCHED axe bundle (plain-color-fields, - # -26 % on a realistic page set -- see builder/PLAN-axe-perf.md), so the - # patch has to be verified before its results are trusted. The patch - # asserts its substitution targets and so fails loudly if an axe-core - # bump moves the code; this catches the other case, where the text still - # matches but the colour maths has changed. The fingerprint gate cannot - # see that -- it compares `incomplete` as a rule-id set. - # - # Cheap (one page, two bundles) and Chromium is already installed. The - # PR that bumps axe-core is exactly when it earns its place. - - name: Verify axe source patch (check_axe_patch_equiv.mjs) - run: node scripts/check_axe_patch_equiv.mjs - # The scan is thirteen pages of ~1,160, so its page list decides what it can - # report at all. This fails when the site grows a markup construct no - # sample page carries -- the drift that let the previous six-page sample - # report a clean pass while 54 pages had violations in constructs it never - # saw. No browser, ~1 s. - - name: Verify a11y sample coverage (pick_a11y_sample.mjs) - run: node scripts/pick_a11y_sample.mjs --check - # Matches check.bat: the build's own link check gates this, and a - # link failure short-circuits before the (slower) browser scan runs. - # Scans _site-offline/, whose relative asset paths actually resolve -- - # _site/ uses root-absolute URLs that render unstyled here, making - # every colour-contrast result meaningless. - - name: Accessibility check (check_a11y.mjs) - run: node scripts/check_a11y.mjs diff --git a/.github/workflows/tbdocs-gh-pages.yml b/.github/workflows/tbdocs-gh-pages.yml index a3736a44..04028a69 100644 --- a/.github/workflows/tbdocs-gh-pages.yml +++ b/.github/workflows/tbdocs-gh-pages.yml @@ -94,108 +94,14 @@ jobs: # online tree gets a base path -- the offline tree's links are all # relative after the rewrite. run: node builder/tbdocs.mjs --src docs --url '${{ steps.pages.outputs.origin }}' --baseurl '${{ steps.pages.outputs.base_path }}' --no-fetch-assets --check-audit-index - # Not a second pass over the site: a nine-file synthetic tree carrying - # one fault of every kind, so scripts/check_links.mjs -- still the - # tool for a tree the build did not produce -- cannot rot unnoticed. - # ~0.3 s. See the same step in checks.yml, which additionally runs the - # comparison against the build's own checker. - - name: Verify the standalone link checker (check_links_diff.mjs) - run: node scripts/check_links_diff.mjs --case fixture --a script --b index - # The publish allowlist (builder/publish-policy.mjs) is enforced - # inside the build above, so a green build already says nothing - # unpublishable is in docs/. It does NOT say the allowlist still - # refuses anything: one widened until it refuses nothing reports the - # same clean pass. This asserts the refusals against named probes -- - # a stray .bak, a .pem, a scratch .md with no frontmatter. No - # browser, no built tree, ~40 ms. - - name: Verify the publish allowlist (check_publish_policy.mjs) - run: node scripts/check_publish_policy.mjs - # The two gate lists on Tools.md against the wrappers that run - # them, plus every gate count stated in prose in README.md or - # under docs/Documentation/. Documented gate counts have rotted - # three times; the third was round 3's own fix pass, on the two - # pages the first version of this gate did not read. Pure text, - # ~50 ms. - - name: Verify the documented gate lists (check_gate_lists.mjs) - run: node scripts/check_gate_lists.mjs - # This workflow and checks.yml against the wrappers: every gate - # test.bat and check.bat run, with the same arguments and in the same - # order, and a build with build.bat's flags. See the script's header - # for the differences it allows. No browser, no built tree. - - name: Verify the workflows run the wrappers' gates (check_ci_workflows.mjs) - run: node scripts/check_ci_workflows.mjs - # An exponentially backtracking regex does not fail a build, it - # stops one -- a render worker sits inside String.replace forever - # the first time a page contains the trigger. VOID_TAGS_RE shipped - # that way, and so did its first fix. See the same step in - # checks.yml for what it gates on and why only exponential. - # - # Here for the same reason check_dot_fit.mjs is: checks.yml runs - # only on pull requests, so this is the one gate a regex merged by - # a direct push to `staging` would otherwise never meet. No browser - # and no built tree, so its position relative to the build does not - # matter. - - name: Verify regex safety (check_regex_safety.mjs) - run: node scripts/check_regex_safety.mjs - # The pre-render rewrites in render.mjs run over raw markdown and - # cannot tell prose from code; four defects of that shape shipped, - # and nothing else looks inside . See the same step in - # checks.yml. Here for the same reason the gate above is: checks.yml - # runs only on pull requests, so this is the one place a rewrite - # merged by a direct push to `staging` would meet it. No browser and - # no built tree. - - name: Verify code regions survive the pre-render rewrites (check_code_regions.mjs) - run: node scripts/check_code_regions.mjs - # The page-count drift guard reports nothing on a healthy tree, so a - # green build says exactly what a guard that had stopped working says. - # These probes make the other assertion, against a scratch baseline - # rather than the committed one. The first replays the defect that - # motivated it: 37 pages of AppGlobalClassObject lost to a blanket - # exclude rule, under a guard that knew only a floor of 836 against a - # real 908. No browser, no built tree. - - name: Verify the page-count drift guard (check_page_baseline.mjs) - run: node scripts/check_page_baseline.mjs - # The book-coverage warnings say nothing when every page has an entry - # in docs/_book.yml, which is also all a check that had stopped working - # would say. Before they existed, whole sections dropped out of the PDF - # without a word. These probes give each of the five findings a fault - # to report, on a manifest and pages built in memory. No browser, no - # built tree. - - name: Verify the book-coverage warnings (check_book_coverage.mjs) - run: node scripts/check_book_coverage.mjs - # The symbol index's rules and drift guard, against fixtures. See the - # same step in checks.yml. - - name: Verify the symbol index and its drift guard (check_symbol_index.mjs) - run: node scripts/check_symbol_index.mjs - # Graphviz sizes each node box from a width table; the browser paints the - # label with a real font, and nothing in the build compares the two -- so a - # mismatch ships as text hanging outside its box on a green build. See the - # same step in checks.yml for how 27 labels once went out that way. - # - # checks.yml only runs on pull requests, so this is the one gate a diagram - # merged by a direct push to `staging` -- or by a manual dispatch -- would - # otherwise never meet. - # - # After the build on purpose: builder/dot.mjs rewrites a stale .svg from - # its .dot in place, so this measures the bytes about to be deployed, not - # the ones that were committed. Chromium is already installed above, and - # the check loads Inter from the committed .woff2 rather than an installed - # face, so it measures the same here as on a dev box. Five diagrams, ~1 s. - - name: Verify DOT diagram fit (check_dot_fit.mjs) - run: node scripts/check_dot_fit.mjs - # check_a11y.mjs injects a PATCHED axe bundle; verify the patch is still - # value-preserving before trusting what it reports. See the same step in - # checks.yml for why the fingerprint gate does not cover this. - - name: Verify axe source patch (check_axe_patch_equiv.mjs) - run: node scripts/check_axe_patch_equiv.mjs - # Fails when the site grows a construct no sample page covers; see the - # same step in checks.yml. No browser, ~1 s. - - name: Verify a11y sample coverage (pick_a11y_sample.mjs) - run: node scripts/pick_a11y_sample.mjs --check - # Same gate as check.bat and the PR workflow. Chromium is already - # installed above for the PDF render, so this costs only the scan. - - name: Accessibility check (check_a11y.mjs) - run: node scripts/check_a11y.mjs + # The gates both workflows run, in one list: see + # .github/actions/run-gates/action.yml, which check_ci_workflows.mjs + # checks against test.bat and check.bat. checks.yml runs only on pull + # requests, so this is the one place a change pushed straight to + # `staging`, or a manual dispatch, meets them. Each gate is its own + # group in this step's log. + - name: Run the gates + uses: ./.github/actions/run-gates - name: Render book PDF run: | mkdir -p _pdf diff --git a/WIP.md b/WIP.md index 35c3d794..11ce5401 100644 --- a/WIP.md +++ b/WIP.md @@ -512,7 +512,10 @@ about what a gate interrogates, not about what it happens to open. Both CI workflows run every one of these as its own step, unconditionally and without invoking the `.bat` files --- so skipping `test.bat` locally changes -what a content edit costs you, never what reaches `staging`. +what a content edit costs you, never what reaches `staging`. The steps are one +list, the composite action `.github/actions/run-gates/action.yml`, which both +workflows call: **a new gate goes into its wrapper, that action and Tools.md's +list**, and `check_ci_workflows.mjs` fails `test.bat` until CI matches. The nav integrity check ([builder/nav.mjs](builder/nav.mjs)) runs during COMPUTE and aborts the build on two failure modes, both otherwise silent: diff --git a/builder/PLAN-TOOLING-REVIEW.md b/builder/PLAN-TOOLING-REVIEW.md index 70cdbcea..b1b7dc7e 100644 --- a/builder/PLAN-TOOLING-REVIEW.md +++ b/builder/PLAN-TOOLING-REVIEW.md @@ -497,6 +497,23 @@ a workflow that stops calling it. **Verify.** The roster gate and its probes. CI: a dispatch of `checks.yml` on `origin`, and the deploy workflow's next run. +**Landed.** All thirteen shared steps moved, the standalone link checker's included, so +`checks.yml` runs its fused fixture step after the action, with its comment saying which step +it now follows. The setup steps stay in each workflow: a local action cannot be used before +the repository is checked out, and the two workflows' installs differ only in how many steps +they take. The action has one comment per gate, merged from the two workflows'. `checks.yml`'s +were the long ones, and three facts only the deploy workflow's comments had are kept: +`check_dot_fit.mjs` runs after the build because `dot.mjs` rewrites a stale `.svg` in place, +Chromium is already installed for the PDF render, and the deploy run is the one place a change +pushed straight to `staging` meets the gates. + +`check_ci_workflows.mjs` reads a step that uses a local action as that action's own steps, and +reports one it cannot read; its probes are 17. With `pick_a11y_sample.mjs`'s step removed from +the action, it reported that gate missing from both workflows and exited 1. `Extending.md`'s +rule for registering a gate goes from four places to three, and names both checks that enforce +it. **The cost:** GitHub shows a composite action as one step, with each gate as a named group +inside its log, so a failure reads as "Run the gates" until the log is opened. + ### C05 — `lint: Biome, correctness rules only, and the fixes it finds` **Decision 4**, first half. The linter comes first because moved and deleted code leaves diff --git a/docs/Documentation/Extending.md b/docs/Documentation/Extending.md index 8f1a4765..3a0b3aa4 100644 --- a/docs/Documentation/Extending.md +++ b/docs/Documentation/Extending.md @@ -60,7 +60,7 @@ partial reload. What *is* watched is everything under `docs/` --- page content a `check_regex_safety.mjs` reads every regex under `builder/`, `scripts/`, `book/`, `eval/` and `wisdom/`, and refuses one that can take exponential time on some input. -It runs in `test.bat` and as a step of its own in both CI workflows, so a regex added +It runs in `test.bat` and as a step of its own in the gates both CI workflows run, so a regex added to the builder can pass the build and `check.bat` and still be refused. The report starts with @@ -655,9 +655,9 @@ Separating 1 from 2 is what stops a broken gate reading as a clean site, and it **Be explicit about what may already have run.** Both wrappers stop at the first failure, so a gate's position decides what it can assume --- and the two do not offer the same guarantees. `check_tree_fresh.mjs` runs first in `check.bat`, so every later gate there may assume `_site-offline/` is current. `test.bat` has no freshness gate at all, and its one gate that opens a built page does not need one: `check_axe_patch_equiv.mjs` loads a single page and never reads that page's DOM, so a stale tree cannot change its result. Nothing in either wrapper may assume the build's own link check passed: a link failure sets the build's exit code without aborting the build, so a tree that failed it is still on disk and still fresh. -**Register it in four places** --- the wrapper it belongs in, `.github/workflows/checks.yml`, `.github/workflows/tbdocs-gh-pages.yml`, and that wrapper's numbered list in [Tools and Scripts](Tools#checkbat). The first three each take a comment saying what the gate protects against, which is the convention already in all four files. Both workflows run every gate as its own step, from both wrappers, so the wrapper choice does not change what CI does. Leaving a gate out of the workflows is a decision, not an omission, and gets the same comment: `check_tree_fresh.mjs` is in neither, because CI builds in the same job and cannot have a stale tree. +**Register it in three places** --- the wrapper it belongs in, `.github/actions/run-gates/action.yml`, and that wrapper's numbered list in [Tools and Scripts](Tools#checkbat). The action is the one list of gates both CI workflows run, so a gate goes into CI once. The first two each take a comment saying what the gate protects against, which is the convention already in both files. CI runs every gate from both wrappers, each as its own step of the action, so the wrapper choice does not change what CI does. Leaving a gate out of CI is a decision, not an omission, and is written down with its reason in `check_ci_workflows.mjs`'s list of allowed differences: `check_tree_fresh.mjs` is left out because CI builds in the same job and cannot have a stale tree. -The fourth is the one that is machine enforced, and the only one you will be told about: [`check_gate_lists.mjs`](Tools#check-gate-lists) compares both wrappers against `Tools.md`'s two numbered lists --- membership, order, and the step count each section states --- so adding a gate without the entry fails `test.bat` naming the disagreement. It then sweeps `README.md` and every page under `docs/Documentation/` for a gate count stated anywhere in prose and fails on those too, which is what makes one edit enough: `Tools.md` owns the lists and nothing else restates them. If you find yourself writing a gate count into a second page, that is the thing not to do. +Two checks enforce the registration, and they are the only ones you will be told about. [`check_gate_lists.mjs`](Tools#check-gate-lists) compares both wrappers against `Tools.md`'s two numbered lists --- membership, order, and the step count each section states --- so adding a gate without the entry fails `test.bat` naming the disagreement. It then sweeps `README.md` and every page under `docs/Documentation/` for a gate count stated anywhere in prose and fails on those too, which is what makes one edit enough: `Tools.md` owns the lists and nothing else restates them. If you find yourself writing a gate count into a second page, that is the thing not to do. [`check_ci_workflows.mjs`](Tools#check-ci-workflows) does the same for CI: both workflows, read through the action, must run every wrapper gate with the same arguments and in the same order, and any difference not on its list of allowed ones fails `test.bat`. ### It must be able to fail diff --git a/docs/Documentation/Tools.md b/docs/Documentation/Tools.md index 3c86dc88..a0f0f65b 100644 --- a/docs/Documentation/Tools.md +++ b/docs/Documentation/Tools.md @@ -451,9 +451,11 @@ Its probes ride along in the ordinary run rather than hiding behind `--self-test The same question as [`check_gate_lists.mjs`](#check-gate-lists), asked of the two CI workflows, which nothing else reads. It requires that `checks.yml` and `tbdocs-gh-pages.yml` each run every gate [`test.bat`](#testbat) and [`check.bat`](#checkbat) run, with the same arguments and in each wrapper's own order; that the two workflows run the same gate steps in the same order; and that each workflow's build passes every argument [`build.bat`](#buildbat) passes, plus `--no-fetch-assets`. A step dropped from a workflow, a gate added to a wrapper and never to CI, or a lost `--check-audit-index` would otherwise leave CI green over a check it had stopped making. +The gates both workflows share are one composite action, `.github/actions/run-gates/action.yml`, and the gate reads a workflow step that uses a local action as that action's own steps. A local action it cannot read is a finding, so a renamed action cannot take its gates out of CI unnoticed. + The differences that are meant are listed in the script, each with where it is recorded: `check_tree_fresh.mjs` runs only locally, because CI builds the tree in the same job; the two `check_links_diff.mjs` fixture steps run only in CI, one of them only in `checks.yml`; and the deploy build adds `--url` and `--baseurl`. CI may also interleave the two wrappers' gates, as long as each wrapper's own order holds. Anything else is a finding, and so is an allowance that no longer matches anything. -Its probes ride along in every run: each plants one defect in a small synthetic set of wrappers and workflows --- a missing gate, a step no wrapper runs, two gates swapped, changed arguments, a build flag lost or added --- and requires exactly the findings it should produce. Pure text: no browser, no built tree. Exits 0 clean, 1 on a finding, 2 when a probe fails or the gate cannot run. +Its probes ride along in every run: each plants one defect in a small synthetic set of wrappers, workflows and actions --- a missing gate, a step no wrapper runs, two gates swapped, changed arguments, a build flag lost or added, a gate missing from the shared action, a workflow that stops calling it --- and requires exactly the findings it should produce. Pure text: no browser, no built tree. Exits 0 clean, 1 on a finding, 2 when a probe fails or the gate cannot run. ### check_page_baseline.mjs {: #check-page-baseline } diff --git a/scripts/check_ci_workflows.mjs b/scripts/check_ci_workflows.mjs index 74f4bc83..7daf1856 100644 --- a/scripts/check_ci_workflows.mjs +++ b/scripts/check_ci_workflows.mjs @@ -85,9 +85,10 @@ function removeEach(list, drop) { /** * The findings for one set of texts, as {kind, where, text}. Pure, so that the * probes can run it on synthetic sets. `workflows` maps a file name to its - * parsed YAML. + * parsed YAML, and `actions` maps a local action's `uses:` path to its parsed + * action.yml; a workflow step that uses one is read as that action's steps. */ -function findings({ testBat, checkBat, buildBat, workflows, allowed }) { +function findings({ testBat, checkBat, buildBat, workflows, actions = {}, allowed }) { const out = []; const add = (kind, where, text) => out.push({ kind, where, text }); @@ -104,7 +105,10 @@ function findings({ testBat, checkBat, buildBat, workflows, allowed }) { const compared = {}; for (const wf of names) { - const steps = workflowSteps(workflows[wf], JOB); + const steps = workflowSteps(workflows[wf], JOB, (uses) => actions[uses] ?? null); + for (const s of steps.filter((x) => x.unreadable)) { + add("action", wf, `uses \`${s.unreadable}\`, which is not a composite action this gate can read`); + } const all = steps.flatMap((s) => s.gates.map(keyOf)); const mine = allowed.ciOnly.filter((e) => e.workflows.includes(wf)); const { rest: core, absent } = removeEach(all, mine.map(keyOf)); @@ -204,6 +208,18 @@ function pair(one, two) { return { "one.yml": one, "two.yml": two }; } +// A workflow whose gates are in a shared composite action, and the action. +function wfAction(build, extra = []) { + const steps = [{ name: "Checkout", uses: "actions/checkout@v5" }, { name: "Build", run: build }]; + steps.push({ name: "Run the gates", uses: "./gates" }); + for (const g of extra) steps.push({ name: g, run: `node scripts/${g}` }); + return { jobs: { [JOB]: { steps } } }; +} + +function actionOf(gates) { + return { runs: { using: "composite", steps: gates.map((g) => ({ name: g, shell: "bash", run: `node scripts/${g}` })) } }; +} + const PROBES = [ ["the recorded differences alone", {}, []], ["a gate missing from one workflow", @@ -242,6 +258,18 @@ const PROBES = [ ["allowances that match nothing", { workflows: pair(wf(GOOD, BUILD_ONE), wf(GOOD, BUILD_ONE)) }, ["stale"]], + ["the gates in a shared composite action", + { workflows: pair(wfAction(BUILD_ONE, ["ci.mjs"]), wfAction(BUILD_TWO)), actions: { "./gates": actionOf(GOOD) } }, + []], + ["a gate missing from the shared action", + { workflows: pair(wfAction(BUILD_ONE, ["ci.mjs"]), wfAction(BUILD_TWO)), actions: { "./gates": actionOf(["a.mjs", "c.mjs --check"]) } }, + ["missing"]], + ["a workflow that stops calling the action", + { workflows: pair(wf(["ci.mjs"], BUILD_ONE), wfAction(BUILD_TWO)), actions: { "./gates": actionOf(GOOD) } }, + ["missing", "differ"]], + ["an action the gate cannot read", + { workflows: pair(wfAction(BUILD_ONE, ["ci.mjs"]), wfAction(BUILD_TWO)), actions: {} }, + ["action", "missing"]], ]; function runProbes() { @@ -275,11 +303,28 @@ console.log(`check_ci_workflows: ${PROBES.length} probes, all pass`); const read = (rel) => readFileSync(path.join(ROOT, rel), "utf8"); const workflows = {}; for (const name of WORKFLOWS) workflows[name] = yaml.load(read(`.github/workflows/${name}`)); +// Every local action the workflows use; one that cannot be read is left out, +// and findings() reports the step that uses it. +const actions = {}; +for (const w of Object.values(workflows)) { + for (const s of w?.jobs?.[JOB]?.steps ?? []) { + if (typeof s.uses !== "string" || !s.uses.startsWith("./") || s.uses in actions) continue; + for (const file of ["action.yml", "action.yaml"]) { + try { + actions[s.uses] = yaml.load(read(path.join(s.uses, file))); + break; + } catch { + // not this name; try the other + } + } + } +} const found = findings({ testBat: read("test.bat"), checkBat: read("check.bat"), buildBat: read("build.bat"), workflows, + actions, allowed: ALLOWED, }); diff --git a/scripts/lib/gate-roster.mjs b/scripts/lib/gate-roster.mjs index d9c06c95..6ce119be 100644 --- a/scripts/lib/gate-roster.mjs +++ b/scripts/lib/gate-roster.mjs @@ -48,15 +48,29 @@ export function buildArgs(text) { return out; } +function stepOf(s, via = null) { + return { + name: s.name ?? s.uses ?? "", + via, + gates: gateSteps(s.run ?? ""), + build: buildArgs(s.run ?? ""), + }; +} + /** * One job's steps from a parsed workflow, each with the gates and the build it - * runs: `{name, gates, build}`. + * runs: `{name, via, gates, build}`. A step that uses a local composite action + * (`uses: ./path`) is replaced by that action's own steps, with `via` naming + * the action; `resolveAction(uses)` returns the parsed `action.yml`, or null + * when it cannot be read, which leaves a step with `unreadable` set rather + * than one that quietly runs nothing. */ -export function workflowSteps(workflow, job) { +export function workflowSteps(workflow, job, resolveAction = () => null) { const steps = workflow?.jobs?.[job]?.steps ?? []; - return steps.map((s) => ({ - name: s.name ?? s.uses ?? "", - gates: gateSteps(s.run ?? ""), - build: buildArgs(s.run ?? ""), - })); + return steps.flatMap((s) => { + if (typeof s.uses !== "string" || !s.uses.startsWith("./")) return [stepOf(s)]; + const action = resolveAction(s.uses); + if (action?.runs?.using !== "composite") return [{ ...stepOf(s), unreadable: s.uses }]; + return (action.runs.steps ?? []).map((a) => stepOf(a, s.uses)); + }); } From 9cb0efe8b9c1dc1476f901f3ba97b4fae6abdef2 Mon Sep 17 00:00:00 2001 From: Kuba Sunderland-Ober Date: Fri, 25 Sep 2026 20:35:27 +0200 Subject: [PATCH 14/53] lint: Biome, correctness rules only, and the fixes it finds --- biome.jsonc | 58 ++++++++++++ book/lib/fast-sync-load.mjs | 2 +- book/lib/measure-pass.mjs | 8 +- builder/PLAN-TOOLING-REVIEW.md | 52 +++++++++++ builder/book.mjs | 2 +- builder/census_attributes.mjs | 4 - builder/cpu-worker.mjs | 2 - builder/nav.mjs | 10 +- builder/offline.mjs | 3 +- builder/render.mjs | 23 +---- builder/symbols.mjs | 2 +- builder/tbdocs.mjs | 12 +-- builder/template.mjs | 1 - builder/write.mjs | 2 +- docs/Documentation/Builder.md | 6 +- docs/assets/js/svg-inline.js | 2 +- docs/assets/js/theme-toggle.js | 6 +- eval/build_corpus.mjs | 1 - eval/nav_hops.mjs | 2 +- package-lock.json | 164 +++++++++++++++++++++++++++++++++ package.json | 1 + scripts/build_package_api.mjs | 2 +- scripts/check_links_diff.mjs | 2 +- scripts/check_regex_safety.mjs | 2 +- scripts/impexp.mjs | 1 + scripts/lib/twin-api.mjs | 6 +- wisdom/extract/merger.mjs | 6 +- wisdom/extract/prep.mjs | 1 - wisdom/extract/state.mjs | 2 +- 29 files changed, 320 insertions(+), 65 deletions(-) create mode 100644 biome.jsonc diff --git a/biome.jsonc b/biome.jsonc new file mode 100644 index 00000000..830dfd81 --- /dev/null +++ b/biome.jsonc @@ -0,0 +1,58 @@ +{ + // Biome as the repository's linter (builder/PLAN-TOOLING-REVIEW.md, + // decision 4): rules that find defects -- the correctness and suspicious + // groups -- and nothing about style. The formatter stays off until the + // review's last phase. scripts/check_lint.mjs runs this configuration. + "$schema": "./node_modules/@biomejs/biome/configuration_schema.json", + "root": true, + "vcs": { "enabled": true, "clientKind": "git", "useIgnoreFile": true }, + "files": { + "includes": [ + "builder/**/*.mjs", + "scripts/**/*.mjs", + "book/**/*.mjs", + "book/lib/progress-handler.js", + "eval/**/*.mjs", + "wisdom/**/*.mjs", + "test/**/*.mjs", + "docs/assets/js/*.js", + // Vendored code, and the two pagedjs-cli ports the review treats as + // vendored: attributed and unmodified. + "!builder/vendor", + "!book/lib/paged.browser.js", + "!book/lib/outline.mjs", + "!book/lib/postprocesser.mjs", + // A Workflow script, not a module: the agent Workflow engine runs it as + // a function body, so it ends in a top-level `return`, which no module + // parser accepts. check_regex_safety.mjs still reads its regexes, through + // acorn's allowReturnOutsideFunction. + "!wisdom/extract/workflow.mjs", + // Trees the gates read as data. + "!test/fixtures" + ] + }, + "formatter": { "enabled": false }, + "assist": { "enabled": false }, + "linter": { + "enabled": true, + "rules": { + "preset": "none", + "correctness": { "preset": "recommended" }, + "suspicious": { + "preset": "recommended", + // `while ((m = re.exec(s)))` is how this tree walks a regex's matches. + "noAssignInExpressions": "off", + // Strings here hold GitHub Actions, PowerShell and JavaScript source + // whose `${...}` is meant literally. + "noTemplateCurlyInString": "off" + } + } + }, + "overrides": [ + { + // ES5-style scripts that ship to readers as written. + "includes": ["docs/assets/js/*.js"], + "linter": { "rules": { "correctness": { "noInnerDeclarations": "off" } } } + } + ] +} diff --git a/book/lib/fast-sync-load.mjs b/book/lib/fast-sync-load.mjs index f05bd760..4a81f97f 100644 --- a/book/lib/fast-sync-load.mjs +++ b/book/lib/fast-sync-load.mjs @@ -128,7 +128,7 @@ if (!PDFParser.prototype.__fastSyncLoadInstalled) { const initialOffset = this.bytes.offset(); try { this.parseIndirectObject(); - } catch (e) { + } catch { this.bytes.moveTo(initialOffset); this.tryToParseInvalidIndirectObject(); } diff --git a/book/lib/measure-pass.mjs b/book/lib/measure-pass.mjs index 293e6887..1fc49c84 100644 --- a/book/lib/measure-pass.mjs +++ b/book/lib/measure-pass.mjs @@ -345,7 +345,7 @@ export class Measurer { if (tag === 1 && IsNumeric[buf[this.pos]]) { const v = this.parseNumberOrRefCapture(); - if (!isNaN(v)) this._stLength[d] = v; + if (!Number.isNaN(v)) this._stLength[d] = v; } else if (tag === 2 && buf[this.pos] === SLASH) { if (this._isNameAt(this.pos + 1, 'ObjStm')) this._stIsObjStm[d] = 1; this.pos++; @@ -353,10 +353,10 @@ export class Measurer { this.numNames++; } else if (tag === 3 && IsNumeric[buf[this.pos]]) { const v = this.parseNumberOrRefCapture(); - if (!isNaN(v)) this._stN[d] = v; + if (!Number.isNaN(v)) this._stN[d] = v; } else if (tag === 4 && IsNumeric[buf[this.pos]]) { const v = this.parseNumberOrRefCapture(); - if (!isNaN(v)) this._stFirst[d] = v; + if (!Number.isNaN(v)) this._stFirst[d] = v; } else { this.parseObject(); } @@ -371,7 +371,7 @@ export class Measurer { } parseArray() { - const d = this._depth++; + this._depth++; if (this._depth > this.maxRecursionDepth) this.maxRecursionDepth = this._depth; this.pos++; diff --git a/builder/PLAN-TOOLING-REVIEW.md b/builder/PLAN-TOOLING-REVIEW.md index b1b7dc7e..32233996 100644 --- a/builder/PLAN-TOOLING-REVIEW.md +++ b/builder/PLAN-TOOLING-REVIEW.md @@ -542,6 +542,48 @@ unused imports and undeclared names behind, and Phases 1 and 2 move and delete a **Verify.** Lint clean over the scope. The tree comparison identical, since the fixes touch `builder/`. `test.bat` and `check.bat` clean. +**Landed** with Biome 2.5.14, exact. Its first run over the scope, with the recommended +correctness and suspicious rules, found 106 diagnostics in eleven rules: +`noAssignInExpressions` 20, `noInnerDeclarations` 19, `noUnusedFunctionParameters` 17, +`noUnusedVariables` 17, `noTemplateCurlyInString` 10, `useIterableCallbackReturn` 7, +`noUnusedImports` 5, `noControlCharactersInRegex` 4, `noGlobalIsNan` 3, +`noShadowRestrictedNames` 1, and two `useBiomeIgnoreFolder` notes on the configuration itself. +None fired on a browser global inside a `page.evaluate` body, so the three kinds of code need +no separate treatment and ESLint was not needed. It checks the scope, 136 scripts, in about +100 ms. + +`biome.jsonc` turns two rules off, each with its reason: `noAssignInExpressions`, because +`while ((m = re.exec(s)))` is how this tree walks a regex's matches, and +`noTemplateCurlyInString`, because strings here hold Actions, PowerShell and JavaScript source +whose `${...}` is literal. `noInnerDeclarations` is off only for `docs/assets/js/`, whose +ES5-style scripts ship as written. Biome 2.5 replaced the `recommended` field with `preset`, +which the configuration uses. **One file is outside the entry's scope:** +`wisdom/extract/workflow.mjs` ends in a top-level `return`, because the agent Workflow engine +runs it as a function body, and no module parser accepts that. It is excluded, and +`check_regex_safety.mjs` still reads its regexes, through acorn's `allowReturnOutsideFunction`. + +The other 54 findings are fixed, or suppressed with a reason. Five unused imports went, and +sixteen unused parameters of fixed callback signatures gained an underscore. Seventeen unused +variables were deleted, among them `census_attributes.mjs`'s `declLine`, assigned on two paths +and never read, and `cpu-worker.mjs`'s `idMapping`, which no worker reads. `measure-pass.mjs` +uses `Number.isNaN`, the same test there, since `parseNumberOrRefCapture` returns only a +number or `NaN`. `twin-api.mjs`'s `unescape`, which shadowed the global, is `unbracket`. Seven +`forEach` callbacks no longer return their expression's value. The two regexes whose control +characters are intended, the code mask's NUL delimiter and `impexp.mjs`'s test for the +characters Windows forbids in a file name, say so in a `biome-ignore` comment. **Two findings +are suppressed rather than fixed**, on the owner's decision: `offline.mjs`'s +`writeOfflinePages` and `writeOffline`'s `precomputed` parameter are A2-1's dead code, and +C14, which deletes them, now removes the two comments as well and moves the function's account +of the nav-block cache into `cpu-worker.mjs`. The main thread still posts `idMapping` to every +worker, and C14 now deletes that too. + +The tree comparison could not be identical, because the commit edits `Builder.md`, the site's +two scripts, and `scripts/impexp.mjs`, which the site publishes as a download. Those are the +only differences: `Builder.html`, the search index and `book.html`; `svg-inline.js` and +`theme-toggle.js`, whose four `catch (e)` became `catch (_e)` to stay ES5; and the +`impexp.mjs` download, whose readers now see its new comment. Every other change built +identical output. + ### C06 — `scripts: check_lint.mjs, a lint gate in test.bat and CI` **Decision 4.** The backstop for C08's hook. @@ -689,6 +731,16 @@ naming them, remain. re-export block that nothing imports (`:55-88`); `search.mjs`'s `writeSearchData` (`:18-24`); `sitemap.mjs`'s `extractSitemapUrls` (`:70-74`); `pdf.mjs`'s `extractImagePaths` (`:146-158`). +- With `writeOfflinePages` and `precomputed` go the two `biome-ignore` comments C05 put on + them and `tbdocs.mjs`'s `precomputed: true` argument. The function's comment on the + nav-block cache (`PLAN-9.md` §5.3, B7, and §7.D11) is the only explanation in the code of + a mechanism that lives on in `cpu-worker.mjs`'s `render`, so it moves above that copy + instead of going with the function. `offline.mjs`'s header (`:6-20`) describes + `precomputed`, `writeOfflinePages`, `buildSitePaths` and the re-exports, and is rewritten + to match. +- `worker-pool.mjs`'s `sendInit` still posts `idMapping` to every worker (`:43-45`, called + from `tbdocs.mjs:1449`), though C05 deleted the only place a worker kept it. The parameter + and the message field go, and `Pipeline-Stages.md:860`'s signature with them. - Delete `tbdocs.mjs`'s unused `makeTimer` export (`:196-209`). `offline.mjs` keeps its private copy (`:98-111`), with a comment that no longer cites the deleted tools. - Delete or correct the comments that name them: `offline-rewrite.mjs:411`, diff --git a/builder/book.mjs b/builder/book.mjs index 46479edc..6ce9c934 100644 --- a/builder/book.mjs +++ b/builder/book.mjs @@ -610,7 +610,7 @@ export function assembleBook(site, pages) { out.push("\n\n"); out.push(renderTitlePage(site)); emitFrontMatter(out, bookData, baseurl, imagePaths); - (bookData.parts ?? []).forEach((part, i) => emitPart(out, part, i, site, baseurl, imagePaths)); + (bookData.parts ?? []).forEach((part, i) => { emitPart(out, part, i, site, baseurl, imagePaths); }); out.push("\n\n\n"); let bookHtml = out.join(""); diff --git a/builder/census_attributes.mjs b/builder/census_attributes.mjs index e1570ecb..c7c53e7c 100644 --- a/builder/census_attributes.mjs +++ b/builder/census_attributes.mjs @@ -301,7 +301,6 @@ function scanFile(file, pkg) { // put the comment text in the report and lost the real target. // blankStrings erases a ' comment, so a blank result means "no code". let decl = blankStrings(decomment(run.rest)).trim() ? decomment(run.rest) : null; - let declLine = run.endLine; if (!decl) { let j = run.endLine + 1; while (j < lines.length) { @@ -314,7 +313,6 @@ function scanFile(file, pkg) { break; } decl = decomment(lines[j] ?? ""); - declLine = j; } const container = stack.at(-1)?.kind ?? "(file)"; const kind = classify(decl, container); @@ -392,8 +390,6 @@ function documentedAttributes() { } // ------------------------------------------------------------------- report -const pct = (n, d) => (d ? ((n / d) * 100).toFixed(1) : "0.0"); - function buildReport(sites, problems, files, projects, meta) { const byAttr = new Map(); for (const s of sites) { diff --git a/builder/cpu-worker.mjs b/builder/cpu-worker.mjs index f22ad314..6a7b74f7 100644 --- a/builder/cpu-worker.mjs +++ b/builder/cpu-worker.mjs @@ -35,7 +35,6 @@ const myLane = workerData?.lane ?? 0; let views = null; // Int32Array views into the scheduling SAB let ctx = null; // { srcRoot, destRoot, opts, workerCount } -let idMapping = null; // { nameToIdx, idxToName, DYNAMIC_BASE, … } let _payloadSAB = null; // SharedArrayBuffer with packed per-task payloads let _sharedSAB = null; // SharedArrayBuffer with packed shared payload @@ -290,7 +289,6 @@ parentPort.on("message", (msg) => { if (msg.init) { views = createViews(msg.sab); ctx = msg.ctx; - idMapping = msg.idMapping; _payloadSAB = null; _sharedSAB = null; _renderEnv = null; diff --git a/builder/nav.mjs b/builder/nav.mjs index 8c943df4..5bda2f41 100644 --- a/builder/nav.mjs +++ b/builder/nav.mjs @@ -228,14 +228,14 @@ function buildNavNode(page, chain, orderedChildren, depth) { // ---------- §5.4 nav-levels ------------------------------------------------ -function computeNavLevels(pages, state) { +function computeNavLevels(_pages, state) { const topIndex = new Map(); - state.topLevel.forEach((p, i) => topIndex.set(p.permalink, i + 1)); + state.topLevel.forEach((p, i) => { topIndex.set(p.permalink, i + 1); }); const childIndex = new Map(); for (const [parentUrl, list] of state.orderedChildren) { const m = new Map(); - list.forEach((c, i) => m.set(c.permalink, i + 1)); + list.forEach((c, i) => { m.set(c.permalink, i + 1); }); childIndex.set(parentUrl, m); } @@ -278,7 +278,7 @@ function levelsFromPath(chain, topIndex, childIndex) { // ---------- §5.5 breadcrumbs ----------------------------------------------- -function computeBreadcrumbs(pages, state) { +function computeBreadcrumbs(_pages, state) { for (const page of state.titled) { page.breadcrumbs = breadcrumbChainFor(page, state.byTitle); } @@ -314,7 +314,7 @@ function resolveParent(parentTitle, grandParentTitle, byTitle) { // ---------- §5.6 children --------------------------------------------------- -function computeChildren(pages, state) { +function computeChildren(_pages, state) { for (const page of state.titled) { const candidates = state.byParentTitle.get(String(page.frontmatter.title)) || []; const filtered = candidates.filter(c => { diff --git a/builder/offline.mjs b/builder/offline.mjs index e4fa767d..0d85e60f 100644 --- a/builder/offline.mjs +++ b/builder/offline.mjs @@ -30,7 +30,6 @@ import * as acornWalk from "acorn-walk"; import { WRITE_LIMIT, - isUnderProject, mkdirRec, runLimited, safeWrite, @@ -114,6 +113,7 @@ function makeTimer() { // §A Top-level orchestration // --------------------------------------------------------------------------- +// biome-ignore lint/correctness/noUnusedFunctionParameters: unread since the diff tools were retired (A2-1); C14 removes it and tbdocs.mjs's argument. export async function writeOffline(pages, staticFiles, site, destRoot, { auxStats, profileOffline = false, precomputed = false, sitePaths, check = false } = {}) { if (!destRoot) { throw new Error("writeOffline requires a destRoot"); @@ -241,6 +241,7 @@ export async function buildOfflineState(pages, staticFiles, site, destRoot, { st // its pre-rewrite nav block matches the cached `input` byte-for-byte. // On miss we fall back to the full rewrite with a warning -- the // cache is purely an optimisation, never a correctness dependency. +// biome-ignore lint/correctness/noUnusedVariables: dead since the diff tools were retired (A2-1); C14 deletes it and moves the nav-block cache comment above cpu-worker.mjs's live copy. async function writeOfflinePages(pages, deps, { precomputed = false } = {}) { const { offlineRoot } = deps; diff --git a/builder/render.mjs b/builder/render.mjs index c8805f2b..72ea7b29 100644 --- a/builder/render.mjs +++ b/builder/render.mjs @@ -136,6 +136,7 @@ export function applyPreRenderRewrites(rawContent) { // continuation needs block context that a pre-render pass does not have, and // guessing would change how real list content renders. That gap is a known, // measured one -- scripts/check_code_regions.mjs covers it. +// biome-ignore lint/suspicious/noControlCharactersInRegex: NUL delimits the placeholders because page text never contains one. const CODE_MASK_RE = /`\u0000CM(\d+)\u0000`/g; export function maskCodeRegions(src) { @@ -456,7 +457,7 @@ export function createMarkdownIt(ctx) { // kramdown emits `style="text-align: left"` with a space after the // colon; markdown-it emits the compact form. Override the th/td // renderers to widen the gap. - const styleSpace = (defaultRule) => (tokens, idx, opts, env, slf) => { + const styleSpace = (_defaultRule) => (tokens, idx, opts, _env, slf) => { const tok = tokens[idx]; const styleIdx = tok.attrIndex("style"); if (styleIdx >= 0) { @@ -464,7 +465,7 @@ export function createMarkdownIt(ctx) { } return slf.renderToken(tokens, idx, opts); }; - md.renderer.rules.th_open = ((defaultRule) => (tokens, idx, opts, env, slf) => { + md.renderer.rules.th_open = ((_defaultRule) => (tokens, idx, opts, _env, slf) => { const tok = tokens[idx]; const styleIdx = tok.attrIndex("style"); if (styleIdx >= 0) { @@ -771,9 +772,6 @@ const SQ_CLOSE_EXCLUDED = new Set([" ", "\\", "\t", "\r", "\n", "[", "{", "(", " // kramdown's SQ_PUNCT character class. const SQ_PUNCT_RE = /[!"#$%&'()*+,\-./:;<=>?@[\\\]^_`{|}~]/; -// Any straight or curly quote character. -const QUOTE_ANY_RE = /["'“”‘’]/; - // Apply kramdown-style smart-quote conversion to a raw HTML body -- // used for content inside `

...` and // similar inline elements where kramdown's HTML parser descends and @@ -869,7 +867,6 @@ function standaloneIalForwardPlugin(md) { // consumed as attrs) and re-target the attrs onto the right // neighbour using token.map to look up the source-line gap. md.core.ruler.after("curly_attributes", "standalone-ial-attach", (state) => { - const srcLines = state.src.split("\n"); const toks = state.tokens; // Lines previously occupied by a now-removed standalone IAL -- // counts as "non-blank" when checking adjacency for a following @@ -1132,7 +1129,6 @@ function looseDeflistPlugin(md) { // to visible. Override both directions: hide when tight, unhide // when loose. md.core.ruler.after("block", "deflist-tightness", (state) => { - const srcLines = state.src.split("\n"); const toks = state.tokens; let dtEndLine = -1; for (let i = 0; i < toks.length; i++) { @@ -1596,17 +1592,6 @@ function splitFragment(href) { return [href.slice(0, i), href.slice(i + 1)]; } -function normalizePosixPath(p) { - const parts = p.split("/"); - const out = []; - for (const part of parts) { - if (part === "" || part === ".") continue; - if (part === "..") { out.pop(); continue; } - out.push(part); - } - return out.join("/"); -} - // Mirrors jekyll-relative-links's File.expand_path-based resolution: a // link whose `..` segments would escape the docs/ root (the Jekyll // source directory) is left unrewritten by the upstream gem because the @@ -1737,7 +1722,7 @@ export function rewriteAdmonitions(src) { const stashed = []; let work = stashCodeFences(src, stashed); - work = work.replace(ADMONITION_RE, (m, leading, indent, typeRaw, bodyRaw) => { + work = work.replace(ADMONITION_RE, (_m, leading, indent, typeRaw, bodyRaw) => { const type = typeRaw.toLowerCase(); const meta = ADMONITION_TYPES[type]; // The body lines all share the same leading indent; strip it plus the diff --git a/builder/symbols.mjs b/builder/symbols.mjs index 5e0d12d3..d2792ea0 100644 --- a/builder/symbols.mjs +++ b/builder/symbols.mjs @@ -572,7 +572,7 @@ export function serializeSymbolIndex({ symbols, interfaces, packages }, api) { const lines = ["{"]; for (const [k, v] of Object.entries(head)) lines.push(` ${JSON.stringify(k)}: ${JSON.stringify(v)},`); lines.push(` "symbols": [`); - symbols.forEach((s, i) => lines.push(` ${JSON.stringify(s)}${i < symbols.length - 1 ? "," : ""}`)); + symbols.forEach((s, i) => { lines.push(` ${JSON.stringify(s)}${i < symbols.length - 1 ? "," : ""}`); }); lines.push(" ]", "}"); return lines.join("\n") + "\n"; } diff --git a/builder/tbdocs.mjs b/builder/tbdocs.mjs index cf51a8cd..f316da50 100644 --- a/builder/tbdocs.mjs +++ b/builder/tbdocs.mjs @@ -564,7 +564,7 @@ const TASKS = { nav: { expected: ["discover"], runOnMain: true, - execute(_, ctx, state) { + execute(_, _ctx, state) { const { navTree } = computeNav(state.pages, state.site.config); state.site.navTree = navTree; return { sidebar: renderSidebar(state.site) }; @@ -579,7 +579,7 @@ const TASKS = { buildInit: { expected: ["discover"], runOnMain: true, - execute(_, ctx, state) { + execute(_, _ctx, state) { return { initData: buildInitConfig(state.site) }; }, submit() {}, @@ -594,7 +594,7 @@ const TASKS = { // is derived from the stub set, and nothing else on this task needs it. expected: ["discover", "vendorAssets", "deriveRedirects"], runOnMain: true, - execute({ deriveRedirects: { stubs } }, ctx, state) { + execute({ deriveRedirects: { stubs } }, _ctx, state) { const linkTables = buildLinkTables(state.pages); const baseurl = String(state.site.config.baseurl || ""); const staticFileSet = new Set(state.staticFiles.map(s => s.srcRel)); @@ -646,7 +646,7 @@ const TASKS = { resolveBookChapters: { expected: ["deriveSitemap"], runOnMain: true, - execute(_, ctx, state) { + execute(_, _ctx, state) { resolveBookChapters(state.site.bookData, state.pages); return {}; }, @@ -659,7 +659,7 @@ const TASKS = { deriveRedirects: { expected: ["discover"], runOnMain: true, - execute(_, ctx, state) { + execute(_, _ctx, state) { return { stubs: deriveRedirectStubs(state.pages, state.site) }; }, submit(out, state) { @@ -676,7 +676,7 @@ const TASKS = { deriveSitemap: { expected: ["dispatch"], runOnMain: true, - execute(_, ctx, state) { + execute(_, _ctx, state) { return { urls: deriveSitemapUrls(state.pages, state.site) }; }, submit() {}, diff --git a/builder/template.mjs b/builder/template.mjs index e4c49c49..33dfb76b 100644 --- a/builder/template.mjs +++ b/builder/template.mjs @@ -441,7 +441,6 @@ function renderNavExternalLinks(config) { // ---------- §5.5 navActivationCss ---------------------------------------- const COLLECTION_PREFIX = ".site-nav > ul.nav-list:first-child"; -const OTHER_COLLECTION_PREFIX = ".site-nav > ul.nav-list:not(:first-child)"; export function navActivationCss(page) { const levels = page.navLevels; diff --git a/builder/write.mjs b/builder/write.mjs index 1b37bede..f0a740ab 100644 --- a/builder/write.mjs +++ b/builder/write.mjs @@ -173,7 +173,7 @@ async function copyTheme(builderAssetsRoot, destRoot, limit, baseurl) { function cssBaseurlTransformer(baseurl) { return (css) => css.replace( /url\((["']?)\/(?!\/)([^)"']*)\1\)/g, - (whole, q, rest) => `url(${q}${baseurl}/${rest}${q})`, + (_whole, q, rest) => `url(${q}${baseurl}/${rest}${q})`, ); } diff --git a/docs/Documentation/Builder.md b/docs/Documentation/Builder.md index 720ab877..7b697e40 100644 --- a/docs/Documentation/Builder.md +++ b/docs/Documentation/Builder.md @@ -416,6 +416,7 @@ A single `package.json` at the repo root contains everything --- the static site ```json { "devDependencies": { + "@biomejs/biome": "2.5.14", "@hpcc-js/wasm-graphviz": "^1.29.1", "acorn": "^8.0", "acorn-walk": "^8.0", @@ -438,10 +439,11 @@ A single `package.json` at the repo root contains everything --- the static site } ``` -No template engine, no framework, no bundler, no postinstall hooks. For the site generator, the `markdown-it-*` packages cover the dialect extensions the legacy parser supported; `gray-matter` splits off page frontmatter and `js-yaml` parses `_config.yml` and `_book.yml`; `fast-glob` finds the source files; `shiki` is the syntax highlighter; `@hpcc-js/wasm-graphviz` is the WASM build of Graphviz that renders `.dot` diagram sources; `sass` is Dart Sass for the SCSS compile; `acorn` + `acorn-walk` parse the upstream `just-the-docs.js` for the AST-based offline patcher; and `htmlparser2` is the SAX parser under the link and integrity check. `puppeteer` + `pdf-lib` + `html-entities` are the PDF renderer's toolchain: puppeteer controls headless Chromium for the paged.js layout pass, and `html-entities` decodes the entities in the PDF outline's entries. `axe-core` + `puppeteer` also back the standalone accessibility checker ([`scripts/check_a11y.mjs`](https://github.com/twinbasic/documentation/blob/main/scripts/check_a11y.mjs)), which runs the same headless Chromium over the built pages, and `recheck` + `acorn` back the regex-safety gate ([`scripts/check_regex_safety.mjs`](https://github.com/twinbasic/documentation/blob/main/scripts/check_regex_safety.mjs)). Neither `axe-core` nor `recheck` is used by `tbdocs` itself. +No template engine, no framework, no bundler, no postinstall hooks. For the site generator, the `markdown-it-*` packages cover the dialect extensions the legacy parser supported; `gray-matter` splits off page frontmatter and `js-yaml` parses `_config.yml` and `_book.yml`; `fast-glob` finds the source files; `shiki` is the syntax highlighter; `@hpcc-js/wasm-graphviz` is the WASM build of Graphviz that renders `.dot` diagram sources; `sass` is Dart Sass for the SCSS compile; `acorn` + `acorn-walk` parse the upstream `just-the-docs.js` for the AST-based offline patcher; and `htmlparser2` is the SAX parser under the link and integrity check. `puppeteer` + `pdf-lib` + `html-entities` are the PDF renderer's toolchain: puppeteer controls headless Chromium for the paged.js layout pass, and `html-entities` decodes the entities in the PDF outline's entries. `axe-core` + `puppeteer` also back the standalone accessibility checker ([`scripts/check_a11y.mjs`](https://github.com/twinbasic/documentation/blob/main/scripts/check_a11y.mjs)), which runs the same headless Chromium over the built pages, and `recheck` + `acorn` back the regex-safety gate ([`scripts/check_regex_safety.mjs`](https://github.com/twinbasic/documentation/blob/main/scripts/check_regex_safety.mjs)). `@biomejs/biome` is the repository's linter, which `biome.jsonc` limits to its correctness and suspicious rules. None of `axe-core`, `recheck` and `@biomejs/biome` is used by `tbdocs` itself. -**Which packages are pinned.** A package is pinned to an exact version where a new release could change what the build produces or what a gate reports without anything failing to say so: where the code patches the package or relies on its internals with no guard that fails when they change, or where the package's own results are what a gate reports. Everything else takes a caret range. Four packages are exact: +**Which packages are pinned.** A package is pinned to an exact version where a new release could change what the build produces or what a gate reports without anything failing to say so: where the code patches the package or relies on its internals with no guard that fails when they change, or where the package's own results are what a gate reports. Everything else takes a caret range. Five packages are exact: +- `@biomejs/biome` --- a new release can add or change a rule, which would change the linter's verdict on code nobody touched. [PLAN-TOOLING-REVIEW.md](https://github.com/twinbasic/documentation/blob/main/builder/PLAN-TOOLING-REVIEW.md), decision 4, records why the pin is exact. - `axe-core` --- the scan injects a copy of its bundle patched at source level, and its rules decide the accessibility gate's verdict. [PLAN-axe-perf.md](https://github.com/twinbasic/documentation/blob/main/builder/PLAN-axe-perf.md) records why the pin is exact. - `pdf-lib` --- the shims under `book/lib/` are line-by-line ports of this release's source, and pdf-lib is no longer maintained; [08-pdf-lib.md](https://github.com/twinbasic/documentation/blob/main/perf/notes/08-pdf-lib.md) records the pin. - `puppeteer` --- the book renderer and the accessibility gate measure what its Chromium renders, and the performance notes reason about that version at source level. It was pinned in the same change as `pdf-lib`. diff --git a/docs/assets/js/svg-inline.js b/docs/assets/js/svg-inline.js index 321bf127..f60a39c7 100644 --- a/docs/assets/js/svg-inline.js +++ b/docs/assets/js/svg-inline.js @@ -50,7 +50,7 @@ function fontData(rel) { if (fontCache[rel]) return fontCache[rel]; var url; - try { url = new URL(rel, SCRIPT_SRC).href; } catch (e) { url = null; } + try { url = new URL(rel, SCRIPT_SRC).href; } catch (_e) { url = null; } if (!url) return (fontCache[rel] = Promise.resolve(null)); fontCache[rel] = fetch(url).then(function (r) { if (!r.ok) throw new Error(r.status + " " + r.statusText); diff --git a/docs/assets/js/theme-toggle.js b/docs/assets/js/theme-toggle.js index 4346777f..3fe78a61 100644 --- a/docs/assets/js/theme-toggle.js +++ b/docs/assets/js/theme-toggle.js @@ -39,7 +39,7 @@ try { var stored = localStorage.getItem(KEY); if (stored === "light" || stored === "dark") return stored; - } catch (e) { + } catch (_e) { // localStorage unavailable (private mode) -- fall through to system. } return "system"; @@ -50,14 +50,14 @@ root.removeAttribute("data-theme"); try { localStorage.removeItem(KEY); - } catch (e) { + } catch (_e) { // ignore: the preference simply won't persist } } else { root.setAttribute("data-theme", choice); try { localStorage.setItem(KEY, choice); - } catch (e) { + } catch (_e) { // ignore: the preference simply won't persist } } diff --git a/eval/build_corpus.mjs b/eval/build_corpus.mjs index a081704d..75a9f2c1 100644 --- a/eval/build_corpus.mjs +++ b/eval/build_corpus.mjs @@ -12,7 +12,6 @@ // // See eval/README.md for how a round uses it. -import { createRequire } from "node:module"; import { fileURLToPath } from "node:url"; import fs from "node:fs"; import path from "node:path"; diff --git a/eval/nav_hops.mjs b/eval/nav_hops.mjs index a01efdcb..30093122 100644 --- a/eval/nav_hops.mjs +++ b/eval/nav_hops.mjs @@ -154,7 +154,7 @@ async function main(argv) { const chain = []; for (let f = hit; f; f = prev.get(f)) chain.unshift(f); console.log(`${t}: ${chain.length - 1} hop(s)`); - chain.forEach((f, i) => console.log(` ${i}. ${show(f)}${pages.urlOf.has(f) ? ` ${pages.urlOf.get(f)}` : ""}`)); + chain.forEach((f, i) => { console.log(` ${i}. ${show(f)}${pages.urlOf.has(f) ? ` ${pages.urlOf.get(f)}` : ""}`); }); } return unreachable ? 1 : 0; } diff --git a/package-lock.json b/package-lock.json index a8178569..38b0aa59 100644 --- a/package-lock.json +++ b/package-lock.json @@ -8,6 +8,7 @@ "name": "twinbasic-docs", "version": "0.0.0", "devDependencies": { + "@biomejs/biome": "2.5.14", "@hpcc-js/wasm-graphviz": "^1.29.1", "acorn": "^8.0", "acorn-walk": "^8.0", @@ -53,6 +54,169 @@ "node": ">=6.9.0" } }, + "node_modules/@biomejs/biome": { + "version": "2.5.14", + "resolved": "https://registry.npmjs.org/@biomejs/biome/-/biome-2.5.14.tgz", + "integrity": "sha512-0FabLIjd4M/dm8VFI86RMaLLdgepzbdfiAL2R8cr7V81OYYrP1w7Z73KfAwPK5h9SrEBXTzNG2y+mBwPo9xRnw==", + "dev": true, + "license": "MIT OR Apache-2.0", + "bin": { + "biome": "bin/biome" + }, + "engines": { + "node": ">=14.21.3" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/biome" + }, + "optionalDependencies": { + "@biomejs/cli-darwin-arm64": "2.5.14", + "@biomejs/cli-darwin-x64": "2.5.14", + "@biomejs/cli-linux-arm64": "2.5.14", + "@biomejs/cli-linux-arm64-musl": "2.5.14", + "@biomejs/cli-linux-x64": "2.5.14", + "@biomejs/cli-linux-x64-musl": "2.5.14", + "@biomejs/cli-win32-arm64": "2.5.14", + "@biomejs/cli-win32-x64": "2.5.14" + } + }, + "node_modules/@biomejs/cli-darwin-arm64": { + "version": "2.5.14", + "resolved": "https://registry.npmjs.org/@biomejs/cli-darwin-arm64/-/cli-darwin-arm64-2.5.14.tgz", + "integrity": "sha512-UnzaXO65L4tsZimFITFP2M121GyhDcWFrT3pL5ZJ5U4XcS/0L5VHYytVohdCf/gnDUFHgl2I9xnt5bV/J1kxHQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT OR Apache-2.0", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=14.21.3" + } + }, + "node_modules/@biomejs/cli-darwin-x64": { + "version": "2.5.14", + "resolved": "https://registry.npmjs.org/@biomejs/cli-darwin-x64/-/cli-darwin-x64-2.5.14.tgz", + "integrity": "sha512-kiy8qA16K93J7uvFfWi4LgjqDpnRKyePAna6A0Y4jxyUga35SYpIZ2cNWlhy0lsfgqRenCpfymnJWEZ6mgpRgA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT OR Apache-2.0", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">=14.21.3" + } + }, + "node_modules/@biomejs/cli-linux-arm64": { + "version": "2.5.14", + "resolved": "https://registry.npmjs.org/@biomejs/cli-linux-arm64/-/cli-linux-arm64-2.5.14.tgz", + "integrity": "sha512-vO/9BaU1n30CiFNLx49gMTMtbCAAqlo/EqFoG0BU3deQInTbJrmXSmTQ9e3FoDZIKQnvSLDVNL4lx1ci7y1T1Q==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT OR Apache-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=14.21.3" + } + }, + "node_modules/@biomejs/cli-linux-arm64-musl": { + "version": "2.5.14", + "resolved": "https://registry.npmjs.org/@biomejs/cli-linux-arm64-musl/-/cli-linux-arm64-musl-2.5.14.tgz", + "integrity": "sha512-SJ9PrZkBnnH9dHJDnxk34vKs0GB2dbsioaft6/hPhWJ7AHpwYH83Isnhj0FSRpFkGgpVfJ1lds23ApB2czUDLQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT OR Apache-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=14.21.3" + } + }, + "node_modules/@biomejs/cli-linux-x64": { + "version": "2.5.14", + "resolved": "https://registry.npmjs.org/@biomejs/cli-linux-x64/-/cli-linux-x64-2.5.14.tgz", + "integrity": "sha512-VHZRa7CCQBUWKxNwUvHJeuW03WFRgN5NWO//SDGOitc9QIeNyfA42V9zlKm+x82xFSG9Bzi5fi+hbhm9anO8yA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT OR Apache-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=14.21.3" + } + }, + "node_modules/@biomejs/cli-linux-x64-musl": { + "version": "2.5.14", + "resolved": "https://registry.npmjs.org/@biomejs/cli-linux-x64-musl/-/cli-linux-x64-musl-2.5.14.tgz", + "integrity": "sha512-2kI5PrMgW5dcEZYrstLPmUmCwkUwZY39rP4BN93Vxbwcs2O57LDQOktUZZYPvvspN5i0mhGr3TJtF5sU26NUWg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT OR Apache-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">=14.21.3" + } + }, + "node_modules/@biomejs/cli-win32-arm64": { + "version": "2.5.14", + "resolved": "https://registry.npmjs.org/@biomejs/cli-win32-arm64/-/cli-win32-arm64-2.5.14.tgz", + "integrity": "sha512-pHgAFffmZtaYoxavEsWEYNvcA7NwOIjLyqw2HXyLbt16xYryX0L4fMV2u0G9ag6LNYPIoqWaWqzUcnc6rLGGXg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT OR Apache-2.0", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=14.21.3" + } + }, + "node_modules/@biomejs/cli-win32-x64": { + "version": "2.5.14", + "resolved": "https://registry.npmjs.org/@biomejs/cli-win32-x64/-/cli-win32-x64-2.5.14.tgz", + "integrity": "sha512-oJWmBhoHsnhUKIke+0gXDX0mltJrWHA1UyHsrTlXwX0TL64ilVZAo+TYZm97baecV0esBdpzy3k96q+09JaZUQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT OR Apache-2.0", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">=14.21.3" + } + }, "node_modules/@hpcc-js/wasm-graphviz": { "version": "1.29.1", "resolved": "https://registry.npmjs.org/@hpcc-js/wasm-graphviz/-/wasm-graphviz-1.29.1.tgz", diff --git a/package.json b/package.json index 6577c035..56a8f5e6 100644 --- a/package.json +++ b/package.json @@ -4,6 +4,7 @@ "private": true, "description": "twinBASIC documentation source, tbdocs static-site builder, and PDF book pipeline.", "devDependencies": { + "@biomejs/biome": "2.5.14", "@hpcc-js/wasm-graphviz": "^1.29.1", "acorn": "^8.0", "acorn-walk": "^8.0", diff --git a/scripts/build_package_api.mjs b/scripts/build_package_api.mjs index 5251576d..18dba58c 100644 --- a/scripts/build_package_api.mjs +++ b/scripts/build_package_api.mjs @@ -146,7 +146,7 @@ function serialize({ build, packages, exportsOf }) { lines.push(` "exports": ${JSON.stringify(exportsOf.get(pkg))},`); lines.push(` "types": [`); const types = packages[pkg]; - types.forEach((t, j) => lines.push(` ${JSON.stringify(t)}${j < types.length - 1 ? "," : ""}`)); + types.forEach((t, j) => { lines.push(` ${JSON.stringify(t)}${j < types.length - 1 ? "," : ""}`); }); lines.push(" ]", ` }${i < names.length - 1 ? "," : ""}`); }); lines.push(" }", "}"); diff --git a/scripts/check_links_diff.mjs b/scripts/check_links_diff.mjs index d04e0e11..917d4cf8 100644 --- a/scripts/check_links_diff.mjs +++ b/scripts/check_links_diff.mjs @@ -369,7 +369,7 @@ const SIDES = { // nothing else would say so. fused: { describe: "tbdocs --check, the build's own pass", - run(argv, { case: name }) { + run(_argv, { case: name }) { const c = CASES[name]; if (!c.fused) throw new Error(`case '${name}' has no fused equivalent`); const all = fusedBuild(c.fused); diff --git a/scripts/check_regex_safety.mjs b/scripts/check_regex_safety.mjs index 87c11812..f5485648 100644 --- a/scripts/check_regex_safety.mjs +++ b/scripts/check_regex_safety.mjs @@ -361,7 +361,7 @@ async function checkAll(regexes) { // cluster by file (render.mjs alone holds 69 of them), so contiguous // slices would leave one shard doing nearly all the work. const buckets = Array.from({ length: shards }, () => []); - regexes.forEach((r, i) => buckets[i % shards].push(r)); + regexes.forEach((r, i) => { buckets[i % shards].push(r); }); const settled = await Promise.all(buckets.map(runShard)); const merged = settled.flat(); // A shard that returns fewer results than it was given would silently diff --git a/scripts/impexp.mjs b/scripts/impexp.mjs index 7644188a..f8b3862b 100644 --- a/scripts/impexp.mjs +++ b/scripts/impexp.mjs @@ -295,6 +295,7 @@ const HINT_IMPORT = ['import replaces the project file with the folder.', // and Windows drops a trailing dot or space, which turns `.. ` into `..`. // Nothing the IDE writes breaks these rules. function isSafeName(name) { + // biome-ignore lint/suspicious/noControlCharactersInRegex: Windows forbids the control characters \x00-\x1f in a file name. return name !== '' && !/[\\/:*?"<>|\x00-\x1f]/.test(name) && !/[. ]$/.test(name); } diff --git a/scripts/lib/twin-api.mjs b/scripts/lib/twin-api.mjs index 25ba5a3c..6bb65c39 100644 --- a/scripts/lib/twin-api.mjs +++ b/scripts/lib/twin-api.mjs @@ -136,7 +136,7 @@ const ENUM_VALUE = new RegExp(String.raw`^(${NAME})\s*(=|$)`); // Types whose procedures have bodies, and so an `End Sub` to wait for. const WITH_BODIES = new Set(["module", "class", "type", "union"]); -const unescape = (name) => name.replace(/^\[(.*)\]$/, "$1"); +const unbracket = (name) => name.replace(/^\[(.*)\]$/, "$1"); function kindOf(keyword) { const k = keyword.toLowerCase(); @@ -222,7 +222,7 @@ export function parseTwin(src, file = "") { const open = TYPE_OPEN.exec(decl); if (open && !/^As$/i.test(open[2])) { const kind = open[1].toLowerCase(); - const name = unescape(open[2]); + const name = unbracket(open[2]); if (kind === "interface" && parent?.kind === "coclass") { parent.interfaces.push({ name, @@ -294,7 +294,7 @@ export function parseTwin(src, file = "") { // Everything in an Interface, CoClass, Enum, Type or Union is public. function add(type, rawName, kind, vis, hidden, line, fallback) { const implicit = ["interface", "coclass", "enum", "type", "union"].includes(type.kind) ? "public" : fallback; - type.members.push({ name: unescape(rawName), kind, vis: vis ?? implicit, hidden, line }); + type.members.push({ name: unbracket(rawName), kind, vis: vis ?? implicit, hidden, line }); } } diff --git a/wisdom/extract/merger.mjs b/wisdom/extract/merger.mjs index 2975fcca..520c720c 100644 --- a/wisdom/extract/merger.mjs +++ b/wisdom/extract/merger.mjs @@ -14,8 +14,8 @@ // Atomic writes: temp file + rename, previous staging.md retained as // staging.md.bak for one generation. -import { existsSync, mkdirSync, readFileSync, renameSync, writeFileSync, copyFileSync, unlinkSync } from 'node:fs' -import { dirname, join } from 'node:path' +import { existsSync, mkdirSync, readFileSync, renameSync, writeFileSync, copyFileSync } from 'node:fs' +import { join } from 'node:path' import { buildEmissionKeySet, emissionKey } from './state.mjs' const STAGING_FILE = 'staging.md' @@ -94,7 +94,7 @@ export function graftAdditions(outDir, additions, state) { */ export function renderSideband(additions) { const parsed = freshStaging() - const stats = mergeIntoParsed(parsed, additions, new Set()) + mergeIntoParsed(parsed, additions, new Set()) ensureUnmappedHeader(parsed) return serializeStaging(parsed) } diff --git a/wisdom/extract/prep.mjs b/wisdom/extract/prep.mjs index ebcfc3d9..34aa91d3 100644 --- a/wisdom/extract/prep.mjs +++ b/wisdom/extract/prep.mjs @@ -46,7 +46,6 @@ export async function runExtract(flags) { // Scan thread .md files const allThreads = [] let skippedByState = 0 - let skippedByChannel = 0 let skippedBySince = 0 const subdirs = readdirSync(threadsDir, { withFileTypes: true }) diff --git a/wisdom/extract/state.mjs b/wisdom/extract/state.mjs index 1fc3a244..53fbc590 100644 --- a/wisdom/extract/state.mjs +++ b/wisdom/extract/state.mjs @@ -27,7 +27,7 @@ // leaves state untouched, so the next run retries the same threads. import { existsSync, mkdirSync, readFileSync, renameSync, writeFileSync } from 'node:fs' -import { dirname, join } from 'node:path' +import { join } from 'node:path' const STATE_FILE = 'extract-state.json' const STATE_VERSION = 1 From f5f1d9d52e68c53014fdce13258e916b0cb13cd4 Mon Sep 17 00:00:00 2001 From: Kuba Sunderland-Ober Date: Fri, 25 Sep 2026 20:49:31 +0200 Subject: [PATCH 15/53] deps: update js-yaml, ws, linkify-it and immutable past their advisories --- builder/PLAN-TOOLING-REVIEW.md | 34 ++++++++++++++++++++++++++--- package-lock.json | 40 +++++++++++++++++++++------------- 2 files changed, 56 insertions(+), 18 deletions(-) diff --git a/builder/PLAN-TOOLING-REVIEW.md b/builder/PLAN-TOOLING-REVIEW.md index 32233996..e8506219 100644 --- a/builder/PLAN-TOOLING-REVIEW.md +++ b/builder/PLAN-TOOLING-REVIEW.md @@ -1150,9 +1150,9 @@ text; `hrefs` returns the same links for every page. ### C40 — `builder, eval: frontmatter through lib/frontmatter; drop gray-matter` -**Decision (a)'s frontmatter half.** `gray-matter@4.0.3` bundles its own `js-yaml@3.14.2`, +**Decision (a)'s frontmatter half.** `gray-matter@4.0.3` bundles its own `js-yaml@3.15.2`, while `data.mjs`, `tbdocs.mjs` and `check_publish_policy.mjs` parse the configuration with -`js-yaml@4.1.1`: page frontmatter and site configuration go through two major versions of one +`js-yaml@4.3.2`: page frontmatter and site configuration go through two major versions of one library. **Change.** `discover.mjs` (`:94-117`, which strips the BOM itself because `matter.test()` @@ -2037,7 +2037,35 @@ review's plan. ## Found while implementing -Nothing yet: defects the review did not have, found by building something this plan asks for. +Defects the review did not have, found by building something this plan asks for. + +- **Four high-severity advisories in the installed packages**, which `npm` reported while C05 + installed Biome. Fixed between C05 and C06 in `deps: update js-yaml, ws, linkify-it and + immutable past their advisories`. All four are denial of service from crafted input, and + every fix is a release inside a range already declared, by `package.json` for `js-yaml` and + by the parent package for the other four, so only `package-lock.json` changed and + Builder.md's Dependencies did not: `js-yaml` 4.1.1 to + 4.3.2, and `gray-matter`'s nested copy 3.14.2 to 3.15.2; `ws` 8.20.1 to 8.21.3, under + Puppeteer; `linkify-it` 5.0.1 to 5.0.2, under `markdown-it`, where it cannot change the + output because `render.mjs` sets `linkify: false`; and `immutable` 5.1.6 to 5.1.9, under + `sass`. + + **`npm ls` reported the fix as already done.** `node_modules/.package-lock.json`, npm's + record of what is installed, had been rewritten with the fixed versions after the Biome + install, most likely by the `npm audit fix --dry-run` that listed them, while the packages + on disk stayed old. npm trusts that file when it is newer than every package folder, so a + real `npm audit fix` could have updated the lockfile and left the old packages in place. + With the file moved aside, `npm ls` showed the old versions, and the fix replaced five + packages. C09 and C40 change installed packages too: read the versions from each package's + own `package.json`, not from `npm ls`. + + **`compare_trees.mjs` cannot see a dependency change**, because both of its worktrees + resolve packages from this checkout's one `node_modules`. One `--keep` run before the update + and one after gave two builds of the same commit, which the same three normalisers found + identical. The book rendered 2,276 pages both times, with the same 2,460 outline entries and + the same extracted text; the two PDFs differ from byte 22.6 MB on, inside the compressed + object streams, and their document dates differ. `build.bat`, `check.bat` and `test.bat` + are clean. ## Open questions diff --git a/package-lock.json b/package-lock.json index 38b0aa59..646ad427 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1433,9 +1433,9 @@ } }, "node_modules/gray-matter/node_modules/js-yaml": { - "version": "3.14.2", - "resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-3.14.2.tgz", - "integrity": "sha512-PMSmkqxr106Xa156c2M265Z+FTrPl+oxd/rgOQy2tijQeK5TxQ43psO1ZCwhVOSdnn+RzkzlRz/eY4BgJBYVpg==", + "version": "3.15.2", + "resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-3.15.2.tgz", + "integrity": "sha512-6EuL879VkRA+1Cz578mKMiKvjPNEuk6+r1JaFzoSWejZmtf7xWbIyw1e3KkxlkzTIt9Taw6JBhEppG7utc1P+w==", "dev": true, "license": "MIT", "dependencies": { @@ -1536,9 +1536,9 @@ } }, "node_modules/immutable": { - "version": "5.1.6", - "resolved": "https://registry.npmjs.org/immutable/-/immutable-5.1.6.tgz", - "integrity": "sha512-q1swsS8K7L8usSHuOqF2TAoCCkonYz0SG38wLAggaa4Wml70zixIvt2ql4coQ2C2B3hTjltJry4r6bULwgAXLQ==", + "version": "5.1.9", + "resolved": "https://registry.npmjs.org/immutable/-/immutable-5.1.9.tgz", + "integrity": "sha512-m8nVez3rwrgmWxtLMt1ZYXB2Lv7OKYn/disyxAlSDYAlKSlFoPPfIAmAM/M5xqL4m4C/wAPw7S2/CNaUii1Hxg==", "dev": true, "license": "MIT" }, @@ -1627,10 +1627,20 @@ "license": "MIT" }, "node_modules/js-yaml": { - "version": "4.1.1", - "resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.1.1.tgz", - "integrity": "sha512-qQKT4zQxXl8lLwBtHMWwaTcGfFOZviOJet3Oy/xmGk2gZH677CJM9EvtfdSkgWcATZhj/55JZ0rmy3myCT5lsA==", + "version": "4.3.2", + "resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.3.2.tgz", + "integrity": "sha512-SFNOvSJ+Dgf/9An904Yx+CgSlIPCkIpao4qo51lpee25TIRejdH3rhR4EZMGoNx3/TP3O+wzWuiTFl4sqbltzA==", "dev": true, + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/puzrin" + }, + { + "type": "github", + "url": "https://github.com/sponsors/nodeca" + } + ], "license": "MIT", "dependencies": { "argparse": "^2.0.1" @@ -1664,9 +1674,9 @@ "license": "MIT" }, "node_modules/linkify-it": { - "version": "5.0.1", - "resolved": "https://registry.npmjs.org/linkify-it/-/linkify-it-5.0.1.tgz", - "integrity": "sha512-wVoTjP4Q6R0NW5hiZkVJaFZPWgtXfoGF+6LucL3/FtiNjmcHhYjEr5f1Kqjirc1nBW07J/ZuRFumqr2oqccEWg==", + "version": "5.0.2", + "resolved": "https://registry.npmjs.org/linkify-it/-/linkify-it-5.0.2.tgz", + "integrity": "sha512-ONTm2jCMAVZjgQa/Fy1kScXsuOoF5NPTsoFBdE1KVIZ2vAh/r9+Bqo+0jINCBYnavTPQZz38QzFTme79ENoN3Q==", "dev": true, "funding": [ { @@ -2710,9 +2720,9 @@ "license": "ISC" }, "node_modules/ws": { - "version": "8.20.1", - "resolved": "https://registry.npmjs.org/ws/-/ws-8.20.1.tgz", - "integrity": "sha512-It4dO0K5v//JtTXuPkfEOaI3uUN87iYPnqo/ZzqCoG3g8uhA66QUMs/SrM0YK7/NAu+r4LMh/9dq2A7k+rHs+w==", + "version": "8.21.3", + "resolved": "https://registry.npmjs.org/ws/-/ws-8.21.3.tgz", + "integrity": "sha512-201TZ/kPWxoPr/OKWjquZR1SWKXcvxdH+e1xrx89b3YbmzLMFCLfnaG1HFIgWzJOEWZ7MvpK++odZufgYR50Rw==", "dev": true, "license": "MIT", "engines": { From 0d42ab3c9d26d143d7eaac0d2e4df46a13ce2a4e Mon Sep 17 00:00:00 2001 From: Kuba Sunderland-Ober Date: Fri, 25 Sep 2026 21:01:32 +0200 Subject: [PATCH 16/53] scripts: check_lint.mjs, a lint gate in test.bat and CI --- .github/actions/run-gates/action.yml | 10 ++++ WIP.md | 9 ++- builder/PLAN-TOOLING-REVIEW.md | 23 ++++++++ docs/Documentation/Builder.md | 2 +- docs/Documentation/Building.md | 1 + docs/Documentation/Tools.md | 29 +++++++--- scripts/check_lint.mjs | 87 ++++++++++++++++++++++++++++ test.bat | 8 +++ 8 files changed, 158 insertions(+), 11 deletions(-) create mode 100644 scripts/check_lint.mjs diff --git a/.github/actions/run-gates/action.yml b/.github/actions/run-gates/action.yml index bcdcb0b9..3c5cbbd4 100644 --- a/.github/actions/run-gates/action.yml +++ b/.github/actions/run-gates/action.yml @@ -54,6 +54,16 @@ runs: - name: Verify the workflows run the wrappers' gates (check_ci_workflows.mjs) shell: bash run: node scripts/check_ci_workflows.mjs + # Biome, pinned, over the tooling, with the rules that find defects and + # none about style (biome.jsonc). Moving and deleting code leaves unused + # imports and undeclared names behind, and nothing else reads the tooling + # for them. Warnings fail as well as errors, since Biome reports an unused + # import as a warning and exits 0 on one; and a scope that matches no + # script fails rather than passing. npm ci installs the Linux binary. No + # browser, no built tree, ~0.25 s. + - name: Lint the tooling (check_lint.mjs) + shell: bash + run: node scripts/check_lint.mjs # A regex that backtracks exponentially does not fail a build, it stops # one: the corpus passes until a page happens to contain the trigger, and # then a render worker sits inside String.replace forever. VOID_TAGS_RE diff --git a/WIP.md b/WIP.md index 11ce5401..dd869063 100644 --- a/WIP.md +++ b/WIP.md @@ -452,7 +452,7 @@ Why the report separates the wedged task from the merely blocked ones, and why - `build.bat` — runs `node builder\tbdocs.mjs --src docs --check-audit-index` (which implies `--check`) and produces three trees in one pass: the online copy at `_site/`, a `file://`-browsable copy at `_site-offline/`, and the sparse pagedjs source at `_site-pdf/`. The offline pass adds ~700 ms and the PDF pass adds ~150 ms on top of the ~2 s online build. Toggle `also_build_offline` / `also_build_pdf` in `_config.yml` (or pass `--no-offline` / `--no-pdf`) to skip a sibling output. `--check` adds ~1.7 s and runs the link + integrity check over the HTML while it is still in worker memory; `build.bat --no-check` gets a plain build. - `serve.bat` — runs `tbdocs --serve`: initial build, then a long-lived process with watcher, debounced rebuilds, and SSE-driven browser auto-reload. Writes to `docs/_serve/` (disjoint from `build.bat`'s `_site*/`) and skips the offline + PDF passes — so a one-off `build.bat` for the PDF or offline mirror doesn't disturb the live preview. Ctrl+C to stop. - `check.bat` — the gates that read the built site: a freshness check that refuses a stale tree (`scripts/check_tree_fresh.mjs`), the DOT diagram fit check (`scripts/check_dot_fit.mjs`), the a11y sample-coverage check (`scripts/pick_a11y_sample.mjs --check`), then the accessibility check (`scripts/check_a11y.mjs`). The link + integrity check moved into `build.bat`. ~37 s. -- `test.bat` — the tests the *toolchain* has to pass: the publish-allowlist self-test (`scripts/check_publish_policy.mjs`), the gate-list check (`scripts/check_gate_lists.mjs`), the CI-workflow roster check (`scripts/check_ci_workflows.mjs`), the regex-safety gate (`scripts/check_regex_safety.mjs`), the code-region gate (`scripts/check_code_regions.mjs`), the page-count drift-guard probes (`scripts/check_page_baseline.mjs`), the book-coverage probes (`scripts/check_book_coverage.mjs`), the symbol-index probes (`scripts/check_symbol_index.mjs`), and the axe source-patch verification (`scripts/check_axe_patch_equiv.mjs`). ~8 s. See [What belongs in test.bat rather than check.bat](WIP.Build.md#what-belongs-in-testbat-rather-than-checkbat). +- `test.bat` — the tests the *toolchain* has to pass: the publish-allowlist self-test (`scripts/check_publish_policy.mjs`), the gate-list check (`scripts/check_gate_lists.mjs`), the CI-workflow roster check (`scripts/check_ci_workflows.mjs`), the lint gate (`scripts/check_lint.mjs`), the regex-safety gate (`scripts/check_regex_safety.mjs`), the code-region gate (`scripts/check_code_regions.mjs`), the page-count drift-guard probes (`scripts/check_page_baseline.mjs`), the book-coverage probes (`scripts/check_book_coverage.mjs`), the symbol-index probes (`scripts/check_symbol_index.mjs`), and the axe source-patch verification (`scripts/check_axe_patch_equiv.mjs`). ~8 s. See [What belongs in test.bat rather than check.bat](WIP.Build.md#what-belongs-in-testbat-rather-than-checkbat). - `book.bat` — renders the PDF from `docs\_site-pdf\book.html` via `node book\render-book.mjs` into `docs\_pdf\twinBASIC Book.pdf`. Run `build.bat` first to populate `_site-pdf/`; `book.bat` refuses a tree older than its sources rather than rendering the previous book (see [The book refuses a stale source tree](WIP.Build.md#the-book-refuses-a-stale-source-tree)). - `examples.bat` — compiles the documentation's own twinBASIC code samples, every `tb` fence marked `check_build`, and reports the ones the compiler refuses against the line in the page they came from. Needs a twinBASIC install and Windows, so it is outside every gate and outside CI; ~110 s over the 1,119 samples marked today. Two modes need no compiler at all: `--census` classifies every fence and says how many classifiable ones are still unmarked, and `--report ` groups a saved `--propose --json` survey by diagnostic, section and unresolved name. `--propose` itself does compile. See [Compiling the reference's own code samples](#compiling-the-references-own-code-samples) and [WIP.ExamplesBuild.md](WIP.ExamplesBuild.md). @@ -470,7 +470,7 @@ build.bat && check.bat On the dev box that is ~4 s of build against ~37 s of check, of which the axe scan is ~20 s. [builder/PLAN-checks.md](builder/PLAN-checks.md) records how the link checker got folded into the build's task graph, what it cost and what it saved; the axe follow-ons are designed there but not implemented. -**If the change touched `builder/`, `scripts/`, `book/`, `eval/`, `wisdom/`, a wrapper or a workflow, run `test.bat` as well** --- another ~8 s. Seven of its nine gates cannot be affected by a content edit at all. **Two can.** `check_gate_lists.mjs` is the easy one to predict: it reads `README.md` and every page under `docs/Documentation/`, so an edit to any developer page that states a gate count can fail it. **`check_code_regions.mjs` is the one worth understanding**, and which half of it a content edit reaches is worth keeping straight. Its corpus sweep has `ROOT = /docs` and tokenises all 906 markdown files, so a page that provokes a rewrite into *altering* a code region fails it --- that half is content-dependent. Its fixed probes are not: they run against their own sources whatever the tree holds, and they cover the **mirror** fault, where a rewrite silently stops firing. The sweep structurally cannot see that one, because text the rewrite skipped is stashed and restored unchanged and every region still matches. So run `test.bat` after adding an unusual code construct --- a fence whose contents include a fence marker, a 4-space indented block, an admonition wrapping a fence --- and read the built page as well, because for the mirror fault the gate is asserting that the stasher still works rather than checking your page: +**If the change touched `builder/`, `scripts/`, `book/`, `eval/`, `wisdom/`, `test/`, the site's scripts in `docs/assets/js/`, a wrapper or a workflow, run `test.bat` as well** --- another ~8 s. Seven of its ten gates cannot be affected by an edit under `docs/` at all. **Three can.** `check_lint.mjs` lints the site's two scripts in `docs/assets/js/` along with the tooling. `check_gate_lists.mjs` is the easy one to predict: it reads `README.md` and every page under `docs/Documentation/`, so an edit to any developer page that states a gate count can fail it. **`check_code_regions.mjs` is the one worth understanding**, and which half of it a content edit reaches is worth keeping straight. Its corpus sweep has `ROOT = /docs` and tokenises all 906 markdown files, so a page that provokes a rewrite into *altering* a code region fails it --- that half is content-dependent. Its fixed probes are not: they run against their own sources whatever the tree holds, and they cover the **mirror** fault, where a rewrite silently stops firing. The sweep structurally cannot see that one, because text the rewrite skipped is stashed and restored unchanged and every region still matches. So run `test.bat` after adding an unusual code construct --- a fence whose contents include a fence marker, a 4-space indented block, an admonition wrapping a fence --- and read the built page as well, because for the mirror fault the gate is asserting that the stasher still works rather than checking your page: ```sh build.bat && check.bat && test.bat @@ -504,6 +504,7 @@ wrapper: | `test.bat` | `check_regex_safety` | no regex in the tree can backtrack exponentially | | `test.bat` | `check_symbol_index` | the symbol index still places each kind of symbol, from fixtures | | `test.bat` | `check_ci_workflows` | both CI workflows run every wrapper gate, with the same arguments and order, and build with `build.bat`'s flags | +| `test.bat` | `check_lint` | Biome finds nothing in the tooling, warnings included, and checked at least one script | | `test.bat` | `check_publish_policy`, `check_gate_lists`, `check_page_baseline`, `check_book_coverage`, `check_axe_patch_equiv` | the gates on the gates | **A gate belongs in `test.bat` rather than `check.bat` if it would still mean @@ -532,6 +533,10 @@ inline `` is content. Favor concise one-line git commit messages. +**Lint before every commit:** `node scripts/check_lint.mjs`, a fraction of a second. It +runs Biome over the tooling and the site's two scripts, and fails on a warning as well as an +error, because Biome reports an unused import as a warning. `test.bat` and CI run it too. + **A bug in twinBASIC itself goes in [BUGS-TO-REPORT.md](BUGS-TO-REPORT.md)**, which is a queue rather than a record: an entry is deleted once it has been filed upstream. Each one carries the build it was seen on and a *narrowed* reproduction --- the compiler crash diff --git a/builder/PLAN-TOOLING-REVIEW.md b/builder/PLAN-TOOLING-REVIEW.md index e8506219..1edf31e4 100644 --- a/builder/PLAN-TOOLING-REVIEW.md +++ b/builder/PLAN-TOOLING-REVIEW.md @@ -596,6 +596,29 @@ and WIP.md's gate table. WIP.md gains the rule: lint before every commit. **Verify.** Clean on the tree; an unused import exits 1; a broken configuration exits 2. The roster gate passes. CI waits for the owner's push. +**Landed** with two things the entry did not foresee, both about what Biome's exit code +means. **Biome 2.5 reports `noUnusedImports` and `noUnusedVariables` as warnings, and exits 0 +on warnings**, so a gate that ran `biome lint` as C05 left it would have passed the entry's +own test case, an unused import. The gate passes `--error-on-warnings`. And the exit code +cannot tell a finding from a gate that checked nothing: Biome exits 1 for a configuration it +cannot read, as for a finding, and 0 for a scope that matches no script, because it counts +`biome.jsonc` among the files it checked and so never reports that no files were processed. +The gate reads the summary Biome writes to a file beside its usual output. No summary means +Biome stopped before linting, and the summary counts the findings and the files checked. +Neither the SARIF nor the JUnit report counts the files checked, and Biome prints a notice +calling its JSON report experimental on every run, so the summary is the report the gate +reads. C08's hook will pass the staged files, where checking none of them is normal, so the +floor of one script belongs to the whole-scope run only. + +Verified: clean on the tree, 0; an unused import, 1, and `test.bat` stops there; a syntax +error, 1; an unknown key in `biome.jsonc`, invalid JSON, and a scope that matches no script, +2; Biome not installed, 2. About 0.25 s. With the gate registered, `check_gate_lists.mjs` and +`check_ci_workflows.mjs` pass. Tools.md's "seven of the nine" became "seven of the ten", not +eight: the lint scope includes `docs/assets/js/`, so an edit under `docs/` can now affect +three gates. The tree comparison differs only in the three pages the commit edits (Tools, +Building, and Builder, whose dependency list now names the gate), the search index and +`book.html`. CI waits for the owner's push. + ### C07 — `scripts: convert_em_dash_separators exits 2 on a crash` **A6-3 (R2).** Its one exit is `process.exit(main())`, with 0 or 1 (`:210,214`), and a crash diff --git a/docs/Documentation/Builder.md b/docs/Documentation/Builder.md index 7b697e40..3d725e96 100644 --- a/docs/Documentation/Builder.md +++ b/docs/Documentation/Builder.md @@ -439,7 +439,7 @@ A single `package.json` at the repo root contains everything --- the static site } ``` -No template engine, no framework, no bundler, no postinstall hooks. For the site generator, the `markdown-it-*` packages cover the dialect extensions the legacy parser supported; `gray-matter` splits off page frontmatter and `js-yaml` parses `_config.yml` and `_book.yml`; `fast-glob` finds the source files; `shiki` is the syntax highlighter; `@hpcc-js/wasm-graphviz` is the WASM build of Graphviz that renders `.dot` diagram sources; `sass` is Dart Sass for the SCSS compile; `acorn` + `acorn-walk` parse the upstream `just-the-docs.js` for the AST-based offline patcher; and `htmlparser2` is the SAX parser under the link and integrity check. `puppeteer` + `pdf-lib` + `html-entities` are the PDF renderer's toolchain: puppeteer controls headless Chromium for the paged.js layout pass, and `html-entities` decodes the entities in the PDF outline's entries. `axe-core` + `puppeteer` also back the standalone accessibility checker ([`scripts/check_a11y.mjs`](https://github.com/twinbasic/documentation/blob/main/scripts/check_a11y.mjs)), which runs the same headless Chromium over the built pages, and `recheck` + `acorn` back the regex-safety gate ([`scripts/check_regex_safety.mjs`](https://github.com/twinbasic/documentation/blob/main/scripts/check_regex_safety.mjs)). `@biomejs/biome` is the repository's linter, which `biome.jsonc` limits to its correctness and suspicious rules. None of `axe-core`, `recheck` and `@biomejs/biome` is used by `tbdocs` itself. +No template engine, no framework, no bundler, no postinstall hooks. For the site generator, the `markdown-it-*` packages cover the dialect extensions the legacy parser supported; `gray-matter` splits off page frontmatter and `js-yaml` parses `_config.yml` and `_book.yml`; `fast-glob` finds the source files; `shiki` is the syntax highlighter; `@hpcc-js/wasm-graphviz` is the WASM build of Graphviz that renders `.dot` diagram sources; `sass` is Dart Sass for the SCSS compile; `acorn` + `acorn-walk` parse the upstream `just-the-docs.js` for the AST-based offline patcher; and `htmlparser2` is the SAX parser under the link and integrity check. `puppeteer` + `pdf-lib` + `html-entities` are the PDF renderer's toolchain: puppeteer controls headless Chromium for the paged.js layout pass, and `html-entities` decodes the entities in the PDF outline's entries. `axe-core` + `puppeteer` also back the standalone accessibility checker ([`scripts/check_a11y.mjs`](https://github.com/twinbasic/documentation/blob/main/scripts/check_a11y.mjs)), which runs the same headless Chromium over the built pages, and `recheck` + `acorn` back the regex-safety gate ([`scripts/check_regex_safety.mjs`](https://github.com/twinbasic/documentation/blob/main/scripts/check_regex_safety.mjs)). `@biomejs/biome` is the repository's linter, which `biome.jsonc` limits to its correctness and suspicious rules, and the lint gate ([`scripts/check_lint.mjs`](https://github.com/twinbasic/documentation/blob/main/scripts/check_lint.mjs)) runs it. None of `axe-core`, `recheck` and `@biomejs/biome` is used by `tbdocs` itself. **Which packages are pinned.** A package is pinned to an exact version where a new release could change what the build produces or what a gate reports without anything failing to say so: where the code patches the package or relies on its internals with no guard that fails when they change, or where the package's own results are what a gate reports. Everything else takes a caret range. Five packages are exact: diff --git a/docs/Documentation/Building.md b/docs/Documentation/Building.md index 71e60153..a51a873a 100644 --- a/docs/Documentation/Building.md +++ b/docs/Documentation/Building.md @@ -71,6 +71,7 @@ Each `.bat` opens with `@pushd "%~dp0"`, which is what lets it be invoked from a node scripts/check_publish_policy.mjs \ && node scripts/check_gate_lists.mjs \ && node scripts/check_ci_workflows.mjs \ + && node scripts/check_lint.mjs \ && node scripts/check_regex_safety.mjs \ && node scripts/check_code_regions.mjs \ && node scripts/check_page_baseline.mjs \ diff --git a/docs/Documentation/Tools.md b/docs/Documentation/Tools.md index a0f0f65b..6ca9c3ea 100644 --- a/docs/Documentation/Tools.md +++ b/docs/Documentation/Tools.md @@ -65,23 +65,25 @@ One of the four does not mean the same thing locally as it does in CI, on any pl test.bat -The tests the toolchain has to pass. Nine steps, each stopping the run if it fails: +The tests the toolchain has to pass. Ten steps, each stopping the run if it fails: 1. [`scripts/check_publish_policy.mjs`](#check-publish-policy) --- verifies the publish allowlist still refuses the types it is meant to. Needs neither a browser nor a built tree, so it goes first. 2. [`scripts/check_gate_lists.mjs`](#check-gate-lists) --- verifies the two gate lists on this page still match the wrappers that run them. 3. [`scripts/check_ci_workflows.mjs`](#check-ci-workflows) --- verifies both CI workflows run the gates the wrappers run, and build as `build.bat` does. -4. [`scripts/check_regex_safety.mjs`](#check-regex-safety) --- refuses a regex that can backtrack exponentially, written as a literal or built from constants. -5. [`scripts/check_code_regions.mjs`](#check-code-regions) --- verifies no pre-render rewrite alters the contents of a code fence or code span. -6. [`scripts/check_page_baseline.mjs`](#check-page-baseline) --- verifies the page-count drift guard still refuses a fall. -7. [`scripts/check_book_coverage.mjs`](#check-book-coverage) --- verifies the build still warns about a page `docs/_book.yml` does not mention. -8. [`scripts/check_symbol_index.mjs`](#check-symbol-index) --- verifies the symbol index still places each kind of symbol, and its drift guard still refuses a lost URL. -9. [`scripts/check_axe_patch_equiv.mjs`](#check-axe-patch-equiv) --- verifies the vendored axe source patch still produces identical colour values. +4. [`scripts/check_lint.mjs`](#check-lint) --- runs Biome over the tooling and fails on any finding, warnings included. +5. [`scripts/check_regex_safety.mjs`](#check-regex-safety) --- refuses a regex that can backtrack exponentially, written as a literal or built from constants. +6. [`scripts/check_code_regions.mjs`](#check-code-regions) --- verifies no pre-render rewrite alters the contents of a code fence or code span. +7. [`scripts/check_page_baseline.mjs`](#check-page-baseline) --- verifies the page-count drift guard still refuses a fall. +8. [`scripts/check_book_coverage.mjs`](#check-book-coverage) --- verifies the build still warns about a page `docs/_book.yml` does not mention. +9. [`scripts/check_symbol_index.mjs`](#check-symbol-index) --- verifies the symbol index still places each kind of symbol, and its drift guard still refuses a lost URL. +10. [`scripts/check_axe_patch_equiv.mjs`](#check-axe-patch-equiv) --- verifies the vendored axe source patch still produces identical colour values. POSIX: node scripts/check_publish_policy.mjs \ && node scripts/check_gate_lists.mjs \ && node scripts/check_ci_workflows.mjs \ + && node scripts/check_lint.mjs \ && node scripts/check_regex_safety.mjs \ && node scripts/check_code_regions.mjs \ && node scripts/check_page_baseline.mjs \ @@ -89,7 +91,7 @@ POSIX: && node scripts/check_symbol_index.mjs \ && node scripts/check_axe_patch_equiv.mjs -**Seven of the nine cannot be affected by an edit confined to `docs/`**, which is why they are separate from `check.bat`. Run this one when the change touches `builder/`, `scripts/`, `book/`, `eval/` or `wisdom/`, a wrapper, or a workflow. Both CI workflows run all nine unconditionally, as they always did, so skipping it locally cannot let a tooling regression reach `staging`. +**Seven of the ten cannot be affected by an edit confined to `docs/`**, which is why they are separate from `check.bat`. Run this one when the change touches `builder/`, `scripts/`, `book/`, `eval/`, `wisdom/` or `test/`, the site's scripts in `docs/assets/js/`, a wrapper, or a workflow. Both CI workflows run all ten unconditionally, as they always did, so skipping it locally cannot let a tooling regression reach `staging`. The two exceptions are [`check_code_regions.mjs`](#check-code-regions) and [`check_gate_lists.mjs`](#check-gate-lists), which reads this page. The first is worth knowing in detail. Its corpus sweep tokenises every markdown file under `docs/`, so a page that provokes a rewrite into altering a code region fails it. Its fixed probes are a different matter: they run against their own sources whatever the tree holds, and they cover the *mirror* fault, where a rewrite silently stops firing. The sweep cannot see that one --- text the rewrite skipped is stashed and restored unchanged, so every region still matches. Add a page with an unusual code construct and run `test.bat`, but read the built page too. @@ -457,6 +459,17 @@ The differences that are meant are listed in the script, each with where it is r Its probes ride along in every run: each plants one defect in a small synthetic set of wrappers, workflows and actions --- a missing gate, a step no wrapper runs, two gates swapped, changed arguments, a build flag lost or added, a gate missing from the shared action, a workflow that stops calling it --- and requires exactly the findings it should produce. Pure text: no browser, no built tree. Exits 0 clean, 1 on a finding, 2 when a probe fails or the gate cannot run. +### check_lint.mjs +{: #check-lint } + + node scripts/check_lint.mjs + +Runs Biome, pinned to an exact version, over the tooling: `builder/`, `scripts/`, `book/`, `eval/`, `wisdom/`, `test/` and the site's two scripts in `docs/assets/js/`, less the exceptions that `biome.jsonc` at the repository root lists and explains. The rules are the ones that find defects --- Biome's correctness and suspicious groups --- and none about style; the configuration names the few it turns off, each with its reason. Moving and deleting code leaves unused imports and undeclared names behind, and nothing else reads the tooling for them. No browser, no built tree, a fraction of a second. + +**Warnings fail as well as errors.** Biome reports an unused import or variable as a warning, and exits 0 on warnings, so a plain `npx biome lint` passes a file full of them. The gate also refuses to pass when Biome could not lint. Biome exits 1 for a broken `biome.jsonc`, as it does for a finding, and 0 for a scope that matches no script at all, so the gate reads the summary Biome writes beside its usual output to tell these apart. Exits 0 clean, 1 on a finding, 2 when Biome could not lint or checked no script. + +Lint before every commit that touches one of those folders. `npx biome lint --write` applies the fixes Biome marks safe. The fixes it offers for an unused import or variable are marked unsafe and need `--unsafe` as well, so read the diff after applying them. + ### check_page_baseline.mjs {: #check-page-baseline } diff --git a/scripts/check_lint.mjs b/scripts/check_lint.mjs new file mode 100644 index 00000000..d6c905e1 --- /dev/null +++ b/scripts/check_lint.mjs @@ -0,0 +1,87 @@ +#!/usr/bin/env node +// The lint gate: the pinned Biome over the scope biome.jsonc names. +// +// The rules are the ones that find defects -- Biome's correctness and +// suspicious groups -- and none about style. Moving and deleting code leaves +// unused imports and undeclared names behind, and nothing else reads the +// tooling for them. +// +// Warnings fail as well as errors. Biome reports noUnusedImports and +// noUnusedVariables as warnings and exits 0 on warnings, so a plain +// `biome lint` passes a file full of unused imports; --error-on-warnings is +// what makes them findings. +// +// Biome's exit code cannot tell a finding from a gate that checked nothing. It +// exits 1 for a finding and also for a configuration it cannot read, and 0 for +// a scope that matches no script at all: it counts biome.jsonc itself as a +// file checked, so it never says that no files were processed. The summary it +// writes to a file beside its usual output tells them apart. A configuration +// it cannot read leaves no summary; the summary counts the findings, and the +// files checked. +// +// node scripts/check_lint.mjs +// +// Exit codes: 0 clean, 1 a finding, 2 Biome could not lint, or checked no +// script. + +import { spawnSync } from "node:child_process"; +import { mkdtempSync, readFileSync, rmSync } from "node:fs"; +import { createRequire } from "node:module"; +import os from "node:os"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; + +process.on("uncaughtException", (err) => { console.error(err); process.exit(2); }); + +const ROOT = path.resolve(fileURLToPath(new URL("..", import.meta.url))); + +function cannotLint(message) { + console.error(`check_lint: ${message}`); + process.exit(2); +} + +let biome; +try { + biome = createRequire(import.meta.url).resolve("@biomejs/biome/bin/biome"); +} catch { + cannotLint("Biome is not installed; run npm install"); +} + +// What Biome's summary reporter wrote, or null when it wrote nothing. +function readSummary(file) { + let text; + try { + text = readFileSync(file, "utf8"); + } catch { + return null; + } + const checked = /\bChecked (\d+) files?\b/.exec(text); + let found = 0; + for (const m of text.matchAll(/\bFound (\d+) (?:errors?|warnings?)\b/g)) found += Number(m[1]); + return { checked: checked ? Number(checked[1]) : null, found }; +} + +const dir = mkdtempSync(path.join(os.tmpdir(), "check_lint-")); +let run; +let summary; +try { + const file = path.join(dir, "summary.txt"); + run = spawnSync( + process.execPath, + [biome, "lint", "--error-on-warnings", "--reporter=default", "--reporter=summary", `--reporter-file=${file}`], + { cwd: ROOT, stdio: ["ignore", "inherit", "inherit"] }, + ); + summary = readSummary(file); +} finally { + rmSync(dir, { recursive: true, force: true }); +} + +if (run.error) cannotLint(`could not start Biome: ${run.error.message}`); +if (!summary) cannotLint(`Biome stopped before linting (exit ${run.status}); its message is above`); +if (run.status !== 0) { + if (summary.found > 0) process.exit(1); + cannotLint(`Biome failed without a finding (exit ${run.status}); its message is above`); +} +// The count includes biome.jsonc. +if (summary.checked === null) cannotLint("Biome's summary does not say how many files it checked"); +if (summary.checked < 2) cannotLint("Biome checked no script: the scope biome.jsonc names matches none"); diff --git a/test.bat b/test.bat index f5e5de90..fb3b1f10 100644 --- a/test.bat +++ b/test.bat @@ -53,6 +53,14 @@ node scripts/check_gate_lists.mjs @rem ~100 ms. node scripts/check_ci_workflows.mjs @if errorlevel 1 goto :fail +@rem Biome over the tooling, with the rules that find defects and none +@rem about style (biome.jsonc). Moving and deleting code leaves unused +@rem imports and undeclared names behind, and nothing else reads the +@rem tooling for them. Warnings fail too: Biome reports an unused import +@rem as a warning and exits 0 on one. Lint before every commit. No tree, +@rem no browser, ~0.25 s. +node scripts/check_lint.mjs +@if errorlevel 1 goto :fail @rem A regex that backtracks exponentially is a hang waiting for the @rem right input, and nothing that reads the site can see it: the corpus @rem passes until some page happens to contain the trigger, and then the From 0ffd7e9aa33d9fd7eed4f1aef931e8e76eb584ff Mon Sep 17 00:00:00 2001 From: Kuba Sunderland-Ober Date: Fri, 25 Sep 2026 21:06:30 +0200 Subject: [PATCH 17/53] scripts: convert_em_dash_separators exits 2 on a crash --- builder/PLAN-TOOLING-REVIEW.md | 13 ++++++++++++- scripts/convert_em_dash_separators.mjs | 4 ++++ 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/builder/PLAN-TOOLING-REVIEW.md b/builder/PLAN-TOOLING-REVIEW.md index 1edf31e4..c56c2bfc 100644 --- a/builder/PLAN-TOOLING-REVIEW.md +++ b/builder/PLAN-TOOLING-REVIEW.md @@ -633,6 +633,15 @@ one helper. **Verify.** A forced throw exits 2; `--check` over `docs/` exits 0; a planted literal dash exits 1. +**Landed** with one difference from the four gates' copies: the handler is installed inside +the entry-point guard (`process.argv[1]` against `import.meta.url`), not at the top of the +module. The tool is written to be importable, and a module that installs a process-wide +handler on import changes how the importing process ends on a crash. C43's helper has to keep +that property. The header now states the three exit codes. A throw forced from a preload +(`node --import`, replacing `fs.promises.readFile`) exited 1 before the change and 2 after, +since a rejected top-level `await` reaches `uncaughtException`; `--check` exits 0 on `docs/` +and 1 with a planted em-dash. The tree comparison is identical. + ### C08 — `githooks: a pre-commit hook that runs Biome on the staged files` **Decision 4.** The owner approved the hook on condition that it runs Biome and nothing else. @@ -1240,7 +1249,9 @@ detail. **Change.** `scripts/lib/gate-probes.mjs` holds the accumulator, the report, the crash handler and `withBaseline`. Probes stay unconditional, and the exit code for a failed probe is a parameter: 1 for these gates, 2 where probes guard a separate sweep. The three gates and -`check_publish_policy.mjs` adopt it, and so do the handlers C07 and C28 added. +`check_publish_policy.mjs` adopt it, and so do the handlers C07 and C28 added. C07's sits +inside `convert_em_dash_separators.mjs`'s entry-point guard because that module is +importable, so the shared handler is installed by a call, never as a side effect of the import. `check_gate_lists.mjs` and `check_regex_safety.mjs` adopt it only if the fit is exact. The re-indenting becomes the shared behaviour: the one change in output, and only in gate text. diff --git a/scripts/convert_em_dash_separators.mjs b/scripts/convert_em_dash_separators.mjs index ba166145..c2bbe9f6 100644 --- a/scripts/convert_em_dash_separators.mjs +++ b/scripts/convert_em_dash_separators.mjs @@ -5,6 +5,9 @@ // node scripts/convert_em_dash_separators.mjs # rewrite in place // node scripts/convert_em_dash_separators.mjs --check # report, change nothing // +// Exit codes: 0 nothing to report, or converted; 1 --check found a literal +// dash; 2 the tool failed, so that a crash cannot read as a finding. +// // The typographer (enabled in builder/render.mjs) renders: // // source `--` -> en-dash @@ -211,5 +214,6 @@ async function main(argv) { } if (process.argv[1] && pathToFileURL(process.argv[1]).href === import.meta.url) { + process.on("uncaughtException", (err) => { console.error(err); process.exit(2); }); process.exit(await main(process.argv.slice(2))); } From 982047b85fc37d570efa7e00bb0f0b13a71cf261 Mon Sep 17 00:00:00 2001 From: Kuba Sunderland-Ober Date: Fri, 25 Sep 2026 21:12:54 +0200 Subject: [PATCH 18/53] githooks: a pre-commit hook that runs Biome on the staged files --- .gitattributes | 5 ++++ .githooks/pre-commit | 11 ++++++++ WIP.md | 2 ++ builder/PLAN-TOOLING-REVIEW.md | 24 ++++++++++++++++++ docs/Documentation/Tools.md | 9 +++++-- scripts/check_lint.mjs | 46 ++++++++++++++++++++++++++-------- 6 files changed, 84 insertions(+), 13 deletions(-) create mode 100644 .gitattributes create mode 100755 .githooks/pre-commit diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 00000000..5441e169 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,5 @@ +# Git hooks stay LF even where core.autocrlf checks files out CRLF. Git for +# Windows runs a CRLF hook without trouble, but a POSIX Git -- WSL on a +# Windows checkout, say -- hands the hook to the kernel, which reads the +# carriage return after #!/bin/sh as part of the interpreter's name. +.githooks/* text eol=lf diff --git a/.githooks/pre-commit b/.githooks/pre-commit new file mode 100755 index 00000000..b27d8a09 --- /dev/null +++ b/.githooks/pre-commit @@ -0,0 +1,11 @@ +#!/bin/sh +# Lints the scripts this commit stages with the pinned Biome, through +# scripts/check_lint.mjs --staged: the check test.bat and both CI workflows +# run over the whole scope, on the files being committed. It runs nothing +# else, and a commit that stages no script returns before Biome starts. +# +# Enable it in a clone: git config core.hooksPath .githooks +# +# Git runs a hook from the top of the working tree. .gitattributes keeps this +# file LF in every checkout, for a POSIX Git's sake: see the reason there. +exec node scripts/check_lint.mjs --staged diff --git a/WIP.md b/WIP.md index dd869063..5b4238a7 100644 --- a/WIP.md +++ b/WIP.md @@ -536,6 +536,8 @@ Favor concise one-line git commit messages. **Lint before every commit:** `node scripts/check_lint.mjs`, a fraction of a second. It runs Biome over the tooling and the site's two scripts, and fails on a warning as well as an error, because Biome reports an unused import as a warning. `test.bat` and CI run it too. +The pre-commit hook in `.githooks/` runs it on the staged scripts and nothing else; enable it +in a clone with `git config core.hooksPath .githooks`. **A bug in twinBASIC itself goes in [BUGS-TO-REPORT.md](BUGS-TO-REPORT.md)**, which is a queue rather than a record: an entry is deleted once it has been filed upstream. Each one diff --git a/builder/PLAN-TOOLING-REVIEW.md b/builder/PLAN-TOOLING-REVIEW.md index c56c2bfc..e0211228 100644 --- a/builder/PLAN-TOOLING-REVIEW.md +++ b/builder/PLAN-TOOLING-REVIEW.md @@ -658,6 +658,30 @@ same check for a clone without it. **Verify.** A staged file with an unused import is refused; a clean commit passes; a commit that stages no JavaScript is not slowed. Time the hook on a typical commit. +**Landed** as the entry describes, with three details. The hook is one line, `exec node +scripts/check_lint.mjs --staged`. The gate's new `--staged` asks git for the scripts the commit +adds or changes (`git diff --cached --diff-filter=ACMR`) and passes them to Biome with +`--no-errors-on-unmatched`, so a staged script outside the scope is skipped and checking none +is clean; the whole-scope run keeps its floor. A partly staged file is linted as it is in the +working tree. A new `.gitattributes` keeps `.githooks/*` LF. Git for Windows ran a CRLF copy +of the hook correctly, through `sh` and through `git hook run`, so the rule is for a POSIX Git +on a CRLF checkout, such as WSL on a Windows tree, whose kernel would read the carriage return +after `#!/bin/sh` as part of the interpreter's name. That case is untested, since this machine +has no WSL. C84, which settles line endings, keeps the rule. The hook is committed executable, +as a POSIX Git requires. + +`core.hooksPath` in this clone's `.git/config` was `D:\OCP\wc\twinBASIC-documentation\.git\hooks`, +a folder holding only Git's samples, and is now `.githooks`. The worktrees under +`.claude/worktrees/` share the setting, and get the hook once their branch has it. + +Verified: with a planted unused import staged, `git hook run pre-commit` exited 1 and +`git commit` was refused with HEAD unchanged; a staged script outside the scope, in `perf/`, +was skipped, exit 0; this commit, which stages `check_lint.mjs`, passed the hook. Timed with +`git hook run`: Git with no hook about 55 ms; the hook with no script staged about 145 ms, the +difference being Node's start and one `git diff`; with one clean script staged about 230 ms. +The gate's whole-scope cases are unchanged. The tree comparison differs only in Tools.html, +the search index and `book.html`. + ## Phase 1: remove, relocate, and fix in place Done ahead of this phase, during the review: the four superseded pdf-lib shims deleted diff --git a/docs/Documentation/Tools.md b/docs/Documentation/Tools.md index 6ca9c3ea..18e6ab4b 100644 --- a/docs/Documentation/Tools.md +++ b/docs/Documentation/Tools.md @@ -463,12 +463,17 @@ Its probes ride along in every run: each plants one defect in a small synthetic {: #check-lint } node scripts/check_lint.mjs + node scripts/check_lint.mjs --staged Runs Biome, pinned to an exact version, over the tooling: `builder/`, `scripts/`, `book/`, `eval/`, `wisdom/`, `test/` and the site's two scripts in `docs/assets/js/`, less the exceptions that `biome.jsonc` at the repository root lists and explains. The rules are the ones that find defects --- Biome's correctness and suspicious groups --- and none about style; the configuration names the few it turns off, each with its reason. Moving and deleting code leaves unused imports and undeclared names behind, and nothing else reads the tooling for them. No browser, no built tree, a fraction of a second. -**Warnings fail as well as errors.** Biome reports an unused import or variable as a warning, and exits 0 on warnings, so a plain `npx biome lint` passes a file full of them. The gate also refuses to pass when Biome could not lint. Biome exits 1 for a broken `biome.jsonc`, as it does for a finding, and 0 for a scope that matches no script at all, so the gate reads the summary Biome writes beside its usual output to tell these apart. Exits 0 clean, 1 on a finding, 2 when Biome could not lint or checked no script. +**Warnings fail as well as errors.** Biome reports an unused import or variable as a warning, and exits 0 on warnings, so a plain `npx biome lint` passes a file full of them. The gate also refuses to pass when Biome could not lint. Biome exits 1 for a broken `biome.jsonc`, as it does for a finding, and 0 for a scope that matches no script at all, so the gate reads the summary Biome writes beside its usual output to tell these apart. Exits 0 clean, 1 on a finding, 2 when Biome could not lint or, over the whole scope, checked no script. -Lint before every commit that touches one of those folders. `npx biome lint --write` applies the fixes Biome marks safe. The fixes it offers for an unused import or variable are marked unsafe and need `--unsafe` as well, so read the diff after applying them. +Lint before every commit that touches one of those folders, or let the pre-commit hook do it. `.githooks/pre-commit` runs this gate with `--staged`, on the scripts the commit adds or changes, as they are in the working tree, and runs nothing else. Biome skips the staged scripts its scope excludes, and a commit that stages no script returns before Biome starts. Enable the hook in a clone with: + + git config core.hooksPath .githooks + +A clone without the hook is still checked, because `test.bat` and both CI workflows run this gate over the whole scope. `npx biome lint --write` applies the fixes Biome marks safe. The fixes it offers for an unused import or variable are marked unsafe and need `--unsafe` as well, so read the diff after applying them. ### check_page_baseline.mjs {: #check-page-baseline } diff --git a/scripts/check_lint.mjs b/scripts/check_lint.mjs index d6c905e1..6a420642 100644 --- a/scripts/check_lint.mjs +++ b/scripts/check_lint.mjs @@ -19,13 +19,20 @@ // it cannot read leaves no summary; the summary counts the findings, and the // files checked. // -// node scripts/check_lint.mjs +// With --staged it lints only the scripts the next commit adds or changes, +// which is how the pre-commit hook in .githooks/ runs it. Biome keeps those +// its scope includes, and a commit whose scripts it excludes checks none and +// is clean. A commit that stages no script returns before Biome starts. A +// partly staged file is linted as it is in the working tree. // -// Exit codes: 0 clean, 1 a finding, 2 Biome could not lint, or checked no -// script. +// node scripts/check_lint.mjs # the whole scope: test.bat and CI +// node scripts/check_lint.mjs --staged # the staged scripts: the hook +// +// Exit codes: 0 clean, 1 a finding, 2 Biome could not lint or, over the whole +// scope, checked no script. import { spawnSync } from "node:child_process"; -import { mkdtempSync, readFileSync, rmSync } from "node:fs"; +import { existsSync, mkdtempSync, readFileSync, rmSync } from "node:fs"; import { createRequire } from "node:module"; import os from "node:os"; import path from "node:path"; @@ -40,6 +47,25 @@ function cannotLint(message) { process.exit(2); } +const argv = process.argv.slice(2); +const staged = argv.length === 1 && argv[0] === "--staged"; +if (argv.length && !staged) cannotLint("usage: node scripts/check_lint.mjs [--staged]"); + +// The scripts the next commit adds or changes that are still on disk, by the +// two extensions the scope in biome.jsonc is made of. +function stagedScripts() { + const r = spawnSync("git", ["diff", "--cached", "--name-only", "--diff-filter=ACMR", "-z"], { + cwd: ROOT, + encoding: "utf8", + }); + if (r.error) cannotLint(`could not run git: ${r.error.message}`); + if (r.status !== 0) cannotLint(`git diff --cached failed: ${r.stderr.trim()}`); + return r.stdout.split("\0").filter((f) => /\.m?js$/.test(f) && existsSync(path.join(ROOT, f))); +} + +const scripts = staged ? stagedScripts() : []; +if (staged && !scripts.length) process.exit(0); + let biome; try { biome = createRequire(import.meta.url).resolve("@biomejs/biome/bin/biome"); @@ -66,11 +92,9 @@ let run; let summary; try { const file = path.join(dir, "summary.txt"); - run = spawnSync( - process.execPath, - [biome, "lint", "--error-on-warnings", "--reporter=default", "--reporter=summary", `--reporter-file=${file}`], - { cwd: ROOT, stdio: ["ignore", "inherit", "inherit"] }, - ); + const args = ["lint", "--error-on-warnings", "--reporter=default", "--reporter=summary", `--reporter-file=${file}`]; + if (staged) args.push("--no-errors-on-unmatched", "--", ...scripts); + run = spawnSync(process.execPath, [biome, ...args], { cwd: ROOT, stdio: ["ignore", "inherit", "inherit"] }); summary = readSummary(file); } finally { rmSync(dir, { recursive: true, force: true }); @@ -82,6 +106,6 @@ if (run.status !== 0) { if (summary.found > 0) process.exit(1); cannotLint(`Biome failed without a finding (exit ${run.status}); its message is above`); } -// The count includes biome.jsonc. if (summary.checked === null) cannotLint("Biome's summary does not say how many files it checked"); -if (summary.checked < 2) cannotLint("Biome checked no script: the scope biome.jsonc names matches none"); +// The whole scope has to reach a script; the count includes biome.jsonc. +if (!staged && summary.checked < 2) cannotLint("Biome checked no script: the scope biome.jsonc names matches none"); From 3a80006372e1ba5f3476796b879a328425b6dd5b Mon Sep 17 00:00:00 2001 From: Kuba Sunderland-Ober Date: Fri, 25 Sep 2026 21:30:23 +0200 Subject: [PATCH 19/53] deps: declare picocolors and pako, which the code imports directly --- builder/PLAN-TOOLING-REVIEW.md | 15 +++++++++++++++ docs/Documentation/Builder.md | 7 +++++-- package-lock.json | 2 ++ package.json | 2 ++ 4 files changed, 24 insertions(+), 2 deletions(-) diff --git a/builder/PLAN-TOOLING-REVIEW.md b/builder/PLAN-TOOLING-REVIEW.md index e0211228..ef7b95e2 100644 --- a/builder/PLAN-TOOLING-REVIEW.md +++ b/builder/PLAN-TOOLING-REVIEW.md @@ -703,6 +703,21 @@ Dependencies gains both rows. **Verify.** `npm ls picocolors pako` shows both as direct dependencies at the same versions, and `package-lock.json` should change only in its root entry. The tree comparison identical. +**Landed** as the entry describes. Each package was installed once, at the version declared, +and both lockfiles agreed with every package's own `package.json` apart from the optional +packages for other platforms, so `npm install` reported the tree up to date and changed no +installed file. `package-lock.json` changed only in its root entry, and `npm ls` shows both +at depth 0. Builder.md's block gains both, its prose says what each is for, and its pinned +list gains `pako`, making six: `fast-inflate.mjs` patches the copy it imports, which reaches +pdf-lib only while the two share one copy, and 1.0.11 is the last 1.x release, the only one +pdf-lib's own `^1.0.11` accepts. Declaring pako 2 instead would put it at the root and nest +pdf-lib's own copy under `pdf-lib/`, and the patch would stop reaching pdf-lib without an +error. + +Verified with two `--keep` runs of the tree comparison, one before the install and one after: +HEAD's two builds are identical under the three normalisers, and the working tree differs from +HEAD only in Builder.html, the search index and `book.html`. + ### C10 — `scripts: move census_attributes.mjs out of builder/` **A8-2 (R2).** It imports `../scripts/lib/tb-packages.mjs` (`:79`) against `builder/`'s rule diff --git a/docs/Documentation/Builder.md b/docs/Documentation/Builder.md index 3d725e96..8dff17c3 100644 --- a/docs/Documentation/Builder.md +++ b/docs/Documentation/Builder.md @@ -430,7 +430,9 @@ A single `package.json` at the repo root contains everything --- the static site "markdown-it-attrs": "^4.3", "markdown-it-deflist": "^3.0", "markdown-it-footnote": "^4.0", + "pako": "1.0.11", "pdf-lib": "1.17.1", + "picocolors": "^1.1.1", "puppeteer": "25.0.4", "recheck": "4.5.0", "sass": "^1.0", @@ -439,12 +441,13 @@ A single `package.json` at the repo root contains everything --- the static site } ``` -No template engine, no framework, no bundler, no postinstall hooks. For the site generator, the `markdown-it-*` packages cover the dialect extensions the legacy parser supported; `gray-matter` splits off page frontmatter and `js-yaml` parses `_config.yml` and `_book.yml`; `fast-glob` finds the source files; `shiki` is the syntax highlighter; `@hpcc-js/wasm-graphviz` is the WASM build of Graphviz that renders `.dot` diagram sources; `sass` is Dart Sass for the SCSS compile; `acorn` + `acorn-walk` parse the upstream `just-the-docs.js` for the AST-based offline patcher; and `htmlparser2` is the SAX parser under the link and integrity check. `puppeteer` + `pdf-lib` + `html-entities` are the PDF renderer's toolchain: puppeteer controls headless Chromium for the paged.js layout pass, and `html-entities` decodes the entities in the PDF outline's entries. `axe-core` + `puppeteer` also back the standalone accessibility checker ([`scripts/check_a11y.mjs`](https://github.com/twinbasic/documentation/blob/main/scripts/check_a11y.mjs)), which runs the same headless Chromium over the built pages, and `recheck` + `acorn` back the regex-safety gate ([`scripts/check_regex_safety.mjs`](https://github.com/twinbasic/documentation/blob/main/scripts/check_regex_safety.mjs)). `@biomejs/biome` is the repository's linter, which `biome.jsonc` limits to its correctness and suspicious rules, and the lint gate ([`scripts/check_lint.mjs`](https://github.com/twinbasic/documentation/blob/main/scripts/check_lint.mjs)) runs it. None of `axe-core`, `recheck` and `@biomejs/biome` is used by `tbdocs` itself. +No template engine, no framework, no bundler, no postinstall hooks. For the site generator, the `markdown-it-*` packages cover the dialect extensions the legacy parser supported; `gray-matter` splits off page frontmatter and `js-yaml` parses `_config.yml` and `_book.yml`; `fast-glob` finds the source files; `shiki` is the syntax highlighter; `@hpcc-js/wasm-graphviz` is the WASM build of Graphviz that renders `.dot` diagram sources; `sass` is Dart Sass for the SCSS compile; `acorn` + `acorn-walk` parse the upstream `just-the-docs.js` for the AST-based offline patcher; `htmlparser2` is the SAX parser under the link and integrity check; and `picocolors` colours `tbdocs`'s terminal output. `puppeteer` + `pdf-lib` + `html-entities` are the PDF renderer's toolchain: puppeteer controls headless Chromium for the paged.js layout pass, and `html-entities` decodes the entities in the PDF outline's entries. `pako` is pdf-lib's own zlib library, declared because the renderer imports it too. `axe-core` + `puppeteer` also back the standalone accessibility checker ([`scripts/check_a11y.mjs`](https://github.com/twinbasic/documentation/blob/main/scripts/check_a11y.mjs)), which runs the same headless Chromium over the built pages, and `recheck` + `acorn` back the regex-safety gate ([`scripts/check_regex_safety.mjs`](https://github.com/twinbasic/documentation/blob/main/scripts/check_regex_safety.mjs)). `@biomejs/biome` is the repository's linter, which `biome.jsonc` limits to its correctness and suspicious rules, and the lint gate ([`scripts/check_lint.mjs`](https://github.com/twinbasic/documentation/blob/main/scripts/check_lint.mjs)) runs it. None of `axe-core`, `recheck` and `@biomejs/biome` is used by `tbdocs` itself. -**Which packages are pinned.** A package is pinned to an exact version where a new release could change what the build produces or what a gate reports without anything failing to say so: where the code patches the package or relies on its internals with no guard that fails when they change, or where the package's own results are what a gate reports. Everything else takes a caret range. Five packages are exact: +**Which packages are pinned.** A package is pinned to an exact version where a new release could change what the build produces or what a gate reports without anything failing to say so: where the code patches the package or relies on its internals with no guard that fails when they change, or where the package's own results are what a gate reports. Everything else takes a caret range. Six packages are exact: - `@biomejs/biome` --- a new release can add or change a rule, which would change the linter's verdict on code nobody touched. [PLAN-TOOLING-REVIEW.md](https://github.com/twinbasic/documentation/blob/main/builder/PLAN-TOOLING-REVIEW.md), decision 4, records why the pin is exact. - `axe-core` --- the scan injects a copy of its bundle patched at source level, and its rules decide the accessibility gate's verdict. [PLAN-axe-perf.md](https://github.com/twinbasic/documentation/blob/main/builder/PLAN-axe-perf.md) records why the pin is exact. +- `pako` --- [`fast-inflate.mjs`](Fixes/PDFLib#fast-inflatemjs) replaces its `inflate` at run time, and the replacement reaches pdf-lib only while the two share one copy of pako. 1.0.11 is the last 1.x release, the only one pdf-lib's own range accepts. - `pdf-lib` --- the shims under `book/lib/` are line-by-line ports of this release's source, and pdf-lib is no longer maintained; [08-pdf-lib.md](https://github.com/twinbasic/documentation/blob/main/perf/notes/08-pdf-lib.md) records the pin. - `puppeteer` --- the book renderer and the accessibility gate measure what its Chromium renders, and the performance notes reason about that version at source level. It was pinned in the same change as `pdf-lib`. - `recheck` --- the regex-safety gate reports its analysis, and finds its native backend itself, because this release cannot find it on Windows; [WIP.Build.md](https://github.com/twinbasic/documentation/blob/main/WIP.Build.md) records the workaround. diff --git a/package-lock.json b/package-lock.json index 646ad427..ab85e354 100644 --- a/package-lock.json +++ b/package-lock.json @@ -22,7 +22,9 @@ "markdown-it-attrs": "^4.3", "markdown-it-deflist": "^3.0", "markdown-it-footnote": "^4.0", + "pako": "1.0.11", "pdf-lib": "1.17.1", + "picocolors": "^1.1.1", "puppeteer": "25.0.4", "recheck": "4.5.0", "sass": "^1.0", diff --git a/package.json b/package.json index 56a8f5e6..bad90605 100644 --- a/package.json +++ b/package.json @@ -18,7 +18,9 @@ "markdown-it-attrs": "^4.3", "markdown-it-deflist": "^3.0", "markdown-it-footnote": "^4.0", + "pako": "1.0.11", "pdf-lib": "1.17.1", + "picocolors": "^1.1.1", "puppeteer": "25.0.4", "recheck": "4.5.0", "sass": "^1.0", From 746cb63138a35d2122d066e86925da08f5afec0e Mon Sep 17 00:00:00 2001 From: Kuba Sunderland-Ober Date: Fri, 25 Sep 2026 21:37:12 +0200 Subject: [PATCH 20/53] scripts: move census_attributes.mjs out of builder/ --- BUGS-TO-REPORT.md | 2 +- WIP.ExamplesBuild.md | 2 +- WIP.Harness.md | 13 +++++-------- WIP.HelpAddin.md | 2 +- WIP.md | 2 +- biome.jsonc | 19 +++++++++++++++++++ builder/PLAN-TOOLING-REVIEW.md | 22 ++++++++++++++++++++++ docs/Documentation/Tools.md | 4 ++-- docs/Reference/Attributes.md | 2 +- scripts/build_package_api.mjs | 2 +- {builder => scripts}/census_attributes.mjs | 4 ++-- scripts/check_tree_fresh.mjs | 8 +------- scripts/lib/tb-fences.mjs | 4 ++-- scripts/lib/tb-packages.mjs | 2 +- scripts/lib/twin-api.mjs | 2 +- 15 files changed, 61 insertions(+), 29 deletions(-) rename {builder => scripts}/census_attributes.mjs (99%) diff --git a/BUGS-TO-REPORT.md b/BUGS-TO-REPORT.md index 074407d8..e1e99eb9 100644 --- a/BUGS-TO-REPORT.md +++ b/BUGS-TO-REPORT.md @@ -427,7 +427,7 @@ file is written and the run ends `... DONE`. **What does not reproduce it:** a long *input* path to `export`, and any output folder short enough that no file path reaches 260 characters and no folder path 248. -**Found by** the attribute census, `builder/census_attributes.mjs`, pointed at a cache folder +**Found by** the attribute census, `scripts/census_attributes.mjs`, pointed at a cache folder inside a deep working directory: `WebView2Package` and the three `cefPackage` versions came back `... FAILED` while the other twelve packages exported. The census used to trust `export`'s exit code, so until it tested for `... DONE` it would have scanned those partial diff --git a/WIP.ExamplesBuild.md b/WIP.ExamplesBuild.md index 56a0d932..8b66bd19 100644 --- a/WIP.ExamplesBuild.md +++ b/WIP.ExamplesBuild.md @@ -82,7 +82,7 @@ probe in `check_examples.mjs`: `Type_Initialize`, `Type_Assignment` and `Type_Conversion`, which is what `Features/Language/UDTs.md` is about. But a UDT *field* may also be called `Type As Long`, which reads as an opener that never closes --- the trap - `builder/census_attributes.mjs` records paying for, so every opener demands a name after + `scripts/census_attributes.mjs` records paying for, so every opener demands a name after the keyword. - **`Overridable` is a modifier**, with 32 uses in the shipped packages and 3 in `docs/`. Leaving it out of the list cost three fences, and they came back as *"End Function closing diff --git a/WIP.Harness.md b/WIP.Harness.md index d34c18dc..39062a7c 100644 --- a/WIP.Harness.md +++ b/WIP.Harness.md @@ -12,7 +12,7 @@ Read it before changing `scripts/tbbuild.mjs`, `scripts/tbrun.mjs`, `scripts/lib/tb-launch.ps1`, `scripts/lib/tb-registry.mjs`, `scripts/lib/tb-ide-copy.mjs`, `scripts/lib/tb-project.mjs`, `scripts/lib/tb-addin.mjs`, `scripts/lib/tb-operate.mjs`, `scripts/lib/tb-lane.mjs`, -anything under `test/addin/`, or `builder/census_attributes.mjs`, and before +anything under `test/addin/`, or `scripts/census_attributes.mjs`, and before concluding anything about twinBASIC syntax from a sweep of exported sources. ## Getting at the `.twin` sources @@ -72,19 +72,16 @@ minutes, against a question that four documentation pages could not settle betwe ## Censusing every attribute at once -[builder/census_attributes.mjs](builder/census_attributes.mjs) --- which sits under -`builder/` by deliberate placement rather than because it renders anything; it is listed in -`check_tree_fresh.mjs`'s `IGNORED_FILES` for exactly that reason, so editing it does not -mark every output tree stale --- does the export above for +[scripts/census_attributes.mjs](scripts/census_attributes.mjs) does the export above for every package of the current install and reports, per attribute, **which enclosing construct and which kind of declaration it decorates**. No arguments needed; it finds the newest `twinBASIC_IDE_BETA_*` the same way `tbbuild` does, caches the export by build number, and re-uses it. ```sh -node builder/census_attributes.mjs --out census.md -node builder/census_attributes.mjs --attr Hidden # one attribute -node builder/census_attributes.mjs --attr Hidden --dump-sites sites.json +node scripts/census_attributes.mjs --out census.md +node scripts/census_attributes.mjs --attr Hidden # one attribute +node scripts/census_attributes.mjs --attr Hidden --dump-sites sites.json ``` Against BETA 983: **661 files, 9,701 attribute sites, 55 distinct attributes**, and every diff --git a/WIP.HelpAddin.md b/WIP.HelpAddin.md index 472f4bc9..c2e78fdc 100644 --- a/WIP.HelpAddin.md +++ b/WIP.HelpAddin.md @@ -734,7 +734,7 @@ at `/tB/Modules/Collection`, not under VBRUN. The index is generated instead. `/tB/Core/Attributes#`. - **From the packages:** the public API from an `export` of the shipped packages --- the export and its cache already exist in - [builder/census_attributes.mjs](builder/census_attributes.mjs). That supplies each + [scripts/census_attributes.mjs](scripts/census_attributes.mjs). That supplies each symbol's package, container and kind, and lists public symbols that have no page, which measures documentation coverage as a side effect. **It must also supply each class's default interface**, because that is what the compiler names a class's members by (P5): diff --git a/WIP.md b/WIP.md index 5b4238a7..29cb871c 100644 --- a/WIP.md +++ b/WIP.md @@ -130,7 +130,7 @@ because an install path contains a username. ```sh "$TB/bin/twinBASIC_win32.exe" export ".twinproj" "C:\out\dir\" --overwrite -node builder/census_attributes.mjs --out census.md # every attribute, by enclosing construct +node scripts/census_attributes.mjs --out census.md # every attribute, by enclosing construct node scripts/tbbuild.mjs C:/probe/Thing.twinproj # does it compile node scripts/tbrun.mjs # what does it print ``` diff --git a/biome.jsonc b/biome.jsonc index 830dfd81..3a37c424 100644 --- a/biome.jsonc +++ b/biome.jsonc @@ -53,6 +53,25 @@ // ES5-style scripts that ship to readers as written. "includes": ["docs/assets/js/*.js"], "linter": { "rules": { "correctness": { "noInnerDeclarations": "off" } } } + }, + { + // builder/ must not depend on scripts/. check_tree_fresh.mjs watches + // docs/ and builder/ as the inputs that decide the built bytes, and would + // not see a change to a module the build imported from scripts/. Biome + // files the rule under style; here it guards a dependency. + "includes": ["builder/**/*.mjs"], + "linter": { + "rules": { + "style": { + "noRestrictedImports": { + "level": "error", + "options": { + "patterns": [{ "group": ["**/scripts/**"], "message": "builder/ must not depend on scripts/." }] + } + } + } + } + } } ] } diff --git a/builder/PLAN-TOOLING-REVIEW.md b/builder/PLAN-TOOLING-REVIEW.md index ef7b95e2..6b06a9c9 100644 --- a/builder/PLAN-TOOLING-REVIEW.md +++ b/builder/PLAN-TOOLING-REVIEW.md @@ -735,6 +735,28 @@ guard rather than a comment. reaches the page. `check_tree_fresh.mjs` clean. `census_attributes.mjs --json` byte-identical before and after (a harness run). +**Landed** as the entry describes. `REPO` needed no change, since `scripts/` sits at the same +depth as `builder/`. Seventeen lines citing the old path changed, in twelve files: the tool's +own header, the three `scripts/lib/` modules and `build_package_api.mjs` that name it, +Tools.md's usage line, the HTML comment in Attributes.md, `BUGS-TO-REPORT.md`, WIP.md and three +WIP siblings. Two more places described the old placement rather than citing the path, and +lost the description: Tools.md's opening paragraph listed the tool as the one executable +under `builder/`, and WIP.Harness.md explained why it sat there and why `IGNORED_FILES` named +it. The review's own files keep the old path, as records of `fe9ce12b`. + +Biome can express the rule. `biome.jsonc` gains an override for `builder/**/*.mjs` that turns +on `style/noRestrictedImports` with the pattern `**/scripts/**`, a dependency guard although +Biome files it under style. A probe under `builder/` showed it catching a static import, a +re-export, a bare side-effect import and a dynamic `import()` into `scripts/`, and passing +`picocolors` and `./render.mjs`; the same file under `scripts/` is not checked. With the +census copied back into `builder/` with its old import, `check_lint.mjs` exits 1. C12's `lib/` +needs a rule of its own, since it may import none of the tree's other folders. + +Verified: the census's `--json` report is byte-identical before and after the move, from the +cached export of BETA 983 (661 files, 9,701 sites; no compiler started). The tree comparison +differs only in Tools.html, Attributes.html (the HTML comment does reach the page), the search +index and `book.html`, which carries both pages. + ### C11 — `scripts: census_attributes finds the install through tb-install` **L2-2 (R1).** `findInstall` (`census_attributes.mjs:99-119`) recognises an install by its diff --git a/docs/Documentation/Tools.md b/docs/Documentation/Tools.md index 18e6ab4b..9df63e33 100644 --- a/docs/Documentation/Tools.md +++ b/docs/Documentation/Tools.md @@ -8,7 +8,7 @@ permalink: /Documentation/Development/Tools # Tools and Scripts {: .no_toc } -One-line-per-tool reference for every executable in the documentation repository: the seven Windows batch wrappers at the repository root, the Node and Python scripts under `scripts/` (cross-platform except for [`tbbuild.mjs`](#tbbuild), which drives the twinBASIC IDE), the `tbdocs` orchestrator and its CLI flags, [`census_attributes.mjs`](#census-attributes) under `builder/`, and the PDF render driver. If you are looking for the day-to-day workflow rather than a cheat sheet, the [Building and Deployment](Building) page is the gentler read; if you are modifying the build pipeline itself, the [tbdocs Internals](Builder) page goes one level deeper. +One-line-per-tool reference for every executable in the documentation repository: the seven Windows batch wrappers at the repository root, the Node and Python scripts under `scripts/` (cross-platform except for [`tbbuild.mjs`](#tbbuild), which drives the twinBASIC IDE), the `tbdocs` orchestrator and its CLI flags, and the PDF render driver. If you are looking for the day-to-day workflow rather than a cheat sheet, the [Building and Deployment](Building) page is the gentler read; if you are modifying the build pipeline itself, the [tbdocs Internals](Builder) page goes one level deeper. * TOC goes here {:toc} @@ -979,7 +979,7 @@ It also writes a key naming the `Attributes.md` line each probe came from, besid ### census_attributes.mjs {: #census-attributes } - node builder/census_attributes.mjs [--ide ] [--src ] [--cache ] + node scripts/census_attributes.mjs [--ide ] [--src ] [--cache ] [--refresh] [--samples] [--attr ] [--json] [--out ] [--dump-sites ] [--quiet] diff --git a/docs/Reference/Attributes.md b/docs/Reference/Attributes.md index aa6b3e38..8ddd1b67 100644 --- a/docs/Reference/Attributes.md +++ b/docs/Reference/Attributes.md @@ -585,7 +585,7 @@ Hides the declaration from certain IntelliSense and other lists. It applies to a > [!NOTE] > A **CoClass** can only be hidden whole. Its body holds nothing but **Interface** lines, and the attribute is refused there with TB5155 --- unlike [**Default**](#default) and [**Source**](#source), which are interface-line attributes. It is likewise refused on an **Enum** or [**Type**](Type) declaration, on a **Type** member, and on a procedure parameter, though an individual **Enum** *member* does accept it. -