` and `` atomically.
-**One gap is deliberate and stated rather than hidden:** `maskCodeRegions` does
-not protect **indented** (4-space) code blocks, because telling one from a
-list-item continuation needs block context a pre-render pass does not have, and
-guessing would change how real list content renders. `check_code_regions.mjs`
+**One gap is deliberate and stated rather than hidden:** the chain does not
+mask **indented** (4-space) code blocks, since it calls `maskCode` without
+`indented: true`. `check_code_regions.mjs`
*does* compare them, so a rewrite that damages one is reported --- and must be
fixed at the rewrite, not by widening the mask.
-`rewriteAdmonitions` deliberately runs **outside** the mask. A fence inside an
-admonition still carries its `> ` markers at that point, so the mask does not
-see it as a fence, and the admonition rewrite is what strips those markers.
+`rewriteAdmonitions` deliberately runs **outside** the mask. It finds an
+admonition's lines by their `>` markers and strips them, and a masked fence
+inside an admonition takes its markers with it into the stash. It asks
+`blockRegions`, with the same parser, which lines are code instead.
### Whitespace inside inline code is content
@@ -420,7 +422,7 @@ node scripts/check_code_regions.mjs --self-test
Two details are load-bearing. **It imports the chain rather than reconstructing
it**, so removing the mask from one rewrite changes what the gate runs and is
-caught --- a gate that exercised `maskCodeRegions` alone would have passed. And
+caught --- a gate that exercised `maskCode` alone would have passed. And
**its seven probes ride along in the normal run**, each a defect this repository
actually shipped, because the corpus is clean: a sweep that finds nothing is
otherwise indistinguishable from a gate that has stopped detecting. Verified by
@@ -473,7 +475,7 @@ comment in [builder/page-baseline.mjs](builder/page-baseline.mjs):
`test/fixtures/check-src`, three pages, to compare the two link checkers.
Against an unkeyed baseline that build reports **905 pages missing** --- a
loud, confident, entirely wrong finding, on the one harness whose whole job is
- noticing when two implementations disagree. `GUARDED_SRC` names the tree the
+ noticing when two front ends disagree. `GUARDED_SRC` names the tree the
numbers are of and every other root is skipped in silence.
- **The build now writes a tracked file, and `check_tree_fresh.mjs` watches
`builder/`.** The write happens after the tree, so without an exclusion the
@@ -521,20 +523,27 @@ comparison matched, the link check passed, and axe has no opinion about a
blockquote. It was found only because a new entry added to that page rendered
the same way and looked wrong.
-The stasher is a line scan now --- CommonMark closes a fence on a line that is
+The stasher became a line scan --- CommonMark closes a fence on a line that is
only the fence character, repeated at least as often as in the opener, which is
a rule about lines rather than something to express as one regex over a whole
-document. Measured across the site, the fix changes four files: `Attributes.html`,
+document. Measured across the site, that fix changed four files: `Attributes.html`,
`search-data.json` (which indexes it), and the two that record build timings.
The same stasher had a second way to fail: it recognised *backtick* fences only,
-reasoning that `maskCodeRegions` knows about tildes --- but `rewriteAdmonitions` runs
-**outside** the mask by design, so nothing protected a tilde fence. `FENCE_OPEN_RE`
-accepts either character now and closes on the one that opened. `docs/` contains no
-tilde fence, which is why the corpus sweep could never have found it --- the same
-blind spot that makes the ADMONITION_PROBES necessary.
-
-Five probes in `check_code_regions.mjs` assert this direction, and **writing one
+reasoning that `maskCodeRegions` knew about tildes --- but `rewriteAdmonitions` runs
+**outside** the mask by design, so nothing protected a tilde fence. Its opener test
+was made to accept either character and close on the one that opened. `docs/`
+contains no tilde fence, which is why the corpus sweep could never have found it ---
+the same blind spot that makes the ADMONITION_PROBES necessary.
+
+**The stasher is gone.** A scan of its own could still disagree with the parser
+that renders the page: it never saw a fence opened after a definition list's `: `,
+so an admonition written inside one as a sample became a live one. `rewriteAdmonitions`
+asks `blockRegions` from `lib/markdown.mjs`, with the site's parser, and leaves an
+admonition alone when its `[!TYPE]` line is in a region. A fence *inside* an
+admonition is a region too, and the rewrite still strips its `>` markers.
+
+Six probes in `check_code_regions.mjs` assert this direction, and **writing one
correctly is not obvious**: a mis-paired opener swallows text only as far as the next
fence marker, so a probe with no fence *after* the admonition passes against the very
stasher it was written to catch. The damage is always to the prose **between** two
@@ -734,8 +743,11 @@ Two judgement calls worth keeping:
*"found `test.bat` documented as three gates when it had four"* as a false
claim, so the verb list is explicit.
-Twelve of its eighteen probes cover the sweep, seven positive and five
-negative, each taken from the real corpus. The verification that means anything
+Thirteen of its nineteen probes cover the sweep, eight positive and five
+negative. Twelve are taken from the real corpus; the thirteenth puts a fenced
+`## ` line, the shape of Wisdom.md's `staging.md` example, inside a wrapper's
+section, since sections are split through `lib/markdown.mjs`'s `splitOnMarker`
+and a heading-shaped line in a region starts none. The verification that means anything
is reverting the offending pages to the commit that shipped them and confirming
the gate names every site.
diff --git a/WIP.ExamplesBuild.md b/WIP.ExamplesBuild.md
index 50ec527c..f512da23 100644
--- a/WIP.ExamplesBuild.md
+++ b/WIP.ExamplesBuild.md
@@ -117,7 +117,7 @@ that ride along on every run:
| property | result |
|---|---|
| a marked fence renders byte-identical HTML to a plain one | **true** |
-| `maskCodeRegions` still hides the body | **true** |
+| `maskCode` still hides the body | **true** |
| the mask round-trips the marked fence | **true** |
| `applyPreRenderRewrites` leaves it byte-identical | **true** |
@@ -129,7 +129,8 @@ Shape --- bare flags and `key=value` pairs after the language token:
```tb check_build slot=module id=getobject-1
**No backticks in it.** CommonMark forbids them in a backtick fence's info string, and
-`maskCodeRegions` skips such a fence outright.
+markdown-it reads such a line as prose rather than as a fence, so the sample would not be
+a fence at all.
| token | meaning | default |
|---|---|---|
diff --git a/WIP.HelpAddin.md b/WIP.HelpAddin.md
index c2e78fdc..4d1e63f5 100644
--- a/WIP.HelpAddin.md
+++ b/WIP.HelpAddin.md
@@ -391,7 +391,8 @@ have: writing into the install, which every new build replaces. Still deferred.
says `*class* **Collection** ... in package VBA` and lists `*[default]* VBA._Collection`.
A variable gives its declaration, `*local variable* Dim c As Collection`. `Debug`,
`Debug.Print` and a statement such as `Dim` give nothing, and a type such as `Long` a line
- about it. Over a procedure's name in its own declaration, hover gives a debug block
+ about it. BETA 983 gives no hover for those three, and BETA 987 a hover whose text is
+ empty, so the add-in must treat both as nothing. Over a procedure's name in its own declaration, hover gives a debug block
instead, `TB-DEBUG CODEGEN SIZE: [NOT-READY]`; over a `ByVal` parameter of a class,
`String`, `Variant` or `Object` it adds a wrong note about `Option Explicit`
([BUGS-TO-REPORT.md](BUGS-TO-REPORT.md)).
diff --git a/WIP.Wisdom.md b/WIP.Wisdom.md
index 9ab68b64..1b3fd8a6 100644
--- a/WIP.Wisdom.md
+++ b/WIP.Wisdom.md
@@ -3,8 +3,9 @@
The Discord knowledge-harvesting tool: a three-phase pipeline (`export` →
`process` → `extract`) that mines the twinBASIC Discord for things the
documentation does not yet say, and drafts additions for human review. Plans in
-`wisdom/PLAN-{1,2,3}.md`; implementation under `wisdom/`. Uses only Node.js
-built-in APIs.
+`wisdom/PLAN-{1,2,3}.md`; implementation under `wisdom/`. Uses Node.js built-in
+APIs, and the repository's npm packages through `lib/markdown.mjs`, which
+`extract/merger.mjs` reads `staging.md` with.
Split out of [WIP.md](WIP.md) because Wisdom runs occasionally and its
invocation detail has no business in the context of a session doing anything
diff --git a/WIP.md b/WIP.md
index 800d8275..611e1a11 100644
--- a/WIP.md
+++ b/WIP.md
@@ -87,11 +87,10 @@ The rest of this file is the maintenance guide for updating existing pages or ad
> to drop `_Images`, swallowed all 37 pages under `_App/` --- the twinBASIC interface
> really is named `_App`, after the COM hidden-interface convention. It is scoped to
> `**/_Images/**` (plus `**/*.af`) now, so **never widen an exclude to a bare
- > underscore prefix.** Separately `index.md` carried a UTF-8 BOM, which sits in front
- > of the `---` and stops `gray-matter` recognising any frontmatter at all, so
- > `discover` filed it as a *static file* and served the raw markdown verbatim;
- > `discover.mjs` strips a leading BOM before parsing now, and editors on Windows add
- > one without being asked.
+ > underscore prefix.** Separately `index.md` carried a UTF-8 BOM in front of the `---`,
+ > which hid the frontmatter, so `discover` filed it as a *static file* and served the raw
+ > markdown verbatim; `lib/frontmatter.mjs` strips a leading BOM before parsing now, and
+ > editors on Windows add one without being asked.
- `docs/Reference/Built-In/tbIDE/` — IDE Extensibility package (this is the **addin SDK**). The package is type-only — it ships **public interfaces + CoClasses** that an addin DLL binds to; every implementation behind them lives in the twinBASIC IDE itself. The user-facing surface is one entry-point factory (`tbCreateCompilerAddin`) plus 23 CoClasses grouped by role: the addin contract (`AddIn`), the root API (`Host`), the loaded `Project`, the editors collection (`Editor` / `CodeEditor` / `Editors`), the virtual file system (`FileSystem` / `FileSystemItem` / `Folder` / `File`), the in-IDE UI surface (`Toolbar` / `Toolbars` / `Button` / `ToolWindow` / `ToolWindows`), the HTML DOM inside a tool window (`HtmlElement` / `HtmlElements` / `HtmlElementProperty` / `HtmlElementProperties` / `HtmlEventProperty` / `HtmlEventProperties`), the `DebugConsole`, `KeyboardShortcuts`, `Themes`, and the single concrete user-instantiable helper class `AddinTimer`. Flat layout — one page per CoClass / Class plus the index landing.
- `docs/Reference/Statements.md` — alphabetical index of language statements.
- `docs/Reference/Procedures and Functions.md` — alphabetical index of procedures/functions.
@@ -452,7 +451,7 @@ Why the report separates the wedged task from the merely blocked ones, and why
- `build.bat` — runs `node builder\tbdocs.mjs --src docs --check-audit-index` (which implies `--check`) and produces three trees in one pass: the online copy at `_site/`, a `file://`-browsable copy at `_site-offline/`, and the sparse pagedjs source at `_site-pdf/`. The offline pass adds ~700 ms and the PDF pass adds ~150 ms on top of the ~2 s online build. Toggle `also_build_offline` / `also_build_pdf` in `_config.yml` (or pass `--no-offline` / `--no-pdf`) to skip a sibling output. `--check` adds ~1.7 s and runs the link + integrity check over the HTML while it is still in worker memory; `build.bat --no-check` gets a plain build.
- `serve.bat` — runs `tbdocs --serve`: initial build, then a long-lived process with watcher, debounced rebuilds, and SSE-driven browser auto-reload. Writes to `docs/_serve/` (disjoint from `build.bat`'s `_site*/`) and skips the offline + PDF passes — so a one-off `build.bat` for the PDF or offline mirror doesn't disturb the live preview. Ctrl+C to stop.
- `check.bat` — the gates that read the built site: a freshness check that refuses a stale tree (`scripts/check_tree_fresh.mjs`), the DOT diagram fit check (`scripts/check_dot_fit.mjs`), the a11y sample-coverage check (`scripts/pick_a11y_sample.mjs --check`), then the accessibility check (`scripts/check_a11y.mjs`). The link + integrity check moved into `build.bat`. ~37 s.
-- `test.bat` — the tests the *toolchain* has to pass: the publish-allowlist self-test (`scripts/check_publish_policy.mjs`), the gate-list check (`scripts/check_gate_lists.mjs`), the CI-workflow roster check (`scripts/check_ci_workflows.mjs`), the lint gate (`scripts/check_lint.mjs`), the regex-safety gate (`scripts/check_regex_safety.mjs`), the code-region gate (`scripts/check_code_regions.mjs`), the page-count drift-guard probes (`scripts/check_page_baseline.mjs`), the book-coverage probes (`scripts/check_book_coverage.mjs`), the symbol-index probes (`scripts/check_symbol_index.mjs`), and the axe source-patch verification (`scripts/check_axe_patch_equiv.mjs`). ~8 s. See [What belongs in test.bat rather than check.bat](WIP.Build.md#what-belongs-in-testbat-rather-than-checkbat).
+- `test.bat` — the tests the *toolchain* has to pass: the publish-allowlist self-test (`scripts/check_publish_policy.mjs`), the gate-list check (`scripts/check_gate_lists.mjs`), the CI-workflow roster check (`scripts/check_ci_workflows.mjs`), the lint gate (`scripts/check_lint.mjs`), the regex-safety gate (`scripts/check_regex_safety.mjs`), the code-region gate (`scripts/check_code_regions.mjs`), the page-count drift-guard probes (`scripts/check_page_baseline.mjs`), the book-coverage probes (`scripts/check_book_coverage.mjs`), the symbol-index probes (`scripts/check_symbol_index.mjs`), the command-line probes and cases (`scripts/check_cli.mjs`), and the axe source-patch verification (`scripts/check_axe_patch_equiv.mjs`). ~9 s. See [What belongs in test.bat rather than check.bat](WIP.Build.md#what-belongs-in-testbat-rather-than-checkbat).
- `book.bat` — renders the PDF from `docs\_site-pdf\book.html` via `node book\render-book.mjs` into `docs\_pdf\twinBASIC Book.pdf`. Run `build.bat` first to populate `_site-pdf/`; `book.bat` refuses a tree older than its sources rather than rendering the previous book (see [The book refuses a stale source tree](WIP.Build.md#the-book-refuses-a-stale-source-tree)).
- `examples.bat` — compiles the documentation's own twinBASIC code samples, every `tb` fence marked `check_build`, and reports the ones the compiler refuses against the line in the page they came from. Needs a twinBASIC install and Windows, so it is outside every gate and outside CI; ~120 s over the 1,129 samples marked as of 2026-09-25. Two modes need no compiler at all: `--census` classifies every fence and says how many classifiable ones are still unmarked, and `--report ` groups a saved `--propose --json` survey by diagnostic, section and unresolved name. `--propose` itself does compile. See [Compiling the reference's own code samples](#compiling-the-references-own-code-samples) and [WIP.ExamplesBuild.md](WIP.ExamplesBuild.md).
@@ -470,7 +469,7 @@ build.bat && check.bat
On the dev box that is ~4 s of build against ~37 s of check, of which the axe scan is ~20 s. [builder/PLAN-checks.md](builder/PLAN-checks.md) records how the link checker got folded into the build's task graph, what it cost and what it saved; the axe follow-ons are designed there but not implemented.
-**If the change touched `builder/`, `scripts/`, `lib/`, `book/`, `eval/`, `wisdom/`, `test/`, the site's scripts in `docs/assets/js/`, a wrapper or a workflow, run `test.bat` as well** --- another ~8 s. Seven of its ten gates cannot be affected by an edit under `docs/` at all. **Three can.** `check_lint.mjs` lints the site's two scripts in `docs/assets/js/` along with the tooling. `check_gate_lists.mjs` is the easy one to predict: it reads `README.md` and every page under `docs/Documentation/`, so an edit to any developer page that states a gate count can fail it. **`check_code_regions.mjs` is the one worth understanding**, and which half of it a content edit reaches is worth keeping straight. Its corpus sweep has `ROOT = /docs` and tokenises all 906 markdown files, so a page that provokes a rewrite into *altering* a code region fails it --- that half is content-dependent. Its fixed probes are not: they run against their own sources whatever the tree holds, and they cover the **mirror** fault, where a rewrite silently stops firing. The sweep structurally cannot see that one, because text the rewrite skipped is stashed and restored unchanged and every region still matches. So run `test.bat` after adding an unusual code construct --- a fence whose contents include a fence marker, a 4-space indented block, an admonition wrapping a fence --- and read the built page as well, because for the mirror fault the gate is asserting that the stasher still works rather than checking your page:
+**If the change touched `builder/`, `scripts/`, `lib/`, `book/`, `eval/`, `wisdom/`, `test/`, the site's scripts in `docs/assets/js/`, a wrapper or a workflow, run `test.bat` as well** --- another ~9 s. Eight of its eleven gates cannot be affected by an edit under `docs/` at all. **Three can.** `check_lint.mjs` lints the site's two scripts in `docs/assets/js/` along with the tooling. `check_gate_lists.mjs` is the easy one to predict: it reads `README.md` and every page under `docs/Documentation/`, so an edit to any developer page that states a gate count can fail it. **`check_code_regions.mjs` is the one worth understanding**, and which half of it a content edit reaches is worth keeping straight. Its corpus sweep reads `DOCS_DIR`, which is `/docs`, and tokenises all 906 markdown files, so a page that provokes a rewrite into *altering* a code region fails it --- that half is content-dependent. Its fixed probes are not: they run against their own sources whatever the tree holds, and they cover the **mirror** fault, where a rewrite silently stops firing. The sweep structurally cannot see that one, because text the rewrite skipped is stashed and restored unchanged and every region still matches. So run `test.bat` after adding an unusual code construct --- a fence whose contents include a fence marker, a 4-space indented block, an admonition wrapping a fence --- and read the built page as well, because for the mirror fault the gate is asserting that the stasher still works rather than checking your page:
```sh
build.bat && check.bat && test.bat
@@ -501,9 +500,10 @@ wrapper:
| `check.bat` | `check_tree_fresh` | the tree is not older than the sources that produced it |
| `check.bat` | `check_dot_fit` | every diagram label sits inside the box Graphviz drew for it |
| `check.bat` | `pick_a11y_sample --check`, `check_a11y` | see [WIP.A11y.md](WIP.A11y.md) |
-| `test.bat` | `check_code_regions` | no source or HTML rewrite altered a code region |
+| `test.bat` | `check_code_regions` | no source or HTML rewrite altered a code region; `lib/markdown.mjs` and `lib/frontmatter.mjs` pass their probes, and the block parse finds what the full parse finds; the count-name check skips code and names the file's line; `discover` warns about an unquoted frontmatter value that ends in `#`; the dash normaliser converts only prose |
| `test.bat` | `check_regex_safety` | no regex in the tree can backtrack exponentially |
| `test.bat` | `check_symbol_index` | the symbol index still places each kind of symbol, from fixtures |
+| `test.bat` | `check_cli` | `lib/cli.mjs` parses as a strict `parseArgs` does, and keeps a lenient tool's leniency where asked; each tool's recorded command-line errors still exit and print as recorded, run with an IDE and a browser that do not exist |
| `test.bat` | `check_ci_workflows` | both CI workflows run every wrapper gate, with the same arguments and order, and build with `build.bat`'s flags |
| `test.bat` | `check_lint` | Biome finds nothing in the tooling, warnings included, and checked at least one script |
| `test.bat` | `check_publish_policy`, `check_gate_lists`, `check_page_baseline`, `check_book_coverage`, `check_axe_patch_equiv` | the gates on the gates |
@@ -565,12 +565,18 @@ they are.
through `<<'EOF'` had every `"\\s+"` delivered as `"\s+"`, matched nothing, and reported
**444 unclassifiable fences against a true 32** --- a number that reads as a finding
about the corpus and was a finding about the quoting.
+
+ **The file tools decode a `\u` escape of four hex digits** into the character it
+ names: a BOM, an em-dash and a replacement character all arrived raw, and only NUL was
+ left as written. A doubled backslash arrives doubled, which is a different string.
+ Write the brace form, `\u{FEFF}`, which arrives intact (a regex needs the `u` flag
+ for it), and check a file you have written for raw non-ASCII.
- Don't push or force-push without explicit user request.
- Don't leave a remote image URL in a finished page. A pasted `https://github.com/user-attachments/assets/...` link is fine to write --- [builder/vendor-assets.mjs](builder/vendor-assets.mjs) downloads it to `docs/assets/attachments/gh-.` on the next local build and rewrites the render to point there; commit the downloaded file with the edit. Any other remote host has no such handling: download it yourself and commit it under the section's `Images/` folder. Remote images cost a network round trip per page view, break the `file://` offline mirror, and **abort the PDF book render** -- the forked paged.js in `book/lib/` dropped async image loading, so an image still in flight when the page-breaking pass runs raises instead of degrading. The build enforces this unconditionally (see [Site integrity check](#site-integrity-check)); `--check-remote-assets` is the standalone checker's flag, not a `tbdocs` one. The check is scoped to `
`; `