Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/workflows/checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,14 @@ jobs:
# real 908. No browser, no built tree.
- name: Verify the page-count drift guard (check_page_baseline.mjs)
run: node scripts/check_page_baseline.mjs
# The book-coverage warnings say nothing when every page has an entry
# in docs/_book.yml, which is also all a check that had stopped working
# would say. Before they existed, whole sections dropped out of the PDF
# without a word. These probes give each of the five findings a fault
# to report, on a manifest and pages built in memory. No browser, no
# built tree.
- name: Verify the book-coverage warnings (check_book_coverage.mjs)
run: node scripts/check_book_coverage.mjs
# Graphviz sizes each node box from a width table; the browser paints
# the label with a real font. Nothing in the build compares the two, so
# a mismatch ships as text hanging outside its box on a green build --
Expand Down
8 changes: 8 additions & 0 deletions .github/workflows/tbdocs-gh-pages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,14 @@ jobs:
# real 908. No browser, no built tree.
- name: Verify the page-count drift guard (check_page_baseline.mjs)
run: node scripts/check_page_baseline.mjs
# The book-coverage warnings say nothing when every page has an entry
# in docs/_book.yml, which is also all a check that had stopped working
# would say. Before they existed, whole sections dropped out of the PDF
# without a word. These probes give each of the five findings a fault
# to report, on a manifest and pages built in memory. No browser, no
# built tree.
- name: Verify the book-coverage warnings (check_book_coverage.mjs)
run: node scripts/check_book_coverage.mjs
# Graphviz sizes each node box from a width table; the browser paints the
# label with a real font, and nothing in the build compares the two -- so a
# mismatch ships as text hanging outside its box on a green build. See the
Expand Down
40 changes: 39 additions & 1 deletion WIP.Build.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ Two implementations of one check is exactly the shape that rots quietly: **a che
node scripts/check_links_diff.mjs --a script --b fused
```

It diffs the two implementations' findings category by category across the real invocations -- `_site/` with sitemap + search + canonical, `_site-offline/` with the forbidden-prefix rule, `book.html`, and a `--baseurl` tree checked with the matching base path. It is deliberately *not* in `check.bat`: the script side costs ~3 s, which is the whole saving.
It diffs the two implementations' findings category by category across the real invocations -- `_site/` with sitemap + search + canonical, `_site-offline/` with the forbidden-prefix rule, `book.html` with the same rule (there it collects the links that leave the book for the website, reported as `OUT OF BOOK`), and a `--baseurl` tree checked with the matching base path. It is deliberately *not* in `check.bat`: the script side costs ~3 s, which is the whole saving.

Two further modes matter:

Expand Down Expand Up @@ -523,6 +523,44 @@ fence marker, so a probe with no fence *after* the admonition passes against the
stasher it was written to catch. The damage is always to the prose **between** two
fences.

### The book-coverage warnings

**A page no `_book.yml` entry selected was left out of the PDF without a word**, and
by September 2026 that had taken 52 pages out of the book. Some were deliberate ---
the 404 page, Videos, Challenges --- and some were not: Data Types, Enumerations and
twinBASIC Additions are as plainly reference material as anything the book carries,
and nothing recorded why they were missing. The IDE section's pages with real prose
went the same way as its placeholders. The only trace was the book pass of the link
check, which listed the 32 links from the book to pages it did not carry as `BROKEN`,
on a pass marked informational --- so the list read as noise.

Two halves fixed it, and the second is what makes the first worth having. **`left_out:`
in `_book.yml` names every page that is out on purpose, with a `reason:`**, and
`bookCoverage()` in `builder/book.mjs` warns about a page that is in neither. Every
page has an entry one way or the other, so a warning is a decision nobody has made.
Without the list the warning fired for 37 pages on every build, which is a warning
nobody reads after the first week.

It reports five things, all empty on a consistent manifest: a page in no entry, a page
in the book and in `left_out:`, a book entry that selects no page, a `left_out:` entry
that matches none (a page renamed or deleted), and a landing or foreword URL no page
publishes at. **They are warnings, not failures**: the book is complete for the
manifest it was given, and a new page should not stop a build. They print under the
`pdf:` summary, and only when the book is built, so `--serve` does not repeat them on
every save. A link from the book to a page left out opens the website instead, and the
link check lists it as `OUT OF BOOK` --- which left-out pages the book still links to.

**"In the book" has to mirror `emitPart`, not the selectors.** A chaptered part's
`landing_page` and a `foreword_page` are emitted by URL rather than selected, so a
check that walked only `_chapters` would report the Features landing and the Packages
foreword on every build. The emission sites are listed once, in `bookCoverage()`, and
two probes pin them.

`scripts/check_book_coverage.mjs` is the gate on it, in `test.bat` and both CI
workflows: twelve probes over pages and a manifest built in memory, so it reads nothing
under `docs/`. Dropping the chaptered-landing site fails ten of the twelve; ignoring
`left_out:` fails nine.

### Build-time counts as named values

`{{tbdocs:pages}}` in a page renders as the number of pages the build
Expand Down
8 changes: 4 additions & 4 deletions WIP.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ The rest of this file is the maintenance guide for updating existing pages or ad
- `docs/Reference/Procedures and Functions.md` — alphabetical index of procedures/functions.
- `docs/LLVM/` — the LLVM section: compiling with the LLVM back end. A top-level section between Features and Reference Section in the nav, at `nav_order: 6` (Features moved to 5 to make room). `index.md` is the landing page and `Getting-Started.md` the only page so far, with its screenshots under `Images/`. Everything it describes arrived in **BETA 984**; the local BETA 983 compiler restricts `+llvm` to standard-module procedures and to Professional/Ultimate, and has no LLVM project settings at all. Both of the page's samples are marked `check_build` and compile clean on 983 --- but the harness asks only the front end, which accepts `[CompilerOptions]` with every CPU flag in it; nothing runs LLVM code generation, so a clean run says nothing about the 984 behaviour the page describes.

**A new top-level section needs four things besides its folder**, and no gate asks for the first three: a `nav_order` between its neighbours; a part in `docs/_book.yml`, since a page no part claims is left out of the PDF without a warning --- the IDE, Challenges and Videos sections are all absent from the book that way; a line in *Where content lives* in `docs/Documentation/Authoring.md`; and the page-count rise that `build.bat` writes to `builder/page-baseline.json`, committed with the pages. A section index lists its topics by hand and sets `has_toc: false`, or the template appends a second, automatic list of its children.
**A new top-level section needs four things besides its folder**, and nothing checks the first and the third: a `nav_order` between its neighbours; an entry in `docs/_book.yml` for every page, in a part or in `left_out:` with a reason, which the build warns about under its `pdf:` summary when one is missing; a line in *Where content lives* in `docs/Documentation/Authoring.md`; and the page-count rise that `build.bat` writes to `builder/page-baseline.json`, committed with the pages. The Challenges and Videos sections are in `left_out:`, and so are the IDE pages that are still screenshots and labels; the IDE part names its pages one by one, so a new IDE page warns until it goes into the part or into `left_out:`. A link from the book to a left-out page opens the website, and the pass over `book.html` lists it as `OUT OF BOOK`. A section index lists its topics by hand and sets `has_toc: false`, or the template appends a second, automatic list of its children.
- Footer rendering — [builder/template.mjs](builder/template.mjs)'s `renderFooterCustom()` renders the copyright line and, when `vba_attribution: true` is set in a page's frontmatter, an additional CC-BY-4.0 attribution line beneath it.
- Contributor authoring guide — [docs/Documentation/Authoring.md](docs/Documentation/Authoring.md) is the public "start here" page that distils this file's authoring conventions (page template, heading levels, formatting, plain-English prose, attribution policy, cross-section linking) for a new contributor. This file remains the exhaustive maintainer source of truth; keep the two in sync when a convention changes.

Expand Down Expand Up @@ -452,7 +452,7 @@ Why the report separates the wedged task from the merely blocked ones, and why
- `build.bat` — runs `node builder\tbdocs.mjs --src docs --check-audit-index` (which implies `--check`) and produces three trees in one pass: the online copy at `_site/`, a `file://`-browsable copy at `_site-offline/`, and the sparse pagedjs source at `_site-pdf/`. The offline pass adds ~700 ms and the PDF pass adds ~150 ms on top of the ~2 s online build. Toggle `also_build_offline` / `also_build_pdf` in `_config.yml` (or pass `--no-offline` / `--no-pdf`) to skip a sibling output. `--check` adds ~1.7 s and runs the link + integrity check over the HTML while it is still in worker memory; `build.bat --no-check` gets a plain build.
- `serve.bat` — runs `tbdocs --serve`: initial build, then a long-lived process with watcher, debounced rebuilds, and SSE-driven browser auto-reload. Writes to `docs/_serve/` (disjoint from `build.bat`'s `_site*/`) and skips the offline + PDF passes — so a one-off `build.bat` for the PDF or offline mirror doesn't disturb the live preview. Ctrl+C to stop.
- `check.bat` — the gates that read the built site: a freshness check that refuses a stale tree (`scripts/check_tree_fresh.mjs`), the DOT diagram fit check (`scripts/check_dot_fit.mjs`), the a11y sample-coverage check (`scripts/pick_a11y_sample.mjs --check`), then the accessibility check (`scripts/check_a11y.mjs`). The link + integrity check moved into `build.bat`. ~37 s.
- `test.bat` — the tests the *toolchain* has to pass: the publish-allowlist self-test (`scripts/check_publish_policy.mjs`), the gate-list check (`scripts/check_gate_lists.mjs`), the regex-safety gate (`scripts/check_regex_safety.mjs`), the code-region gate (`scripts/check_code_regions.mjs`), the page-count drift-guard probes (`scripts/check_page_baseline.mjs`), and the axe source-patch verification (`scripts/check_axe_patch_equiv.mjs`). ~8 s. See [What belongs in test.bat rather than check.bat](WIP.Build.md#what-belongs-in-testbat-rather-than-checkbat).
- `test.bat` — the tests the *toolchain* has to pass: the publish-allowlist self-test (`scripts/check_publish_policy.mjs`), the gate-list check (`scripts/check_gate_lists.mjs`), the regex-safety gate (`scripts/check_regex_safety.mjs`), the code-region gate (`scripts/check_code_regions.mjs`), the page-count drift-guard probes (`scripts/check_page_baseline.mjs`), the book-coverage probes (`scripts/check_book_coverage.mjs`), and the axe source-patch verification (`scripts/check_axe_patch_equiv.mjs`). ~8 s. See [What belongs in test.bat rather than check.bat](WIP.Build.md#what-belongs-in-testbat-rather-than-checkbat).
- `book.bat` — renders the PDF from `docs\_site-pdf\book.html` via `node book\render-book.mjs` into `docs\_pdf\twinBASIC Book.pdf`. Run `build.bat` first to populate `_site-pdf/`; `book.bat` refuses a tree older than its sources rather than rendering the previous book (see [The book refuses a stale source tree](WIP.Build.md#the-book-refuses-a-stale-source-tree)).

- `examples.bat` — compiles the documentation's own twinBASIC code samples, every `tb` fence marked `check_build`, and reports the ones the compiler refuses against the line in the page they came from. Needs a twinBASIC install and Windows, so it is outside every gate and outside CI; ~110 s over the 1,119 samples marked today. Two modes need no compiler at all: `--census` classifies every fence and says how many classifiable ones are still unmarked, and `--report <survey.json>` groups a saved `--propose --json` survey by diagnostic, section and unresolved name. `--propose` itself does compile. See [Compiling the reference's own code samples](#compiling-the-references-own-code-samples) and [WIP.ExamplesBuild.md](WIP.ExamplesBuild.md).
Expand All @@ -470,7 +470,7 @@ build.bat && check.bat

On the dev box that is ~4 s of build against ~37 s of check, of which the axe scan is ~20 s. [builder/PLAN-checks.md](builder/PLAN-checks.md) records how the link checker got folded into the build's task graph, what it cost and what it saved; the axe follow-ons are designed there but not implemented.

**If the change touched `builder/`, `scripts/`, `book/`, `eval/` or `wisdom/`, run `test.bat` as well** --- another ~8 s. Four of its six gates cannot be affected by a content edit at all. **Two can.** `check_gate_lists.mjs` is the easy one to predict: it reads `README.md` and every page under `docs/Documentation/`, so an edit to any developer page that states a gate count can fail it. **`check_code_regions.mjs` is the one worth understanding**, and which half of it a content edit reaches is worth keeping straight. Its corpus sweep has `ROOT = <repo>/docs` and tokenises all 906 markdown files, so a page that provokes a rewrite into *altering* a code region fails it --- that half is content-dependent. Its fixed probes are not: they run against their own sources whatever the tree holds, and they cover the **mirror** fault, where a rewrite silently stops firing. The sweep structurally cannot see that one, because text the rewrite skipped is stashed and restored unchanged and every region still matches. So run `test.bat` after adding an unusual code construct --- a fence whose contents include a fence marker, a 4-space indented block, an admonition wrapping a fence --- and read the built page as well, because for the mirror fault the gate is asserting that the stasher still works rather than checking your page:
**If the change touched `builder/`, `scripts/`, `book/`, `eval/` or `wisdom/`, run `test.bat` as well** --- another ~8 s. Five of its seven gates cannot be affected by a content edit at all. **Two can.** `check_gate_lists.mjs` is the easy one to predict: it reads `README.md` and every page under `docs/Documentation/`, so an edit to any developer page that states a gate count can fail it. **`check_code_regions.mjs` is the one worth understanding**, and which half of it a content edit reaches is worth keeping straight. Its corpus sweep has `ROOT = <repo>/docs` and tokenises all 906 markdown files, so a page that provokes a rewrite into *altering* a code region fails it --- that half is content-dependent. Its fixed probes are not: they run against their own sources whatever the tree holds, and they cover the **mirror** fault, where a rewrite silently stops firing. The sweep structurally cannot see that one, because text the rewrite skipped is stashed and restored unchanged and every region still matches. So run `test.bat` after adding an unusual code construct --- a fence whose contents include a fence marker, a 4-space indented block, an admonition wrapping a fence --- and read the built page as well, because for the mirror fault the gate is asserting that the stasher still works rather than checking your page:

```sh
build.bat && check.bat && test.bat
Expand All @@ -495,7 +495,7 @@ wrapper:
| `check.bat` | `pick_a11y_sample --check`, `check_a11y` | see [WIP.A11y.md](WIP.A11y.md) |
| `test.bat` | `check_code_regions` | no source or HTML rewrite altered a code region |
| `test.bat` | `check_regex_safety` | no regex in the tree can backtrack exponentially |
| `test.bat` | `check_publish_policy`, `check_gate_lists`, `check_page_baseline`, `check_axe_patch_equiv` | the gates on the gates |
| `test.bat` | `check_publish_policy`, `check_gate_lists`, `check_page_baseline`, `check_book_coverage`, `check_axe_patch_equiv` | the gates on the gates |

**A gate belongs in `test.bat` rather than `check.bat` if it would still mean
something with no documentation in the tree.** That is the whole rule; it is
Expand Down
Loading
Loading