diff --git a/BUGS-TO-REPORT.md b/BUGS-TO-REPORT.md index 28cb029f..2690397c 100644 --- a/BUGS-TO-REPORT.md +++ b/BUGS-TO-REPORT.md @@ -347,7 +347,9 @@ the tree. Five of the 48 project and package files the IDE ships have one --- `WinNativeCommonCtls` (which embeds `VBComDlg`), samples 8, 17 and 23, and the *Standard EXE (plus VBCCR v1.8)* project template --- and each was measured: `export` succeeds, and `import` of the tree it has just written stops as above. None of them round-trips through -the command line, and neither does any project created from that template. +the command line, and neither does any project created from that template. Nor does any +export written by the IDE's **Export Project**, which always adds the compiler packages under +`Packages` (see *Export Project writes the compiler packages*, below). **Found by** checking `scripts/impexp.mjs` against the compiler's `import` for line-ending handling: a probe tree with a made-up `Packages\Nested\` folder never produced a project to @@ -666,3 +668,312 @@ variable on the right both reproduce it. the other way round: `"2" \ b` is the `Long` -2. **Found by** the result-type probe for `Reference/Operators.md`. + +--- + +## Export Project follows a directory junction in its folder and deletes what it points to + +**Build:** BETA 983 +**Severity:** data loss outside the folder the user chose. Export Project empties its folder +before writing, as the *Export Path* setting warns; it does not stop at a junction. + +1. In the export folder, make a junction to another folder that holds a file: + `mklink /J \linked `, with `\precious.txt`. +2. Run **File → Export Project** into ``, with *Export Verbose* on. +3. The Debug Console shows: + ``` + [EXPORT] DELETED: \\?\\linked\precious.txt + [EXPORT] DELETED: \\?\\linked + ``` + and `` is empty afterwards. + +**What does not reproduce it:** the command-line `export` verb, which deletes nothing. + +**Found by** the Export Project probe for round 8's UC-55, which drove the IDE's own +`exportProjectTo()` over DevTools on a scratch folder. + +--- + +## Export Project stops at a read-only file after deleting everything before it, and the IDE reports nothing + +**Build:** BETA 983 +**Severity:** a partly emptied folder, with the only record in the Debug Console. On a Git +working copy it breaks the repository, because Git makes its object files read-only. + +1. Put a read-only file in the export folder among other files. +2. Run **File → Export Project** into it. +3. The Debug Console shows: + ``` + [EXPORT] DELETE FAILED: \\?\\readonly.txt + [EXPORT] ERROR: unable to clean the output folder + [EXPORT] export failed. + ``` + The files and folders that sort before the read-only one are already deleted, nothing is + exported, and no dialog appears: the compiler's response to the IDE is code 0. + +On a `git init` working copy with a commit, it deletes `.git\config`, `HEAD`, `index`, `hooks` +and `info`, then stops at the first object file. `git status` there reports +`fatal: not a git repository`. + +**What does not reproduce it:** a folder with no read-only file, which is emptied and exported +completely --- `.git` included, with no prompt. + +**Found by** the same probe. + +--- + +## Export Path refuses `${SourcePath}` alone, but not the same folder written as a path + +**Build:** BETA 983 +**Severity:** the project file is deleted when the export folder is the folder that holds it. + +The Settings editor's check on `project.exportPath` in `ide/main.js` compares the text with +`${sourcepath}` and `${sourcepath}\`, with the message "This would DELETE the project file, as +the `Export Project` command empties the output folder before exporting". It does not resolve +the path. The compiler applies no check of its own: an export into the project's own folder +logged `[EXPORT] DELETED: \\?\\.twinproj` and completed. A **Save** afterwards +wrote the file back; closing without saving loses it. + +**Found by** the same probe. The compiler's side was measured, by calling `exportProjectTo()` +with the folder; that the editor accepts the same folder typed as a path is read from the +check's code, not tried. + +--- + +## Export Project writes the compiler packages, which the project does not hold, and the command line cannot pack the result + +**Build:** BETA 983 +**Severity:** the IDE's export of a project cannot be packed back into a project by the +supported tool, so it cannot serve for version control; and a two-file project exports as +477 files. + +**File → Export Project** writes a `Packages` folder holding the full source of the compiler +packages the project uses: `VB`, `VBA`, `VBRUN` and `AppGlobalClassProject` for a project +with the default references --- 475 of the 477 files an export of a two-file project wrote. +The project file does not hold them. A `.twinproj` the IDE saved holds only the packages the +project embeds, and `twinBASIC_win32.exe export` of it writes only those: for a project +embedding WinDevLib, `Packages\WinDevLib` and no other package. + +Then, on that 477-file export: + +- `twinBASIC_win32.exe import x.twinproj \` stops with exit code 999 and writes + nothing (the `import` entry above), as it does for any folder under `Packages`; +- the standalone script packs it, into a 4,220,723-byte project, against 2,055 bytes for the + same export with `Packages` removed. The project now embeds its own copy of the four + compiler packages. It compiles with no errors; which copy the IDE then uses was not + measured. + +**What does not reproduce it:** the command line's own `export`, which writes what the +project file holds. + +**Found by** checking round 9's UC-62 answer, which sets up *Export After Save* into a Git +repository and rebuilds the project from a fresh clone with the tB executable. The export was +round 8's, written by the IDE's `exportProjectTo()` over DevTools. + +--- + +## An out-of-range index raises `&H8002000B` or `&H80004005`, not VBA's error 9 + +**Build:** BETA 983 --- the IDE and a compiled EXE alike +**Severity:** VBA code that handles `Err.Number = 9` does not recognise the error, with no +diagnostic. + +``` +Dim a(5) As Long +On Error Resume Next +a(7) = 1 +Debug.Print Err.Number, Hex$(Err.Number), Err.Description +``` + +prints `-2147352565 8002000B Invalid index.`. Every case measured, reading `Err.Number` in +the program: + +| access | twinBASIC | VBA, per VBA-Docs' *Subscript out of range (Error 9)* | +|---|---|---| +| past a fixed or dynamic array's bound, a `Variant` array's, or `Split("x y")(5)` | -2147352565 (`8002000B`) *Invalid index.* | 9 | +| an element of an array never dimensioned: `Dim u() As Integer: u(8) = 234`, VBA-Docs' own example | -2147467259 (`80004005`) *Unspecified error* | 9 | +| a `Collection` member by a missing index or key | -2147467259 *Unspecified error* | 9 for a missing member | +| `Forms(99)`, `Forms.Item(-1)` | -2147467259 *Unspecified error* | --- | + +**What does not reproduce it:** `UBound` of an erased array, `Printers(99)` and `Err.Raise 9` +all give 9, and division by zero gives 11. The IDE's run-time error panel shows the same number +`Err.Number` holds, for the array case. An erased array behaves as one never dimensioned: +`-2147467259` for an element, 9 from `LBound` and `UBound`. `Printers` raises 9 past its end but +`-2147467259` for a negative index and for an unknown name. The description of `-2147467259` +varies between runs --- *Unspecified error* in one, *Automation error* in another. + +--- + +## Reading `Forms` by index returns a broken reference, and the process then crashes + +**Build:** BETA 983 --- the IDE and a compiled EXE alike +**Severity:** crash (`0xC0000005`), from a form of access the documentation shows. + +With one form loaded (`Load Form1`): + +``` +Dim s As String +s = Forms(0).Name ' s is "", and the process later dies with 0xC0000005 +``` + +`Set f = Forms(0)` followed by `f.Name` does the same, and so does `s = Forms(n).Name` with +`n` a variable. Inside `For k = 0 To Forms.Count - 1`, `Set f = Forms(k)` corrupts the loop +variable: `k` read 0, 0, 0, then 8195702. + +**What does not reproduce it:** `n = 0: Set f = Forms(n)` outside a loop returns the form +(`f.Name` is `Form1`) and the program exits 0; `For Each f In Forms` and `Unload Forms(i)` work; +`Printers(0)` with a literal index works. + +**Found by** the fix pass for round 8's error-number findings: an EXE that logs a line before +each statement, run once per case with crash dialogs suppressed. The crash itself was +reproduced by the orchestrator; the loop-variable corruption was measured by the fix agent only. + +**Found by** the IDE debugging probe for round 8's UC-61, then a probe of its own run in the IDE +and as the built EXE, with identical results. + +--- + +## A step key pressed on the line that raised an error leaves a step pending + +**Build:** BETA 983 +**Severity:** the debugger stops where it was not asked to, and one command no longer means one +thing. + +1. Run a procedure that raises an untrapped error inside a loop, and let the error panel open. +2. Press F8 (or F10, F11, SHIFT+F8) on the failing line. The line re-runs, the error recurs, + and the mark does not move --- as expected. +3. Now choose **Ignore (Resume Next)**. It stops at the next line instead of running on. + Moving past the line with **Set Next Statement** (CTRL+F9) instead, each F5 then advances + one line. + +Seen in three runs. In the one followed to the end, it lasted until the procedure returned; in +another, a fix-then-F5 stopped once. Which of the two applies was not isolated. + +**What does not reproduce it:** choosing **Ignore** without pressing a step key first, which +runs on from the next line as the panel says. + +**Found by** the IDE debugging probe for round 8's UC-61, driving real keys over DevTools. + +--- + +## Stop at a run-time error ends only the procedure that raised it + +**Build:** BETA 983 +**Severity:** the program goes on running after the user asked it to stop. + +A `Sub Main` that calls a procedure which raises an untrapped error, and prints a line after +the call. At the error panel, choose **Stop** --- the panel's button or the toolbar's. The +failing procedure ends, and `Main`'s following `Debug.Print` still runs. Three runs, the same +each time. + +**What does not reproduce it:** **Stop** at an ordinary break (a breakpoint or a step), which +prints `aborted` and ends the whole run. + +**Its worst consequence is a false pass.** At a failed `Assert` --- whose error is raised by the +assertion's own procedure --- **Stop**, and **Run → End** too, end only that procedure: the test +carries on past the failed check, and a runner in the shape `Testing-with-Assert.md` teaches then +prints `All PadLeft tests passed.` Three trials, one per button (**Ignore (Resume Next)** does the +same, as it should). Moving execution to the test's `End Sub` with **Set Next Statement** and then +choosing **Run → End** makes it an ordinary break, and the run is aborted (two trials). + +**Found by** the same probe; the assertion case by the fix pass for the Assert tutorial. + +--- + +## A `Static` declaration cannot initialise with a constructor that takes arguments + +**Build:** BETA 983 +**Severity:** a valid declaration does not compile; the workaround is a `Static` without an +initialiser and a `Set` on first use. + +``` +Private Class Dog + Private m_Name As String + Public Sub New(ByVal Name As String) + m_Name = Name + End Sub +End Class + +' in a procedure: +Static s As Dog = New Dog("Rex") +``` + +fails with TB5074, *Could not bind to parameterized constructor of class 'Dog'. No compatible +Sub New() method found*, at the `New`. + +**What does not reproduce it:** the same initialiser on `Dim` (`Dim d As Dog = New Dog("Rex")`), +and on a module-level `Private` or `Public`; a `Static` initialised with a constructor that takes +no arguments (`Static c As Collection = New Collection`); and a `Static` of a value type +(`Static n As Long = 5`). All compile and run. + +**Found by** the fix pass for round 8's UC-59, measuring the forms `New.md` documents. + +--- + +## *Import from file...* leaves the imported package unticked + +**Build:** BETA 983 +**Severity:** the package is imported but not referenced, and the documentation says it is. + +Settings → References → Available Packages → *Import from file...*, and choose a `.twinpack`. The +compiler answers the IDE's `importPackage` request with +`success: true, body: { packageSymbol: "DocProbePkg" }`, and the package appears in the list +unticked, so nothing in the project can use it until it is ticked by hand. +`packageLoadFromFile` in `ide/main.js` reads `packageSymbol` from the response itself rather +than from its `body`, which is consistent with what is seen; that part is read, not traced. + +**Found by** the package probe for round 8's UC-60, which drove the import over DevTools with +the file's path in place of the native picker. + +--- + +## Replacing an embedded package under one Apply keeps running the old copy + +**Build:** BETA 983 +**Severity:** the project builds and runs the old package after the user has replaced it. + +1. A project embeds a package built locally, `DocProbePkg` v1. +2. In Settings → References, untick it; *Import from file...* its v2; tick v2; apply once. +3. The console shows only `[COMPILER] Project settings updated` --- no restart and no save. Builds + keep running v1. Save All and then a compiler restart give v2; a restart *without* saving + brings v1 back, under a reference numbered 1.1.0.0. + +Two runs of two, on a machine with no linked copy of the package. + +**What does not reproduce it:** the same steps with a linked copy of the package present in +`%APPDATA%\twinBASIC\packages` (six runs of six), and an apply after the untick followed by +another after the import and tick (every run): each of those restarts the compiler and saves, +and v2 runs at once. + +**Found by** the same probe. + +--- + +## An error in the body of a generic procedure names neither the type nor the call that caused it + +**Build:** BETA 983 +**Severity:** a diagnostic that points at correct code. In a project with many calls to a +generic procedure, nothing says which call to fix. + +``` +Module GenMax +Public Function Max(Of T)(a As T, b As T) As T + If a > b Then + Return a + Else + Return b + End If +End Function +End Module +``` + +With one call, `Set m = Max(Of Collection)(c1, c2)`, the project fails to compile with +`TB5092 Missing argument 'Index'`, reported twice, both times at `[3,14]` of the module that +holds `Max` --- the line with `>`. The message comes from `Collection`'s default member, +`Item`. Neither error names `Collection`, and neither names the line of the call. + +**What does not reproduce it:** calls with `Long`, `Double` and `String`, deduced or given +with `(Of ...)`, which compile and return the larger value. + +**Found by** probing round 9's UC-65 answer, whose `Max` uses `>` on a type parameter with +nothing to say which types it accepts. diff --git a/WIP.Harness.md b/WIP.Harness.md index fe5a0758..32e16adc 100644 --- a/WIP.Harness.md +++ b/WIP.Harness.md @@ -363,7 +363,7 @@ above with the one symptom that detects it removed. `tbrun` pins the path in its copy, which is why it insists on a source tree it can edit rather than a packed project it cannot. -Three smaller things it knows, each of which cost a run: +Four smaller things it knows, each of which cost a run: - **`element.click()` on `#buildIcon` does nothing.** It is a plain DIV behind the IDE's own pointer handling and needs real `Input.dispatchMouseEvent` presses at its centre. @@ -382,6 +382,18 @@ Three smaller things it knows, each of which cost a run: and the linker writes there *after* the build, so without a clear you capture your output interleaved with `[LINKER]` lines. The script warns rather than guessing which lines are yours. +- **A failed build is not output.** A build that fails after a clean compile never runs the + probe, and the IDE's own log stays in the console: `[BUILD] Starting...`, + `[TYPELIB] failed to finalize typelibrary. Disk error?`, `[LINKER] FAILED to create type + library`, `[BUILD] failed`. `tbrun` returned exactly that as the probe's output, with exit 0, + twice in round 8's fix pass --- five runs going at once on ports 9740--9744, and both passed + when repeated. It now exits 2 on a `[BUILD] failed` or `[LINKER] FAILED` line, which the + probe's own `Debug.Cls` would have erased. What made the type library fail was not isolated. + +A reader of the console that is not `tbrun` should **compare the whole console before and +after, not read on from an index**: new text can be appended to an entry that is still open. +Round 8's export probe missed the first `[EXPORT] exporting...` line of every session that +way. `tbrun` re-reads the whole backing array on every poll, which is why it never did. It settles on a quiet period rather than a sentinel, so no probe has to print a marker the script knows about. Distinct `--port` values let probes run concurrently, exactly as diff --git a/builder/REVIEW-USECASES-5b4cd37.md b/builder/REVIEW-USECASES-5b4cd37.md new file mode 100644 index 00000000..32641068 --- /dev/null +++ b/builder/REVIEW-USECASES-5b4cd37.md @@ -0,0 +1,588 @@ +# Use-case review, round 8 --- isolated evaluators, an export that empties a Git repository, and constructors that fail in silence, at `5b4cd37` + +Branch `staging` · reviewed 2026-09-24 · 13 cases + +The eighth round of [the harness in `eval/`](../eval/README.md), and the first run with +evaluators that cannot see `WIP.md`. Round 7's four named re-runs (UC-55, UC-56, UC-57, +UC-50) and two more of its cases (UC-54, UC-58); round 1's four lowest-scoring cases, never +re-measured since (UC-06, UC-14, UC-15, UC-16); and three new site cases (UC-59, UC-60, +UC-61). The corpus was built at `5de91d0`, which rebasing `staging` onto the IDE help add-in +branch turned into `5b4cd37`; the round ran in two sessions, and the Round 8 section of +[eval/usecases.md](../eval/usecases.md) records both. + +## Verdict + +**No isolated evaluator walked into a hazard it was set, and every re-run improved on at least +one axis --- but the round's most serious findings are behaviours of the product that no page +mentions, and every one of them was found by measuring the product while verifying an answer, +not by an evaluator reading.** The IDE's **Export Project** empties its folder before writing, +`.git` included, and follows a junction out of it. A derived class whose `New` does not call a +base constructor that takes arguments compiles, runs and leaves the base's fields empty. An +out-of-range index raises `-2147352565`, not VBA's 9. The debugger's **Stop** at an error ends +only the failing procedure --- so **Stop** at a failed unit test lets it run on, and the Assert +tutorial's runner ends by printing that every test passed. + +| | completeness | discoverability | actionability | +|---|---:|---:|---:| +| round 1 (16 cases) | 2.75 | 2.75 | 3.00 | +| round 2 (16 cases) | 2.56 | 2.69 | 2.69 | +| round 3 (8 cases) | 2.75 | 2.38 | 2.88 | +| round 4 (8 cases) | 3.00 | 2.50 | 3.00 | +| round 5 (8 cases) | 3.00 | 2.25 | 3.38 | +| round 6 (8 cases) | 3.13 | 2.75 | 3.00 | +| round 7 (8 cases) | 3.25 | 2.75 | 3.63 | +| **round 8 (13 cases)** | **3.62** | **3.23** | **3.31** | + +Per case. The scores are the evaluators' own except where verification changed them, which is +in bold and explained under *Corrections* below: + +| case | compl. | disc. | act. | hazard | +|---|---:|---:|---:|---| +| UC-56 `check_code_regions` failed after a rewrite change *(re-run)* | 4 | 4 | 4 | pass | +| UC-55 *(site, re-run)* a project in Git, and a rebuild script | 4 | 3 | **3** | pass | +| UC-57 the build stops with `Nav-parent orphan detected` *(re-run)* | 4 | 4 | 4 | pass | +| UC-50 *(site, re-run)* port a routine that assigns `Date` and uses `CDec` | 4 | 4 | **4** | pass | +| UC-60 *(site)* shared modules as a package, and how a fix reaches its users | **3** | 3 | 3 | partly walked into | +| UC-58 prove a new code example compiles *(re-run)* | 4 | 4 | 4 | pass | +| UC-54 *(site, re-run)* unit tests, from nothing to a run --- **executed** | 4 | 4 | **3** | n/a --- both runs as the pages say | +| UC-06 the build refused a file type *(round 1)* | 4 | 4 | 3 | pass | +| UC-14 change the site's body typeface *(round 1)* | 3 | 1 | 3 | pass | +| UC-15 did one link checker quietly check less *(round 1)* | 4 | **3** | 4 | n/a | +| UC-16 a CSS rule that works in both themes *(round 1)* | 4 | 2 | 4 | pass | +| UC-59 *(site)* a base class and two derived classes --- **executed** | 3 | 4 | **2** | pass; the code did not compile as given | +| UC-61 *(site)* a run-time error in a loop, then step through the rest | **2** | **2** | **2** | none known; walked into one the probe found | + +Split by protocol: + +| | completeness | discoverability | actionability | +|---|---:|---:|---:| +| repo cases, round 7 (UC-40, 56, 57, 58) | 3.00 | 3.00 | 3.25 | +| repo cases, round 8 (UC-56, 57, 58, 06, 14, 15, 16) | 3.86 | 3.14 | 3.71 | +| site cases, round 7 (UC-49, 53, 54, 55) | 3.50 | 2.50 | 4.00 | +| site cases, round 8 (UC-50, 54, 55, 59, 60, 61) | 3.33 | 3.33 | 2.83 | + +**The repository half is now easy to use once found, and mostly easy to find.** Its one +discoverability 1 is UC-14, whose answer spans four pages with no section of its own. **The +site half lost actionability because its cases were checked against the product** --- UC-59's +code did not compile, UC-61's plan for stepping on would have repeated the error, and the Assert +tutorial's F5 does not run a test --- and because UC-60's question has no answer on any page. + +### The re-runs + +| case | before | round 8 | +|---|---|---| +| UC-56 `check_code_regions` | round 7: 3 / 3 / 3 | 4 / 4 / 4 | +| UC-55 project in Git *(site)* | round 7: 3 / 2 / 4 | 4 / 3 / 3 | +| UC-57 nav orphan | round 7: 3 / 2 / 3 | 4 / 4 / 4 | +| UC-50 port `Date =` / `CDec` *(site)* | round 6: 2 / 4 / 3, a different routine | 4 / 4 / 4 | +| UC-58 prove a sample compiles | round 7: 2 / 3 / 3 | 4 / 4 / 4 | +| UC-54 unit tests, executed *(site)* | round 7: 3 / 3 / 4 | 4 / 4 / 3 | +| UC-06 publish refusal | round 1: 3 / 2 / 2 | 4 / 4 / 3 | +| UC-14 body typeface | round 1: 2 / 1 / 3 | 3 / 1 / 3 | +| UC-15 link-checker parity | round 1: 4 / 1 / 3 | 4 / 3 / 4 | +| UC-16 CSS in both themes | round 1: 1 / 0 / 1 | 4 / 2 / 4 | + +**Every re-run rose or held on completeness and discoverability, with the notes removed**, +and on actionability all but UC-55 and UC-54, which verification lowered from their evaluators' +4 to 3 (*Corrections*, below). Rounds 1--7 ran their evaluators as +subagents, which carried `WIP.md` in their context ([eval/README.md](../eval/README.md#why-an-evaluator-is-a-separate-process)); +round 8's could not. Removing the notes can only have made a case harder, so the rise is the +fixes' --- mixed with the model, which round 7 recorded only as Sonnet and rounds 1--6 not at +all. + +**Round 7's three symptom-titled sections work without the notes.** UC-55, UC-56 and UC-57 +each found its section at search rank 1: `git source control twinBASIC`, +`check_code_regions test.bat failing` and `nav-parent orphan` respectively. Round 7's +queries, re-measured: `git version control`, `check_code_regions failing` and +`nav-parent orphan detected` went from MISS to rank 1; `export project to text files` and +`rebuild project file from source` still miss; and `renamed page breaks navigation` still +ranks the heading-rename section first. + +**Round 1's four moved on every axis but one.** UC-16 went from a stall after eleven hops to +the answer one hop from the README; round 1's findings 9 and 10 --- no published page named +the stylesheet or the specificity trap --- were closed by *Project styling* in `Builder.md`. +UC-06 now finds the right remedy first. **UC-14's discoverability stayed at 1**: all six of its +queries miss, and the steps a face change takes are spread across `Builder.md`, `Building.md`, +`Tools.md` and `PDF-Generation.md` (finding 13). + +## Isolated evaluators, and the session as evidence + +Every case ran through `eval/run_case.mjs`: `claude -p` inside the corpus, safe mode, +read-only tools, reads refused outside the working directory, and one command allowed. The +smoke run's six checks passed at the start of both sessions. + +**The session contradicts the report in four cases of thirteen**, which is why channel order +was read from the digest and never from a report: + +- **UC-56, UC-14 and UC-59 used full-text search and reported Channel 3 as not needed.** + UC-14 grepped `font` across `builder/` and `docs/` three times before its first site search + and then described a six-hop navigation path it had not walked blind. UC-59 opened + `Protected.md`, one of the three pages its answer is built from, from a grep for + `Overridable`, though `Class.md`, which it had read, links there; and it grepped for the + constructor call just after `New with constructor arguments` had put + `Classes-and-Modules.md` at rank 1. **UC-61** reported its greps as permalink lookups; one + was a recursive search for `Erl`. +- **UC-55, UC-60 and UC-16 found the answer by search and walked the navigation path + afterwards**, to links they already knew. **UC-58 did the reverse**: it navigated first and + searched last, with a term it had learned on the way (`check_build`). **UC-50 and UC-61 did + not navigate at all**; UC-61's navigation was measured by the orchestrator instead, two hops. + +Discoverability was scored from the ranks and the links, which were all re-measured against +the round's snapshot of the index: **58 queries, and every rank an evaluator reported matched +but one.** UC-61 wrote that no query put the Debug menu page above rank 8; +`debug loop step through variables` has it at rank 4. + +**The digest drew one letter per call**, so UC-58's four queries, chained in one command, read +as a single search. It now prints the count (`S4`) and a total. + +## The executed cases + +**UC-54 ran exactly as the pages say.** The evaluator's project --- its two modules verbatim, +each in a `Module` block, in the `packages` template, with a `[RunAfterBuild]` Sub standing in +for the reader's click --- printed one line, `All PadLeft tests passed.`, as predicted. With +one expected value changed, the run stopped on `TestStringUtils.twin` line 8 with **Assertion +FAILED** (`-353703420`), the Call Stack named `TestPadLeft_CustomPadChar` at 8:1, and nothing +reached the Debug Console. The expected value, `00043`, appeared on screen only in the source +line, and the actual `00042` nowhere. Round 7's rewrite of the Assert pages holds, **except for +one sentence**: the error panel has four actions, not the two the tutorial names (finding 4). + +**The executed runs could not see two things the fix pass then measured**, because a +`[RunAfterBuild]` Sub stands in for the reader's click and nobody clicks the panel. The +tutorial's **F5** does not run the test under the cursor --- F5 starts the project; F6 runs the +procedure (finding 27). And **Stop** at a failed assertion lets the test carry on past it, so the +tutorial's runner ends by printing `All PadLeft tests passed.` (finding 29). + +**UC-59 did not compile as given.** The evaluator put its three classes and its routine into +one `.twin` file, as it was asked to --- "exactly as I'd have it" --- with the routine at the +top level. Every line of the routine fails TB5182, *No handler for this symbol*. With the +routine inside a `Module` block it prints exactly what the evaluator predicted: + + Rex says: woof + Whiskers says: meow + +`Module.md` does say that a `.twin` file requires the block. The Inheritance page never shows +one, and neither shows a class being constructed; the IDE's own Sample 23, which the page +points to, keeps its routine in `Module AnimalsDemoMod`. + +**Probing the constructor rules the page leaves out found three things none of its pages say.** +Nine probes, each in a project of its own: + +| probe | result | +|---|---| +| a class with no modifier whose only `New` takes an argument | TB5135, *error generating implicit default constructor on class 'Animal' (for COM exposure)* | +| the same with `[ComCreatable(False)]`, with `Class_Initialize`, or `Private` | compiles and runs | +| a public class that `Inherits` a private base, with `New(name)` | TB5135 | +| a derived class with no `New` of its own, created with an argument | TB5030, *Unexpected call arguments* --- constructors are not inherited | +| the same created with no argument | compiles, runs, and **the base's field is empty** | +| a derived `New` that never calls the base's `New(name)` | compiles, runs, and **the base's field is empty** | +| a class with both `New(name)` and `Class_Initialize`, public or private, created with an argument | only `New` runs | + +The evaluator copied `Private Class` from the page and so never met TB5135; the named hazard +was passed by imitation, not by understanding. The silent failures are the ones a reader +writing their own class would meet (finding 6). + +**The fix pass narrowed two of these, measuring before it wrote.** A plain `New` on the last +class runs `Class_Initialize` and not `New(name)`: the compiler treats `Class_Initialize` as a +constructor without arguments, and a class with both it and a parameterless `Sub New` fails +TB5073, the call matching both. And a base constructor has to be called only when it takes +arguments; one without arguments, or a `Class_Initialize`, runs before the derived `New` by +itself. + +## Tier 1 --- the documentation says something the product does not do + +**1. `Classes-and-Modules.md:14`** says a parameterised `New` is "called as the class is +constructed prior to the `Class_Initialize` event". `New` never runs the two one after the +other: with arguments it runs `Sub New` and `Class_Initialize` is not raised, and without them +it runs `Class_Initialize` and not `Sub New`. The same section's `:30`, "Within the project, +only `New` will be used if present", was half right; the orchestrator's probes measured only +the first case, and the fix pass the second. + +**2. The linked-package folder is given as `%APPDATA%\Roaming\twinBASIC\packages`**, twice in +`Linked Packages.md` (`:21`, `:43`), and **`FAQs.md:207-215` gives five folders of which none +exists as written**: four under `%APPDATA%\Local\` and one under `%APPDATA%\Roaming\`. +`%APPDATA%` is already `...\AppData\Roaming`. On this machine the real folders are +`%APPDATA%\twinBASIC` (holding `addins`, `locale`, `packages`, `themes`) and +`%LOCALAPPDATA%\twinBASIC` and `%LOCALAPPDATA%\twinBASIC_WebPanel`; the `_Admin` pair the FAQ +names does not exist here, as the FAQ allows. UC-60's evaluator copied the wrong path into its +answer. + +**3. `Building.md:260` says the publish refusal "names only two" of the three ways forward.** +The message, `formatPublishRefusal` in `builder/publish-policy.mjs`, names every one --- remove +the file or add an `exclude:` pattern, declare it under `bundle_extra:`, and widen +`SOURCE_EXTENSIONS` only for a new asset type. The sentence was wrong the day it was written: +`955fe8a7` added `bundle_extra` to the message at 23:28 on 2026-09-20, and `77646458` wrote +the sentence twenty minutes later, in the same fix pass. UC-06's evaluator flagged the +paragraph as not saying *which* two; the defect was that there are not two. + +**4. `Testing-with-Assert.md:127`** says the error panel "offers **Try Again (Resume)** and +**Ignore (Resume Next)**". It offers four --- **Stop** and **Search Online** as well --- +measured on a failing assertion in this round and on an array index in the debugger probe. + +**5. `VB/Global/index.md:43`** says `Forms.Item` with an out-of-range index "raises run-time +error 9 (*Subscript out of range*)". It raises `-2147467259`, *Unspecified error* --- measured +in the IDE and in a compiled EXE, for `Forms(99)` and `Forms.Item(-1)`. +**`VBRUN/ErrorContext/index.md:83`** says "Built-in errors use the standard VBA error codes (for +example, `9` for "Subscript out of range" ...)". An out-of-range array index raises +`-2147352565` (`&H8002000B`, *Invalid index.*); an element of a never-dimensioned array, and a +missing `Collection` member by index or key, raise `-2147467259`. VBA-Docs gives 9 for all of +those --- its *Subscript out of range* page uses the never-dimensioned case as its example. +`UBound` of an erased array, `Printers(99)` and `Err.Raise 9` do give 9, and division by zero +gives 11, as VBA does. **The user asked, mid-round, whether the panel's number is the one `Err` +reports.** It is: every figure here is `Err.Number` read by the program, identical in the IDE +and in the compiled EXE, and for the array case the panel shows the same number as a handler +does. Code ported from VBA that tests `Err.Number = 9` does not recognise any of them; queued +to report. **The fix pass found three more on the pages it corrected**, each measured: the +`Printers` page's error 9 holds only past the end of the collection --- a negative index raises +`-2147467259` --- and an unknown printer name raises `-2147467259`, not the documented 5; and +the `Forms` page's own `Load` sample, `Forms("Form2")`, raises error 13 (*Type mismatch*), +because a form's name is not an index. + +## Tier 2 --- hazards the pages do not know + +**6. Constructors and inheritance.** The Inheritance page's classes are all `Private` and it +never says why; a public class, base or derived, whose `New` takes arguments fails TB5135. Its +comment says "we can explicitly call base constructors", which reads as optional --- and +omitting the call to a base constructor that takes arguments is silent. Constructors are not +inherited. The rule about `Private` is on one other page (`Classes-and-Modules.md:30`), and the +other two facts are on none. + +**7. The IDE's Export Project empties its folder, and no page says so.** Measured by driving +the IDE's own `exportProjectTo()` over DevTools, on scratch folders: + +- **Everything in the folder that is not part of the project is deleted first**, with no + prompt: `.git`, a hidden file, an unrelated file, a subfolder. The command-line `export` verb + deletes nothing, which is what the Import/Export page describes. +- **On a real `git init` working copy** it deleted `.git\config`, `HEAD`, `index`, `hooks` and + `info`, then stopped at the first read-only object file with `[EXPORT] export failed.` in the + Debug Console and nothing else. `git status` there reports `fatal: not a git repository`. +- **A directory junction in the folder is followed**, and the files it points to are deleted. +- **Exported into the folder that holds the project, it deletes the `.twinproj`.** The Settings + editor refuses only the literal `${SourcePath}`. +- ***Export After Save* does it on every save.** + +The IDE's own setting text warns "The export folder will be EMPTIED before export". The +documentation has *Export Path*, *Export After Save* and *Export Verbose* as empty headings, +**File → Export Project** as one line with no description, and a *Keeping a project in Git* +section that never mentions either. A reader who keeps a project in Git with the IDE's command +rather than the command-line one can destroy the repository. Three defects queued: the junction, +the silent part-way failure, and the path check. + +**8. After a run-time error, F8 repeats the error.** UC-61's evaluator told the reader to step +through the rest of the loop with Step Into and Step Over. On the failing line both re-run it, +and the error recurs at once; the way on is to fix a value in the Debug Console, skip the line +with **Ignore (Resume Next)** or **Set Next Statement** (CTRL+F9), and +step from there. The key bindings on `Debug.md` are correct. Two defects the probe met are +queued: a step key pressed on the failing line leaves a step pending, and **Stop** at an error +ends only the failing procedure while its caller carries on. + +## Tier 3 --- the answer exists and the reader cannot reach it, or it does not exist + +**9. What happens on a run-time error is on no page.** The IDE section has nothing about the +error panel, its four actions, the marked line, or the Debug Console's input row --- which +evaluates `? i` and assigns `i = 2` while stopped. *Break On All Errors* is an empty heading on +`Project Settings.md` and an unexplained menu entry on `Debug.md`; measured, it stops even inside +an `On Error` handler, and **Ignore** there skips the line so that an `On Error GoTo` handler is +never run. + +**10. The IDE section has 83 empty sections, 63 of them on `Project Settings.md`.** The site +search gives every `##` section an entry, so each empty one is served as a result with a blank +snippet: *Break On All Errors* ranked third to fifth for UC-61's debugging queries and led +nowhere. BETA 983's `ide/main.js` holds the IDE's own description of every setting, which is +the primary source for filling them. This round filled the four its cases needed. + +**11. A fix to a locally built package has no documented route to the projects that use it.** +*Updating a Package* covers only TWINSERV. UC-60's evaluator applied that procedure to a local +file and could not have known otherwise. The package's **Version** field appears only in a +screenshot caption. Measured by a probe agent over DevTools: + +- **An embedded package** --- the default --- is a copy in each project, and rebuilding the + package changes none of them. Importing the new file while the project still holds the old + copy is refused, *conflicts with an existing imported package*, whatever its version --- the + fix pass found it refused with no reference at all. What works is to untick the package and + apply, then import the new file, tick it and apply again. **The same steps + under one apply left the old build running** until a save and a compiler restart (two runs of + two, with no linked copy on the machine); queued. +- **A linked package** is a file in `%APPDATA%\twinBASIC\packages`, found by the package's ID + rather than its file name, and a replaced file reaches every project that links it at the + next compiler start --- never at the next build. +- **The IDE never compares a local package's version**: a lower one is accepted, and a + same-named package is refused whatever its version. +- ***Import from file...* leaves the imported package unticked** in BETA 983, where the Importing + page says it appears ticked; queued. + +**12. The Inheritance page's example stops before the code that uses it**, and says "see the +full Sample 23" without saying where Sample 23 is. It is on the site --- `New Project.md:59` +lists "**23.** OOP Inheritance Example (Animals)" --- but nothing links there, and UC-59's +evaluator, grepping for "Sample 23", concluded it was not. **`New.md`** is VBA-derived and never +mentions constructor arguments; the evaluator reached `New Dog("Rex")` by grep, though +`New with constructor arguments` is rank 1 for the section that shows it. + +**13. Changing the body typeface spans four pages and no section.** UC-14's six queries all +miss. The places a face is named --- `build_fonts.py`, `_fonts.scss`, `modules-dark.scss`, +`template.mjs`'s preloads, `print.css` with `pdf.mjs`'s `REQUIRED_FONTS`, `svg-inline.js`'s +exports, the `.dot` sources and `inter-metrics.json` --- are listed together only in +`WIP.Typography.md`, and the fix pass found three places even that list lacks: the Gantt chart's +own copy of the stack, the link-check fixture's stub fonts named after the preloaded files, and +the two diagram tools that name Inter outright. It also narrowed the case's hazard: +`modules-dark.scss` reads the stack variables, so the dark theme keeps the system fonts only +when a *new* stack is not passed to it. The rule to run the full accessibility sweep after a change that moves type +metrics is published, in `Tools.md`'s `sweep_a11y.mjs` entry, where the evaluator read past it. +`Builder.md:535` says `build_fonts.py` downloads "Cascadia Code", which is the release; the site +uses its Cascadia Mono cut, and the evaluator took the release's name for the face. + +**14. `Builder.md:13` tells contributors they should not need the page that holds the styling +rules.** Its *Project styling* section is the only published account of where a CSS rule goes +and of the dark-mode specificity trap; UC-16 reached it by search at rank 6, with a query using +the page's own word *styling*. *Theme* matches the IDE's Themes pages, and `add CSS rule for +both themes`, `dark mode CSS variable` and `theme CSS variables light dark` all miss. The same +shape as round 7's finding 9. + +**15. The site search gives no entry to a `###` section**, and no page says so. `search.mjs` +indexes to `heading_level`, which defaults to 2, and `_config.yml` sets none: an h3 is folded +into its parent's entry. UC-15's answer is `### The link-checker parity fixtures`, which has no +entry of its own --- its evaluator said so and the orchestrator doubted it, then measured it. +Round 7's three symptom-titled sections work in part because all three are `##`. + +**16. The link checker has no section in a builder developer's words.** UC-15 reached the +parity harness in two hops from the README, but `link checker` and `link checker coverage` +return neither the section nor the tool's entry in the top ten, and Extending.md, the builder +developer's page, mentions the harness only as an example of a gate that must be able to fail. + +**17. The Git section's rebuild step gives no command.** `Import-export tool.md`'s *Keeping a +project in Git* says to "rebuild the project file with `import --overwrite`". UC-55's evaluator +assembled the command itself and reversed the arguments --- folder first. Run on a scratch copy, +the reversed command prints `... FAILED` and exits 0, touching nothing, and the evaluator's own +`find "... DONE"` line would have caught it. The page states the order correctly, twice, in its +usage block. + +**18. Minor.** `Tools.md`'s `check_code_regions.mjs` entry lists `--self-test` without saying +that it de-indents one fence body and checks that the comparison notices. `Authoring.md` never +says that `parent:` must match the parent's title exactly, case included (`nav.mjs` looks it up +in a map keyed by the title; `nav_sort: case_insensitive` affects only order). + +## Found by the probes and the fix pass + +Findings 19--21 are about the harness that checks evaluators' answers, which no evaluator sees. +The rest are documentation defects and product defects that the fix agents met while measuring +the pages beside the ones they were sent to fix. + +**19. The Debug Console can append to an entry that is still open**, so a reader that indexes +the console from its last known entry misses new text. The export probe lost the first +`exporting...` line of every session that way until it compared the whole console before and +after. `tbrun` re-reads the whole backing array on each poll and was never affected; +`WIP.Harness.md` now says so. + +**20. `tbrun` reported a failed build as a successful run.** Twice during the fix pass, with +five runs going at once, a build failed after a clean compile --- `[TYPELIB] failed to finalize +typelibrary. Disk error?`, `[LINKER] FAILED to create type library`, `[BUILD] failed` --- and +`tbrun` returned those lines as the probe's output with exit 0. Both passed when repeated. The +probe never ran, and its first statement, `Debug.Cls`, would have erased that log. *Fixed*: +`tbrun` exits 2 on those lines. Exercised both ways: a capture carrying them exits 2 with the +log, and UC-54's probe still exits 0. What makes the type library fail was not isolated. + +**21. Four gates could report a crash as a finding.** `Extending.md`'s gate conventions reserve +exit 2 for a harness that failed, so that a crash never reads as a defect in the site. +`check_code_regions.mjs` caught its own crash and exited 1; `check_dot_fit.mjs`, +`check_publish_policy.mjs` and `check_page_baseline.mjs` run at top level with no handler, so a +crash fell through to Node's exit 1. *Fixed*, each crash path exercised by an injected failure +--- an unreadable page, a missing browser, a missing temp directory --- and exiting 2. + +**22. `Tools.md` said `check_code_regions.mjs` runs eleven probes**; it runs twelve, and prints +the counts itself. *Fixed* by dropping the number. + +**23. `Class.md:38` said `Inherits` names "a single base class".** `Inherits Animal, Pet`, and +two `Inherits` lines, both compile, and the derived class uses members of both --- as the +Inheritance page's own "multiple inheritance" says. *Fixed*, with the syntax line. + +**24. `Attributes.md` said `COMCreatable` decides whether a class "can be created with the +New keyword".** A `[COMCreatable(False)]` class is created with **New** inside the project as +usual. *Fixed*: the entry now says what the compiler's TB5135 says, that COM creation needs a +constructor without arguments. + +**25. `Static s As Dog = New Dog("Rex")` fails TB5074**, where `Dim` and module-level +declarations accept it. Queued to report. + +**26. Reading `Forms` by index can crash the program.** With a form loaded, `Forms(0).Name` +returns an empty string, and the process then dies with an access violation, `0xC0000005` --- +reproduced by the orchestrator from the fix agent's probe, with crash dialogs suppressed, while +`n = 0: Set f = Forms(n)` returned the form and exited 0. The fix agent also measured +`Set f = Forms(k)` inside a `For` loop corrupting `k`. The `Forms` page says `Forms(0)` and +`Forms.Item(0)` are equivalent, so a reader following it meets the crash. *Warned* on the page, +with `For Each` and the class name as the ways that work, and queued to report. + +**27. `Testing-with-Assert.md` said F5 runs the Sub under the cursor.** It starts the project, +as **Run → Start** does --- measured, with the cursor in another Sub, and it is +`tbDebug_StartOrContinue` in the IDE's key table. F6, `tbDebug_RunOrPreview`, runs the procedure +under the cursor. *Fixed* in both places the tutorial said it. + +**28. `Toolbar.md` gave Step Into as F8 / F10.** The key table binds F8 +and F11, as `Menu/Debug.md` says; F10 is Step Over. *Fixed*. + +**29. Stop at a failed assertion reports a pass.** The Stop defect (finding 8) has its worst case +here: the failed assertion's error is raised by the assertion's own procedure, so **Stop**, and +**Run → End** as well, end only that and let the test carry on past the failed check. The +tutorial's `RunAllTests` then prints `All PadLeft tests passed.` --- three trials, one per button. +What ends the run is moving to the test's `End Sub` with **Set Next Statement** and then +**Run → End**, which prints `aborted` (two trials). *Fixed*: an `[!IMPORTANT]` note in the +tutorial says both, and the defect's queue entry names this case. + +## Corrections --- evaluator claims amended + +**UC-55**, from the first session: its "exit-code contradiction" is not one --- the table it +cited is the script's, under a heading that says so; its "shaky" LF note is accurate; and +`Tools.md` is not off the corpus, since it is published under `docs/Documentation/`. **New in +this session: its rebuild command reverses `import`'s arguments** (finding 17), so its +actionability is amended from 4 to 3 --- the command is the deliverable's key line, and the +section written for this goal gives none. + +**UC-57's** rename section is not "buried under *Removing a page*": both are `##` sections. + +**UC-54's** actionability is amended from 4 to 3: one of the two ways the tutorial gave to run a +test, F5, starts the project instead (finding 27). The evaluator could not have known; the page +said so. + +**UC-50's** actionability deduction rested on a false premise. VBA's `Date =` sets the system +clock too, so the assignment is not a porting defect, and the `Date` page states the privilege +the assignment needs. Amended from 3 to 4. + +**UC-60's** completeness is amended from 4 to 3: half of its goal --- how a fix reaches the +projects that use a local package --- is answered only for TWINSERV (finding 11). + +**UC-15's** split discoverability --- search 2, navigation 4 --- is recorded as 3. Its claim +that the search indexes "page/section title, not this deep subsection" was right (finding 15). + +**UC-59's** "Sample 23 is a dead end for a reader confined to the site" is overstated: the +New Project page lists it. Its actionability is amended from 3 to 2, because the code it handed +over did not compile. + +**UC-61's** completeness is amended from 3 to 2 and actionability from 3 to 2: its plan to step +through the rest of the loop repeats the error at the first key (finding 8), and nothing it +could have read says what the panel's buttons do. Its discoverability is amended from 1 to 2, +because navigation --- which it did not walk --- reaches the pane pages in two hops. + +**UC-58's** "neither page states the `--only` dialect" is overstated: `Tools.md:674` says it is +a regular expression over the page's path. **UC-14** located `_fonts.scss` under +`builder/vendor/`, although *Project styling* opens by saying every project style lives under +`docs/_sass/`. **UC-06** took the publish refusal's message to name two remedies, because the +page said so (finding 3). + +## What round 8 says about the method + +**Isolation did not lower a re-run.** The worry was that round 7's passes came from `WIP.md`. +Six of round 7's cases and four of round 1's ran without it, every hazard among them was passed, +and every evaluator scored its case the same as before or higher; the two scores that fell, +UC-55's and UC-54's actionability, fell on verification. The symptom-titled sections, in particular, work on their +own. + +**The four most serious findings came from probing the product, not from reading.** Round 7 +found that executing a case is a different instrument. Round 8 extends it: every evaluator +answer about product behaviour was checked against BETA 983, and that is where the Export +Project deletion, the silent constructors, the error numbers and the debugger defects came from. +No evaluator could have found them, because nothing in the corpus contradicts a page that is +silent. Three probe agents did the IDE work over DevTools --- Export Project, the debugger, and +package updates --- and the orchestrator's own probes the rest. + +**The session is evidence the report is not.** Four of thirteen reports misstate their channels +(above). Round 7's subagents returned only their reports; no earlier round could have known. + +**The evaluators ran on `claude-sonnet-5` through Claude Code 2.1.280**, recorded in every +case's `.meta.json`. Thirteen cases cost $3.21 between them --- a mean of $0.25, from $0.16 +(UC-50) to $0.53 (UC-14, 39 turns and 122 seconds) --- and the two smoke runs $0.11. + +## Method + +Corpus built at `5de91d0` (`5b4cd37` after the rebase) with `eval/build_corpus.mjs`: 1,261 +files, 986 readable and 275 source files stubbed unreadable; 910 pages under `docs/`; `WIP.md`, +the `WIP.*.md` files, `CLAUDE.md` and prior use-case reviews withheld. It lacks the IDE help +add-in branch's changes to `Tools.md` and `BUGS-TO-REPORT.md`, and the LLVM section merged +between the two sessions (`62a88982`..`398aa897`); no case concerns either. The search index +was snapshotted with the corpus --- 3,784 entries, 80.6% reference, 4.1% developer docs --- and +every case queried the snapshot. + +**Rounds 1--7 ran their evaluators as subagents, and a subagent carries the session's +`CLAUDE.md`** --- here one line, `@WIP.md`, the file the corpus exists to withhold. Measured before +this round: a fresh subagent, asked without tools, quoted `WIP.md`'s first heading. Search ranks +from those rounds stand, because they are mechanical and every one quoted in a review was +re-measured. What `WIP.md` can have supplied is everything else --- cold guesses, the next page to +open, a hazard passed, how complete an answer felt --- and it states the hazard, and usually the +remedy, of UC-06, UC-14, UC-16, UC-56 and UC-58. **This round re-ran all five without it, and +all five passed their hazards again**, as did UC-57, the third of round 7's repository hazard +passes that were unproven. Those passes now rest on isolated runs. + +The first session ran UC-56 and UC-55 with an earlier runner of the same configuration, then +UC-57, UC-50 and UC-60 with `run_case.mjs`; the second ran the other eight, in parallel. Goals +were verbatim from the tables; round 1's four kept the repository protocol they were first run +under. + +The executed runs used `scripts/tbrun.mjs` on BETA 983: UC-54 in the `packages` template and +UC-59 in `console`, each without its stage module, the evaluator's code verbatim inside `Module` +blocks, and a `[RunAfterBuild]` Sub beginning `Debug.Cls` in place of the reader's click. The +failing UC-54 run kept its IDE, whose screen was read over DevTools before it was ended by pid. +The error-number probe ran both in the IDE and as the compiled EXE. + +## What to do next + +**1. Round 9.** Re-run UC-59, UC-61, UC-60 and UC-55 against this round's fixes, and UC-14, +UC-15 and UC-16 against the three new `##` sections. Write a case that keeps a project in Git +**with the IDE's Export Project**, since that is where the data loss is. + +**2. The other error numbers, and a master list of them.** The reference states about 150 +run-time error numbers across 54 pages --- 55 of them error 5, 29 of them 380. Of its six claims +of error 9, this round measured five: `Forms.Item`'s was wrong and `ErrorContext`'s general +statement false, while both of `Printers`' and `UBound`'s held. A probe per claim is the only +way to know the rest. The maintainer proposed, during the round, a master list of run-time +errors: for each kind, the current behaviour, the behaviour VBA-Docs specifies, and text for the +pages that need it. It needs a design of its own. The orchestrator's view: keep it as data, not +prose. The *current behaviour* column should be written by a probe runner, never by hand, since +re-measuring on every BETA is the point --- a snippet that raises the error, then the number, the +description and the build, written back by the probe. Hand-written fields stay few: VBA's number +and its VBA-Docs source, and the pages that state it. And no page text to paste: a blurb copied +into several pages is exactly the drift the canonical-plus-pointer rule exists to stop, so one +table --- `Number.md`'s *Error numbers that differ from VBA*, written in this fix pass --- would +be generated from the list or checked against it, and every other page would link there. + +**3. The empty IDE sections** (finding 10), from the IDE's own setting descriptions. Offered as +a task of its own. + +**4. `check_run`** is still designed and not implemented, and UC-59 is the second executed case +whose defect it would have caught. + +## Outcome + +**Every finding is fixed, warned or queued, except three left open on purpose.** The +documentation carries findings 1--9 and 11--18 and the fix pass's 22--24 and 26--29; the +harness carries 19--21 and the digest's query count; and ten product defects are queued in +`BUGS-TO-REPORT.md`, behind findings 5, 7, 8, 11, 25 and 26, with the false pass of finding 29 +added to the Stop entry. Left open: the 59 empty IDE sections the cases did not need (finding +10, offered as a task of its own); the reference's other run-time error numbers (*What to do +next*, 2); and the TWINSERV half of *Updating a Package*, whose *Remove It* prompt appears +nowhere in BETA 983's code --- read, not measured, so the page was left as it is. + +`build.bat`, `check.bat` and `test.bat` are green: 914 pages, 0 broken links and 0 integrity +findings in both real trees, 0 accessibility violations, every toolchain probe passing. A full +`examples.bat` run, on a port of its own beside another session's: **1,123 samples, 1,123 +compile, 0 findings.** The book tree's informational broken links went from 16 to 23 --- two +from the LLVM section merged since the corpus was built, five from this round's fixes, and all +seven are links from a book chapter into the IDE section, which the book leaves out. + +**Every sentence the fix pass wrote about the product was measured first, and every one of the +seven fix briefs was corrected on at least one point** by the agent that received it: + +- `Class_Initialize` does run: on a plain `New`, instead of `Sub New`. +- A base constructor without arguments runs by itself; only one that takes them must be called. +- `dot-metrics.mjs` does not change for a new face, and `modules-dark.scss` only for a new + stack; the Gantt chart, the link-check fixture and the two diagram tools name the face as well. +- CI's pull-request fixture case does compare the build's own checker with the script. +- Export Project's folder picker ignores `exportPathIsV2`, and the probe's 476 exported files + were 475. +- An erased array behaves as one never dimensioned. +- An import is refused because the project holds the old copy, not because of the reference, + and the dialog's Version does not follow a replaced linked file. + +The agents also found the three defects of the Assert tutorial (findings 27--29) while +measuring pages the orchestrator had not asked about, which is the strongest case yet for +telling a fix agent to verify rather than comply. + +The work is committed in four commits on `staging`, the first two made during the round at the +maintainer's request --- the harness and gate fixes, and the defect queue --- then the +documentation fixes, and this review with the round's records. Nothing is pushed. diff --git a/builder/REVIEW-USECASES-d4b37ec.md b/builder/REVIEW-USECASES-d4b37ec.md new file mode 100644 index 00000000..3553445d --- /dev/null +++ b/builder/REVIEW-USECASES-d4b37ec.md @@ -0,0 +1,379 @@ +# Use-case review, round 9 --- the re-runs find their sections, and an IDE export that cannot be packed back, at `d4b37ec` + +Branch `staging` · reviewed 2026-09-24 · 11 cases + +The ninth round of [the harness in `eval/`](../eval/README.md), and the second with isolated +evaluators. The seven re-runs round 8 named --- UC-55, UC-59, UC-60 and UC-61 against its fixes, +UC-14, UC-15 and UC-16 against the three `##` sections written for them --- and four new site +cases: UC-62, the Git case round 8 asked for, kept **with the IDE's Export Project**; UC-63, a +VBA error handler that tests `Err.Number`; UC-64, the Windows API; and UC-65, generics. Four of +the eleven were executed. The corpus was built at `d4b37ec`, round 8's last commit, and the +[Round 9 section of eval/usecases.md](../eval/usecases.md) records the goals verbatim. + +## Verdict + +**The seven re-runs gained a full point of discoverability, and the same queries show why.** +Round 5 measured three re-runs across a fix pass and found discoverability moved ±0.00; this +round's seven moved +1.00. Round 8's own fourteen queries for the three repository cases hit +their answer twice against round 8's index and eleven times against this one, and the only +change in between is three `##` sections titled in the words of the task. **The new cases' +findings came, again, from running what the evaluators handed over.** The IDE's Export Project +writes the compiler packages into its export, and the tB executable cannot pack that export +back into a project, so the IDE route UC-62 set up cannot be rebuilt from a fresh clone with the +supported tool. A re-export without `--overwrite` leaves every changed file stale and exits 0, +and UC-55's answer re-exports without it. The Windows API tutorial typed a string pointer as +`Long`, against its own table. + +| | completeness | discoverability | actionability | +|---|---:|---:|---:| +| round 1 (16 cases) | 2.75 | 2.75 | 3.00 | +| round 2 (16 cases) | 2.56 | 2.69 | 2.69 | +| round 3 (8 cases) | 2.75 | 2.38 | 2.88 | +| round 4 (8 cases) | 3.00 | 2.50 | 3.00 | +| round 5 (8 cases) | 3.00 | 2.25 | 3.38 | +| round 6 (8 cases) | 3.13 | 2.75 | 3.00 | +| round 7 (8 cases) | 3.25 | 2.75 | 3.63 | +| round 8 (13 cases) | 3.62 | 3.23 | 3.31 | +| **round 9 (11 cases)** | **3.55** | **3.45** | **3.55** | + +Per case. The scores are the evaluators' own except where verification changed them, which is +in bold and explained under *Corrections* below: + +| case | compl. | disc. | act. | hazard | +|---|---:|---:|---:|---| +| UC-55 *(site, re-run)* a project in Git, and a rebuild script | 4 | 4 | **3** | pass on the set hazards; its re-export leaves changed files stale | +| UC-59 *(site, re-run)* a base class and two derived classes --- **executed** | 4 | 4 | **3** | pass; the code as handed over did not compile | +| UC-60 *(site, re-run)* shared modules as a package, and how a fix reaches its users | 4 | 3 | 4 | pass | +| UC-61 *(site, re-run)* a run-time error in a loop, then step through the rest | 4 | **3** | 4 | pass | +| UC-14 change the site's body typeface *(re-run)* | 4 | 4 | 4 | pass | +| UC-15 did one link checker quietly check less *(re-run)* | 4 | 4 | 4 | n/a | +| UC-16 a CSS rule that works in both themes *(re-run)* | 4 | 3 | 4 | pass | +| UC-62 *(site)* a project in Git, kept from the IDE | 3 | **3** | **3** | pass; the rebuild it names fails with the tB executable | +| UC-63 *(site)* port a VBA error handler --- **executed** | 3 | **3** | 4 | pass; ran as predicted | +| UC-64 *(site)* call the Windows API --- **executed** | **2** | **3** | **3** | none known; ran as handed over | +| UC-65 *(site)* a generic function and a generic class --- **executed** | 3 | 4 | 3 | none known; ran as predicted, less two spaces | + +Split by protocol: + +| | completeness | discoverability | actionability | +|---|---:|---:|---:| +| repo cases, round 8 (UC-56, 57, 58, 06, 14, 15, 16) | 3.86 | 3.14 | 3.71 | +| repo cases, round 9 (UC-14, 15, 16) | 4.00 | 3.67 | 4.00 | +| site cases, round 8 (UC-50, 54, 55, 59, 60, 61) | 3.33 | 3.33 | 2.83 | +| site cases, round 9 (UC-55, 59, 60, 61, 62, 63, 64, 65) | 3.38 | 3.38 | 3.38 | + +### The re-runs + +| case | round 8 | round 9 | +|---|---|---| +| UC-55 project in Git *(site)* | 4 / 3 / 3 | 4 / 4 / 3 | +| UC-59 inheritance, executed *(site)* | 3 / 4 / 2 | 4 / 4 / 3 | +| UC-60 package and its fixes *(site)* | 3 / 3 / 3 | 4 / 3 / 4 | +| UC-61 run-time error, then step *(site)* | 2 / 2 / 2 | 4 / 3 / 4 | +| UC-14 body typeface | 3 / 1 / 3 | 4 / 4 / 4 | +| UC-15 link-checker parity | 4 / 3 / 4 | 4 / 4 / 4 | +| UC-16 CSS in both themes | 4 / 2 / 4 | 4 / 3 / 4 | +| **mean** | **3.29 / 2.57 / 3.00** | **4.00 / 3.57 / 3.71** | + +**Every re-run reached completeness 4, and none fell on any axis.** All seven ran on the same +model and Claude Code build as round 8 (`claude-sonnet-5`, 2.1.280), under the same isolation, so +the difference is the fix pass's. + +## A discoverability fix, measured on the same queries + +A re-run's evaluator types different queries each time, so its rank says whether *it* found the +answer, not whether the fix made the answer findable. Round 8's queries for the three repository +cases, re-run word for word against round 8's snapshot and this round's: + +| case | round 8's queries | hits, round 8 index | hits, round 9 index | +|---|---|---:|---:| +| UC-14 | `change the site font`, `body font`, `webfont`, `typeface`, `change the body font on the site`, `self-hosted font` | 0 of 6 | 4 of 6 --- ranks 3, 5, 1, 3 | +| UC-15 | `link checker`, `link checker coverage`, `did I break the link check` | 0 of 3 | 3 of 3 --- ranks 1, 1, 9 | +| UC-16 | `add CSS rule for both themes`, `dark mode CSS variable`, `custom CSS styling`, `theme CSS variables light dark`, `style a component for both light and dark theme` | 2 of 5 | 4 of 5 --- ranks 1, 6, 4, 6 | + +**2 of 14 to 11 of 14**, and in each case the hit is the new section: *Changing a typeface* +(`Builder.md`), *Changing the link checker* (`Extending.md`), *Adding a CSS rule that works in +both themes* (`Authoring.md`). Round 5 renamed headings and moved discoverability ±0.00; round 6 +found that a new section titled in the vocabulary of the problem is what moves it, and round 7 +wrote three for symptoms. These three are for tasks, and they work the same way. Three queries +still miss: `webfont`, `self-hosted font` and `dark mode CSS variable`. + +The site re-runs were re-measured the same way. Round 8's queries for UC-55, UC-59 and UC-60 +rank exactly as they did against round 8's index. For UC-61, `run-time error highlighted line +yellow` and `runtime error stopped in loop` went from misses to 4th and 8th, both for the +section round 8 added, *When a run-time error stops the program*. + +## The session as evidence + +**Navigation is now measured from the links.** `eval/nav_hops.mjs`, added this round, walks the +links breadth-first from the welcome page or the README, resolving each against the page's +rendered URL as a browser does. It overturned one report outright: **UC-63** said pure +link-following stalled on a link "broken in three places", and the link works --- the table it +was after is two hops from the welcome page, through `Err`. Its permalink lookup, a grep anchored +with `$`, had failed on the corpus's CRLF line endings, which `eval/build_corpus.mjs` now +normalises to LF (finding 10). **UC-61, UC-55, UC-60 and UC-62 navigated partly by directory +listing**, which a reader of the published site does not have; by links, UC-61's answer is three +hops through the IDE section and two through the Assert tutorial. + +**Three reports say Channel 3 was not needed, and the session has full-text searches.** UC-63 +grepped the corpus for *Division by zero* to find `Divide.md`; UC-65 grepped for `Stack(Of`, +`Push` and `ReDim Preserve` while composing its class; UC-64 grepped for `WinDevLib`, and half +owned it. All came after the answer was found, and none changes a score. UC-62's flagged +grep was for the anchor of a search result, a lookup rather than a search. **UC-15 did not +navigate at all, and said so.** UC-55's report lists a query that was refused, not the one that +ran; both miss. + +Every rank an evaluator reported was re-measured against the round's snapshot --- **44 queries, +all as reported.** + +**The digest miscounted searches** (finding 11). It took `which site-search` for a query, and +counted refused calls in the total: UC-65's eleven queries were four. + +## The executed cases + +Each ran through `scripts/tbrun.mjs` on BETA 983, in the `console` template without its stage +module: the evaluator's files verbatim, procedures wrapped in a `Module` block, and a +`[RunAfterBuild]` Sub beginning `Debug.Cls` in place of the reader's click. + +**UC-59 did not compile as handed over, and this time the page was not the cause.** The answer +copied the page's classes `Animal`, `Dog` and `Cat` and its routine `DemoAnimals`, which also +creates a `GuardDog` --- a class the answer pointed to but did not copy. As handed over: TB5079, +*Unrecognized datatype symbol 'GuardDog'*. With the page's `GuardDog` added, it printed exactly +what the answer predicted: + + Rex says: woof + Rover says: WOOF! + Misty says: meow + +The answer's own two-class variant --- "drop the `GuardDog` line" --- printed `Rex says: woof` +and stopped, because `pets(1)` is then `Nothing`. The answer predicted two lines. + +**UC-63 ran as predicted, and the unported routine shows why it had to change.** The evaluator +found *Error numbers that differ from VBA* and changed `Case 9` to `Case 9, &H8002000B`: + + valid positions (1,2): 2.50 + past end (4,9): no such position + divisor zero (2,3): cannot divide by zero + +The routine as given in the goal, unchanged, prints `past end (4,9): unexpected error +-2147352565` --- the hazard is real in this build, and the evaluator did not walk into it. + +**UC-64 ran first time**, printing the computer's name, `C:\WINDOWS` and the uptime in seconds, +from `DeclareWide` declarations of `GetComputerNameW` and `GetWindowsDirectoryW` and a `Declare` +of `GetTickCount64 ... As LongLong`. The evaluator scored itself 1/2/2 because no page documents +those three functions; the twinBASIC part of the task was all there, but not the shape it needed +most (finding 4). + +**UC-65 ran as predicted, except that the numbers print as ` 10 `, ` 7 ` and ` 3.5 `.** A +number is printed with a sign space before it and a space after it, which `Debug.md:40` states; +the evaluator never opened that page. Its generic `Max(Of T)` compares with `>`, which it +doubted would compile; it does, for **Long**, **Double** and **String** alike (finding 6). + +## Tier 1 --- the documentation says something the product does not do + +**1. `Tutorials/Windows-API.md:185-189`** declared `GetWindowTextW`'s text pointer as `ByVal +lpString As Long`. Line 155 of the same page gives **LongPtr** for `LPWSTR`, and line 160 says to +"always use `LongPtr` for handle and pointer parameters", because a `Long` "fails or crashes in +64-bit mode". Found while checking UC-64's answer against the tutorial; the evaluator used +`DeclareWide` instead and never met it. The harness builds only 32-bit, where a `Long` holds a +pointer, so the 64-bit failure was not measured; the corrected declaration was compiled and run. +*Fixed.* + +## Tier 2 --- hazards the pages do not know + +**2. A second export needs `--overwrite`, and the Git section never said so.** Its steps give the +`import` command in full and no `export` command at all, so UC-55's evaluator took the one in the +usage block, which has no `--overwrite`, and told the reader to export again after each change. +Measured with the tB executable: into a folder that holds an earlier export, `export` without +`--overwrite` prints `[EXPORT] ERROR: output file already exists and --overwrite not set` for +each existing file, writes the files that are new, ends `... FAILED` and exits 0. The file edited +in the project kept its old contents; with `--overwrite` the same export ends `... DONE` and +updates it. The product side --- a refused export that still writes part of the tree --- was +already queued. *Fixed*: step 2 now gives the command, with `--overwrite` and the `... DONE` +check. + +**3. The IDE's Export Project cannot be packed back into a project by the tB executable.** Its +export holds a `Packages` folder with the compiler packages --- `VB`, `VBA`, `VBRUN` and +`AppGlobalClassProject` for a project with the default references, 475 of the 477 files round 8's +export of a two-file project wrote. A `.twinproj` the IDE saved does not hold them: the tB +executable's `export` of one writes only the packages the project embeds. And the tB executable's +`import` stops at the first folder under `Packages\`, writes nothing and exits 999 --- exit 231 +in Git Bash, which reports it modulo 256 --- so it can never rebuild a project from the IDE's +export. The standalone script can, into a 4,220,723-byte project that embeds its own copy of the +four packages, against 2,055 bytes for the same export with `Packages` removed; that project +compiles. UC-62's evaluator set up *Export After Save* into a `src` folder correctly --- never the +repository's top folder --- and then told the reader to rebuild "with the separate tB executable +or Node/Python script". No page connected the two facts, which were on different pages. *Fixed* +in *Export Project* and in the Git section's warning; queued, with a cross-reference from the 999 +entry. + +## Tier 3 --- the answer exists and the reader cannot reach it, or it does not exist + +**4. The Windows API tutorial never showed a function that returns a string.** Most do it the +same way --- the caller passes a buffer and the function reports the characters it wrote --- and +UC-64's goal needed two. The tutorial's only string example was the `GetWindowTextW` +declaration of finding 1, with no call. The evaluator composed its buffers from a `wsprintfW` +example on the API Declarations page, and `GetComputerName` and `GetWindowsDirectory` returned +no search result at all. WinDevLib, which declares common APIs ready-made, is linked from the +64-bit and Windowless pages but not from the tutorial. *Fixed*: a `##` section, *Functions that +return a string*, with both usual ways the length comes back, executed; WinDevLib under *Where to +go next*. Both API names and `WinDevLib` now rank 1. + +**5. The error-number table was not found by the name of the error.** A porter whose handler +says `Case 9 ' subscript out of range` searches for that: `subscript out of range` ranked the +table 23rd, behind scroll-bar *range* pages, and `error 9` 17th. UC-63 found it with `divide by +zero error number`, a query about a different error. Eight new headings were tried against the +real index and query logic, by swapping the entry's title and rebuilding the index in memory, +before one was chosen; the ranks below are the rebuilt site's: + +| query | *Error numbers that differ from VBA* | *Error 9, Subscript out of range: numbers that differ from VBA* | +|---|---:|---:| +| `subscript out of range` | 23 | **1** | +| `error 9` | 17 | **4** | +| `Err.Number 9` | 2 | **1** | +| `error numbers VBA` | 1 | 3 | +| `error numbers differ` | 1 | 4 | +| `Err.Number different from VBA` | 1 | 3 | +| `porting error handling from VBA` | 2 | 10 | +| `divide by zero error number` | 3 | 14 | + +A longer title costs the queries that match only its old words, so the change trades the second +half of the table for the first. The anchor is kept with an explicit id, since five pages link +to it. *On Error*, the first page a porter reads, pointed nowhere near the table, and now has a +note; it ranks 8th for `porting error handling from VBA`. *Fixed.* + +**6. Generics: the body is compiled for each type, and no page said so.** UC-65's evaluator +doubted that `>` on a type parameter would compile, and the page gives nothing to settle it. It +does; and a type the body cannot handle is a compile error reported **in the body**, not at the +call: `Max(Of Collection)(c1, c2)` fails with TB5092, *Missing argument 'Index'*, twice, at the +`>` --- the message comes from `Collection`'s default member, `Item`. Nothing names the type or +the call. *Fixed* with an executed example; the diagnostic is queued. + +**7. An override is final unless it is also `Overridable`.** UC-59's evaluator asked why the +page's `Dog` override repeats `Overridable` and `GuardDog`'s does not. Measured: a class that +overrides `Cat`'s `GetSound`, which is not marked `Overridable`, fails TB5068, *procedure is not +marked as Overridable*. The rule was on the `Sub`, `Function` and `Property` pages and not on the +Inheritance page the evaluator read. *Fixed.* + +**8. The Features *Debugging* page did not point to the Debug menu**, where stepping and +run-time errors are covered; UC-61's evaluator noted that a reader searching for debugging lands +there and finds neither. *Fixed* with one sentence. + +**9. Left open.** Three phrasings of referencing a package --- `reference a package in my +project`, `add a package reference`, `use a package in my project` --- miss, with the Add-Ins +pages on top; `library references` ranks Project Settings' section first, which points to the +Packages pages. And round 8's `webfont`, `self-hosted font` and `dark mode CSS variable` still +miss. + +## The harness + +**10. The corpus kept the checkout's CRLF line endings**, so a permalink grep anchored with `$` +matched nothing, and UC-63 reported a working link as broken three times. The repository stores +LF; the CR comes from a Windows checkout with `autocrlf`. *Fixed*: `build_corpus.mjs` writes +readable files with LF, through a `latin1` round trip that changes no other byte --- all 988 +checked against their sources. + +**11. The digest's search counts were wrong in three ways.** `which site-search` and `type +site-search` counted as queries; a refused call counted in the total; and the first site search +could be a refused one, which decides whether a full-text search came "before" it. *Fixed*: +only the search box in command position counts, by name or by a path to the shim; refused calls +are counted apart; the first search is the first that ran. + +**12. Navigation had no mechanical measure**, so every hop count so far was a report's or was +checked by hand. *Added*: `eval/nav_hops.mjs`. + +## Corrections --- evaluator claims amended + +**UC-55's** actionability is amended from 4 to 3: its export command has no `--overwrite`, so +every export after the first leaves changed files as they were (finding 2). + +**UC-59's** actionability is amended from 4 to 3: the code as handed over does not compile, and +its two-class variant stops after one line. The page it copied from runs as predicted. Its note +that `check_build` means the page's samples are *run* is overstated: they are compiled. + +**UC-61's** split discoverability --- search 3, navigation 4 --- is recorded as 3, because its +navigation went by directory listing; by links it is three hops. + +**UC-62** gave no actionability score; it is recorded as 3, for finding 3, and because the +answer never says where the `.twinproj` goes --- which matters, since an export into the folder +that holds it deletes it. Its split discoverability, search 2 and navigation 4, is recorded as 3. + +**UC-63's** discoverability is amended from 2 to 3: the navigation stall was its own lookup +failing (finding 10), and by links the table is two hops. Its "broken link, repeated three +times" is withdrawn. + +**UC-64's** scores are amended from 1/2/2 to 2/3/3. The signatures it could not find are +Microsoft's to document; what twinBASIC's pages owe --- the forms of `Declare` and `DeclareWide`, +**LongLong**, `Err.LastDllError` --- was there, and the code ran as handed over. Its claim that +WinDevLib is not reachable inside the corpus is right of the tutorial and wrong of the site: the +64-bit and Windowless pages link it. + +**UC-65's** scores stand. Its predicted output lacks the spaces `Debug.md:40` documents. + +## What round 9 says about the method + +**A re-run measures the evaluator; the same queries measure the fix.** The seven re-runs' +1.00 +discoverability is the evaluators' search, which varies; round 8's fourteen queries going from +two hits to eleven is the index, which does not. Both point the same way this round. The second +kind is the one to keep, and it takes a snapshot of each round's index --- round 8's was still on +disk. + +**Run what the answer hands over, and its alternatives.** UC-59's main answer failed on a +missing class and its offered variant on a `Nothing` element; the page it drew on was correct. +Round 8's UC-59 failed because of the page. Both are actionability failures, and only running +the code tells them apart. + +**The product findings came from following an answer one step further than the evaluator +went.** UC-62's answer stops at the setting; the fresh clone it implies is where the export +breaks. UC-55's stops at the first export; the second one is where the files go stale. + +**The evaluators ran on `claude-sonnet-5` through Claude Code 2.1.280**, recorded in every case's +`.meta.json`. Eleven cases cost $2.85 --- a mean of $0.26, from $0.13 (UC-59) to $0.44 (UC-63) --- +and the smoke run $0.05. + +## Method + +Corpus built at `d4b37ec` with `eval/build_corpus.mjs`: 1,267 files, 988 readable and 279 +stubbed unreadable, 262 binary omitted; `WIP.md`, the `WIP.*.md` files, `CLAUDE.md` and the +eight prior use-case reviews withheld. The corpus was built before finding 10's fix, with CRLF +line endings. The search index was snapshotted with it --- 3,807 entries, 80.3% reference, 4.1% +developer docs --- and every case queried the snapshot. The smoke run's six checks passed; the +eleven cases then ran in parallel, all through `eval/run_case.mjs`, goals verbatim from the +tables, round 1's three on the repository protocol they were first run under. + +Executed runs used `scripts/tbrun.mjs` on BETA 983 on ports 9771--9775, four at a time, with +`tbbuild` on 9776 and `examples.bat` from 9790: UC-59 as +handed over, with the page's `GuardDog`, with the `GuardDog` line dropped, and a subclass of +`Cat` overriding `GetSound`; UC-63 as ported and as given; UC-64; UC-65, and `Max(Of Collection)`. +The two examples the fix pass added were run the same way before they were written, and every +`check_build` sample on the six changed pages that have one compiles (`examples.bat`, 29 +samples). The +export measurements used `bin\twinBASIC_win32.exe` on scratch copies: a project packed from the +`console` template; round 8's IDE export and its `.twinproj`; and a copy of a project the IDE +had saved, which embeds WinDevLib. + +## What to do next + +**1. Round 10.** Re-run UC-62, UC-63, UC-64 and UC-65 against this round's fixes, and UC-55 +against the export command. Write the fresh-clone case UC-62 stopped short of: a project kept +with the IDE's *Export After Save*, rebuilt on another machine. Re-run round 8's and round 9's +queries against round 10's index, as this round did. + +**2. Still open from round 8:** the master list of run-time error numbers, the 59 empty IDE +sections, and `check_run`. + +**3. The phrasings that still miss** (finding 9). + +## Outcome + +**Every finding is fixed or queued, except the misses left open in finding 9.** The +documentation carries findings 1--8; the harness carries 10--12; and two product defects are +queued in `BUGS-TO-REPORT.md` --- the IDE's export of the compiler packages (finding 3) and the +generic body's diagnostic (finding 6) --- with a cross-reference added to the 999 entry. + +`build.bat`, `check.bat` and `test.bat` are green: 914 pages, 0 broken links and 0 integrity +findings in both real trees, 0 accessibility violations, every toolchain probe passing. The book +tree's informational broken links are 23, as before. + +The work is committed in four commits on `staging`: the harness, the defect queue, the +documentation fixes, and this review with the round's records. Nothing is pushed. diff --git a/docs/Documentation/Authoring.md b/docs/Documentation/Authoring.md index c8793734..3c754e23 100644 --- a/docs/Documentation/Authoring.md +++ b/docs/Documentation/Authoring.md @@ -261,6 +261,8 @@ Do **not** jump from `#` straight to `###`. That old "house style" --- an h1 fol The repair is a re-levelling of the whole page, not a patch to one heading. The plugin raises **every** heading of level 3 or deeper until each one sits exactly one level below the heading it belongs under, closing every gap in a single pass: `#` / `###` renders as h1 / h2, and `#` / `###` / `#####` renders as h1 / h2 / h3. Only the h1 chapters are left as they are, since a page may legitimately have several. +**Only `#` and `##` headings get an entry of their own in the site search.** The search index cuts each page at its h1 and h2 headings and gives each piece one entry, titled with its heading. A `###` or deeper heading gets no entry: its text is folded into the entry of the nearest `#` or `##` above it. So a section a reader should be able to find by searching for its subject needs a `##` heading. The index is built from the rendered page, after the normalizer has run, so on an old-style page a `###` that renders as h2 does get an entry. + ### Editing a page that still uses the old style The repair is conditional, and the condition is easy to break without noticing. **The plugin runs only on a page that uses `#` and `###` and no `##` anywhere** --- a single `##` and it does not run at all. More than four hundred pages on this site are currently in that state, so on most of them, adding one `##` section disarms the normalizer for the whole page: every `###` that was already there stops being repaired and becomes a live heading-order defect, in the same edit that added a correctly-levelled section. @@ -432,6 +434,17 @@ Diagram exports carry the font with them. The Download / Copy SVG and PNG button **Do not hand-edit a diagram's `.svg`.** It is a build artifact: the `.dot` beside it is the source, and the next build overwrites your edit. Changing the face is the edit that looks most harmless and is not --- Graphviz sizes each box to the text it measured, so a diagram whose labels are painted in a font the layout never saw has text hanging outside its boxes. `check.bat` fails on that; see [Diagrams](#diagrams) below. +## Adding a CSS rule that works in both themes +{: #css-rules } + +A style rule for something new on the site goes in `docs/_sass/custom/custom.scss`. Do not put it in the vendored theme under `builder/vendor/just-the-docs/`, and do not start a stylesheet of your own, which no page would load: everything under `docs/_sass/` is compiled into `just-the-docs-combined.css`, which every page does load. `serve.bat` rebuilds it each time you save. + +**The dark theme is a second copy of the whole theme, not a set of CSS variables.** The build compiles the theme twice and emits the dark copy inside a theme selector such as `html[data-theme="dark"]`, so every theme rule is more specific in dark mode than it is in light. A rule you write with a single class can therefore beat the theme in light mode and lose to it in dark, with no error: `.reversefootnote` did exactly that. Prefix the selector with `.main-content` --- `.main-content .reversefootnote` --- and it wins in both. [The specificity trap](Builder#the-specificity-trap) covers the cases where that is not enough. + +**Check it in both themes.** Run `serve.bat`, open a page that uses the rule, and switch themes with the theme button in the page header rather than with your operating system's setting, because the button is what exercises the `[data-theme]` rules. A rule that fails only in the dark theme is usually cosmetic, and no gate reports it. + +[Project styling](Builder#project-styling) is the full account: which file under `docs/_sass/` holds what, why the theme is compiled twice, and how to verify a style change. + ## Checking that a sample compiles Nothing in the ordinary build looks inside a code fence. The link check, the accessibility @@ -968,8 +981,11 @@ does not. It lists each page it could not place, with the reason: Nav-parent orphan detected in 12 page(s): Features/Example/Child.md: no page titled "Old Title" exists -**The match is on the parent's title, not its file.** Changing a page's `title:` leaves -every page whose `parent:` names the old title without a parent, although nothing moved. +**The match is on the parent's title, not its file, and it is exact, case included:** +`parent: Strings module` does not find the page titled `Strings Module`. Setting +`nav_sort: case_insensitive` in `_config.yml` would not change that, because the build +reads it only to order the sidebar. Changing a page's `title:` leaves every page whose +`parent:` names the old title without a parent, although nothing moved. Change each of those lines to the new title, in the same commit as the rename. One search finds them, and the `grand_parent:` lines that name it as well: diff --git a/docs/Documentation/Builder.md b/docs/Documentation/Builder.md index 078b4c44..985e33f8 100644 --- a/docs/Documentation/Builder.md +++ b/docs/Documentation/Builder.md @@ -10,7 +10,7 @@ permalink: /Documentation/Development/Builder # tbdocs Builder {: .no_toc } -Detailed technical documentation for the `tbdocs` static site generator at [`builder/`](https://github.com/twinbasic/documentation/tree/main/builder). Read this when modifying the build pipeline itself; content contributors who only need to build, preview, and ship documentation should not need any of it. +Detailed technical documentation for the `tbdocs` static site generator at [`builder/`](https://github.com/twinbasic/documentation/tree/main/builder). Read this when modifying the build pipeline itself. Content contributors who only build, preview and ship documentation need none of it, with one exception: anyone adding or changing a CSS rule needs [Project styling](#project-styling), the full account of where a rule goes and of why one that works in the light theme can silently do nothing in the dark one. Module-level documentation lives next to the code: @@ -523,6 +523,22 @@ Run `serve.bat` and look at the page **in both themes**. This is not a formality Then `build.bat && check.bat`. A malformed rule surfaces as an SCSS compile failure, which warns with the source location and flips the exit code rather than aborting --- so the previous build's CSS lingers in `_site/` and the site appears to still work; read the build output, do not judge by the page. `check.bat`'s accessibility scan covers every sample page in both themes for exactly the reason above, and its `target-size` and `color-contrast` rules are where a geometry or palette change lands. If the new component introduces markup the site has not used before, also add a construct family to `scripts/pick_a11y_sample.mjs` --- see [Tools and Scripts](Tools#pick-a11y-sample) --- or no axe rule keyed on it will run anywhere. +## Changing a typeface + +This follows on from [Project styling](#project-styling). The site uses three faces --- Inter for text, Cascadia Mono for code, and Source Serif 4 for the body text of the PDF book --- and the build names them in far more places than the stylesheet. Only some of those places fail loudly when one is missed. In the order a change would go: + +1. **The font files** (all three faces). [`scripts/build_fonts.py`](Tools#build-fonts) downloads each face's pinned release, verifies its SHA-256, subsets it, and writes the `.woff2` files and their licences into `docs/assets/fonts/`. A new face is an entry in its `SOURCES` table --- the release URL, the hash and the licence --- and one entry per file in `FACES`: the file inside the archive, the pinned axes, the Unicode ranges and the output name. Keep the output `.woff2`, which is the only font format the [publish allowlist](Building#what-the-build-refuses-to-publish) accepts. +2. **The web stylesheet** (Inter and Cascadia Mono). `docs/_sass/custom/_fonts.scss` holds the `@font-face` rules, inside the `emit-font-faces` mixin, and the `$tb-body-font-family` and `$tb-mono-font-family` stacks, which `docs/assets/css/just-the-docs-combined.scss` passes into the theme. `docs/_sass/modules-dark.scss` passes them again for the dark compilation, and `docs/assets/css/just-the-docs-dark.scss` uses the mono stack once more for `pre`, `kbd` and `samp`. All of these read the two variables, so replacing a face inside an existing stack changes nothing in them. A new stack has to be passed in both compilations, or the dark theme keeps the system fonts --- the [specificity trap](#the-specificity-trap) again. +3. **The preloads** (the two roman web faces). `fontPreloads()` in `builder/template.mjs` preloads `inter-variable.woff2` and `cascadia-mono-variable.woff2` by name, from its `PRELOAD_FONTS` list. It sets `crossorigin`, which a font preload needs even from the same origin: without it the browser downloads the file twice. A preload of a file that does not exist is a broken link on every page, and the build's link check reports it. The link-check fixture under `test/fixtures/check-src/assets/fonts/` holds stub files under the same two names, so renaming either file means renaming its stub too, or [`check_links_diff.mjs`](Tools#check-links-diff) fails on every pull request. +4. **The book** (all three faces). `docs/assets/css/print.css` is the whole design of the PDF and loads nothing from the Sass build, so it has its own `@font-face` block and its own stacks. `REQUIRED_FONTS` in `builder/pdf.mjs` names the same six files and copies them into the sparse `_site-pdf/` tree. Keep the two in step: the PDF pass aborts on a listed file that does not exist, and a face `print.css` uses that was not copied fails to load, which aborts the book render. +5. **Diagram exports** (Inter and Cascadia Mono). `FONT_FILES` in `docs/assets/js/svg-inline.js` maps each family name to its weight range and file names, and the Download and Copy buttons embed those files in an exported SVG or PNG. A family it does not list is exported without its face and without a message; a listed file that no longer exists costs one console warning. +6. **The Gantt chart** (Inter). `builder/gantt.mjs` writes a `