Attribute sweep, compileProject, and a canary for check_examples

.gitattributes

@@ -3,3 +3,8 @@
 # Windows checkout, say -- hands the hook to the kernel, which reads the
 # carriage return after #!/bin/sh as part of the interpreter's name.
 .githooks/* text eol=lf
+
+# The Workflow tool refuses a script that contains a carriage return, and the
+# Wisdom extract script is passed to it by path, so a CRLF checkout of it
+# cannot be run.
+wisdom/extract/workflow.mjs text eol=lf

.github/actions/run-gates/action.yml

@@ -151,6 +151,15 @@ runs:
     - name: Verify the twinBASIC source and attribute scanners (check_twin_parsers.mjs)
       shell: bash
       run: node scripts/check_twin_parsers.mjs
+    # The attribute sweep asks the compiler where every attribute is legal, and
+    # none of its failures announces itself: a wrong site skeleton reads as
+    # "every attribute is refused here", an ignored control as a recognised
+    # attribute, a refusal taken for an acceptance as a finding about the
+    # compiler. The IDE is what cannot run here, so the parts that decide what
+    # an answer means are probed on fixed inputs: no IDE, no tree, no install.
+    - name: Verify the attribute sweep's logic (check_attribute_sweep.mjs)
+      shell: bash
+      run: node scripts/check_attribute_sweep.mjs
     # Nothing else tests how a tool reads its command line, which is how a
     # value flag given no value came to be read as NaN or as the next flag.
     # lib/cli.mjs's probes, then each tool's recorded command-line errors, each

BUGS-TO-REPORT.md

@@ -1253,3 +1253,45 @@ IDE's add-in samples leaves the id out.
 
 **Observed** on 2026-09-25 with the panes probe's third button, operated by
 `test/addin/panes.test.mjs`, which reads `toolWindowsById` over CDP.
+
+## `[PopulateFrom]` with no arguments crashes the compiler
+
+**Build:** BETA 987
+**Severity:** the compiler process dies while the project is being parsed, which
+`tbbuild` reports as a crash (its exit code 4), so a person who forgets the arguments is not
+told what is missing.
+
+The whole reproduction is one Enum:
+
+```
+Public Module CrashProbe
+    [PopulateFrom]
+    Public Enum E
+    End Enum
+End Module
+```
+
+The Enum's body does not matter: the same crash comes with a member in it, and with the
+Enum inside a Class instead of a Module. The documented shape is five strings,
+`("json", "/Resources/PROBE/Strings.json", "events", "name", "id")`, and the other wrong
+shapes tried are handled:
+
+| argument list | result |
+|---|---|
+| none, `[PopulateFrom]` | **the compiler crashes** |
+| `(True)`, `(False)`, `(1)` | TB5155 `This attribute is not supported in this context` |
+| `("probe")`, on the reproduction above | TB5083 `unsupported data source` |
+| the documented five strings, with a resource that exists | compiles |
+
+So a missing argument list is the one wrong shape that is not checked.
+
+**Observed** on 2026-09-30 in two ways. `scripts/sweep_attributes.mjs` builds every attribute
+at every declaration site in batches of 400 and halves a batch the compiler crashes on; each
+of the three Enum sites (an Enum with a member, an empty Enum, an Enum in a Class) was
+narrowed to one probe beside the three canaries the tool adds to every batch, which build
+clean without it. The four-line reproduction above was then built **exactly as written**, in
+a project holding only it and a two-line `Sub Main`, with no resources: `tbbuild` exits 4,
+`the compiler crashed 2x -- this project takes it down`, `last parsing: CrashProbe.twin`.
+The same project with `[PopulateFrom("probe")]` builds and reports the one TB5083 row. The
+rows for `(True)`, `(False)` and `(1)` come from the sweep's batches, not from that
+project.

WIP.ExamplesBuild.md

@@ -408,6 +408,77 @@ Two things a batch runner must do that a single-fence runner need not:
   the same finding and the other eight still reporting. Until then the name went unread ---
   `buildStaged` kept `tbbuild`'s report and nothing looked at it --- so every crash paid for
   the whole bisect.
+- **A batch that reports nothing is not a clean batch: a canary rides in every one.**
+  `waitForCompile` reads the IDE's window --- the status counters and the Problems panel of
+  the open project --- once the compiler status is OPERATIONAL and five one-second samples
+  match. It waits for no build. An IDE under load can sit OPERATIONAL with an empty panel
+  before it has published anything, and every sample of that batch then reads as compiling,
+  with nothing to tell it from a batch that has no errors. `sweep_attributes` met it first,
+  because its canaries exist to: one build in 136 of its last full run drew `ACCEPT` for all
+  three of them, including the invented name that can only ever draw an error, while
+  `build.bat`, `check.bat` and `test.bat` ran on the same machine (a coincidence, not a
+  measurement of the cause; none of the earlier full runs had one). `check_examples` had no
+  such protection, so that read would have passed every sample of its batch. Each batch now
+  carries `tbxCanary`, a module with `[EnforceWarnings(TB0005)]` and a `#Warning` directive,
+  which must draw the warning TB0005. Its rows are taken out, at any severity, before
+  anything else reads them.
+
+  **It is a warning, and enforced, because that measured robust and leaves a clean batch
+  clean** (BETA 987, `canary-warn.mjs`, ten cases). TB0005 appeared beside an unterminated
+  `Sub`, an `If` with no `End If`, a stray `End Sub` with garbage after it, a class inheriting
+  itself with an unknown type, ten undefined names, under Option Explicit off, and beside a
+  module with `[IgnoreWarnings(TB0005)]`. A plain `#Warning` vanished in a project that
+  ignores TB0005 and became an ERROR in one that promotes it; the enforced form stayed a
+  warning in both. No template changes `project.warnings` today (the defaults leave every
+  warning on), so the attribute is insurance, not a fix. An error canary put an error in
+  every batch, so `tbbuild` never exited 0; with the warning a clean batch does.
+
+  The history: the first version carried two error canaries, an unknown attribute (TB5182)
+  and an undefined symbol (TB5079). The second is only the warning TB0002 under **Option
+  Explicit off**, where the name is declared implicitly, and stopped a full run on
+  `Core/Deftype.md:56`, a sample that compiles, in the `[implicit]` template. The next version
+  kept TB5182 alone; the warning replaced it at the user's suggestion. A batch that crashes the
+  compiler returns no rows at all, canary included; `runBatch` hands a crash to
+  `isolateCrash` before it reads the canary, so that case never reaches it.
+
+  **Measured with real failures** (a temporary page of 14 samples, deleted after): ten
+  failing samples across the console and `[implicit]` templates, among them an unterminated
+  `Sub`, garbage after a stray `End Sub` and an `If` with no `End If`, were each reported
+  with their own errors, with no canary event and no split, in about 13 s --- with the TB5182
+  canary and again with the warning, the same result both times.
+
+  **The canary is required only by a read with no errors in it** (the user's rule). It
+  proves that the IDE published something, never that it published everything --- a read
+  that includes it and misses a later file's diagnostics would still pass that file --- so a
+  read holding real errors has already shown what the canary would, and is taken as read:
+  no rebuild, no split, no stop. `heard()` decides what is real: an ERROR in one of the
+  batch's samples, or an unattributed row the template does not draw by itself. **The
+  template's own rows do not count**, or a template that always draws one would switch the
+  canary off for every batch of it; an earlier version let an early read holding only such
+  a row pass the sample (found by review). A canary missing *beside* real errors is printed
+  as a note and not acted on: it has never been seen, and would be the first sign of a read
+  that holds some files and not others.
+
+  A silent read (no errors, no canary) is built once more; then split, as for a crash, until
+  each part reports the canary or errors of its own. A unit that is still silent stops the
+  run (exit 2, its name) without being blamed, since its clean may be false and a finding
+  would claim something about its code that a build with no diagnostics cannot. A
+  `projname` group is one program and cannot be split, and needs no splitting: if any member
+  fails, the read was not silent. This is what keeps a corpus that legitimately fails (a
+  `--propose` survey) from ever being stopped or slowed by the canary; the version before
+  it stopped the run on a group with only some members failing.
+
+  `ownRowsOf`'s empty-template build follows the same rule: a silent read is read again and
+  refused when silent twice, and a read with rows of its own needs no canary. It is
+  memoised for the whole run, and a silent read there believed would leave `own` empty, so
+  every batch with a template-own row would bisect to single samples and blame each one.
+
+  Tested against a fake lane: an early read answered by one build; a sample that hides it,
+  with no errors of its own, found and named; the same sample with errors of its own taken as
+  read in one build, with the note; a group with only some members failing taken as read in
+  one build; the hiding sample with only the template's own row named; two that hide it only
+  together separated by halving; a batch that never reports; the empty template read again,
+  refused when it never reports, and believed in one read when it has rows of its own.
 - **A crash that needs two samples used to vanish.** Halving separates any pair by the time
   it reaches single samples; both halves then build clean, and every sample in the batch
   counted as compiling --- the false clean that the crash check exists to prevent, one level

WIP.Harness.md

@@ -143,6 +143,75 @@ confusion publishes a wrong number with nothing to notice it by. The bar is that
 report's unresolved count is **0**, which it currently is; a non-zero one is a scanner bug,
 not a corpus oddity.
 
+## Sweeping every attribute at every site
+
+A census says where the packages *use* an attribute, and `gen_attribute_probes.mjs` probes
+only the targets `Attributes.md` already claims, so an entry that was too short stayed too
+short: `[ComExport]` was documented as "constants in a Module" because a Sub and a Const were
+the two targets tried, and an API `Declare` never was.
+[scripts/sweep_attributes.mjs](scripts/sweep_attributes.mjs) asks every question --- every
+name (the page's, the compiler's token table's, `--names`) at each of about 60 sites in
+[scripts/lib/attribute-sites.mjs](scripts/lib/attribute-sites.mjs), in each argument shape ---
+and lays the answers against the page. Its reader-facing description is
+[Tools and Scripts](docs/Documentation/Tools.md#sweep-attributes); this is why it is built as
+it is.
+
+**Against BETA 987: 34,526 probes in 134 builds, 8 to 16 minutes on four lanes** (three runs
+took 667, 946 and 489 seconds; the middle one carried a stage 2 inflated by a noise site since
+removed). That is the whole matrix, and nothing in it was slow enough to need the sampling the
+design first allowed for. **A run of one name is a minute or a few, not always one:** an
+attribute the compiler allows once per project (`[RunAfterBuild]`, `[RunBeforeStartupObject]`)
+goes in one probe per batch, so it needs about a hundred builds by itself.
+
+**The token table is a string, and it holds the names you cannot guess.** The compiler binary
+has one 2,496-character run of 261 pipe-separated identifiers, from `On|Off|Explicit` to
+`UserDefinedTypeIsAnAlias`, with `DllExport|ComExport` in the middle of it; it is where
+`ComExport` was first seen. It mixes
+keywords, attributes and object members (`Debug`, `Circle`, `PSet`), so a name in it is only a
+candidate. Swept, exactly one name the page does not document was accepted anywhere:
+`[PropertyPage]`, at eighteen sites, all of them members of a Class or an Interface. The
+other 193 were refused at every site.
+
+**Six things the sweep had to learn, each of which produced a wrong report first**, found by
+the Opus review that was run over the tool and by its own first output:
+
+- **`parseCli` camel-cases its keys.** `values["dry-run"]` is `undefined`, so `--dry-run` was
+  ignored and the first "dry run" was a full four-lane sweep. Read `values.dryRun`; a new
+  tool that takes a hyphenated option should be run once with each of them before it is
+  trusted.
+- **A control must be refused for a site to mean anything.** An unknown name is built at every
+  site, and a site whose control *compiles* is voided. An Enum body accepts any own-line
+  `[...]` --- `[ClassId("guid")]` and `[Hidden(True)]` included, neither of which can be a
+  member name --- so what it does with the line is unchecked, and nothing accepted there is
+  evidence. Inline, `[Name] X = 1`, the compiler refuses every attribute, `[Hidden]` included,
+  though the page documents it on an Enum member. An Enum member target is therefore reported
+  as one the sweep cannot test, which is true. (The mechanism is not established; only the
+  behaviour was measured.) A local variable is different: an own-line `[Name]` in a Sub is a
+  *call* statement, so that variant was dropped and only the inline one is kept.
+- **A probe that draws what the control draws is a refusal, whatever the code.** The signature
+  compared has the attribute's name (whole word, any case) and the probe's own generated names
+  (`S000062`, `S000062_U`) taken out, or the control's message never equals the probe's.
+- **What a canary must draw is fixed in the script.** Reading it from the preflight lets a
+  build that contains the very masking the canaries exist to catch calibrate the check to it.
+  The tool builds the canaries alone first and stops unless they draw what is recorded.
+- **Batch by shuffle, and one probe per singleton.** `[RunAfterBuild]` is once per project
+  (TB5114), read as acceptance if a second reaches the same batch, and `[PopulateFrom]` fills
+  an enum with the same two members every time, and enum members are project-global. Member
+  names are unique per probe (`Probe` becomes `S000062_m`) for the same reason.
+- **A form nobody built cannot be refused.** A cell whose other forms were refused and whose
+  one form able to pass never compiled is inconclusive, not refused, and the same holds for a
+  run cut short.
+
+**It found a compiler crash the documentation never would:** a bare `[PopulateFrom]` on an
+Enum kills the compiler, at all three Enum sites; the tool isolated it by halving to one
+probe beside the canaries. It is in [BUGS-TO-REPORT.md](BUGS-TO-REPORT.md).
+
+**Read the report's "accepted at most sites" section before believing an acceptance.**
+`[Description]` is taken at 53 of 61 sites and `[Hidden]` and `[Restricted]` at 42: either
+they apply nearly everywhere or the compiler tolerates what it does not check. The sweep
+cannot say which, and neither can a clean build, which is also why *accepted but not
+documented* is a list to read and not a list to copy into the page.
+
 ## Compiling a twinBASIC project without the IDE in front of you
 
 Exported sources say what the compiler *accepts today*; they cannot answer a question no
@@ -190,6 +259,34 @@ on it. Moving the code there was checked against 14 fixture cases run before and
 every exit code and every line of output the same, apart from the two fixes below --- and
 against a full `examples.bat` run.
 
+**A build is also a function, [scripts/lib/tb-build.mjs](scripts/lib/tb-build.mjs)'s
+`compileProject`**, which is `tbbuild` without its command line and returns
+`{code, message, rows, counts, dialogs, crashFiles, ...}`. `check_examples` and
+`sweep_attributes` used to start `tbbuild` as a subprocess and read its JSON and its stderr
+back, each with its own copy of that parse, and `check_examples` took the crashed files out of a
+regex over the stderr text. The cost of the subprocess was never speed (about 100 ms against an
+IDE start of about 10 s); it was that parse, and that a harness failure and a compile error
+both arrived as an exit code. Moving `tbbuild` onto the function was checked the same way as the
+move above: seven cases (clean, errors, `--json`, warnings only, a compiler crash, `--arch
+win64`, a relative path) run before and after with every stdout, stderr and exit code
+identical, and a full `check_examples` run (1,135 samples, no findings; the old code took 174 s
+for 1,134 and the new 145 s, on a machine that was not quiet, so no speedup is claimed).
+
+Running in one process changes four things, each of which is a rule now:
+
+- **The function never tidies the registry and never exits.** The caller owns both; a tool that
+  builds many projects calls `startTidy` once, and `tbbuild` does it for its one.
+- **It ends its IDE with `shutdownIdeAsync`.** `shutdownIde`'s `taskkill` and its wait hold the
+  event loop still (up to five seconds), which with four lanes lets the others' CDP timers run
+  out with their answers unread in a socket. The sync version stays for the paths that end in
+  `process.exit()`.
+- **An uncaught exception ends every lane's work, not one child's.** `exitOnCrash(cleanup)` runs
+  a cleanup first; `sweep_attributes` uses it to write the report of what it had learned
+  (`salvage`), and `tb-cdp` drops a frame that is not JSON instead of throwing from an event
+  callback.
+- **The name.** tb-ide already exports a `buildProject(c)` that builds an exe through an open
+  connection, and the notes below mean that one; the new function is `compileProject`.
+
 Two bugs came out of the move, and neither had been noticed:
 
 - **A relative project path never loaded.** The IDE is given the resolved path and echoes
@@ -604,11 +701,12 @@ leave everything as it was found**, and it takes four forms:
   `JSON.stringify`, which is how the IDE writes it, so the other entries keep their exact
   text and order, and the write is refused if the value changed after it was read.
 
-**One process owns the registry per run.** `check_examples` runs four lanes of `tbbuild`
-children at once; each restoring its own snapshot would put back whatever the registry held
-when that lane started, in whatever order the lanes finished. `startTidy` sets
-`TB_REGISTRY_OWNER`, the children inherit it and leave the registry alone, and the owner
-sweeps once after the last lane. An owner pid that is no longer running does not count, or a
+**One process owns the registry per run.** `check_examples` and `sweep_attributes` run four
+lanes of builds at once, in one process (`compileProject`, which never tidies); each
+restoring its own snapshot would put back whatever the registry held when that lane started,
+in whatever order the lanes finished. The tool calls `startTidy` once, which sets
+`TB_REGISTRY_OWNER` --- a `tbbuild` started from that process would inherit it and leave the
+registry alone --- and sweeps once after the last lane. An owner pid that is no longer running does not count, or a
 variable left set in a shell would switch tidying off for good. Under `--keep` nothing is
 tidied, because the kept IDE is still writing. `shutdownIde` waits for the IDE's process to
 be gone before anything is tidied, because `taskkill` only asks.
@@ -789,9 +887,10 @@ WebView2 processes and two console hosts. WebView2 runs inside the job without c
 the children it spawns into a kill-on-close job of its own, so when the Node process ends,
 the launcher ends with it, closing the IDE's job. Normally that is exactly what is wanted:
 killing only `tbbuild` in the middle of a compile now takes its whole IDE down with it,
-where before the IDE lived on, on a desktop nobody could see. `check_examples` should get
-the same protection one level up, since its `tbbuild` children die with it by the same
-mechanism; that step has not been measured separately.
+where before the IDE lived on, on a desktop nobody could see. `check_examples` and
+`sweep_attributes` build in their own process (`compileProject`), so their IDEs are
+launched by it and go when it does, by the same mechanism; that has not been measured
+separately.
 
 **A kept IDE is the exception, and it gets no job.** Both other arrangements were tried, and
 both fail: