Repository navigation
fix(named-controls, monkey): exit 2 means not measured, with the cause on the row (#660, #679) - #694
Merged
Merged
Conversation
…e on the row (#660, #679) named-controls.js: pipe and replay the runner's stdout/stderr, keep its last INSTRUMENT FAULT / SETUP line, score runner exit 2 as FAULT with that line, and exit 2 when any row is FAULT (1 = NOT PROVEN, 0 = all PROVEN). monkey.js: exit 2 before launching a browser when project.config.js's NAV still equals the template sample (config.js TEMPLATE_NAV / navIsTemplate(), ALLOW_TEMPLATE_NAV=1 overrides), and exit 2 when 0 of N targets were reached. verify-module.sh maps rc 2 to FAULT, so a stale NAV now reads INCOMPLETE. Fixtures: test-named-controls.sh (EXIT2 -> FAULT, cause line, CLI exit codes, real-runner credentials abort), new test-monkey-template-nav.sh, and test-monkey-oracle.sh's exit-code assertion follows exitCode(). Field run: the scratch-project repros in #660/#679 (template NAV, TEST_PASS unset). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jQvaonym9V1wKKfAEPmRr
This was referenced Nov 6, 2026
Keep both sides of CHANGELOG.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013jQvaonym9V1wKKfAEPmRr
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
#660 —
project-tests/e2e/named-controls.jsstdio: ['inherit','pipe','pipe']. Its output is replayed to the caller after it exits. SETUP lines go to stdout and INSTRUMENT FAULT to stderr, so both streams are piped.faultLine()keeps the last line that starts withINSTRUMENT FAULTorSETUP. The match is anchored, so the summary text "Fix the SETUP line(s) above" is not taken as the cause.status === 2gives aFAULTrow whose text includes that line, whether or not a findings file was written.judge(2, …)returns FAULT too.#679 —
project-tests/e2e/monkey.js,config.jsconfig.jsnow hasTEMPLATE_NAVandnavIsTemplate(), the same pattern asTEMPLATE_PASSWORDS. The comparison ignores key order, andALLOW_TEMPLATE_NAV=1overrides it.SETUP NAV is the unedited sample …line and exits 2 beforeH.launchBrowser(). No browser starts and no login is attempted.exitCode()returns 1 on a crash-class finding, 2 when 0 of N targets were reached (this used to exit 0), and 0 otherwise. Code 3 still means an unknown--target.Fixtures
tests/wave2/test-named-controls.sh: the EXIT2 case now expects FAULT. New cases cover:journey-runner.jsaborting on credentials, with the SETUP cause on the NAMED CONTROLS line.tests/wave2/test-monkey-template-nav.sh:navIsTemplate()returns true for the template and false for an edited NAV and for the override;TEMPLATE_NAVmatches the template's NAV (drift guard);exitCode()table;page.goto/login output appears.tests/wave2/test-monkey-oracle.sh: the monkey.js scores "spinner still up after 8s" as crash-class, but monkey-test.md defines a crash as only a page exception, an error page or dialog, a 5xx, or an ERROR span: a legitimately slow action (a ~10 s batch job) is reported as a crash #562 assertion ("exit code counts crash-class only") now acceptsexitCode({ crashes: crashes.length, … }).Proof (scoped runs only)
page.gotoin login)Repo checks: check-scripts, check-portability, leak guard (
LEAKGUARD_ALLOW_NO_DENYLIST=1 LEAKGUARD_BASE=origin/master) and check-pr-discipline are all clean.Time cost (#687)
The new checks run per module run, inside existing rungs. Both make runs faster:
Regression risk after
sync-project.shmonkey.jsexits 2 (it used to exit 0) when NAV equals the template sample, or when 0 of N targets were reached.named-controls.jsexits 2 (it used to exit 1) when any row is FAULT. That includes the existing persona-mismatch and not-*.control.jsonrows.verify-module.shmaps rc 2 to FAULT (*:2sits beforeinfo:*). A project still on the sample NAV, or whose NAV reaches no menu item, now shows the monkey rung as INSTRUMENT FAULT, and the module verdict becomes INCOMPLETE where it was green. This is intended: that run measured nothing. It is named in the CHANGELOG line.ALLOW_TEMPLATE_NAV=1covers an app whose menu really is the sample's.named-controls.jsinbin/orproject-bin/parses its exit code; it is run by hand perskills/journey-proof.md.process.execPathis still spawned directly, with no bare wrapper name.Fixes #660
Fixes #679
Review: pending independent review
Generated by Claude Code