Skip to content

fix(conformance, look-ledger): a built row is not a regression; mark-built at module close; dropped pages are not owed (#671, #681) - #697

Merged
MendixMau merged 3 commits into
masterfrom
fix/ledger-built-671
Nov 6, 2026
Merged

MendixMau merged 3 commits into
masterfrom
fix/ledger-built-671

Conversation

@MendixMau

@MendixMau MendixMau commented Nov 6, 2026 •

Copy link
Copy Markdown
Owner

Fixes #671. Fixes #681.

Changes

#671: project-bin/conformance-check.sh

  • (a) A row that was OK and is now UNDERSTATED (stored planned/not built/blocked, element PRESENT) is now keyed as OK in the baseline. Before, every correct build of a planned row read "was OK, now UNDERSTATED" (exit 1), and --update-baseline refused the update as a loosening (exit 2).
    • The lag is now reported as a note: ledger lags: N row(s) built but marked planned — run --mark-built. It counts only planned/not built rows. A PRESENT blocked (…) row gets a separate note, because --mark-built never flips blocked rows.
    • No other regression rule changed. A built row whose element later disappears still reads OK → STALE and exits 1 (case 15).
  • (b) New --mark-built flag. It flips the status cell of each row whose element was measured PRESENT and whose stored status is planned or not built to built, in the ledger file the row came from. It handles the per-BRD, per-module, flat (<M>-coverage-ledger.md) and project-level ledger shapes.
    • Rows are matched by file and line number, and the pointer on that line is checked again. Two rows that share a pointer therefore flip independently (case 17, from the review).
    • The line is split on | and joined again with |, so no other byte changes. Backticks around a status are kept. Prose rows and blocked rows are not touched. Running it twice changes nothing the second time.
  • project-bin/module-close.sh runs conformance-check.sh --module <M> --mark-built once per module, before verify-module.sh, and copies the result line into the report. verify-module.sh still writes nothing.
  • project-bin/verify-module.sh: the comments that said "~25s per row (22 rows ≈ 9 min)" are updated with the figure measured below. Comment-only change; the line count is the same.
  • skills/iterative-build-loop.md step 16 now says that module-close does the ledger flip. The always-loaded word count is unchanged at 65053 (render-routing.sh --check).

#681: project-bin/look-ledger.sh

  • look_pages_in now reads DROP PAGE [IF EXISTS] <name>. When a script has several statements on one page, the last one decides.
  • owe removes the owed.tsv rows of every page that a script ends by dropping, so the gate-check Stage 5 check no longer counts it. A later create page appends a new row, and the page is owed again.
  • RENAME is not covered. There is no captured MDL rename page statement to test against.

Time cost (#687)

  • Granularity: --mark-built runs once per module, inside module-close, not once per row.
  • Measured on 2026-11-06 with a 22-row module ledger and a stub mxcli, which matches the module size quoted in verify-module's old comment:
    • 22.7 s with the stub answering instantly.
    • 22.7 s with the stub sleeping 0.144 s per probe, the per-DESCRIBE time measured in the field and quoted in conformance-check.sh's header.
    • That is about 1 s per row. Almost all of it is the probe watchdog's 1 s poll: its sleep 1 holds the $(…) pipe open until it ends.
  • The pass removes the per-row false reds (UNDERSTATED read as a regression) and the hand edit of each status cell.
  • The DROP handling adds one awk pass per exec that drops a page. Its cost is not measurable.
  • Follow-up, not in this PR: redirecting the watchdog subshell's output to /dev/null in run_guarded brought the same 22-row pass from 22.7 s to 4.0 s in a local try. It would speed up every conformance run, so it should be its own change with its own review.

Regression risk after sync-project.sh

  • Verdicts that move from FINDING to PASS:
    • A conformance run that exits 1 only because rows went from OK to UNDERSTATED now exits 0 and prints the note.
    • A --update-baseline refused only because of UNDERSTATED rows now goes through, and those rows are written as OK.
    • No verdict moves from PASS to FAIL.
  • Baseline rows already stored as UNDERSTATED stay that way until they are rewritten. That is no looser than before.
  • Stage 5 look: goes from FAIL to MANUAL for projects whose only unseen pages were dropped, but only once a new script drops them. owed.tsv rows for pages dropped earlier stay until then, or until removed by hand as the look ledger keeps owing pages that were dropped, gate counts them as never looked at #681 workaround describes.
  • The --mark-built write is opt-in. It changes only status cells in the ledger, never the .mpr. module-close.sh ignores its exit code.
  • Both layouts: paths still resolve through _common.sh. Only bash 3.2-safe constructs are used, and check-portability is clean.
  • Fixtures were updated as listed below. No existing assertion was changed.

Proof

  • tests/wave2/test-conformance-baseline-scope.sh, cases 9–17b: 21/21 on this branch.
    • Against master's script, 8 fail.
    • Against the previous PR head (8ec6398), the 4 cases added in review (10b, 14b, 17, 17b) fail.
  • tests/wave2/test-look-ledger.sh, D1–D5: 26/26 on this branch. Against master, D1, D2 and D4 fail; D4 is the gate-check Stage 5 FAIL from look ledger keeps owing pages that were dropped, gate counts them as never looked at #681.
  • Other scoped fixtures, all green: test-module-close 10/10, test-conformance-absent-marker 24/24, test-conformance-per-brd-ledger 20/20, test-bug09-13 22/22 (the skill), test-docs-numbering 18/18.
  • check-scripts, check-portability and check-pr-discipline are clean. The leak guard is clean; its only warning is the new capitalised word "DROPs". render-routing --check is in sync. No full suites were run.

Review: pending independent review

🤖 Generated with Claude Code

…built at module close; dropped pages are not owed (#671, #681)

#671: conformance-check.sh keyed a PRESENT `planned` row as UNDERSTATED, so every correct
build read "was OK, now UNDERSTATED" (exit 1) and --update-baseline was refused as a
loosening (exit 2). UNDERSTATED is now keyed OK in the baseline and reported as a note
("ledger lags: N row(s) built but marked planned — run --mark-built"); OK -> STALE still
fails. New --mark-built flips status cells planned/not built -> built for rows measured
PRESENT, in per-BRD, per-module, flat and project ledgers; idempotent, status cell only,
blocked rows untouched. module-close.sh runs it once per module; iterative-build-loop.md
step 16 updated, word-neutral (baseline 65053 -> 65053).

#681: look-ledger.sh owe reads DROP PAGE [IF EXISTS]; the last statement on a page in a
script decides, dropped pages' owed.tsv rows are removed, a re-create owes again. RENAME
not handled: no captured MDL statement shape.

Fixtures: test-conformance-baseline-scope.sh 9-16 (8 fail on master), test-look-ledger.sh
D1-D5 (3 fail on master). Field run: the #671 / #681 repros (mxcli nightly-20261105,
Mendix 11.15.0) as reported.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013jQvaonym9V1wKKfAEPmRr
…ows get their own note (PR #697 review)

- --mark-built matched by pointer + file, so two rows of one ledger sharing a pointer (one
  PRESENT, one ABSENT) both flipped to built and the next run exited 1 with STALE. extract now
  carries FNR; the rewrite matches file + line and re-checks the pointer on that line.
- The "ledger lags" note counted PRESENT `blocked (...)` rows that --mark-built never flips, so
  it never went away. It now counts planned/not built only; blocked rows get their own note.
- verify-module.sh: the "~25s per row (22 rows ~ 9 min)" comments predate the 2026-09-02
  watchdog fix. Measured 2026-11-06: 22.7s for a 22-row ledger (~1s per row).
- Fixture cases 10b, 14b, 17, 17b (5 fail on the previous PR head).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013jQvaonym9V1wKKfAEPmRr
Keep both sides of CHANGELOG.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013jQvaonym9V1wKKfAEPmRr
@MendixMau
MendixMau merged commit 2a68b63 into master Nov 6, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants