[johnml1135]

Summary

This adds a conformance suite of synthetic HermitCrab grammars, with CI gates for parse results, coverage claims, and generated ledgers. This revision adds named parser-settings variants. The runner creates a fresh morpher for each run, applies its settings from constructor defaults, and compares complete parse-identity multisets and statuses against that run's expectations. A variant supplies the same ordered words as the default, with complete expectations of its own. Diagnostics name the variant; variants add no fixture or semantic-coverage credit.

Supported settings are DelReapps, MaxRoots, MakePartialsFinal, MergeAnalyses, and explicit per-call GuessRoots. The main review risk is accidentally claiming support for a setting an adapter cannot apply, or treating incomplete output as a valid empty result. Unsupported settings, missing rows, and zero-comparison variants fail; PanGloss also rejects capped or timed-out output.

• PROTOCOL.md, the schema, and WordsYamlLoader: closed settings and strict structure; the loader also checks name uniqueness and ordered inputs.
• WordsYamlMorpherSettingsTests: malformed and mismatched inputs, wrong variant expectations, and variants whose guesses leave no comparable rows.
• Five selected fixtures exercise all supported settings; each engine compares all expected identities and statuses for every run.
• Coverage credit and manifest fixture counts remain independent of variant runs.

No parser engine changed. MaxAlternatives remains unsupported because PanGloss does not implement HC's typed overflow behavior; MergeMSAs remains unsupported because it requires the FieldWorks project loader. MergeAnalyses currently verifies output preservation only; it does not measure merge-fold counts. The new C# and PanGloss contract is documented in Machine's variant protocol.

The PR targets master; review freshly fetched origin/master at 2f70c04698e13f7cf942e4245ae20d77de19ee5a (merge base a2f80e28debaffda8a65b3b11875c80479768bc7). Head: 184bcd46622571a06f1a1fdbdff2fb155714a1c6. The companion PanGloss adapter tested at e2627c9 is on feat/conformance-settings-variants.

Validation at the pinned head: local_check.sh --agent-strict passed; Release had 0 warnings/errors; HermitCrab 634 passed, SIL.Machine 887 passed / 3 skipped, Thot 93 passed, and SentencePiece 4 passed. All 72 C# runs across 62 fixtures passed. The selected five-fixture matrix passed 15/15 runs and 81 word-run rows in both C# HC and PanGloss. PanGloss harness tests: 53 passed; pinned parser tests: 117 passed / 10 marked skipped. The all-scope pin gate passed across 130 fixtures, 1,414 default words, and 10 variants, with three existing pathological/crash exclusions. Broader Rust workspace and Foma suites were not run; production parser engines are unchanged.

Oracle observations and verification detail

C# HC observations were collected on 2026-10-22 with harness draft 1836d23ccb86f1e9da123a43feb661ea76e85d0b, atop the then-current PR #480 head 72dbd4f9bd686d1193d11cf8f908d50ffa878ba1; the HC engine was unchanged. The assembly SHA-256 was 2547068882BEC56873F0678922D11D7F81ED4D68B3B99DCDB71EBF5863714444. The five measured fixture/settings pairs are deletion reapplications (DelReapps), compounding breadth (MaxRoots), final-template partial discriminators (MakePartialsFinal), final-template merge key (MergeAnalyses), and guess-roots settings (GuessRoots). Exact witness outputs and reproduction commands are in the durable observations.

The adapter matrix used the same five fixtures, each with its default plus two variants: 15 complete runs and 81 word-run rows per engine. C# self-check included pathological fixtures. PanGloss validation also included the managed pg-conformance-fixtures and pg-parse checks (both exit 0) and the 130-fixture pin gate. The C# coverage report passed with 31/31 constructs and 15 allowed dead rules. CSharpier checked the 23 changed C# files explicitly. The variant protocol forbids capped or timed-out output from matching an expected empty result; the PanGloss tests include that negative control.

Deferred suite work and prior stack notes

Deferred until after the Hermit Crab Optimizations release. Nothing stacks on this PR any more: #534, #540, #522 and #519 were retargeted to master, and #543 was closed. Their suite-side work is held back and will be added here when this resumes, each once its own PR has merged:

• Trace HermitCrab failure reasons and template slot outcomes #534: FailureRuleAttributor credits through Trace.Allomorph; FieldWorks-producibility rows for the four new failure reasons.
• Fix narrowing-rule unapplication across an interposed optional node #540: the edge-cases/interposed-optional-multi-rhs fixture.
• Remove HermitCrab's analysis-cascade memo #522: removing the harness's --no-memoization flag along with the memo.
• Add conformance fixtures for the #504 and #507 evidence gaps #543: the syntactic-feature-inversion-controls (HermitCrab: track exact analysis feature inversion and chained override parse loss #504) and final-template-gap-coverage (HermitCrab: complete cross-engine correctness controls for final-template pruning #507) fixtures.

The earlier body also recorded these stack descriptions:

• Trace HermitCrab failure reasons and template slot outcomes #534 restores failure-allomorph attribution through its own Trace.Allomorph and classifies its four new FailureReason values in the FieldWorks-producibility ledger.
• Fix narrowing-rule unapplication across an interposed optional node #540 adds interposed-optional-multi-rhs, red on master and green with the fix.
• Remove HermitCrab's analysis-cascade memo #522 removes the analysis memo and the harness's --no-memoization flag.
• Prune reduplication matches whose copies cannot agree #519 was rebased only.
• Add conformance fixtures for the #504 and #507 evidence gaps #543 adds fixtures for the HermitCrab: track exact analysis feature inversion and chained override parse loss #504 and HermitCrab: complete cross-engine correctness controls for final-template pruning #507 evidence gaps.

Suite review map

Where to look

• .github/workflows/conformance.yml -- the gate. Runs on any change under src/SIL.Machine.Morphology.HermitCrab*, src/SIL.Machine, the HermitCrab tests or conformance/: self-check of every fixture, the coverage report, and a freshness check on the generated CSVs.
• conformance/dead-rules-allowed.tsv -- the coverage report now exits 1 on a rule no word exercises unless it is listed here with a reason, and on a listed rule that is exercised again. Today: 15 entries, all deliberate (inactive decoys, refused controls, crash pins).
• conformance/PROTOCOL.md -- the adapter contract a second engine implements.
• src/SIL.Machine.Morphology.HermitCrab.Conformance/Runner.cs -- self-check: fixtures run in-process against HermitCrab as the oracle.
• global.json -- new: pins the SDK to 10.0.100 with latestFeature roll-forward, for the whole repository.

Reading order

About 25k lines are harness code (src/SIL.Machine.Morphology.HermitCrab.Conformance) and 16k are its tests. The rest is data: fixtures (conformance/edge-cases, conformance/languages, about 21k), generated ledgers and evidence cards (about 28k), and docs. Generated files are rewritten by hc-conformance --write-* and pinned by tests, so they need checking for freshness, not reading line by line.

Earlier suite revision and validation decisions

• Rebased onto master (be5407d).
• Dropped Trace.FailureAllomorph: an engine API change does not belong in a test-suite PR, and Trace HermitCrab failure reasons and template slot outcomes #534 adds the same fact as Trace.Allomorph. Generated CSVs regenerate byte-identical without it.
• The coverage report could not fail; it now gates dead rules and uncovered constructs.
• ci.yml is back to master. Its added fixture run duplicated ConformanceFixtureGateTests under dotnet test, and parity-check.py's remaining check is now enforced by the coverage report.
• The counterfactual sweep read mutant output in the console code page, so "katibɯd" came back as "katib?d" and a mutation that changed nothing looked like evidence. It now reads UTF-8 (CounterfactualGate, plus a regression test). Regenerating the ledger demoted two surfaces that had only that false evidence: Gloss to Unobservable and HeadFeatures to RequiredByDtd. The gap ratchet moves from 25 to 26 for Gloss.
• local_check.sh ran --semantic-coverage, which exits 1 by design while the catalog is incomplete, so it could never pass. It now runs the same gates as conformance.yml.
• chained-output-feature-override-loss said it fails on master on purpose. It has passed since Change Add to PriorityUnion #494; checked by running it at 52d069f (red) and 7109531 (green). The header now says so.

Earlier decisions, and why

• PCRE.NET (now on master) ships an IIncrementalGenerator. The compilation-graph loader admits it only by content hash, and a probe runs it and fails closed on any generated output.
• An expect_crash fixture passes only if the same engine completes an ordinary fixture in the same run, so a broken adapter command cannot pass as a crash (conformance/PROTOCOL.md).
• *.fwdata is marked binary in .gitattributes; its pinned hash depended on core.autocrlf.
• Hand-maintained fixture totals were removed from the README and docs; the manifest is the count.
• Four fixture grammars keep trailing whitespace: their hashes are pinned in the manifest.

🤖 Generated with Claude Code

---

This change is

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.93%. Comparing base (2599204) to head (fa8d9b2).

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #480      +/-   ##
==========================================
+ Coverage   74.57%   74.93%   +0.36%     
==========================================
  Files         453      556     +103     
  Lines       38314    56481   +18167     
  Branches     5280     7714    +2434     
==========================================
+ Hits        28573    42325   +13752     
- Misses       8576    12291    +3715     
- Partials     1165     1865     +700     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[johnml1135]

Added two independently justified conformance fixtures in a20bce1: final-template-partial-discriminators (10 words; #507) and template-category-sharing (6 words; #505). Fresh Release build: zero warnings/errors. Both fixtures pass with memoization on and off, zero skipped; 22 focused manifest/coverage tests pass. Four grammar counterfactuals produce the independently predicted mismatches, documented under conformance/docs. These are grammar-sensitivity controls, not proof that pruning, cache hits, or same-key feature collisions execute. No parser code changed. Generated inventory also restores previously omitted rows for the already-existing Exact-inverse fixture; its valid zudiua loss remains tracked in #504, not relabeled as correct.

[johnml1135]

Added the shared-template unconstrained-suffix conformance fixture in f150e2a. The grammar has a slot-only feature-unconstrained suffix shared by mandatory TV/IV templates, an ordinary N-to-IV verbalizer, four positive rows, and six rejection controls. The companion rationale explains complete forward derivations and loader reachability (not a FieldWorks UI round trip).

Fresh C# harness: all 10 rows pass in both template orders, memoization on/off (40 comparisons; no skips). Focused template/manifest tests: 22 passed, no skips. Regenerated manifest is current at 40 fixtures; coverage/rule reports updated. No Machine parser code changed.

PanGloss main now contains the narrow gate-only alignment (fc357daf), with this fixture mirrored unchanged until its old Machine submodule pin is upgraded. Direct state assertions provide red/green evidence; full-parse fixture checks pass both before and after the change. Issue #505 remains open for collision/correlation and broader merge-equivalence evidence.

[johnml1135]

Added five clitic conformance fixtures (all edge-cases/), each pinning one behaviour, plus ledger evidence for the final-template gate:

• clitic-proclitic-after-final-template: a prefix clitic in the Clitics stratum attaches outside a final template; an ordinary prefix is refused after the template and allowed before it.
• clitic-enclitic-and-proclitic: a proclitic and an enclitic on the same word, reachable in either order in the unordered Clitics stratum.
• clitic-stem-entry: a clitic stem entry of the Clitics stratum needs no Morphology-stratum template, takes Clitics-stratum rules, and refuses Morphology-stratum rules.
• clitic-flex-layout-blocking-morphology: FieldWorks layout (Morphology, Clitics, Surface) with a phonological rule in Morphology. That stratum is blocking, so the final-template state resets and an isCliticRule="false" rule may follow the template in the next stratum.
• non-clitic-after-final-template-layout: the same layout without phonology. The state carries through the non-blocking strata, so ordinary rules in Morphology, Clitics and Surface are refused after the final template; nouns without a template are the controls.

NonPartialRuleProhibitedAfterFinalTemplate is now evidenced (Blocked: sagdt, Control: kolt). Severing final or partial can never flip a word to a parse, so I added MorphologicalRule.isCliticRule to the attributes feeding both template gates in EngineGateInventoryLedger. Manifest, coverage.csv/rules.csv, the ledgers and the pinned counts are regenerated (135 to 140 fixtures).

EveryConformanceFixturePassesSelfCheck still fails only on the truncate-morphotactic fixtures that wait on #556 (the four named above, plus truncate-morphotactic itself). Those fixtures give different engine output from run to run, so I kept their Gloss/HeadFeatures/ordering rows out of the counterfactual and evidence ledgers.

[claude]

Minor: F35. Five truncate-fixture rows flipped from Unobservable to Evidenced only because those fixtures give different output from run to run. Removing the interface had no effect. This row records 'a': ok::++|a;++|a -> ok::++|a;+|a. The -identity baseline for a at line 483 (A+ROOT+B;B+ROOT) is not the parse its words.yaml declares. The pins at InterfaceWitnessLedgerTests.cs:47-51 include these flips: evidenced +13 = 8 clitic + 5 truncate. A fresh recompute can therefore disagree. Leave the truncate fixtures out of this sweep until #556 merges, as you did for the counterfactual and evidence ledgers.

[claude]

FYI: F36. ka and kaz (line 41) are tagged as exercising "clitic designation by stratum name alone", but neither word depends on it. ka applies no rule. For kaz, no template applied before mrEnc, so IsLastAppliedRuleFinal is null and the gate in SynthesisAffixProcessRule.cs:63 cannot fire, clitic or not. Renaming the Clitics stratum would leave both words unchanged. Tag them with the construct they actually show (a stem rooted in a later stratum). Unverified by a run.

[claude]

FYI: F37. sagdis claims "Clitic rules (isCliticRule / Clitics stratum) after a final template", but mrEnc's clitic status cannot matter here. Morphology is blocking, so the final-template state is reset before the Clitics stratum, and sagdit shows the non-clitic mrNotClitic getting through in the same position. The note describes phonology feeding the next stratum, and that is what the tag should name.

[claude]

Request changes. Follow-up on ed5cf05. The branch moved forward by four commits (fast-forward). They add five clitic conformance fixtures, add MorphologicalRule.isCliticRule to the template-gate inventory, and regenerate the ledgers and pins. I reviewed ed5cf05..4a66977.

1. Verdict: request changes. The conformance gate still fails at this head because of F32, whose code this round did not touch.
2. Most important: F32, conformance/edge-cases/truncate-morphotactic/words.yaml:157. Four truncate fixtures still expect output that only the unmerged Fix a dropped morpheme ID when a zero-width rule wraps one ending at the same node #500 and Fix directional annotation tie boundaries #556 produce. New this round: F35, conformance/interface-witness.tsv:655. Those same fixtures, whose output changes from run to run, turned five ledger rows to Evidenced, and the pins now include them.
3. Counts: 1 Critical open (F32). 1 Important new (F35). 2 Low new (F36, F37).
4. Ran:
   • gh pr checks 480: both Conformance suite runs fail, and both Comment hygiene runs pass. Both Build jobs and claude-review are pending.
   • Locally, dotnet test tests/SIL.Machine.Morphology.HermitCrab.Tests -c Release filtered to the fixture gate, manifest and the six ledger test classes passed 66 of 67. The one failure is EveryConformanceFixturePassesSelfCheck, on truncate-morphotactic-copy-u, -identity and -subsumption only. All five new clitic fixtures pass.
   • /code-review high on the delta returned 9 candidates. Three are posted. The other six I dropped: a duplicate of F35, a kpsag tag that works as a negative control, pinned duplicate parses (a deliberate fixture convention), the evidence ledger choosing a multiplicity witness (selector code unchanged), a loosely worded gate-inventory comment, and Unmapped construct claims (651 rows already are).
   • I confirmed the F35 arithmetic: the new clitic fixtures contribute 8 Evidenced and 8 Unobservable rows, against a pinned +13 and +3.
5. Could not verify: F36 rests on reading the code. I did not rename the Clitics stratum and rerun. I did not run ./local_check.sh --agent-strict or a fresh interface-witness recompute.

Public API, target framework, package, or sillsdev/machine.py parity: None verified.

Finding	Severity	Status
F1	Critical	addressed (cc947db)
F2	Important	addressed (bcb3d62)
F3	Important	addressed (bcb3d62)
F4	Important	accepted, unverified
F5	Low	addressed (bcb3d62)
F6	Low	addressed (067de6f)
F7	Low	addressed (0a1212b)
F8	Low	addressed (067de6f)
F9	Low	addressed (067de6f)
F10	Important	addressed (cd0c909)
F11	Low	accepted, unverified
F12	Low	addressed (cd0c909)
F13	Low	addressed (0a1212b)
F14	Low	addressed (0a1212b)
F15	Low	addressed (067de6f)
F16	Low	addressed (067de6f)
F17	Low	addressed (0a1212b)
F18	Low	addressed (0a1212b)
F19	Critical	addressed (e7fe733)
F20	Low	addressed (8b5d74e)
F21	Low	addressed (1c5c3a5)
F22	Low	addressed (a3b4c0e)
F23	Important	addressed (49ba481)
F24	Important	addressed (49ba481)
F25	Low	addressed (9671b28)
F26	Low	addressed (641f613)
F27	Low	addressed (8b5d74e)
F28	Critical	addressed (dd5f00f, 2cd68e8)
F29	Low	addressed (bb42ff4)
F30	Low	addressed (f592706)
F31	Low	addressed (6c6fbb3)
F32	Critical	open
F33	Low	addressed (b9c7c8a, ed5cf05)
F34	Low	addressed (382168a)
F35	Important	new
F36	Low	new, unverified
F37	Low	new

Reviewed at 4a66977