diff --git a/docs/hermitcrab-2026-10-21-search-cost-round.md b/docs/hermitcrab-2026-10-21-search-cost-round.md new file mode 100644 index 000000000..9c235a161 --- /dev/null +++ b/docs/hermitcrab-2026-10-21-search-cost-round.md @@ -0,0 +1,377 @@ +# HermitCrab search cost, 2026-10-21 round + +**Disposition: three large reductions in search work stand up. Two of them were bugs in the Rust +port, not algorithmic gains.** HC-Rust ([sillsdev/PanGloss](https://github.com/sillsdev/PanGloss)) +is a port of `SIL.Machine.Morphology.HermitCrab` under test against `hc.dll`. On one private +grammar, HC-Rust did far more search than C# because two kinds of shape node had lost the segment +identity that C# gives them. Fixing both made HC-Rust match C# and removed most of that grammar's +cost. The one algorithmic change that survived, sharing trailing affix-template slot batches, was +built in HC-Rust and then ported to C# as [Machine PR #546](https://github.com/sillsdev/machine/pull/546). +The lead ranked first in the [search-asymptotics research](hermitcrab-search-asymptotics-research.md), +lexically-dead branch pruning, was measured first and rejected on its ceiling. + +This record follows the [optimization ledger](hermitcrab-optimization-ledger.md): one entry per +attempt, the number that settled it, and what differs from earlier attempts. Ledger rows 24-30 +summarise it. + +## Data and units + +**Grammars.** The public reference grammars are Sena, Amharic, Mbugwe, Aweti and Indonesian. The +round's motivating grammar was a **private featureless project (Austronesian, ~30k lexical entries, +3 strata)**. It was measured in three variants: *base*, *v1*, and a *deletion variant* that adds a +rule deleting two stop classes after nasals and has no compounding. Each variant was run on three +word lists: 200 nonwords, a 563-word dev list and a 7,206-word held list. Only aggregate numbers +from it appear here; no word forms, glosses or lexical entries. + +**Units.** The owner rule for this round is that gains are stated as **parser work**: "N% less work +to parse the words". +- **Parser work** is HC-Rust's per-word `work_steps`, summed over a word list. A work unit is one + reservation against HC-Rust's per-word work cap. Every analysis attempt reserves one, and so does + every inner matcher step, segmentation step, root-lookup step and confirmation step. It is + deterministic, so it survives machine contention, unlike wall time. +- **Steps** are analysis attempts (`steps <= work_steps`). They are a secondary figure. +- **Milliseconds** appear only as labelled secondary figures. +- **C# figures** are in C#'s own counters (slot batches, lookups), never in HC-Rust work units. They + are not converted. + +**Parity comparators.** HC-Rust used `rust/tools/parse_compare.py`, which compares complete +analyses per word and reports each row IDENTICAL or not. Entries 090 and 103 ran it with +`--multiset`; entry 095 records only `parse_compare.py`. PR #546 compared ordered morpheme-ID/surface signature lists, +ordinal-sorted, with multiplicity. Neither is the legacy morph-count `WordAnalysisSignature` that the +September disposition calls out. + +**Runs.** Unless noted, HC-Rust runs were `pangloss batch --threads 1` with `HC_STEP_STATS=1`. +Change A's full-list parity runs used `--threads 2` with the default step cap and no wall timeout. + +## What stands up + +| # | Change | Kind | Where | Effect, in parser work | Parses | +|---|---|---|---|---|---| +| A | Analysis-side affix literals keep `StrRep` identity on wide featureless tables | HC-Rust port bug. C# already exact | PanGloss divergence entry 090 (analysis-side `Lhs` follow-up) | Private project, 7 spot words: base **98.5% less** (2,365,602 -> 35,913); worst variant **99.8% less** (50,778,583 -> 83,677). Steps on the full lists fell 46-55x | IDENTICAL on all three private lists and on Amharic, Sena, Indonesian | +| B | A re-inserted segment-class node keeps its members on a featureless grammar | HC-Rust port bug. C# already exact | PanGloss entry 103, [`414ac6b5`](https://github.com/sillsdev/PanGloss/commit/414ac6b5), [record](https://github.com/sillsdev/PanGloss/blob/main/docs/divergences/103-class-deletion-reinsertion-identity.md) | Deletion variant, held list: **94.7% less** (303,966,311 -> 16,220,190). Its 16 words that hit a 1 s limit: **99.9% less** | IDENTICAL on every list. Byte-identical work wherever it cannot fire | +| C | Shared trailing slot trie in the affix-template battery | Algorithmic, no memo | PanGloss entry 095, [record](https://github.com/sillsdev/PanGloss/blob/main/docs/divergences/095-shared-template-slot-batches.md); C# [PR #546](https://github.com/sillsdev/machine/pull/546) | Sena **11.5% less** (all 6,146 words); Mbugwe **18.1% less** (1,364-word partial run); Aweti 0.35% less; zero elsewhere | IDENTICAL in HC-Rust; zero signature differences in C# samples | + +The census instrumentation that settled the rejected lead is +[`d1690c13`](https://github.com/sillsdev/PanGloss/commit/d1690c13). It is gated behind +`HC_BRANCH_CENSUS=1` and inert when unset: parser work and parses are identical to main on six +lists. + +## The lesson: diff the port on the oracle's own export before optimizing the algorithm + +The bugs were found by a comparison, not by a profile. HC-Rust normally loads a FieldWorks project +through its own snapshot reader. Here it was also run on the XML that C# itself exports for the same +project. On the private project's slowest word: + +| Engine and input | Lexical lookups | +|---|---| +| C#, its own export | 49 | +| HC-Rust, C#'s export | 18 | +| HC-Rust, its own snapshot | 1,655 | + +These counts were supplied by the round's brief; this write-up did not reproduce them. A ~90x gap +between two inputs to the same engine is a loading or compilation difference, not a search-strategy +difference. Following it led to both identity bugs: + +- **A (entry 090 follow-up).** HC-Rust's analysis-side affix literals carried identity only through + a single 64-bit id lane. That lane does not exist above 64 character definitions. Provisional + letters pushed the private project to 70, and the grammar has no phonological features, so + nothing else distinguished segments. An enclitic literal therefore matched *any* final segment. + Every word long enough was unapplied against it, and synthesis rejected each bogus candidate. C# + gives a featureless segment a `StrRep` `EqualTo` constraint (`CharacterDefinitionTable.cs:68` at +`a4b29742`, the same snapshot the research doc's anchors use), so + its literal matches only itself. The fix applies HC-Rust's chunked `StrRep` matcher on this path, + only for featureless grammars. Featured wide tables (Amharic) keep the feature-only + over-approximation, which C# shares, because a featured node has no `StrRep`. +- **B (entry 103).** `NarrowAnalysisRewriteRuleSpec.Unapply` re-inserts a deletion rule's LHS + constraint as an optional node. For a `SegmentNaturalClass` that node carries the union of its + members' structures, which on a featureless grammar is "`StrRep` in members", and + `RootAllomorphTrie` unifies against it. HC-Rust gave the node no identity, and its root trie also + ignored a query node's member set. Analysis and root lookup could therefore take any segment at + every re-insertion site. The fix sets the member set on featureless grammars only, and the trie + refuses an edge only when C#'s unification would fail. + +**Neither fix is a C# optimization.** Before the fixes HC-Rust did more work than C#; afterwards it +does what C# does. They are recorded here because they decided where this round's effort went, and +because they explain several figures in the research brief: +- **The deletion variant's 1 s timeouts.** The brief reported 200-1,600 inputs reaching a 1 s + per-input timeout, from an earlier build that had neither fix; that count was not re-measured. + What was measured is narrower: after fix A and before fix B, 16 held-list words still hit a 1 s + limit, and fix B cut their parser work by 99.9%. Both port bugs are implicated, and the two counts + are not comparable. +- **Where the time went.** In a two-word profile of the deletion variant before fix B, analysis was + 0.1-0.3% of parser work, root lookup about 10% and synthesis about 90%. Zero branches were + lexically dead. No analysis-side pruning could have helped. + +One residual is **not explained**. On the slowest word, HC-Rust on C#'s export made 18 lookups where +C# made 49. Parses were identical, but an efficiency nobody can argue for is not a result. It is +listed under open leads rather than claimed. + +**Method rule.** Before measuring an algorithmic lead on a port, run the port on the oracle's own +export of the same grammar and compare a cheap count, such as lookups per word. A large gap between +the port's two inputs is a port bug, and its fix can dwarf any algorithmic change. This round, it +did so twice out of three. + +## C1: prune analysis branches that cannot reach a root (research lead #1) — rejected on its ceiling + +**Expected.** A sound summary of the remaining transformations would prune analysis nodes from +which no lexical lookup can find a root. This is the research doc's rank-1 candidate, residual +viability. It differs from ledger rows 10 and 11 only if compounding and the remaining morphology +are modelled. + +**Measured.** A hindsight census charged each unapplication's steps and work to its input node. +Folded alternatives alias their canonical. A node is *live* when a lookup on it, on an alias or on +a descendant found a root. Dead cost is split three ways: +- **Dead leaves:** a failing rule battery plus one failing lookup. +- **Dead, may compound:** fewer non-heads than `MaxStemCount`, with a compounding rule in this or a + deeper stratum. +- **Dead internal, plain:** the residual. + +Only the plain bucket is what a sound, compound-aware summary could actually remove at a profit. A +dead leaf costs about as much to prove dead as to run. A may-compound node is row 10's +disqualifier. + +| List (words) | Parser work | Analysis / lookup / synthesis | Lexically-dead ceiling | Dead leaves | Dead, may compound | **Dead internal, plain** | Repeat-lookup work | +|---|---|---|---|---|---|---|---| +| Private, base, held (7,206) | 50,345,810 | 52.7 / 11.8 / 35.2% | 22.6% | 6.1% | 14.6% | **1.9%** | 2.4% | +| Private, v1, held (7,206) | 105,768,202 | 70.5 / 6.7 / 22.6% | 38.7% | 19.9% | 6.6% | **12.2%** | 1.8% | +| Private, deletion variant, held (7,206), before fix B | 303,966,311 | 8.5 / 35.7 / 55.8% | 16.0% | 12.7% | 0.0% | **3.4%** | 10.4% | +| Indonesian (70) | 1,906,012 | 85.8 / 2.1 / 12.0% | 41.0% | 15.3% | 23.8% | **1.9%** | 0.7% | +| Amharic (669 completed) | 39,395,506 | 97.2 / 1.4 / 1.4% | 66.1% | 20.3% | 45.8% | **0.0%** | 0.6% | +| Sena (5,134 completed at 30 s) | 271,894,826 | 89.9 / 3.0 / 7.1% | 55.5% | 11.9% | 39.0% | **4.7%** | 2.2% | +| Mbugwe, first 400 (394 completed) | 610,569,774 | 73.6 / 3.3 / 23.0% | 45.5% | 9.7% | 35.8% | **0.0%** | 3.3% | +| Aweti (183 of 206 completed at 30 s) | 488,089,456 | 93.6 / 1.0 / 5.3% | 74.9% | 35.1% | 0.0% | **39.8%** | 1.0% | + +The census builds include fix A but predate change C. Template-heavy shares (Sena, and possibly +Amharic and Mbugwe) are therefore against an older, larger denominator. For scale, Sena was 271.9M +work units on 5,134 completed words then, and is 240.7M on all 6,146 now. + +**Verdict: not built.** +- **Plain bucket.** It is 0-5% of parser work on six of eight lists. The exceptions are v1 at 12.2% + and Aweti at 39.8%, the latter on 183 completed words. +- **Most dead work is unprunable.** It sits in leaves, or in nodes that could still split off a + compound non-head. Amharic, Sena and Mbugwe put 36-46% of parser work there. +- **Any sound summary has a large scope.** It must model reduplication, infixes, truncation, + deletion re-insertion and compounding before it is allowed to prune anything. + +**What is different from prior attempts.** +- **Row 11 ceiling.** Row 11 found 23.5% pooled dead steps. Dead *steps* here are 43-94% of steps, + so dead branches are common. But they are cheap: the ceiling falls once it is weighted by work + and the unprunable buckets are removed. +- **Row 11's other failure mode persists:** the root is reachable but synthesis fails. On the + deletion variant before fix B, 86% of synthesis work went to canonicals whose lookups hit but + never matched the surface. On Mbugwe the share is 98%. No lexical oracle can prune either. + +**C# transfer.** C#'s `Morpher`, `AnalysisStratumRule` and `AnalysisRewriteRule` run the same +search, so the same buckets should apply. No Machine PR unless the shadow experiment under open +leads shows a sound summary deciding a meaningful share of the plain bucket. + +### Per-parse distinct-shape lookup — rejected + +The brief's "97,575 lookups over 56 shapes" came from trace mode, where analysis merging is off. The +research doc warned that a trace-mode count cannot establish an untraced duplicate rate, and the +untraced census confirms it: +- **Calls per key.** The private project makes 1.13-1.49 lookup calls per distinct + (stratum, shape). +- **Ceiling.** Exact repeat-lookup work is 0.5-3.3% of parser work on every list, or 10.4% on the + deletion variant before fix B. +- **Policy.** Avoiding it would be a per-parse memo, which this work excludes. + +C#'s `LexicalLookup` repeats lookups the same way, so there is no Machine PR. Dead-candidate lookups +are 30-75% of lookup work, but lookup is only 1-12% of parser work (35.7% on the +deletion variant before fix B), so they were not pursued either. + +## C2: share trailing slot batches inside one battery call (research lead #2) — kept, ported to C# + +**Expected.** The battery walks each template's slots last-first. Templates whose trailing slots +use the same rules, in the same order, with the same optionality, run the same slot batch on the +same word. Factoring the templates into a trie runs that batch once. This is the research doc's +rank-2 candidate. It is control factoring, not a cache: nothing is keyed by a visited word, and +nothing outlives one battery call. + +**Battery share of all parser work (HC-Rust, before C):** + +| Grammar | Battery share | +|---|---| +| Mbugwe | 61% (86 step-capped words are about 2/3 of the total) | +| Private project, v1, held | 45% | +| Sena | 23% | +| Amharic | 21% | +| Private project, base | 8% | +| Private project, loaded from C#'s export | 9% | +| Private project, deletion variant | 6.5% | +| Indonesian | 0 (no templates) | + +**Static census.** This counts rule applications on the battery's own input, before and after +sharing: +- **Shared tails:** Sena 111 -> 45, Mbugwe 397 -> 196, Aweti 143 -> 100. +- **No sharing possible:** Amharic 50 -> 50, and the private project 30 -> 30. Each has one template. + +**Dynamic ceiling, Sena.** A trie-exact key (same trailing slots and same full word state) predicts +448,573 repeated slot batches, which is 30.8M work units or 11.3% of all Sena work. The built trie +avoided exactly 448,573. + +### HC-Rust (PanGloss entry 095) + +Run with `--threads 1 --step-cap 2000000` (Aweti 500000) and no wall timeout. + +| Grammar | Words compared | Parses | Parser work | Steps | Fire count | +|---|---|---|---|---|---| +| Sena | 6,146 | IDENTICAL | **11.5% less** | 31.6% fewer | 171,319 shared, 448,573 avoided (= predicted ceiling) | +| Mbugwe | 1,364 (partial run) | IDENTICAL | **18.1% less** | 19.3% fewer | - | +| Aweti | 170 | IDENTICAL | 0.35% less | 0.16% fewer | 1,939 shared, 2,159 avoided | +| Private project (4 grammar variants: base, v1, deletion, C#'s export; x 3 lists) | all | IDENTICAL | 0 | 0 | 0 | +| Amharic, Indonesian | all | IDENTICAL | 0 | 0 | 0 | + +**Tests.** +- **Differential test.** Compares the shared walk with each template's own walk for all words up to + 5 segments. It fails when optionality is dropped from the trie key. +- **Fire-count test.** Fails with sharing disabled. +- **Conformance.** `-Scope all` passed 2043/2043. Fixtures at the PR #480 head showed the same + failures as the baseline. + +### C# (Machine PR #546) + +`AnalysisAffixTemplateRule` has the same shape, so the trie ported directly to sequential +`ApplySlots`. It is restricted to untraced, sequential walks with no alternative limit. Tracing, +`ParallelApplySlots`, finite alternative limits, custom compiled templates, changed slot +optionality, and grammars without shared tails all keep the original battery. Row 2's battery memo is no +longer on master, so there is no memo for this change to compose with. + +The figures below are C# slot batches on deterministic 200-row samples, single-threaded and warmed, +with mode order alternated per input: + +| Input | Completed pairs | Slot batches before -> after | Fewer batches | Parse time, secondary (ms under concurrent load) | +|---|---|---|---|---| +| Synthetic shared-tail fixture (10 words x 100) | 1,000 | 2,000 -> 1,000 | 50.0% | 1.27 s -> 1.05 s | +| Sena (200 sampled rows) | 172 | 47,072 -> 32,287 | 31.4% | 20.07 s -> 16.67 s | +| Mbugwe (200 sampled rows; 61 pairs capped at 2 s in either mode, excluded) | 139 | 72,467 -> 58,486 | 19.3% | 58.43 s -> 51.33 s | + +- **Parity in the samples.** Every completed pair had identical signature lists, with zero + differences and zero errors. +- **Differential test.** `SharedWalkMatchesPerTemplateWalkInOrder` compares ordered `Word` equality + and complete `AnalysisStateKey` sequences across four grammar arrangements and 2,904 input/state + combinations. +- **Fire test.** `UntracedBatteryRunsSharedSlotBatchOnce` fails with sharing disabled (`Expected: 1; + But was: 2`). +- **Unit tests.** HermitCrab passed 126/126. +- **Conformance.** The PR #480-head fixtures were run on the branch assembly and on the unchanged + base `aab237b2`. With memoization on, both were 58 passed / 2 failed. With memoization off, the + branch was 58/2 and the base 59/1. Both assemblies fail traced-rule attribution in + `boundary-leftover-in-narrowing-lhs-strrep`. `deep-optional-affix-nesting` crossed its 15 s budget + in three runs; it has no shared slot tails and takes the original walk. + +These two engines measure different things, so do not compare the columns. C#'s "31.4% fewer slot +batches" on Sena and HC-Rust's "31.6% fewer steps" are not the same unit, and neither is HC-Rust's +11.5% less parser work. + +**Conformance pins.** The C# 60/60 self-check below is `hc.dll` at `72dbd4f9`, the PR #480 head. The +58/60 above is PR #546's base `aab237b2`. They are different assemblies, and the two counts are not +in conflict. + +**What is different from prior attempts.** +- **Row 2.** The battery memo reduces battery *invocations*. C reduces repeated slot batches + *within one invocation*, with no `(Word, state)` table. +- **Row 18.** C is not a cross-word cache. +- **The research doc's prediction holds.** It said this would not turn 43,641 battery calls into + 4,988. It does not; it shrinks each call. + +**Rejected alongside.** A rule-level memo per battery call was no bigger than slot-level sharing on +Sena (33.7M vs 32.9M work units of repeats), and it is a memo. + +## Conformance state at the round's close + +The run used the PR #480 head, Machine `72dbd4f9`. + +- **C# self-check.** `hc-conformance --include-pathological` passed **60/60 fixtures**: 609 entries + in the `words.yaml` files, none skipped. Upstream fixtures grew from 44 to 60 since the previous + pin; nothing was removed. +- **HC-Rust.** One fixture failed: `edge-cases/clitic-rule-designation`, 5 words. Machine PR #529 + added `MorphologicalRule@isCliticRule`, which HC-Rust has not yet ported. It is a parity gap, not a + divergence, and the port is under way. Every other new or modified fixture passes in both engines. + No fixture passes in HC-Rust and fails in C#. + +## How today's measurements bear on the six research reports + +The integrated ranking is in the [research doc](hermitcrab-search-asymptotics-research.md). This +section records only what changed. + +| Lead (report) | Status after this round | +|---|---| +| **1. Residual viability**: abstract interpretation with staged refinement (R6), lazy residual-lexicon product (R3), magic sets / goal propagation (R4), extension oracle (R5) | **Ceiling measured, rejected for now** (C1). The plain dead-internal bucket is 0-5% on six of eight lists. Reopens only through the Aweti/v1 shadow experiment below | +| **2. Template battery as a shared control trie** (derived from R2/R3) | **Built, kept, ported** (C; Machine PR #546) | +| **3. Trace-class POR with recovery**: Mazurkiewicz traces (R1), sleep/source-set DPOR (R1), critical-pair certificates (R1); verified POR (R6) | Not pursued. No new measurement. The earlier 0.00-0.05% eligibility still stands as the warning | +| **4. Exact finite-state compilation of a verified subgrammar** (R3, R6, R2 #3) | Not pursued as an exact engine. A narrower **FST-derived precheck** is the best remaining lead for single-template grammars (open leads) | +| Fixed-cap root-result cache (R5 #2; R4 aside) | **Rejected**: repeat-lookup work is 0.5-3.3% of parser work untraced, and the cache is a memo. The 97,575/56 premise was a trace-mode artefact | +| Count-vector chart DP with packed provenance (R2 #1); packed trail forest for synthesis (R2 #2); `AnalysisStateKey` graph with ZDD trails (R4 #1); analysis-DAG x synthesis-state product (R5 #1) | Not pursued. They are search-growing tables, a closed family. The finding that 86-98% of synthesis work is root-reachable failure is consistent with the reports' own warning that one output analysis does not bound product states | +| Proven rule-precedence DAG (R4 #2) | Not pursued. It needs complete-path certificates (research doc section 1) | +| Symmetry reduction (R1 #4) | Not pursued. The reports themselves rated it a probable no-fit | +| Meet-in-the-middle (R3, R4), A*, k-best and k-shortest paths, reverse search (R4, R5), Earley/CKY, scalar semirings (R2), e-graphs (R4), bimachines, two-level morphology, compile-replace (R3) | Still not recommended. Unchanged | + +**Prediction scored.** The research doc said the trace-mode lookup ratio could not establish an +untraced duplicate rate. That was right: 0.5-3.3%. + +**The finding the reports could not have made.** Where it was measured, much of the private +project's cost was two port bugs: 98.5-99.8% less work on the spot words, and 94.7% less on the +deletion variant's held list. No ranked lead addressed that, because each one assumed the port's +search matched the oracle's. It is not the whole story: with fix A in place, v1's held list still +costs 105.8M work units, 45% of it in a one-template battery that C cannot touch (open lead 1). + +## Open leads, not pursued + +Each lead is tagged **[C# analogue]** where the same question applies to `hc.dll`, and +**[HC-Rust]** where it concerns the port only. + +**Work reductions, with a ceiling to measure first:** +1. **FST-derived precheck before slot-rule traversal, for single-template grammars** [C# analogue]. + C cannot help a grammar with one template, and on the private project's v1 variant the battery + is still 45% of parser work. A necessary condition derived from each slot rule's compiled matcher + could reject a rule before traversal begins. This differs from row 22's edge-segment prefilter in + being judged in work units, since a traversal charges per arc, and in being derived from the + whole matcher rather than its first and last segments. Its ceiling, the work spent in + zero-match traversals, has not been measured. +2. **Shadow summary experiment on Aweti and v1** [C# analogue]. These are the only lists whose + plain dead-internal bucket is material (39.8% and 12.2%). Run a sound summary in shadow mode and + count the plain-bucket work it would actually decide, along with the summary's own cost. Close + research lead 1 (C1) if the decided share is small. +3. **Mbugwe's step-capped words** [C# analogue]. 86 of 1,645 words hit the 2M step cap and account + for about 2/3 of Mbugwe's work. They have not been examined individually. +4. **Root-reachable synthesis failure** [C# analogue]. 86% (deletion variant, before fix B) and 98% + (Mbugwe) of synthesis work goes to candidates whose lookup hits but whose synthesis never matches + the surface. No lexical oracle reaches this. It is the largest unaddressed bucket on Mbugwe. + +**C# port follow-ons to #546** [C#]: + +5. **`ParallelApplySlots`** needs its own design for its layered breadth-first queue. +6. **Share compiled slot rules across templates**, in addition to slot batches. +7. **Measure in a deterministic work counter.** The C# gain is measured in slot batches on samples, + not in a work counter on full lists. + +**Time-only costs (no work units charged, so outside the owner metric):** +8. **`ana_allomorph_matches`** deep-clones segments with `segs.to_vec()` and rebuilds + `format!("p{i}")` part names on every allomorph call [HC-Rust]. About 5.1M segment-vector builds + are avoidable per private-v1 held run: 5.48M slot-rule attempts against 365K slot batches. + Hoisting `segs_of` or borrowing segments in `Transduce::new` removes them. +9. **`root_trie::search_segs_ident`** checks its active set with `Vec::contains`, which is quadratic + [HC-Rust]. A bitset would be a pure time win. + +**Identity over-approximations left in HC-Rust** [HC-Rust]. Synthesis re-validates all of them, so +none can change a parse, but each over-explores relative to C# on some grammar: + +10. An `InsertContext` (natural-class insertion) in analysis-side `Lhs` still carries no identity + above 64 character definitions (entry 090's remainder). +11. Narrowing leftovers from `ana_narrow_general` keep no class identity (entry 103's remainder). No + measured grammar exercises a class LHS there. +12. `union_narrowing_node` checks only `char_def`, so it ignores an LHS class node's member set. +13. RHS class nodes in synthesis (`syn_narrow`, `syn_epenthesis`) lack identity on featureless + grammars. They were left alone because they affect rendering. + +**Unexplained observations:** +14. **The lookup gap** [HC-Rust vs C#]. HC-Rust on C#'s export made 18 lookups on the slowest word + against C#'s 49. Parses were identical; the cause was not investigated. +15. **Sena SKIPPED rows** [HC-Rust]. A census run with a 30 s wall limit reported 1,012 SKIPPED + rows, while no-limit A/B runs completed all 6,146 words in about 50 s. Not investigated. + +## Data handling + +Private-project word lists, per-word logs and exports stay on the measuring machine. This record +carries only aggregate counts. Conformance fixtures are synthetic. diff --git a/docs/hermitcrab-2026-10-27-analysis-round.md b/docs/hermitcrab-2026-10-27-analysis-round.md new file mode 100644 index 000000000..780d36bcd --- /dev/null +++ b/docs/hermitcrab-2026-10-27-analysis-round.md @@ -0,0 +1,395 @@ +# HermitCrab analysis-side optimization, 2026-10-27 round + +**Disposition: six candidates are PRs (R1 #575, R2 #572, T1 #571, R5 #574, R3 #576, R1 on #546 #577), one is a +dead end (R4), and one is not built (R6).** This round follows the +[synthesis-to-analysis round](hermitcrab-2026-10-27-synthesis-to-analysis-round.md), whose conclusion was that +synthesis is 1.0% (Aweti) to 9.0% (Mbugwe sample) of total work and the dominant cost is analysis. Here the +target is the analysis side: the template battery, rule outputs that cannot reach a root, matcher work per +attempt, and trace memory. Measurements are in steps, never ms. + +This record follows the [optimization ledger](hermitcrab-optimization-ledger.md). Ledger rows 38-46 point here. + +## Unit and method + +- **Work** is analysis events (the census harness's `ITraceManager` callbacks) plus synthesis steps for the same + words. "% less work" is `(off - on) / off`, per grammar, never pooled. Where a check does work of its own, a + net figure is given in one unit and the unit is named. +- Harness: `research/analysis-cost-census` @ `38e9622e`, based on master `c2454760` (includes PR #480's + conformance suite). Each candidate is an internal Off / Shadow / On mode on `Morpher`, product default On. + The same binary runs both arms; census mode is traced and unmerged, `censusmerge` uses the merged walk. +- Grammars: Indonesian (all 70 words), Aweti (stride 2), Amharic (limit 28), Mbugwe (stride 40, 150 s per + word), Sena (stride 100), the conformance suite (150 runs), and one synthetic stress fixture per candidate. + Words that abort in either arm are dropped from both and listed in the result files. +- Soundness: a shadow mode counts violations (a parse below a fired prune). Any violation stops the candidate. + Parses are compared per word by a signature of root, rule trail and allomorphs. +- Gates for every PR: the HermitCrab test project on the branch and on master (every branch failure also fails on + master), conformance per-fixture verdicts identical, `dotnet csharpier check .`, comment hygiene. The full + `local_check.sh --agent-strict` stops at the SentencePiece build, which needs `cmake`; CI is the first full run. + +## Decisions + +| Candidate | Issue | PR | Decision | +|---|---|---|---| +| R1 required-slot look-ahead | #561 | #575 | **KEEP.** Kill rule waived, see R1 | +| R2 root-substring prune | #562 | #572 | **KEEP**, with a probe skip | +| T1 lean trace | #563 | #571 | **KEEP.** Lands after #534 | +| R5 edge-segment prefilter | #564 | #574 | **KEEP.** The plan's score of 0 was wrong | +| R4 subsequence fallback | #568 | none | **DEAD END** | +| R6 merging while tracing | #565 | none | Not built. Owner product decision | +| R3 ordered look-ahead | #567 | #576 | **KEEP.** Stacks on #575 | +| R1 on #546 | #566 | #577 | **KEEP.** Stacks on #546 and #575 | + +Earlier-round PRs: **#560** (E1, allomorph-loop cutoff) and **#570** (MPR and stem-name trail replay). E3 (template +replay) was dropped on cost; its code stays at `perf/hc-lookup-replay-templates` @ `9efd059c`. + +## What the census said to aim at + +From the [analysis-cost census](hermitcrab-optimization-ledger.md) the plan took these facts: + +- The template battery is 69-98% of analysis events on Aweti, Mbugwe and Sena. Mbugwe has 63.8% of its events in + template partials that never reach lookup (Sena 22.1%, Aweti 11.7%). +- Not-unapplied attempts are 70-91% of events. Skipping one with a cheaper test removes no step; it makes a step + cheaper. The plan therefore scored prefilters 0 in steps (R5). That scoring turned out to be the lesson of R5. +- The root-substring predicate at rule outputs had 13.6-38.5% ceilings with 0 violations on the five real + grammars, but 29 success-path nodes on conformance, so it needed static soundness flags (R2). +- PR #546 shares trailing slot batches only in untraced, sequential walks, so it is inert under tracing. + +Eight issues were filed in sillsdev/machine (#561-#568), each checked against open and closed issues and PRs for +duplicates. Linked as related, not duplicates: #485, #512, #551, PRs #546, #534, #490, #480. + +## R1. Required-slot look-ahead: KEEP (issue #561, PR #575) + +**Claim.** A template partial that reaches a required slot where no rule can unapply dies there, after paying for +every optional slot in between. A necessary condition for a required slot to unapply later is that, for some rule +and allomorph in it, every inserted affix segment unifies with some Segment node of the current shape. The check +runs after the final-template guard, before recursing in `ApplySlots`, and before pushing work in +`ParallelApplySlots`. It is used for slot j only when every slot walked after j is pure (copy and insert only, no +part that can match zero segments). Optional nodes count as present. Memory is grammar-sized tables only: no +per-word state, no cache. + +**Soundness.** Shadow mode, 0 violations and 0 completions under a fired check: + +| grammar | fired | violated | +|---|---:|---:| +| Mbugwe | 803,228 | 0 | +| Sena | 6,402 | 0 | +| Aweti | 5,948 | 0 | +| Amharic | 584 | 0 | +| Indonesian | 0 | 0 | +| conformance (150 runs) | 298 | 0 | +| stress-bantu-tam | 689 | 0 | + +Parses were equal per word, traced and merged. Mutation checks: removing the purity gate fails the impurity +tests, and disabling the mode fails the prune tests. + +**Event-based result** (analysis events, fewer is better; synthesis steps unchanged): + +| grammar | analysis events off to on | % less events | % less work (events plus synthesis) | +|---|---|---:|---:| +| Mbugwe (40 words) | 5,289,909 to 4,940,905 | 6.60% | 6.29% | +| Sena (51) | 110,986 to 76,809 | 30.79% | 30.27% | +| Aweti (99) | 1,596,848 to 1,475,588 | 7.59% | 7.52% | +| Amharic (24) | 8,560 to 7,800 | 8.88% | 7.73% | +| conformance | 38,393 to 37,796 | 1.55% | 1.28% | +| Indonesian (70) | 7,468 to 7,468 | 0% | 0% | +| stress-bantu-tam (60) | 45,342 to 11,497 | 74.64% | 74.35% | + +**Unification-work result.** The unit is top-level `FeatureStruct` unification calls (`IsUnifiable` plus `Unify`) +over the whole parse, look-ahead checks included, counted by census-only counters. Positive is less work: + +| grammar | calls off to on | % less unification work | direction | +|---|---|---:|---| +| Mbugwe | 21,515,185 to 21,444,795 | +0.33 | break-even, slightly less | +| Sena | 1,308,998 to 1,288,517 | +1.56 | less | +| Aweti | 17,699,599 to 16,902,830 | +4.50 | less | +| Amharic | 411,073 to 413,674 | **-0.63** | **more work** | +| Indonesian | 662,201 to 662,201 | 0.00 | none | +| conformance | 303,738 to 303,313 | +0.14 | break-even | +| stress-bantu-tam | 162,015 to 100,593 | +37.91 | less | + +Mbugwe's large event saving is mostly walk and trace work that costs few unifications, so little of it shows in +this unit. The check cost 1,112,761 unifications on Mbugwe against 349,004 events saved. Mbugwe reaches only +10.3% of its no-lookup ceiling, because its required slots mostly insert one segment (`-a`, `a-`), which is nearly +always present somewhere in the shape. R3 is the follow-up. + +**Waived kill rule.** The plan said to close R1 if check unifications exceeded 3x the events saved on the grammar +with the largest saving. That fired on Mbugwe at 3.19x. It was **waived**, because it compared unification calls +with walk and trace events, which are unlike units. A same-unit comparison replaced it (table above). The earlier +ratios (1.5x to 12.9x) are withdrawn. The review asked for this to be stated plainly in the PR, with a direction +word on every figure, and it is. + +**Tried and rejected.** Skipping slots whose every allomorph inserts a single segment: it did not cut Mbugwe's +check cost (1,119,073 against 1,112,761 check unifications) and halved Sena's saving (17.75% against 30.79% of +events; unification work -0.70% instead of +1.56%). The product keeps the uncut test. + +**Gates.** HermitCrab tests 168 passed, 0 failed on the branch (master 151 passed). Conformance 146 passed and 4 +failed on both, the four being the nondeterministic `truncate-morphotactic` family. csharpier clean, comment +hygiene clean. Product branch `perf/hc-required-slot-lookahead` @ `0bf9e646` at the time of the result (changed by +review); measurement branch `research/hc-rsl-census` @ `6ad11702`; the unification counters live only there. + +**Risks.** Amharic is net more work in unification calls, on 24 words. The stress fixture is synthetic. PR #534 +edits the same file; the diff is three hooks and one helper. Aborted words (the heaviest) are dropped from both +arms, so the event saving is a floor for them but unproven in unification calls. + +## R2. Root-substring prune: KEEP (issue #562, PR #572) + +**Claim.** When a node's shape contains no root allomorph as a contiguous substring, and nothing that can still be +unapplied below it can restore, insert or rejoin stem material, no lookup below it can hit. The probe runs at +the three analysis rule outputs. Three static per-stratum conditions turn it off: a rule that drops a part that +can hold segments, copies a part more than once, copies out of order or modifies a part; deeper phonological +rules; and lexical-pattern roots. A rule type it does not know also turns it off. + +**Why it is not row 27.** Row 27 closed root reachability summaries (the prunable bucket was 0-5% on 6 of 8 lists, +and nodes may still compound). A substring test passes a compound, because the compound contains its head root; +it runs at every rule output, including template-slot outputs; and the flags turn it off where stem material can +be restored. Row 27's reopening condition, a shadow summary deciding a meaningful share, is met on Sena (17.7%) and +the stress fixture (67.7%), with 0 violations. + +**Where the flag is on** (first failing condition per stratum): Sena all three strata; the stress fixture's main +stratum; conformance the main stratum in 126 of 150 runs. Off: Indonesian (a rule copies a part more than once; +deeper phonology), Mbugwe (a rule drops a part that can hold segments), Amharic (a rule drops a part), Aweti (a +rule drops a part). Census ceilings for those four, from the plan, were Mbugwe 38.5%, Indonesian 29.6%, Aweti 14.8% +and Amharic 13.6%, which are not reachable soundly (see R4). + +**Net of probes.** After review, each probe is charged as one event: + +| list | gross % less work | net % less work | probes | note | +|---|---:|---:|---:|---| +| Sena (51 words) | 17.7 | **15.3** | 2,698 | parse sets equal | +| stress, early-lexicon agglutinative (60) | 67.7 | **66.5** | 245 | 13,584 events saved | +| conformance | 0.2 | **-15.3** (more work) | 7,237 | buys 92 events; 2 words save more than they probe | +| Indonesian, Mbugwe, Amharic, Aweti | 0 | **0** | 0 | flag off | + +The conformance result is the honest cost: it is a suite of tiny words and small lexicons, so the probes exceed the +events saved, and there is no adaptive off switch. Only Sena among real grammars gains. + +**Probe skip.** Added after review: the probe is skipped when the output keeps at least as many segments as its +input within the same stratum. It is sound (a skip only keeps a word). It removed only 18 of 7,255 probes. + +**Rejected: a root-count gate.** Turning the prune off for strata with fewer than 16 roots would have hidden the +conformance loss. The coordinator rejected it: it is tuned to the toy lexicons in the conformance suite, and no +real grammar would trigger it. Its census was cancelled, so it has no measured result. + +**Gates.** HermitCrab tests 161 passed, 0 failed (master 151). 10 new tests in `RootSubstringPruneTests`; +removing each flag condition turns exactly its test red. Conformance 146 passed and 4 failed on both arms, per +fixture identical. Branch `perf/hc-root-substring-prune` @ `d58a8e88`. + +**Risks.** Pruned analyses no longer count toward the analysis-side alternative limit, so a parse that threw +`MaxAlternativesExceeded` may now succeed, never the reverse. Try-a-Word traces lose the subtree below a pruned +output. Soundness depends on `AnalysisMorphologicalTransform.GenerateShape` semantics; a future analysis step that +changes shape outside rules and stratum entry would break it silently. + +## T1. Lean trace: KEEP, lands after #534 (issue #563, PR #571) + +**Claim.** The default `TraceManager` pins about 1 GB on Mbugwe yɔɔna (1,031.95 MiB retained for 21,200 analysis +events and 41,130 synthesis steps, from the trace-memory study) because every node pins a `Word`, either live or +as one of six deep clones. An opt-in `TraceManager.RetainWords = false` keeps shape text (`Trace.InputText`, +`Trace.OutputText`) and no `Word`. Steps are unchanged by construction. The default stays `true`: `Trace.Input` +and `Trace.Output` are public and would be null in lean mode, so flipping is an owner decision. The CLI opts in. + +Retained trace memory, same word, same run, product arm against lean arm: + +| grammar | words | mean full MiB | mean lean MiB | % less (sum) | max full to lean MiB | +|---|---:|---:|---:|---:|---| +| Mbugwe (stride 40) | 40 | 407.2 | 25.10 | 93.8% | 5,485.6 to 389.3 | +| Aweti (stride 2) | 99 | 35.6 | 2.28 | 93.6% | 656.2 to 29.4 | +| Sena (stride 100) | 51 | 4.35 | 0.35 | 92.0% | 27.5 to 2.0 | +| Amharic (limit 28) | 24 | 6.86 | 0.08 | 98.8% | 17.5 to 0.2 | +| Indonesian | 70 | 2.24 | 0.03 | 98.7% | 18.1 to 0.3 | +| conformance | 1,476 | 0.45 | 0.01 | 97.4% | 329.0 to 7.5 | +| stress deep-derivation | 10 | 200.36 | 4.99 | 97.5% | 343.0 to 7.8 | + +yɔɔna: 1,035.0 to 22.6 MiB (97.8% less); peak falls from 1,038 to 89 MiB. The kill criterion (under 50% on +yɔɔna) is not met. What still holds the lean memory is the node objects and two strings per node, about 190 bytes +a node on yɔɔna (an estimate from totals, not a heap count). Allocation is flat: lean adds text formatting and saves +clones. + +**Steps are identical.** Parse counts and node counts equal for every compared word; the census rows for Indonesian +(70 words) and yɔɔna are byte-equal (md5) between base and product; conformance per-fixture verdicts are identical; +CLI output is byte-identical to master with one processor apart from the "Parse time" line. With default +parallelism master itself orders some synthesis nodes differently run to run. + +**Landing order.** PR #534 rewrites nearly every `TraceManager` callback, `Trace.cs` and `ParseCommand.cs`, so a +textual conflict is certain. Land #534 first, then rebase this onto it, projecting #534's fields in lean mode: +copy strings or ids at the callback (summarise `FailureObject`, keep the allomorph id, the blocking entry name and +the slot rule name) instead of references. + +**Gates.** HermitCrab tests 156 passed, 0 failed (master 151). 5 new `LeanTraceTests`; forcing lean mode to store +words fails 2 of 5. csharpier and hygiene clean. Product commit `9f7828e8` on `perf/hc-lean-trace`. + +**Risks.** Lean text of an unfrozen word is taken at callback time; full mode formats at read time. An unfrozen +word mutated after the callback would differ. An instrumented run found none: unfrozen words reach callbacks only +as private pre-rule clones, compounding temporaries and phonological outputs the manager clones. Default mode still +pins about 1 GB on yɔɔna. + +## R5. Edge-segment prefilter: KEEP (issue #564, PR #574) + +**Claim.** Port ledger row 22's analysis affix prefilter to master. A pattern anchored at both ends must match its +first and last segment by unification, so a mandatory edge segment that cannot unify rejects the attempt before +the matcher runs. It skips boundaries, lets optional edge segments through, and emits the existing failed +unapplication trace callback. An internal Off/On switch defaults On. + +**The plan scored it 0, and that was wrong.** Census steps (analysis events and synthesis steps) are **unchanged** +on all 1,796 completed pairs per mode, so by the owner's step metric the candidate removes nothing. It makes each +failed attempt cheaper. The first measurement reported gross matcher work and left out the prefilter's own +unifications; review required the net. The headline is **net matcher work**: traversals plus arc checks, with each +prefilter unification charged as one arc check. This is the lesson: a step count is blind to a change that +cheapens each step. + +Census mode (unmerged), pairs completed in both arms: + +| list | traversals off to on | arc checks off to on | prefilter unifications | gross less | net less | +|---|---|---|---:|---:|---:| +| Aweti | 1,539,228 to 444,856 | 18,959,730 to 9,102,289 | 1,221,061 | 53.43% | **47.47%** | +| Mbugwe | 4,894,285 to 2,464,790 | 18,994,380 to 13,973,576 | 2,763,590 | 31.19% | **19.62%** | +| Sena | 110,368 to 30,893 | 1,297,930 to 1,014,492 | 87,560 | 25.77% | **19.55%** | +| Amharic | 10,143 to 6,931 | 345,463 to 300,000 | 3,305 | 13.69% | **12.76%** | +| Indonesian | 21,107 to 18,767 | 654,511 to 610,183 | 2,682 | 6.91% | **6.51%** | +| stress | 20,284 to 3,589 | 343,290 to 58,511 | 19,154 | 82.92% | 77.65% | +| conformance | 43,682 to 35,667 | 252,488 to 175,788 | 16,248 | 28.60% | 23.12% | + +`censusmerge` gives the same ordering (Aweti 49.24%, Mbugwe 21.24%, Sena 19.83%, Amharic 12.94%, Indonesian 6.60%). +Counts are unweighted, not a timing claim. The user-specified kill criterion (net under 5% on every real grammar) +is not met: the smallest net is Indonesian at 6.51%. + +**Parity.** All 1,796 completed pairs per mode have identical parse counts and signature sets. The +`edge-cases/truncate-morphotactic` family is excluded (repeated runs differ on master itself). Exclusions are +listed in `edge-prefilter-evidence/census-net-summary.txt` of the working set. + +**Memory.** Mbugwe has 874 compiled analysis affix-rule instances, 1,620 allomorph specs and 3,648 list-capacity +slots, giving a conservative 108.4 KiB of retained metadata (below 159 KiB with abandoned list backing arrays). +Counts measured, byte figures are 64-bit layout estimates, not a heap profile. Nothing per word, no cache. + +**Gates.** HermitCrab tests 169 passed (master 151); tests cover the exact failure trace on matcher and prefilter +rejection, alpha-variable constraints, a group with an optional leading constraint, and a multi-segment suffix +ending in the `ts` digraph. Conformance verdicts identical (146 passed, 4 failed on both). Both `local_check.sh` +runs fail at the SentencePiece `cmake` step. Branch `perf/hc-edge-segment-prefilter` @ `bb525683`. + +## R4. Subsequence fallback and insertion classes: DEAD END (issue #568) + +**Claim tested.** R2 is off on four of five real grammars because their rules restore, reorder or duplicate stem +material. R4 asked whether a weaker sound test still prunes: "some root is a subsequence of the shape" for +order-preserving removal, with the segment classes that a dropped part can restore treated as wildcard gaps. +Ceilings from the census, minus any R1 overlap: Mbugwe 38.5%, Indonesian 29.6%, Aweti 14.8%, Amharic 13.6%. + +**Built.** `RootPruneRuleShape` classifies each rule as substring-safe, subsequence-safe or order-free-safe, with +the segment classes its dropped parts restore. `RootSubsequenceTrie` holds the roots of the stratum and deeper +strata; root segments that unify with a gap class are skipped, since a restorable segment may be re-inserted +anywhere; the earliest-match search runs depth first; order-free mode ignores positions. Each stratum gets the +tier Substring, else Subsequence, else Order-free, and turns off for rules that modify a part, unknown rules, +deeper phonology, lexical-pattern roots, or when every root is made of restorable segments. Memory is one small +trie per stratum (Mbugwe 165 roots). Product branch `perf/hc-root-subsequence-prune`, stacked on R2's `d58a8e88`, +left unsquashed with the order-free code at **`71cba028`**. No PR. + +**Cost accounting.** One probe is 1 step and each trie arc examined is 1 step (`RootSubsequenceSteps`, a hook +that is null in production). + +Sound subsequence tier with gaps, net: + +| list | status | analysis events off to on | net % less work | +|---|---|---|---:| +| Indonesian | off (REDUP-meN copies parts out of LHS order) | 7,468 to 7,468 | 0.0 | +| Amharic | off (a rule modifies a part) | 8,560 to 8,560 | 0.0 | +| Aweti | off (a rule copies parts out of LHS order) | 1,596,848 to 1,596,848 | 0.0 | +| Mbugwe | off (every root is made of restorable segments) | 3,500,941 to 3,500,941 | 0.0 | +| Sena | on, R2's tier unchanged | 110,986 to 91,010 | 15.3 | +| stress | on | 19,959 to 6,375 | 66.5 | +| conformance | on in many fixtures | 38,393 to 38,225 | -19.1 | + +With the order-free tier added (rules that also reorder parts): + +| list | status | events off to on | gross % | net % | probes | arcs | shadow violations | +|---|---|---|---:|---:|---:|---:|---:| +| Indonesian | on (Clitics, Surface off: deeper phonology) | 7,468 to 4,956 | 30.8 | **-182.9** (more work) | 631 | 16,779 | 0 | +| Aweti | off (every root is restorable) | unchanged | 0.0 | 0.0 | 0 | 0 | n/a | +| Mbugwe, Amharic | off, as above | unchanged | 0.0 | 0.0 | | | | +| stress | on | 19,959 to 6,375 | 67.7 | 66.5 | 245 | 0 | 0 | +| conformance | on | 38,393 to 38,225 | 0.4 | -19.1 | 7,432 | 1,657 | 0 | + +**Kill criterion.** Under 5% net on every one of Mbugwe, Indonesian, Aweti, Amharic: met (0.0, -182.9, 0.0, 0.0). +No parse differences outside the known truncate-morphotactic family. Verdict DEAD-END. + +**Why.** + +- **The ceilings are unsound.** The flag-ignoring census (fires of the plain subsequence predicate, share of events + below the topmost pruned nodes) is Indonesian 55.2%, Mbugwe 36.1%, Aweti 21.7%, Amharic 50.9%, Sena 16.8%, + stress 50.6%. Most of it in the three restoring grammars is unsound: **violation events 334,508 (Mbugwe), + 50,187 (Aweti)**, 2,264 (Amharic) and 1,105 (Indonesian). A raw subsequence prune would lose real parses. +- **The insertion-class (wildcard gap) variant never fires** on Mbugwe, Aweti or Amharic: their dropped parts + describe classes that cover every root segment, so no word can be ruled out. Position-specific gaps were not + tried. +- **Amharic is blocked** by a rule that modifies a part, for which there is no sound test. +- **Indonesian is the one grammar where the sound tier works** (30.8% fewer events, 0 shadow violations), and it + costs 26.6 trie arcs per probe against about 4 events saved per probe: a net loss at one step per arc. Every + cheaper structure considered (a set of present token classes, a bitmask per root) costs about the same 25-40 steps + per probe as there are token classes. + +**Revival would need** a test that costs well under one step per saved event on Indonesian, position-specific gaps +for the restoring grammars, and a sound treatment of rules that modify a part. None is in sight. + +**Known nondeterminism.** Conformance parse signatures differ for the same words between off-arm census and +censusmerge runs (`truncate-morphotactic-identity#default` word 2), and the on arms differ only on words in the +`truncate-morphotactic` family. Parse counts are equal. + +## R6, R3 and R1 on #546 + +- **R6, merging while tracing (#565): not built.** Census merging changes analysis events by only 0.2-2.4%, but + under tracing lookups are 1.67x (Mbugwe) to 2.19x (Aweti) more frequent. Whether Try-a-Word shows folded + duplicates is an owner product decision, and it is low in steps. It interacts with #534 and #563. +- **R3, ordered and edge-anchored look-ahead (#567): KEEP, PR #576.** It targets the 90% of Mbugwe's no-lookup + ceiling that R1 leaves, where required slots insert a single segment. Results are in the section above. +- **R1 on #546 (#566): KEEP, PR #577; results in the section above.** It measures R1 on the shared-slot walk and reports what #546 leaves. The rule + set in advance: if R1 saves under 3% of slot events on the #546 walk where it saved over 5% on master, the two + overlap and only one should land. Further sharing of non-trailing slots needs a key on the shape, which is a memo, + so any residual is reported as a ceiling and not proposed. + +## R3 and R1 on #546: KEEP, and the two stack (issues #567, #566; PRs #576, #577) + +- **R3, ordered and edge-anchored look-ahead (#567; PR #576, stacked on #575).** Within a rule, the required inserted + segments must embed in order; the check is anchored at the edge it can reach; and a passing required slot narrows the + window for the slots behind it. Shadow: 0 violations on every grammar, and parses are equal per word. Events plus + synthesis steps, less work than master: Sena **40.9% less**, Mbugwe **7.3% less**, Aweti **11.0% less**, Amharic + **11.9% less**. Unification calls, less than master: Amharic **2.75% less** (R1 alone was 0.63% MORE), Sena + **5.22% less**, Aweti **9.27% less**, Mbugwe **1.83% less**. Mbugwe gains little over R1 because most of its required + slots are not testable; the next step is an R1 purity extension with its own soundness argument. PanGloss port: + sillsdev/PanGloss #65. +- **R1 on #546 (#566; PR #577, stacked on #546 and #575, commit `bbb81a00`).** The look-ahead runs on the shared trie + walk, so a template whose required slot cannot unapply is dropped for its whole subtree. Reviewed LAND. Shadow: 0 + violations. Events plus synthesis steps, less work than master: Sena **55.64% less**, Mbugwe **32.15% less**, Aweti + **20.06% less**, Amharic **9.10% less**. Unification calls, less than master: Sena **4.81% less**, Mbugwe **15.19% + less**, Aweti **12.18% less**; Amharic **0.63% MORE**. Against #546 alone, Mbugwe **1.07% MORE** and Sena **1.40% + MORE** unification calls. The two stack in events, but on Mbugwe and Sena the check's own unifications cost more than + the walk work they save. PanGloss port: sillsdev/PanGloss #66 (for #546 and #566). + +## Cross-cutting lessons + +1. **Count a check's own cost in the same unit as what it saves.** R1, R2 and R5 all first reported gross numbers. + Net, R1 broke even on Mbugwe and lost on Amharic; R2 lost 15.3% on conformance; R5 fell from 6.91% to 6.51% on + Indonesian and from 31.19% to 19.62% on Mbugwe. R1's 3x kill rule compared unification calls with walk and trace + events and was waived for that reason, with a same-unit comparison in its place. R4 was killed by the same + accounting: 30.8% gross became 182.9% more work. +2. **Truncate-morphotactic conformance fixtures are nondeterministic on master and must be excluded from parity.** + `truncate-morphotactic`, `-copy-u`, `-identity` and `-subsumption` give different output run to run with no + change. Their mismatch detail text differs between two base runs, so only per-fixture PASS and FAIL is + comparable, and the same four fail on both arms of every experiment. PanGloss waives the same family. +3. **Census steps miss work that cheapens each step.** R5 scores 0 in steps and saves 6.5% to 47.5% of matcher + work. Choose the unit the change acts on, report steps beside it, and do not rank by steps alone. +4. **No memoization (owner rule).** It is why E3's replay was dropped instead of cached, why R1 and R5 keep + grammar-sized tables only, and why shared work beyond #546 is a reported ceiling and not a proposal. +5. **A port can expose a divergence in the other implementation.** The PanGloss port of machine #548 found and + fixed a real Rust divergence in alpha-environment selection (PanGloss PR #13). Porting a change to the second + engine is a cheap second oracle. +6. **A ceiling measured with the flags ignored can be mostly unsound.** R4's census ceilings (up to 55.2%) were + mostly restoration violations; the sound share on the four target grammars was zero. +7. **A gate tuned to the test lexicons is overfitting.** R2's root-count gate would have removed the conformance + loss and no real-grammar gain, so it was rejected. + +## Open questions + +- R3 and R1-on-#546 results (issues #567, #566). +- Whether a sound subsequence test can be made cheap enough for Indonesian-style grammars, where the sound tier + prunes 30.8% of events but costs 26.6 arcs per probe. +- R6: whether Try-a-Word should show folded duplicates (owner decision). +- Landing order of #571 and #534, and whether the default of `RetainWords` should ever flip (a major-version + decision). +- R2 on conformance-like workloads: the prune has no adaptive off switch, and loses 15.3% there. diff --git a/docs/hermitcrab-2026-10-27-synthesis-to-analysis-round.md b/docs/hermitcrab-2026-10-27-synthesis-to-analysis-round.md new file mode 100644 index 000000000..f49172e83 --- /dev/null +++ b/docs/hermitcrab-2026-10-27-synthesis-to-analysis-round.md @@ -0,0 +1,722 @@ +# HermitCrab synthesis-to-analysis pruning, 2026-10-27 round + +**Disposition: round finished; no product code is proposed for this PR.** This round answers John +Maxwell's question about which synthesis tests could move into analysis. It records a failure-reason +census, gives a direct answer for each of his four candidates, and ranks six experiments. Five were +built and measured on throwaway branches (none pushed); E6 stays research-only. The results are in +[Results of the round](#results-of-the-round). Measurements are in steps, never time. + +This record follows the [optimization ledger](hermitcrab-optimization-ledger.md). Ledger rows 31-37 +point here. The analysis-side round that followed is +[hermitcrab-2026-10-27-analysis-round.md](hermitcrab-2026-10-27-analysis-round.md); it records that E3 was dropped on cost. + +## Answer to John Maxwell's questions + +**Failure-reason statistics.** They exist now (census below, harness `research/failure-reason-stats` +@ `af97348b`). On the five real grammars, pooled, 71.8% of failed synthesis branches and 43.5% of +synthesis steps end in `PartialParse / ApplicableTemplatesNotApplied`, and 25.6% of steps end in +`SurfaceFormMismatch`. Your four reasons together are 0.1% of failed branches and 0.1% of steps. +Caveats from the method section that bind every number: the counts come from traced parses (tracing +turns merging off, so more analyses are explored); a step is a count, and `RequiredMprFeatures` +events are mostly allomorph selection, not failure; branch outcomes are not candidate outcomes; 22.1% +of steps have no terminal event and are unattributed; the samples are strides of five grammars, with +heavy words dropped by the per-word budget; and the census counts synthesis steps only, so it does +not see analysis work. + +**Excluded co-occurrence (E4): sound, cheap, effective only where exclusions are dense.** The check +is sound: parses are identical on every corpus, and the zero-width "other" trap you might expect does +not occur for affix-process allomorphs. It is the only experiment of the round that removes analysis +events. On the five real grammars it removes 0% (none has an affix-only exclude rule), on the +conformance suite 0.23% of total work. On the brief's own stress fixture it gives 3.2%, and on two +deliberately dense synthetic grammars 31.3% (Athabaskan-style prefix complex) and 19.8% +(Bantu-style extension chain). Verdict PARK: land it, if at all, as a cheap safety net for grammars +with many ad hoc co-prohibitions. + +**DisjunctiveAllomorph: already mostly early; no experiment.** The environment-free `break` in the +synthesis allomorph loop already stops later allomorphs, and what remains needs the surface form. +Disjunctions conditioned by MPR features are covered by E2, and the loop itself is shortened by E1. +Real-grammar terminal share is 0.0%. + +**Environments (E5, the input-word prefilter): dead end as specified.** The soundness scan has to +disable the prefilter on 5 of 5 real grammars (four have ambiguous digraph segmentation, one has +non-uniform feature sets) and on 145 of 150 conformance runs. Forced on against that scan it is net +negative on two of three stress fixtures once matcher invocations are counted, and about neutral on +Aweti (net -0.4%). See E5 below for what would have to change. + +**ExcludedMprFeatures (covered by E2, the MPR and stem-name trail replay): sound; worth keeping, +but small in total work.** Excluded MPR features are replayed with the required ones at lexical +lookup. The prune is sound (a shadow census found no successful candidate that it rejects), and it +saves 56.9% of synthesis steps on Aweti and 11.4% on Mbugwe. Those are shares of synthesis, which is +a small part of the whole parse (next paragraph). + +**What the data says matters instead.** Synthesis steps are a small fraction of total work, which is +analysis events plus synthesis steps. On the Aweti sample synthesis is 1.0% of the total (15,940 of +1,560,180). On the Mbugwe sample it is 9.0% (130,647 of 1,449,301). E1, E2 and E3 save synthesis +steps only, so their "% less work" is a share of synthesis. In total work they save at most 1.1% on a +real grammar (table below). The dominant cost is analysis. The next round should target analysis: for +example, extending E4-style monotone checks earlier, and the `ApplicableTemplatesNotApplied` analysis +paths that the census ranks first. + +## The question + +HermitCrab is generate-and-test. Analysis unapplies rules to generate candidate analyses, and +synthesis re-applies them to test each candidate. A test that runs in analysis prunes a candidate +before it is expanded, looked up and synthesized. [PR #519](https://github.com/sillsdev/machine/pull/519) +did this for reduplication copy agreement. Maxwell asked whether other tests could move. His +candidates were: + +- excluded `MorphemeCoOccurrenceRules` (monotone: once a morpheme is present it stays); +- `DisjunctiveAllomorph`; +- `Environments`: check a required environment against the input word in linear time, finding the + left context and then the right context after it. An excluded environment would need its left and + right contexts to be present everywhere; +- `ExcludedMprFeatures`. + +His principle is that an excluded property can be checked early if nothing later can remove it, and +a required one usually cannot. He also asked whether there are statistics on failure reasons. There +were none: `FailureReason` does not appear in any document in PR #490. This round adds them. + +## Scope: the target is the space of grammars, not the five samples + +The owner set the scope during the round. The five sampled reference grammars (Sena, Amharic, +Aweti, Mbugwe, Indonesian) rarely use co-occurrence rules, environments, disjunctive allomorphs or +MPR features. A census of their XML finds 0-2 co-occurrence rules each, required MPR features on 0-9 +allomorphs, and no `Overwrite` MPR groups. The target is every grammar HermitCrab can represent that +some language family would plausibly use. Each candidate is therefore ranked by soundness and by its +benefit across that space. The measured samples are one data point, and they do not decide the +ranking. For every candidate this record names: + +- (a) the language types that lean on the feature; +- (b) the worst-case blow-up when the check runs late compared with early, in steps; +- (c) a stress fixture that models such a language at realistic density. + +Every experiment's measurement must include its stress fixture as well as the conformance suite and +the five real grammars. + +## Failure-reason census + +**Reproduce.** Branch `research/failure-reason-stats` @ `af97348b` is master `c2454760` with PR #480 +merged. Its harness is test-only: +`tests/SIL.Machine.Morphology.HermitCrab.Tests/Research/{FailureReasonStatsHarness.cs, +FailureStatsTraceManager.cs, run_failure_stats.sh, make_failure_stats_report.py}`. Run +`bash tests/SIL.Machine.Morphology.HermitCrab.Tests/Research/run_failure_stats.sh OUT_DIR` from the +repository root. It writes one TSV per corpus and a `tables.md`. No product code changed. + +**Definitions.** +- A *step* is one rule-level synthesis trace event: a rule applied, or one subrule attempt that did + not apply. Steps are counted, never timed. +- A *terminal outcome* is a synthesis branch that ends in `Failed(reason)`, `Successful` or `Blocked`. +- *Steps wasted per reason* attaches each step to its word through `Word.CurrentTrace` lineage, then + splits it equally among that word's terminal descendants. +- *Rule-attempt reasons* (`RequiredMprFeatures`, `Pattern`, `AllomorphNotApplicable`, ...) mean one + subrule did not fire. The word goes on, so these are never terminal by themselves. + +**Corpora.** The real grammars are samples, not whole corpora: Indonesian all 70 words, Aweti every +2nd (103), Amharic the first 28, Mbugwe every 40th (42), Sena every 100th (62). The conformance suite +has 150 fixture runs and 1,492 words. + +### Headline numbers + +Real grammars, pooled (27,912 failed and 422 successful terminal outcomes; 279,872 synthesis steps): + +| Terminal reason | % of failed branches | % of synthesis steps | +|---|---:|---:| +| PartialParse / ApplicableTemplatesNotApplied | 71.8% | 43.5% | +| SurfaceFormMismatch | 4.9% | 25.6% | +| (no terminal event) | - | 22.1% | +| (success) | - | 3.8% | +| PartialParse / MorphologicalRulesPending | 14.8% | 2.1% | +| PartialParse / NonFinalTemplateAppliedLast | 5.2% | 1.6% | +| PartialParse / StratumRulesRemaining | 3.3% | 1.2% | +| RequiredStemName, Environments/Require, BoundRoot, DisjunctiveAllomorph | 0.1% together | 0.1% together | + +Rule-attempt events, real grammars: `RequiredMprFeatures` 41.4%, `Applied` 21.7% (morphological) +plus 1.7% (phonological), phonological `Pattern` 17.0%, `AllomorphNotApplicable` 11.6%, morphological +`Pattern` 5.2%, and `ExcludedMprFeatures` 2 events (0.0%). + +Per grammar, the step shares differ widely. Unpooled: + +| Grammar | Largest wasted-step buckets | +|---|---| +| Mbugwe | ApplicableTemplatesNotApplied 46.3%, SurfaceFormMismatch 27.2%, no terminal 22.8% | +| Aweti | MorphologicalRulesPending 30.4%, NonFinalTemplateAppliedLast 28.5%, success 16.0% | +| Amharic | MorphologicalRulesPending 60.0%, SurfaceFormMismatch 16.9% | +| Indonesian | success 51.9%, SurfaceFormMismatch 37.9% | +| Sena | no terminal 56.3%, MorphologicalRulesPending 13.3%, ApplicableTemplatesNotApplied 6.8% | + +The conformance suite is dominated by one pathological fixture. `deep-optional-affix-nesting` +contributes 5,556 of 6,789 failed branches. Pooled, StratumRulesRemaining has 40.7% of the suite's +steps, success 44.1% and SurfaceFormMismatch 6.9%. Maxwell's four reasons together have 0.9% of +conformance steps. + +### Caveats on the method + +These are read before any number above is used as a ceiling. + +1. **Tracing changes the search.** `Failed(...)` is raised only while tracing. Tracing turns off + equivalent-analysis merging (`AnalysisStratumRule.cs:142`), so traced parses explore more + analyses than production parses. The counts describe a traced parse. +2. **A step is a count, not a cost.** A `RequiredMprFeatures` event is one set lookup + (`SynthesisAffixProcessRule.cs:144-160`). An `Applied` or `AllomorphNotApplicable` event is a full + allomorph pattern match plus an output shape (`:179`). Both count as one step. Ledger row 7 is the + precedent: 100% of a rejection histogram cost 0.2% of wall time. +3. **Analysis work is outside the step total.** On Mbugwe the sampled words had 4,598,689 + analysis-side morphological rule events (unapplied plus not unapplied) against 260,070 synthesis steps. A + synthesis-only saving of N% is not N% of the work to parse the words. Every brief reports + analysis events and synthesis steps separately. +4. **Most `RequiredMprFeatures` events are allomorph selection, not failure.** Mbugwe has + `requiredMPRFeatures` on only 9 allomorphs, yet it logged 113,419 such events against 56,405 + morphological applications. The allomorph loop logs one event for each gated allomorph it skips + on the way to the one that fires. These events show where the loop spends time. They are not why + candidates die. +5. **Branch outcomes are not candidate outcomes.** `ApplicableTemplatesNotApplied` fires on one path + through a stratum. `SynthesisStratumRule.Apply` (`:58`, `:104-140`) runs the templates both on the + input and after each morphological-rule output, so a candidate can lose one path and succeed on + another. A prune that drops whole candidates needs a candidate-level census: the steps spent in + candidates with zero successes. That census was not run. The 2026-10-21 round measured the + HC-Rust analogue at 86-98% of synthesis work (ledger row 27, "root-reachable synthesis failure"). +6. **The 22.1% "no terminal event" share is unattributed.** It is steps on branches that ended with + no terminal callback. Template slot paths that die inside `SynthesisAffixTemplateRule.ApplySlots` + while a sibling path survives are one source. Silently dropped words are another: the realizational + gate at `SynthesisAffixTemplatesRule.cs:27-28` and `HashSet` duplicates. This split is an + inference and was not measured. Treat the share as mostly template-path failure. +7. **Samples and budgets.** A 30 s per-word budget dropped 2 Mbugwe words, so the heaviest words are + under-represented. `run_failure_stats.sh` runs the five grammars concurrently, so which words hit + the budget can vary between runs. A/B runs must be sequential and must drop a word that aborted + in either arm. Amharic stops at word 28 (word 29 hangs). + +## Direct answers to Maxwell's four candidates + +All four are sound to move in at least a restricted form, and the census predicts little benefit +from any of them on the samples. On the samples the work goes to template-path failures and surface +mismatches. + +| Candidate | Sound early? | Where | Terminal share on the samples | Who needs it | +|---|---|---|---|---| +| Excluded `MorphemeCoOccurrenceRules` / `AllomorphCoOccurrenceRules` | Yes for `Anywhere`, with a zero-width caveat; not for `Somewhere*` or `Adjacent*` without linear order | Analysis, per rule (H1) | Real 0; conformance 6 branches (0.1% of steps) | Polysynthetic and Bantu morphotactics; FLEx "ad hoc co-prohibitions" written to kill spurious parses | +| `DisjunctiveAllomorph` | Mostly already early; the rest needs the surface | Synthesis loop | Real 13 (0.0%); conformance 13 (0.2%) | Romance and Semitic suppletion, IE stem allomorphy | +| Required `Environments` (input-word prefilter) | Yes, under two conditions (below) | Analysis, per allomorph (H1) | Real 8 (0.0%); conformance 15 (0.4%) | Turkic and Uralic harmony, Bantu, Austronesian nasal substitution | +| `ExcludedMprFeatures` | Yes, by trail replay after lexical lookup | After lookup (H4) | Real 0 terminal, 2 rule events; conformance 11 rule events | IE declension and conjugation classes, Bantu noun class | + +Hook points (origin/master `c2454760`): **H1**, per analysis rule, after `MorphologicalRuleUnapplied` +at `MorphologicalRules/AnalysisAffixProcessRule.cs:92`. **H3**, end of analysis, +`Morpher.cs:179-193`. **H4**, after lexical lookup and `ExpandAlternatives`, once per alternative, just +before `_synthesisRule.Apply` at `Morpher.cs:353-358` (sequential) and `:386-395` (parallel). The filter +must see each alternative, not the canonical word. With merging on, which is the production setting, +`ExpandAlternatives` (`Word.cs:440-482`) rebuilds merged analyses with their own rules and +allomorphs, so dropping the canonical would drop analyses with different trails. Traced runs hide +this trap, because tracing turns merging off. + +**One principle covers all four.** A successful candidate applies every entry of its trail +(`Word._mruleApps`) in a fixed order, with the allomorph analysis recorded for it +(`Word.IsAllomorphApplicable`, `Word.cs:254-261`, added by PR #548). Any quantity that depends only +on the root, the trail and the recorded allomorphs therefore has one value on every successful path. +A predicate that is false for that value on every successful path can drop the candidate without +losing a parse. Maxwell's "cannot be removed later" is the special case where analysis can compute +that value before the root is known. + +### Excluded co-occurrence + +`MorphCoOccurrenceRule.IsWordValid` (`MorphCoOccurrenceRule.cs:82-87`) rejects an `Exclude` rule +when every listed other co-occurs with the key (`:92-170`). Under `Anywhere` that is set membership, +and adding morphs never undoes it, so it is anti-monotone in the Apriori sense. There is one trap the +monotonicity argument misses. The "others" are read from `Word.AllomorphsInMorphOrder` (`Word.cs:127`), +which is built from morph annotations. `MarkMorph` (`Word.cs:274-296`) creates no annotation for a +zero-width allomorph. A null affix on the trail is therefore invisible to the rule in synthesis. An +early check may count an "other" only when its recorded allomorph always leaves a morph annotation. +`SomewhereToLeft/Right` need surface linear order, which the trail gives only for simple prefixes +and suffixes, so they stay late. `Require` and `Adjacent*` are not monotone and stay late. The key is +always checked, because `Morpher.IsWordValid` iterates `word.Allomorphs` (`Morpher.cs:661`), which +includes zero-width allomorphs. + +Benefit: zero on the samples. In a grammar that uses co-prohibitions it is the only candidate here +that prunes analysis work as well as synthesis. Experiment E4. + +### DisjunctiveAllomorph + +Most of this check already runs early. An environment-free earlier allomorph with no required +syntactic FS blocks later ones at rule-application time, through the `break` at +`SynthesisAffixProcessRule.cs:234-241`. The test left in `Allomorph.IsWordValid` (`Allomorph.cs:127-151`) +covers earlier allomorphs whose environment holds on the final surface. That needs the morph's +surface position, which analysis does not have. Where the disjunction is conditioned by MPR features +(Romance conjugation classes, for example), the MPR replay in E2 covers it. No separate experiment. +E1 shortens the same loop. + +### Environments + +Environments are checked on the **final** synthesized word, after every stratum's phonology +(`Morpher.cs:358` runs `IsWordValid` on `_synthesisRule.Apply(...)`, and `Allomorph.cs:110-125` checks +the morph's neighbours there). For any success, `IsMatch` (`CharacterDefinitionTable.cs:274-282`) +has accepted that shape against the input word. A1's worry, that later rules could change the +context, therefore does not apply here, because no rule runs after the check. Maxwell's prefilter is +sound only under two conditions that the code imposes: + +1. **Boundaries.** The environment matchers accept `Boundary` and `Anchor` annotations + (`AllomorphEnvironment.cs:38-41`, `:55-58`), which the input shape lacks. A boundary node in a + context must match the empty string in the prefilter. +2. **Underspecified surface segments.** `IsMatch` compiles the synthesized shape to a regex, so an + underspecified node (an archiphoneme such as N that no rule fills in) can match an input character + whose features conflict with the context. For example, a [+cons] node unifies with a [+voice] + context and also matches input `t`. Run against the input, the prefilter would then reject a valid + parse. The prefilter is sound only when every surface node is fully specified. Featureless + grammars meet that by `StrRep` identity. Grammars without underspecified segments in lexical + entries, inserted affix material or rule outputs also meet it. Otherwise the prefilter must stay + off. + +Excluded environments cannot be prefiltered this way, as Maxwell said. Benefit: on the samples, 23 +terminal branches across all corpora. The real case is a zero allomorph with an environment: analysis +can unapply it anywhere in any word, and today only `IsWordValid` disposes of it. Experiment E5. + +### ExcludedMprFeatures + +On a successful path, MPR features evolve deterministically along the trail. They start from the +root's set (`Word.cs:152-153`), and each applied allomorph calls `AddOutput` (`MprFeatureSet.cs:29-44`; +`SynthesisAffixProcessAllomorphRuleSpec.cs:209`). This is exact except where something outside the +trail touches the set: realizational rules (not on the trail, `Word.cs:340`), compounding +(`SynthesisCompoundingRule.cs:178-179`), `ChooseInflectionalStem` swapping in a relative's entry +(`SynthesisAffixTemplatesRule.cs:83-115`), and `Overwrite` groups (which remove features, so a +feature can be lost). Excluded MPR features can therefore be checked at H4, but not before lookup, +because the root contributes MPR features. Required MPR features can be checked at the same point, +which matters more: required MPR features are how FLEx inflection classes gate affix allomorphs. +Benefit on the samples: no terminal failure is attributed to either reason. Required failures may be +hidden inside the PartialParse buckets, and E2's shadow census measures that. Experiment E2. + +## Literature that bears on this + +The full survey is in this round's working notes, and only the parts that change a decision are kept +here. Unless marked, the sources were checked at the abstract level only. + +- **Anti-monotone pruning** (Agrawal & Srikant 1994, Apriori). A set that contains a forbidden subset + is forbidden. This justifies E4 and the excluded half of E2. It does not apply to required + properties, which are monotone in the other direction. +- **Nogoods and forward checking in CSP search** (Dechter; Schiex & Verfaillie 1993, cited + second-hand). A failed partial assignment is a nogood. Ledger row 14 already found 0 subsumable + nogood pairs, so nogood *learning* is closed. Forward checking ("can this still be satisfied?") is + the model for E2 and E3. Both ask whether a trail can still succeed once the root is known. +- **Constraint automata intersected with the input** (Koskenniemi 1990; Bird & Ellison 1994; Kaplan & + Kay 1994). This is the principled form of Maxwell's environment check, and E5 is its cheapest + instance: a greedy left-then-right scan, linear in word length for each context. Baeza-Yates (1991) + subsequence automata pay off only when one word is tested against many contexts. +- **Maxwell (1994), "Parsing Using Linearly Ordered Phonological Rules"** (arXiv cmp-lg/9411015, full + text not read). It is HermitCrab's own ancestor: generate-and-test with ambiguity encoded into the + form being parsed. Reading where it rejects candidates is open. +- **Billot & Lang (1989)** warn that look-ahead added to chart parsing can cost more than it saves. + This is why every experiment here has a kill criterion in steps, including the cost of the check. + +No published source was found on tracking morph spans through rule inversion. That is what a +position-aware environment check would need. + +## Ranked experiments + +Ranked by soundness plus benefit across the grammar space, with cost as the tiebreak. "Ceiling" is +the census share that the change could at most remove. It is not a prediction. Every experiment is +measured in steps: synthesis steps, analysis events and candidates reaching synthesis, each reported +as "N% less work" per corpus, unpooled. Every experiment's parses must be identical, per word, on the +conformance suite, the five real grammars and its stress fixture. + +### E1. Stop the synthesis allomorph loop after the recorded allomorph + +- **Hypothesis.** `SynthesisAffixProcessRule.Apply` keeps applying allomorphs after the one analysis + recorded. Each of those applications is a full pattern match and output shape, and its result is + thrown away at `:199` (`AllomorphNotApplicable`). Breaking the loop once `i` passes the recorded + index removes them, along with their MPR checks. +- **Soundness.** An output from allomorph `i > r` fails `IsAllomorphApplicable` and is never added. + `appliedAllomorphIndices` (`:138`, `:196-197`) is copied into the outputs at the moment each is + built (`Word.MorphologicalRuleApplied`, `Word.cs:374-375`, `UnionWith`). Indices added after `r` + therefore never reach `r`'s output. When the recorded allomorph is null, nothing changes. Earlier + allomorphs (`i < r`) are untouched, because their application decides the disjunctive `break` and + the recorded indices. +- **Hook.** `MorphologicalRules/SynthesisAffixProcessRule.cs:140-247`. It needs a `Word` accessor for + the recorded allomorph at the current trail position (`Word.cs:254-261`). +- **Language types, and blow-up.** Any affix with an ordered allomorph list. Examples are Turkic + -DI/-lAr sets written as 4-8 environment-conditioned allomorphs (environment-bearing allomorphs + never `break`, so today every one is applied), Bantu class prefixes, and IE case endings by + declension class. Late: each rule application costs `A` allomorph applications. Early: `r + 1`. + The waste is linear, up to a factor of `A`, for every synthesis of every candidate, including the + successful ones. +- **Stress fixture.** New synthetic fixture `stress-allomorph-list-density`. It has a harmony-style + suffix grammar with four suffix slots, each with 8 environment-conditioned allomorphs, and with the + recorded allomorph spread across positions in the lists. `languages/suffixing-vowel-harmony` and + `edge-cases/disjunctive-recheck` are the soundness controls. +- **Expected savings.** The ceiling is the `AllomorphNotApplicable` events with `i > r` plus the + `i > r` part of `RequiredMprFeatures`. `AllomorphNotApplicable` is 11.6% of real-grammar synthesis + steps (Mbugwe 12.3%, Sena 11.4%, Aweti 2.6%) and 1.0% of conformance steps. The census did not split + by `i > r`. On the stress fixture the ceiling approaches (A-1)/A of allomorph applications. +- **Cost.** Very low: about 10 lines plus a `Word` accessor. +- **Kill.** Less than 2% fewer synthesis steps on every real grammar *and* less than 20% on the + stress fixture, or any parse difference. + +### E2. Replay MPR features and stem names along the trail at lexical lookup + +- **Hypothesis.** Once the root is known (H4), the MPR set that each trail entry will see on a + successful path can be computed without shapes. A candidate whose recorded allomorph fails its + required MPR features (or matches its excluded ones), or whose rule's `RequiredStemName` differs + from the root's, can be dropped before synthesis. +- **Soundness.** Replay in synthesis order (trail from last to first). Start from the root's + `MprFeatures` and add each recorded allomorph's `OutMprFeatures`. On a successful path this gives + the set the synthesis check sees (`SynthesisAffixProcessRule.cs:144-177`), except for contributions + the replay can only bound. The required test runs against an upper bound: that set plus every + realizational and compounding output, with `Overwrite` removals ignored. The excluded test runs + against a lower bound, and only when the grammar has no `Overwrite` group and the trail has no + compounding entry. If the root's entry has a `Family`, the prune stays off, because + `ChooseInflectionalStem` replaces the root. With a null recorded allomorph, the candidate is dropped only if every allomorph + of the rule fails. Stem names: `SynthesisAffixProcessRule.cs:107` compares only with + `input.RootAllomorph.StemName`, which is known exactly at H4. +- **Hook.** H4, per alternative (`Morpher.cs:353-358` and `:386-395`), after the `MaxAlternatives` + count so that the exception behaviour does not change. Both paths call it. +- **Language types, and blow-up.** IE declension and conjugation classes (Latin and Russian + declensions; Romance -ar/-er/-ir conjugations), Bantu noun class written as MPR features, and + Australian conjugation classes. With `c` inflection classes and endings that are homophonous + across classes, each ending's analysis meets roots of every class. Late, each wrong-class root + costs a synthesis up to the gated rule, plus every template path. Early, it costs a set replay of + length |trail|. With agreement prefixes gated in `k` slots, late synthesis grows as `c^k` + candidates. The early check is still `c^k` replays, but each one is trivially cheap. +- **Stress fixture.** New synthetic `stress-mpr-inflection-classes`: five declension classes as MPR + features on 20 roots, case and number endings gated by required MPR features, deliberate homophony + across classes, one excluded-MPR exception and one `Overwrite` group (which must disable the + prune). Soundness controls: `edge-cases/mpr-overwrite-order-dependence`, + `edge-cases/mpr-group-overwrite-without-realizational`, `edge-cases/mpr-gated-exception`, and + `languages/fusional-realizational-morphology`. +- **Expected savings.** On the samples the floor is near 0: no terminal failure has an MPR reason. + The ceiling is the MPR-caused part of PartialParse (48.4% of real-grammar steps across all its + causes). That part is unknown, and phase 1 (a shadow census) measures it. Mbugwe and Aweti are the + only samples with gated allomorphs (9 and 6 required, 41 and 26 excluded). On the stress fixture, + expect most of the wrong-class syntheses, roughly (c-1)/c of candidates. +- **Cost.** Low to medium: one replay function and a `Morpher` switch defaulting on, following PR + #519's `PruneDisagreeingCopies`. +- **Kill.** The shadow census decides less than 3% of synthesis steps on every real grammar *and* + less than 30% on the stress fixture. A shadow run in which any successful candidate fails the + predicate is a soundness bug: stop and fix it, do not tune it. + +### E3. Replay the template and final-state outcome at lexical lookup (after a dead-candidate census) + +- **Hypothesis.** Most synthesis steps on the template-heavy samples go to candidates that die on + shape-free conditions. The conditions are an applicable template whose required slot no pending + rule fills (`SynthesisAffixTemplatesRule.cs:57-63`), a non-final template applied last + (`SynthesisStratumRule.cs:70-74`), rules left in a stratum (`:75-87`), and trail rules never + consumed (`Morpher.cs:639`). An abstract replay at H4 can follow the trail pointer, template + finality, `IsPartial` and the syntactic FS while assuming every pattern match succeeds. That + replay over-approximates success, so it can prove a candidate dead. +- **Soundness.** Abstract interpretation of `SynthesisStratumRule.Apply` and + `SynthesisAffixTemplatesRule.Apply`. Assuming every allomorph pattern matches only adds successful + paths, and so does treating each realizational rule as both applied and not applied. If the + abstract replay finds no path, the concrete synthesis finds none either. Template applicability + (`:37`) needs the syntactic FS at that point. That FS is exact on successful paths: the root entry's + FS (`Word.cs:151`) is unified with each `Required` and then `PriorityUnion`-ed with each `Out` + (`SynthesisAffixProcessRule.cs:123`, `:182-183`). It is not exact when realizational FS output or + `ChooseInflectionalStem` intervene, and the replay must bail out there. +- **Phase 0 (must come first).** A candidate-level census: synthesis steps in candidates (root + allomorph lookups) with zero successes, split by the first shape-free reason the replay would + give. This settles caveats 5 and 6 above. +- **Hook.** H4, as in E2. E3 extends E2's replay. +- **Language types, and blow-up.** Every grammar with affix templates and required slots. FLEx + templates are the normal way to write Bantu verbs, Athabaskan and Algonquian position classes, + Mayan and Quechuan inflection. Late, each dead candidate pays lookup, `ExpandAlternatives` and + synthesis of every stratum below the failing one, plus each template path. Early, it pays a replay + of length |trail| times the number of templates per stratum. On Mbugwe, 19,734 candidates reached + synthesis and 149 branches succeeded. +- **Stress fixture.** New synthetic `stress-required-template-slot`. It models a Bantu-like verb: + one stratum, a template with a required TAM slot and optional subject and object slots, noun and + verb roots with overlapping shapes, and words whose bare-stem and partial analyses find roots. + `languages/prefixal-discontinuous-slot-dependency` and `edge-cases/final-template-partial-*` are + the soundness controls. `edge-cases/deep-optional-affix-nesting` is the pathological control. +- **Expected savings.** The branch-attributed ceiling is PartialParse at 48.4% of real-grammar + synthesis steps plus part of the 22.1% unattributed share. Mbugwe has 46.3% plus 22.8%, Aweti 58.9% + (MorphologicalRulesPending plus NonFinalTemplateAppliedLast) and Amharic 60.0%. Ledger row 19 is + the counterweight: shape-free die points were 0.0-6.0% of cost on heavy Sena probes and 0.4% on 28 + Amharic words. Sena and Amharic are therefore expected low and Mbugwe and Aweti decide. +- **Cost.** Medium to high. It mirrors two synthesis classes' control flow for Linear and Unordered + strata. It is the riskiest experiment for soundness, and the one with the largest measured target. +- **Kill.** Phase 0 shows that shape-free reasons decide less than 10% of dead-candidate synthesis + steps on both Mbugwe and Aweti *and* less than 30% on the stress fixture. Or any successful + candidate fails the abstract replay. + +### E4. Excluded co-occurrence during analysis (Maxwell) + +- **Hypothesis.** An `Exclude` + `Anywhere` morpheme (or allomorph) co-occurrence rule whose key and + others are all on an analysis word's trail rejects that word and everything analysis derives from + it. Checking it at H1 prunes the analysis subtree as well as the lookups and syntheses. +- **Soundness.** As argued above (anti-monotone under `Anywhere`), with three restrictions. An + "other" counts only if its recorded allomorph always leaves a morph annotation. A rule whose key or + others include a root morpheme is checked at H4, not H1. Allomorph rules count only entries whose + allomorph is recorded (not null). `Somewhere*`, `Adjacent*` and `Require` stay late. +- **Hook.** `MorphologicalRules/AnalysisAffixProcessRule.cs:92` (after `MorphologicalRuleUnapplied`, + before `output.Add` at `:96`). Root-involving rules use H4. +- **Language types, and blow-up.** Polysynthetic prefix incompatibilities (Athabaskan, Iroquoian), + Bantu negation against tense, Quechua and Aymara suffix incompatibilities, and any FLEx project + that uses ad hoc co-prohibitions to suppress spurious parses. Take `k` optional slots with `m` + fillers each. Late, every analysis that holds a forbidden pair is expanded below that point, looked + up and synthesized. Early, the subtree is cut where the pair completes. The worst case is + `m^(k-d)` analysis nodes saved per cut at depth `d`, so it is exponential in the slots that remain. +- **Stress fixture.** New synthetic `stress-excluded-cooccurrence`: six optional prefix slots with + four fillers each, about ten `Exclude`/`Anywhere` rules, and a zero-width affix as an "other" in one + rule. That word must still parse. `languages/suffixing-evidential-adjacency-chain`, + `languages/templatic-root-modification` and `edge-cases/morphotactic-attribute-breadth` are the + soundness controls (they hold 3, 1 and 8 exclude rules). +- **Expected savings.** Samples: about 0. Conformance: 2 `MorphemeCoOccurrenceRules/Exclude` and 4 + `AllomorphCoOccurrenceRules/Exclude` branches, 14 steps (0.2%). The stress fixture should show a + reduction in analysis events, which no other experiment here can produce. +- **Cost.** Low. Rule lists can be indexed per morpheme at `Morpher` construction. +- **Kill.** Less than 25% fewer analysis events plus synthesis steps on the stress fixture, or any + parse difference. A zero effect on the samples is expected and is not a kill. + +### E5. Required-environment prefilter on the input word (Maxwell) + +- **Hypothesis.** For each allomorph whose environments are all `Require`, compute once per input + word whether any of them can match anywhere: the left context ending at some position `p`, and the + right context starting at some `q >= p`. Allomorphs that cannot match are skipped during analysis. +- **Soundness.** Environments are checked on the final shape, and `IsMatch` ties that shape to the + input, as argued above. Three restrictions apply. Boundary nodes match empty. The prefilter is + enabled only when the grammar cannot produce underspecified surface segments (a static scan of + lexical entries, inserted affix material and rule outputs; featureless grammars qualify). An + allomorph with any `Exclude` environment is never skipped. Skipping analysis allomorph `i` is sound + only because PR #548 restricts synthesis to the recorded allomorph. +- **Hook.** Compute the set per word in `Morpher.ParseWord` (`Morpher.cs:167-179`). Consult it in the + allomorph loop of `AnalysisAffixProcessRule.Apply` (`:61-102`), skipping `_rules[i]`. +- **Language types, and blow-up.** Environment-conditioned allomorphy is everywhere: Turkic and + Uralic harmony, Bantu, Korean, Austronesian nasal substitution. It matters most for **zero + allomorphs with environments**, which analysis can unapply in any word. With `z` optional + zero-allomorph affixes, late checking multiplies the analysis tree by up to `2^z` before + `IsWordValid` disposes of the extras. The prefilter removes each one whose context is absent from + the whole word. It is weak when contexts are common, because it does not know positions. +- **Stress fixture.** New synthetic `stress-env-zero-allomorphs`: three affixes, each with an overt + allomorph and a zero allomorph conditioned by an environment (for example ∅ / _ [+nasal]), with + words that mostly lack the conditioning contexts. Controls: `edge-cases/discontinuous-morph-environment`, + `edge-cases/diacritic-segments` and `languages/suffixing-vowel-harmony`. Sena (72 required-environment + sets) is the real grammar most likely to move. +- **Expected savings.** Samples: `Environments/Require` has 98 real steps (0.0%) and 32 conformance + steps (0.4%). Pruning at H1 also removes analysis events, which the terminal counts do not show. + The stress fixture is the real test. +- **Cost.** Low to medium. The left matcher runs right to left from each position, with the existing + compiled patterns reused on the input shape. +- **Kill.** Less than 20% fewer analysis events plus synthesis steps on the stress fixture, or the + grammar condition disables it on four of the five real grammars and on most conformance languages. + Any parse difference is a soundness bug. + +### E6. Intermediate-shape check for SurfaceFormMismatch (research only, no brief) + +`SurfaceFormMismatch` is the largest terminal reason that no experiment above touches: 25.6% of +real-grammar steps, 37.9% on Indonesian and 27.2% on Mbugwe. It fails last, after a full synthesis. A +check at each stratum boundary that the synthesized shape still matches the analysis shape the +candidate came from could stop a candidate early. There is no soundness argument yet. Analysis +shapes are over-approximations with optional and underspecified nodes, and ledger row 8 (a +synthesis-side length bound) lost valid parses. Run a census only: at each boundary, record whether +the check would fire, and whether any successful candidate ever fails it. **Kill** if any success +fails it on any corpus. + +### Not an experiment + +- **Cheaper `RequiredMprFeatures` rejection** as such: each rejection is one set lookup (caveat 2), + and ledger row 7 is the precedent. E1 removes the ones past the recorded allomorph as a side effect. +- **Nogood learning over candidates**: ledger row 14 found 0 subsumable pairs. +- **Moving constraints into the analysis merge key**: ledger row 15. Every synthesis rejection depends + on the root, so H4, not the analysis key, is the place. + +## Results of the round + +Five experiments were built, each on its own unpushed branch from `origin/master` `c2454760`, with the +harness commit `af97348b`. Every result is steps only. "Synth" is synthesis steps, "total" is +analysis events plus synthesis steps for the same words in the same arm pair, and "% less work" is +the relative drop. Parses were identical per word in every arm pair on every corpus, except that the +four `truncate-morphotactic*` conformance fixtures are nondeterministic on master (see Open problems). + +### Decisions + +| Experiment | Branch @ SHA | Decision | +|---|---|---| +| E1 allomorph-loop cutoff | `perf/hc-allomorph-loop-cutoff` @ `d21af37e` | **KEEP.** Candidate for its own PR | +| E2 MPR and stem-name trail replay | `perf/hc-trail-mpr-replay` @ `4fbbfb14` (product `84f76cdf`) | **KEEP.** Candidate PR, to combine with E3: both use the same post-lookup hook | +| E3 template replay | `perf/hc-template-replay` @ `6edc4375` | **KEEP, without memoisation.** See E3 | +| E4 excluded co-occurrence | `perf/hc-excluded-cooccurrence` @ `02873608` (product `b07fc4d6`) | **PARK** | +| E5 required-environment prefilter | `perf/hc-env-input-prefilter` @ `9d55dade` (product `1af79e20`) | **DEAD END** as specified | +| E6 intermediate-shape check | none | Not run (research only) | + +None of the branches is pushed. A branch name and SHA is the citation. + +### Synthesis share of total work + +Synthesis steps and analysis events for the identical words, switch off: + +| Corpus | Synth | Analysis events | Synth share of total | +|---|---:|---:|---:| +| Aweti stride 2 | 15,940 | 1,544,240 | 1.0% | +| Mbugwe stride 40 (E2-E4 word set) | 130,647 | 1,318,654 | 9.0% | +| Sena stride 100 | 1,908 | 103,277 | 1.8% | +| Amharic limit 28 | 1,277 | 7,403 | 14.7% | +| Indonesian | 677 | 6,552 | 9.4% | +| Conformance | 8,246 | 27,765 | 22.9% | + +The E1 run used a Mbugwe word set with 3,451,016 analysis events (a different drop set), which is +why its Mbugwe row differs below. Analysis events never changed in E1, E2, E3 and E5-off arms. + +### E1. Allomorph-loop cutoff: KEEP + +`SynthesisAffixProcessRule.Apply` now stops after the recorded allomorph (`Morpher.CutAllomorphLoopAtRecorded`, +default on). Kill criterion not met. The soundness argument held: no parse changed. + +| Corpus | Synth off to on | % less synth | % less total work | +|---|---|---:|---:| +| Mbugwe stride 40 | 151,505 to 138,737 | 8.43% | 0.35% | +| Sena stride 100 | 1,908 to 1,772 | 7.13% | 0.13% | +| Aweti stride 2 | 15,940 to 15,831 | 0.68% | 0.007% | +| Amharic limit 28 | 1,277 to 1,271 | 0.47% | 0.07% | +| Conformance (minus 4 flaky) | 8,150 to 8,109 | 0.50% | 0.12% | +| Indonesian | 677 to 677 | 0% | 0% | +| `stress-allomorph-list-density` (Turkic-style) | 566 to 337 | 40.46% | 8.45% | +| `stress-prefix-class-density` (Bantu-style) | 341 to 190 | 44.28% | 9.49% | + +Cost not in the counters: one `IndexOf` over the rule's allomorphs per rule application. Unaffected: +analysis events and candidates (equal in both arms). + +### E2. MPR and stem-name trail replay: KEEP + +`TrailMprReplay` and `Morpher.PruneByTrailReplay` (default on) drop a candidate at lexical lookup +when its trail cannot satisfy required or excluded MPR features or the stem name. Two refinements to +the brief's predicate: the excluded test is disabled only when an `Overwrite` group is reachable from +a rule output, and the stem-name check stops after a compounding entry. Family roots are never +pruned. The shadow census agreed exactly with the on arm and found no successful terminal under a +rejected candidate anywhere. Kill criterion not met. + +| Corpus | Synth off to on | % less synth | % less total work | Replay checks | +|---|---|---:|---:|---:| +| Aweti stride 2 | 15,940 to 6,873 | 56.9% | 0.58% | 14,064 | +| Mbugwe stride 40 | 130,647 to 115,747 | 11.4% | 1.03% | 26,299 | +| Conformance | 8,246 to 8,218 | 0.3% | 0.08% | 251 | +| Sena, Amharic, Indonesian | unchanged | 0% | 0% | 0 | +| `stress-mpr-inflection-classes` (IE declensions) | 494 to 278 | 43.7% | 6.45% | 180 | +| `stress-mpr-bantu-concord` (three gated slots) | 3,376 to 382 | 88.7% | 31.85% | 652 | + +Replay checks are set operations, a different unit from synthesis steps (a step is a pattern match), +so they are reported beside the savings and not netted. The Aweti and Mbugwe gain comes from required +and excluded gates (Mbugwe 41 excluded, Aweti 26 excluded). + +### E3. Template replay: KEEP, without memoisation + +`TemplateReplay` and `Morpher.PruneByTemplateReplay` (default on) walk the stratum, template and rule +control flow over abstract states (trail pointer, partial, final), assuming every pattern match +succeeds. It bails out on family roots, non-affix-process rules on the trail, any realizational rule +in the grammar and a null stratum. The shadow census found no successful terminal under a rejected +candidate. Phase 0 census (steps in candidates with no success and shape-free): Mbugwe 40.0%, Aweti +73.4%, so the kill rule for the census was not met. + +| Corpus | Synth off to on | % less synth | % less total work | Replay states | +|---|---|---:|---:|---:| +| Aweti stride 2 | 15,940 to 5,112 | 67.9% | 0.69% | 22,308 | +| Mbugwe stride 40 | 130,647 to 114,833 | 12.1% | 1.09% | 133,606 | +| Amharic limit 28 | 1,277 to 532 | 58.3% | 8.58% | 718 | +| Sena stride 100 | 1,908 to 1,790 | 6.2% | 0.11% | 4,166 | +| Conformance | 8,246 to 8,181 | 0.8% | 0.18% | 10,622 | +| Indonesian | 677 to 667 | 1.5% | 0.14% | 389 | +| `stress-derivation-chain-homophone-roots` (Turkic-like) | 300 to 176 | 41.3% | unmeasured | 948 | +| `stress-bantu-extension-required-slot` | 279 to 225 | 19.4% | unmeasured | 763 | +| `stress-required-template-slot` (control) | 155 to 155 | 0% | 0% | 329 | + +The three stress rows do not give analysis event counts in the result files, so their total-work +share is unmeasured. The control has no gain because dead candidates there cost 0 steps. + +The brief's blow-up kill rule says to kill if walk states exceed steps saved, and it fired on every +corpus (Mbugwe 133,606 states against 15,814 steps saved; Aweti 22,308 against 10,828). The result +file therefore said PARK. The owner overrode the literal rule: a replay state is an integer set +operation and a saved step is a pattern match, so the two are different units and cannot be compared +by count. **Decision: KEEP, without memoisation.** Memoising verdicts by root entry and trail is +ruled out: the owner will not accept a memory footprint above a few MB, and earlier memo rows (18, 28, +29) show how such tables grow. Next step: cost the replay walk in units comparable to a step, then cut +the walk itself without a cache, for example by skipping it when the grammar has no required slots or +by stopping at the first violated constraint. + +### E4. Excluded co-occurrence during analysis: PARK + +`ExcludedCoOccurrencePruner` indexes active `Exclude` + `Anywhere` morpheme and allomorph +co-occurrence rules at `Morpher` construction. Rules that name only affixes are checked in analysis, +right after unapplying (H1). Rules that name a root are checked per alternative after lookup (H4). +It is sound and it is the only experiment that cuts analysis events. + +| Corpus | Analysis events off to on | Synth off to on | % less total work | +|---|---|---|---:| +| Aweti, Mbugwe, Sena, Amharic, Indonesian | unchanged | unchanged | 0% | +| Conformance (1,492 words) | 27,765 to 27,705 | 8,246 to 8,223 | 0.23% | +| `stress-excluded-cooccurrence` (brief's fixture) | 4,690 to 4,605 | 326 to 250 | 3.2% | +| `stress-athabaskan-prefix-complex` | 8,937 to 6,347 | 891 to 404 | 31.3% | +| `stress-bantu-extension-chain` | 10,552 to 8,548 | 515 to 332 | 19.8% | + +The brief's kill criterion (under 25% on its own fixture) fires on 3.2%. The two dense fixtures were +added at the owner's request. **Decision: PARK.** Land it only as a cheap safety net for +co-occurrence-heavy grammars. Findings that affect any landing: the "zero-width other is invisible" +trap does not occur for affix-process allomorphs (synthesis marks the morph on the last output node); +merged analyses must be filtered per alternative, not on the canonical word; with pruning on, +`MaxAlternatives` counts fewer candidates, so its exception can stop firing; and the rule index is +read at construction, so a rule added later is not pruned early. + +### E5. Required-environment input prefilter: DEAD END as specified + +The code is recoverable at `1af79e20`. The sound static scan disables the prefilter on 5 of 5 real +grammars (four with ambiguous digraph segmentation, one with non-uniform feature sets) and on 145 of +150 conformance runs, so it gives 0% on every real grammar. Forced on, with the scan bypassed (an +unsound licence), Sena saves 4.0% of synthesis steps and 7.3% of analysis events (net 5.7% after +counting matcher invocations) and Aweti is net -0.4%. + +| Fixture | Synth | Analysis | Gross % less total | Net % less work (matcher calls counted) | +|---|---|---|---:|---:| +| `stress-env-zero-allomorphs` | 87.4% less | 74.1% less | not given | 65.2% | +| `stress-env-bantu-concord` | 1,344 to 732 | 2,036 to 1,207 | 42.6% | -3.8% | +| `stress-env-austronesian-nasal-prefix` | 936 to 883 | 2,206 to 1,471 | 25.1% | -37.4% | + +Forced on, it is net negative on two of the three fixtures. Soundness findings: the brief's +counterexample direction (an archiphoneme `[+cons]` against a `[+voice]` context) does not hold under +subsumption. The real unsound case is a synthesized node more specific than the input character +definition (test word "atpa": `D` against `t`); forcing the prefilter on loses that parse, and the +static scan is what prevents it. + +**What would have to change to revive it.** (1) A scan that stops rejecting digraph grammars, which +needs a segmentation of the input that is unambiguous per grammar, or a prefilter that tests all +segmentations. (2) A treatment of non-uniform feature sets and of synthesized nodes more specific +than the input segment definitions. (3) A prefilter cost small enough that matcher invocations do not +cancel the saving, since two of three fixtures go net negative. (4) Optionally, root allomorph +environments (67 in Sena) prefiltered at lookup, which was not built. Absent (1) and (2) the +eligible set on real grammars is empty (indonesian 0 of 13, aweti 47 of 394, amharic 0 of 93, mbugwe +0 of 415, sena 5 of 117). + +### E6. Intermediate-shape check: not run + +No soundness argument yet; `SurfaceFormMismatch` (25.6% of real synthesis steps) is still untouched. + +### Conclusion + +Each of E1-E3 is sound on every corpus and cuts synthesis steps by a large amount on grammars that +lean on the feature (inflection classes, ordered allomorph lists, template slots). But synthesis is +1.0% of total work on Aweti and 9.0% on the Mbugwe sample, so on real grammars each removes about 1% or less of +total work (E2: 0.58% Aweti, 1.03% Mbugwe; E3: 0.69% and 1.09%; E1: 0.007% and 0.35%). E2 and E3 overlap +and were not measured together. **The dominant cost is analysis-side.** The next round should +target analysis, for example by extending E4-style monotone checks earlier, and the +`ApplicableTemplatesNotApplied` analysis paths. E4 shows an analysis check can cut work, but only +where the grammar has the constraint; the question is which constraints real grammars have. + +### Open problems found at baseline + +1. The four `truncate-morphotactic*` conformance fixtures (`truncate-morphotactic`, `-copy-u`, + `-identity`, `-subsumption`) are nondeterministic on master: their output varies run to run with no + change. They fail in both arms of every experiment, and `EveryConformanceFixturePassesSelfCheck` + fails on them. +2. `FieldworksProducibilityLedgerTests.CoversExactlyEveryFailureReasonAndEveryInterfaceAttributeInThisRepo` + fails on master because PR #548 added `AllomorphNotApplicable` without updating that ledger. +3. The stress fixtures live under `tests/SIL.Machine.Morphology.HermitCrab.Tests/Research/stress-fixtures/`, + not under `conformance/`, because registering a fixture needs ledger and coverage regeneration + (a full test cycle takes 34 to 53 minutes under load). E5's branch registered three fixtures and + broke 8 pinned ledger tests (row totals, fixture count, layer counts, interaction and inventory + ledgers, real-corpus pair counts); E2's registration attempt broke 7. Moving the fixtures is a + ledger-regeneration job. +4. Full-suite evidence: E3's run finished at 215 of 217 passing; the two failures are the truncate + flake and a SIGTERM under load. E4's run was 723 of 725, with the truncate flake and the ledger + failure of item 2. `local_check.sh --agent-strict` could not finish on these machines because the + SentencePiece step needs `cmake`; comment hygiene and formatting passed. + +## Open questions + +- The candidate-level dead-step census (E3 phase 0) was run: shape-free dead steps are 73.4% on Aweti + and 40.0% on Mbugwe. What is left is the cost of the replay in comparable units, cut without a memo (E3). +- Which analysis-side constraints real grammars carry, and whether they can be checked before a + candidate is expanded. This is the next round's target. +- Whether synthesis phonological deletion removes morph annotations. E4's zero-width restriction may + need to cover deleted morphs too. The E4 brief requires a test. +- Whether `ChooseInflectionalStem` families occur in FLEx exports at all. None of the five samples has + one. E2 and E3 bail out on them. diff --git a/docs/hermitcrab-optimization-ledger.md b/docs/hermitcrab-optimization-ledger.md new file mode 100644 index 000000000..eaddf40f9 --- /dev/null +++ b/docs/hermitcrab-optimization-ledger.md @@ -0,0 +1,369 @@ +# HermitCrab optimization ledger — tried, closed, do not retry + +**Purpose: stop this work being redone.** One row per optimization attempted, what was expected, +what actually happened, and the number that settled it. If you are about to try something on this +list, read its row first — several of these look irresistible on paper and three of them have +already been independently rediscovered and re-retracted. + +Deliberately in-repo and durable, not evicted into a PR accordion. A closed avenue is only closed +if the next person can find out cheaply that it is closed. + +**This is the only report.** The working docs it was distilled from +(`hermitcrab-packed-forest-research.md`, `hermitcrab-forest-memo-plan.md`, +`hermitcrab-forest-memo-ceiling.md`, `hermitcrab-synthesis-fold-probes.md`) are chronological +process records containing retracted intermediate claims, and are deliberately **not** merged. +They are preserved only as this record. **The evidence branches were deleted after the round +closed**, along with all instrumentation: research scaffolding is not worth maintaining in two +product assemblies, since the probes gate on `volatile bool` reads inside `Matcher.cs`, a hot inner +loop in `SIL.Machine` used by every consumer. + +**`docs/hermitcrab-probe-design.md` is the replacement.** It carries the probe architecture and its +two-assembly gate trap, the seven exclusive wall-time buckets and why `synForward` must be recorded +net of cascade and battery, the per-node decomposition, the census designs, all three key designs +with field-level audits, the classifier's direction trap, and the full A/B methodology with the +three measurement failures that forced each part of it — enough to rebuild any probe in a day. The +expensive part was never the code; it was learning what to measure and which measurements lie. + +**No instrumentation is merged, by design.** The probes gate on `volatile bool` reads inside +`Matcher.cs` — a hot inner loop in `SIL.Machine` used by every consumer — which is an acceptable +cost for a measurement run and not one to carry in product code indefinitely. Rebuilding a probe +from a branch is cheap; maintaining research scaffolding in two product assemblies is not. + +## Unmeasured search-asymptotics research + +[Analysis-search research](hermitcrab-search-asymptotics-research.md) integrates six independent reports, +checks their code claims, and separates new experiments from this ledger's closed families. It ranks +residual lexical viability, a grammar-compiled template battery, recoverable trace-class reduction, and +exact finite-state compilation for a verified subgrammar. It maps them to lexicon-driven pruning during +unapplication and making the affix-template battery cheaper without memoization. The research doc itself claims no measurement. Its first two candidates were measured on +2026-10-21 (rows 27 and 26): see [the search-cost round](hermitcrab-2026-10-21-search-cost-round.md). + +## Final disposition of the September follow-up + +The follow-up branch does deliver product improvements, retained on `perf/hc-optimization-archive` and not +merged to master. Its authoritative summary is `hermitcrab-perf-2026-09-counts.md`, and each retained +optimization has one record under `docs/optimizations/`. The retained set is: deferred template cloning, +feature-structure traversal-state reuse, frozen-flat-tree equality, frozen-flat-tree cloning, single-pass +`Shape.CopyTo` mapping, copy-on-write shape sharing, flat FST register arrays, the analysis edge-segment +prefilter, and copy-on-write syntactic feature structures. + +Four attractive experiments were deliberately excluded from the final product diff and are recorded under +`docs/rejected-optimizations/`: the negative two-pass nondeterministic traversal, the inert final-template +prune, the zero-value alternatives memo/hoist, and the low-ceiling compounding split prefilter. This final +disposition supersedes intermediate “ships” language for those experiments later in this historical ledger. + +The minimized branch passed all 603 runnable tests in the conformance-inclusive suite (one expected Windows +symlink-privilege skip). A fixed three-word sample matched on `WordAnalysisSignature` on Sena, Amharic, +Indonesian, Mbugwe, and Aweti; on Sena, Mbugwe and Amharic that signature compares morph counts, not +analyses (`hermitcrab-perf-2026-09-disposition.md`). Only Amharic (1.76x) and Mbugwe (1.83x) cleared the +recorded spread and timing reliability rules. The other three ratios are retained as directional evidence +only. + +## Earlier ledger round: net speed delivered none + +No optimization in the "closed" section shipped, and none of them made anything faster. The +fastest thing tried was 4% **slower**. The pre-existing wins in this codebase (rows 1–3) predate +this work. What this effort produced is the measurement, the closed family, the located target, +and this ledger. + +--- + +## Earlier shipped results + +| # | Optimization | Expected | Outcome | Number | +| --- | --- | --- | --- | --- | +| 1 | Memoize analysis morphological-rule cascade | Same state reached by many rule orders; cache it | Works, but the cascade was never the cost | 2,555 expansions vs a 2,546-state floor; 1.4 s of a 30.5 s word | +| 2 | Memoize affix-template battery (same key) | Battery was 93% of wall time | **The big win.** Reduced how *often* the battery runs | 38,840 runs -> 2,581; word 30.5 s -> **6.1 s** | + +### Retracted historical prototype + +Row 3 in the original ledger claimed that shape sharing through `CloneShareFrozenShape` had shipped. No +reachable production commit carried that implementation. Its historical prototype measured 4.5–7.6% fewer +allocated bytes and no wall-time change. The September follow-up independently built a different, retained +copy-on-write design through `CloneForEngine`; see `docs/optimizations/shape-sharing-on-clone.md`. + +## Closed — measured, does not pay + +| # | Optimization | Expected | Why it failed | Number | +| --- | --- | --- | --- | --- | +| 4 | **Narrow `AnalysisStateKey`** (drop strictly-shrinking rules' un-application counts) | `MaxApplicationCount` defaults to 1, so the count multiset is "which subset of rules was used" — 2^27 states. Dropping 19 of 27 rules gives 2^8 | Shape + syntactic FS + realizational FS *already* discriminate. The count component was implied by its neighbours. A worst-case bound collapsing is not the realised state count collapsing | Realised state collapse **Sena 1.12x** (state-weighted) vs a 1.3 gate; **4 of 7** heavy probes exactly 1.00, including the 2,556-state probe. The three that moved were 1.13x, 1.16x, and **1.38x**, and all returned zero parses: R tracks failure, not size. **Reopening condition:** a grammar with genuinely high R; the census is cheap and is the right first question to ask of any new grammar | +| 5 | **Fold-step sharing memo** (share deterministic synthesis fold steps by computed value) | Probe measured 3.22x and 8.10x shareable steps; ~50% wall ceiling on two reliable fixtures | A sound key must carry the **ordered remaining trail**. Two candidates then have to agree on their entire future to share a step, which almost never happens | **hits = 0** on the best fixture; **0.96x** (4.5% *slower* — pure key-construction cost); 1.06x on the second, inside 21.9% noise | +| 6 | **Order-insensitive synthesis-input dedupe** | 9,774x duplicate synthesis inputs | HermitCrab's morphological rules are **not order-invariant**: same rule multiset, different order, different output. Merging loses parses | 2 violations on Sena, 12 independently on Indonesian. Sound version: **15–40%** | +| 7 | **Trail-position indexing of synthesis rules** | 11,445,538 of 11,445,538 rejections are "this rule is not the pending trail rule" — 100% of the histogram | Each rejection is one array index plus one reference compare. Count is not cost | 11.4M x 29 ns = 0.33 s of 143 s = **0.2%** | +| 8 | Synthesis-side length bound (Gate A) | Reject candidates that cannot reach the surface length | At the comparison point the candidate is still the bare root; its affix trail applies later inside `_synthesisRule.Apply`. Rejected valid parses | Unit suite 64 -> 34 passing. Reverted | +| 9 | Analysis length ceiling (Gate B) | Same idea, analysis side | Sound and correct. No available corpus contains the pathology it prunes | Byte-identical output, **no measurable speedup** (~4–7% slower, within noise) | +| 10 | Lexical reachability gate (Phase 5) | Prune candidates from which no lexicon root is reachable | Both reference corpora have real compounding in their deepest stratum, which disqualifies the gate everywhere | Proven **no-op** on both corpora | +| 11 | Tandem lexical intersection (T2) | Kill doomed branches early via the lexicon | The oracle only sees lexical unreachability. The expensive words fail on checks that run *after* lexical lookup succeeds | Pooled **23.5%** dead steps vs a 30% gate; one failure word at exactly **0.0%** dead | +| 21 | **Memo-key caching on the frozen `Word`** | Keys are built ~28x per distinct state, which looked like a cheap win. | High count did not imply high cost; caching would add state and invalidation complexity around negligible work. | **CLOSED:** all `AnalysisStateKey` construction and hashing on Amharic measured **3 ms of 81,707 ms = 0.0%**. | +| 12 | Pool small short-lived collections | Fewer allocations, faster | `HashSet/Dictionary.Clear()` is O(capacity); Gen0 already beats pooling at this size | -15–17% bytes but **+8.6% wall**, +12% on the parallel battery. Reverted | +| 23 | **`(State, AnnotationIndex)` traversal dedupe** (PR #511) | `Advance` forks per `Optional` annotation, so instances are exponential in optional count and independent of `|States|`; keying on state+position bounds it at `N * |States|` | The key omits what filters the traversal. Variable bindings gate later arcs; registers carry the captures `AllSubmatches` consumers read | **112 of 20,000 differential-fuzz cases change `Match()`** (108 bindings, 4 shortened ranges) while `AllMatches().First()` is identical in all 20,000. Narrowed to the deterministic method with no capture groups: **0 divergences**, full bound preserved. Extension censuses: environment matchers are **0.20%/1.18%** of instances (Amharic/Mbugwe); the analysis rules' apparent **93.95%** collapse is **99.7% alternate captures**. See `rejected-optimizations/state-position-traversal-dedup.md` | + +## Closed before building — motivation refuted by measurement + +| # | Optimization | Expected | Why it died | +| --- | --- | --- | --- | +| 13 | Surface-length pruning inside the synthesis fold (P2) | "The only lever on Amharic's ~160 ms per synthesis run" | Amharic's forward synthesis is **14 ms**. The 160 ms was arithmetic on an unmeasured denominator — dividing 30 s by 186 synthesis inputs and assuming the time was there | +| 14 | Nogood lattice subsumption (P4) | Nogood hits dominate positive replays **434,628 to 25,102** — the engine mostly proves subtrees *empty*, and generalising "empty" over the feature lattice would turn nogoods from points into regions | **CLOSED on direct evidence** (it was previously closed on adjacent evidence, which is why it was reopened). Two heavy Sena probes produced 16,176 expansions -> 16,176 distinct keys -> **16,176 singleton buckets, 0 subsumable (0.0%)**, and 12,595 -> 12,595 -> **12,595 singletons, 0 subsumable**. Not one pair of nogood entries even shares a `(shape, stratum, counts, non-head)` bucket, so there is nothing for a subsumption ordering to act on. Breadth across all 33 conformance fixtures agrees: 7,493 empty expansions, 7,398 distinct keys, **0 subsumable (0.0%)** pooled, every non-empty bucket a singleton. Direction checked explicitly as `earlier.Subsumes(later)` (E subsumes Q, general over specific); the reverse would be unsound. Independently reconfirms row 4 from a different angle: the keys are maximally discriminating | +| 15 | Move constraints into the analysis key (Maxwell & Kaplan's category-splitting) | Their biggest measured win: make the chart prune what the constraint solver otherwise would | Everything synthesis rejects on is root- or realization-dependent, and the root is unknown until `LexicalLookup`. Also a Sena analysis state costs ~0.73 ms while a synthesis input costs at most ~0.12 ms — **states are dearer than synthesis inputs** | + +## Reopened this round, and their outcomes + +| # | Optimization | Expected / observation | Outcome | +| --- | --- | --- | --- | +| 22 | **Analysis-side rule prefilter** | 94.4% of Sena allomorph pattern matches failed, so a cheap necessary-condition check had a large direct-work ceiling. | **RETAINED after the earlier self-time cap was refuted by direct work counts.** The original ~10% ceiling counted matcher self-time but omitted traversal allocation, feature unification, and GC booked elsewhere. The implemented edge-segment prefilter removed **87% of FST traversals**, 54% of arc checks, and 85% of traversal instances on 45 Sena words, with `WordAnalysisSignature` parity on those words (a morph-count comparison on Sena; see `hermitcrab-perf-2026-09-disposition.md`) and all 464 conformance-fixture words. See `docs/optimizations/edge-segment-prefilter.md`. | +| 16 | **`ExpandAlternatives` dedupe** | 20.7% of Sena wall, the largest bucket nobody had instrumented. Apparent census: 395,026 alternatives -> **61** distinct (0.0%), 85% of duplicates same-analysis-word | **CLOSED on the sound census.** With the pending-trail content added to the key, 395,026 -> **351,414 distinct (89.0%)** -- only **11%** are duplicates at all, and all 43,612 of them are same-analysis-word. Sound ceiling = 20.7% x 0.110 = **2.28% of Sena wall**, against a gate of >=10% to build and <5% to close. **The fourth apparent redundancy to evaporate under a complete key** | +| 17 | **Per-node cost in the analysis cascade** | 84% of the Amharic cascade is not linguistic computation; clone is 46.7% and 37.4% was unattributed | **Partly settled, remainder still open.** (a) The remainder split **MISSES its <15% gate by ~2.5x** — still ~38% on Amharic `anCascade`. Memo, Trail and **Equality each measured <0.2%**, so `Word.ValueEquals` behind the cascade's `HashSet` is *not* the missing mass, contrary to the standing hypothesis. **Two caveats the first write-up missed.** (i) The ~38% figure comes from a *contended* run — three timing probes were live — and descheduled wall time lands preferentially in the remainder while deflating every self-time share, so **CPU starvation is itself an unbracketed candidate biasing in exactly the direction observed**. (ii) The claim that GC pause time is unmeasurable is **wrong at the measurement point**: only the library assemblies are netstandard2.0; the probe harness lives in `tests/SIL.Machine.Morphology.HermitCrab.Tests`, which targets **net10.0**, so `GC.GetTotalPauseDuration()` and `GC.GetGCMemoryInfo().PauseTimePercentage` are directly callable. Next probe, gated: re-run serial — remainder <20% means contention was the missing mass and 17(a) recloses on the cheaper-clone target; still >=30% then bracket with `GetTotalPauseDuration()` deltas — ΔPause >=50% of the remainder's absolute ms confirms GC (build target converges with clone/freeze at 46.7%), <=15% refutes it and the next split is enumerator/LINQ orchestration (the 2026-07 alloc audit already flagged `ExecuteCommands` enumerator boxing); a collection census over 24 Amharic words gives gen0 8,217 / gen1 2,729 / gen2 301, an unusually high gen1+gen2 ratio consistent with — not proof of — GC as the remainder. (b) **Clone-discard fraction = 66.2%** (created 2,505, discarded 1,659), a count and therefore contention-immune, against a >=70% gate: **misses by ~4 points, so the build is "make a clone cheaper", not "defer cloning"**. Scoped to `AnalysisAffixTemplateRule`'s eager per-template clone, the one clone-before-verification site; the other four analysis-side sites clone only after a successful match and were excluded rather than blended in | +| 18 | Corpus-scope memoization | `AnalysisScope` dies per word, but `AnalysisStateKey` is word-independent by construction, so entries are reusable across a corpus. Nogood hits dominate positive replays 10-17x, making cross-word nogood reuse the cheapest available form of reuse | **CLOSED, and worse than a no-op.** Indonesian, 121 words, **serial on a clean machine: 1.015x, 1.4% wall-clock reduction** against a >=20% build gate and <5% close gate. Costs **68 MB** added peak managed memory. Cross-word reuse is real but small: 39 memo, 73 nogood, 237 template-nogood hits. Token-vs-type: **121 tokens, 121 distinct**, so no type-level dedupe is available either. **A first, contended run of the same harness reported 2.385x / 58.1%** with a 210.9% spread on the shared arm against 13.7% on the baseline; two testhosts were live. The counts were byte-identical across both runs — only the timing was contaminated, and min-of-N on a high-spread arm picks the luckiest sample. Analysis-set equality held throughout, so the key genuinely is word-independent. **Sena, first 300 words, is the decisive case: the shared scope is 28-32% *slower* than per-word scopes** (two clean serial runs, no contention: 282.7s vs 372.2s and 308.9s vs 396.7s baseline-vs-shared; deterministic counters byte-identical across both runs), and costs **~29.3 GB** added peak managed memory against a 2 GB budget. Root cause: `AnalysisScope.MaxMemoEntries` (100,000) is a *per-scope*, not per-word, budget. Two heavy early words (#74, #76 of 300) alone fill the shared `Memo` table to the cap and `TemplateMemo` to 55,071; every word after that loses the free within-word memoization a fresh per-word scope would have given it for nothing, because `Store` silently no-ops past the cap (see its comment) with no eviction. Cross-word hits (shared minus baseline) are **negative** — memo -77,648, nogood -747,607 — i.e. sharing produced *fewer* total hits than 300 independent fresh-table runs would have on their own; the small `+820` template-hit / `+9` template-nogood gain does not come close to covering it. Amharic, capped at 28 words (word 29 hangs, excluded per standing guidance): **0.9% wall-clock reduction**, table nowhere near the cap (Memo 1,251, TemplateMemo 652 of 100,000), consistent with row 17's finding that Amharic's cost is dominated by clone/allocation per node rather than distinct-state count, so avoiding a handful of cross-word re-derivations barely moves wall time. Token-vs-type on all three corpora, at the runs' own sizes and on each corpus's full word list: **zero duplicate tokens** — whole-word caching is not available as a cheaper alternative anywhere here. No analysis-set divergence on any corpus, at any scope regime: the stop condition that would have put rows 1-3 in question did not fire. **A shared-but-capped memo table is not "the same optimization, smaller ceiling" — past the cap it actively takes away the free per-word memoization the product already has, and retains every live entry for the run's full duration instead of per-word, which is where the 29 GB comes from.** Any future attempt needs real eviction (LRU, generational, or per-word sub-budgets) before this is worth re-measuring, and even then the >=20% gate looks unreachable given Indonesian's ceiling under an uncapped-in-practice regime was already only 1.4% | +| 19 | Contexted constraints / abstract feature-only replay | Pack without merging states by replaying only shape-independent constraints. | **CLOSED.** Cost-weighted P1b die-point histograms put shape-free work at **6.0%** and **0.0%** on two heavy Sena probes and **0.4%** on 28 Amharic words; all fire the <30% close gate. All 33 conformance fixtures were measured for breadth: 24 close, 3 nominally fire the build gate, 3 are undecided, 2 have zero rejections, and 1 has zero words. The three build-gate fixtures are all sub-50 ms and cannot carry timing claims. One reliable fixture confirms that count and cost can diverge, but die-point rejection cost itself is only ~0.1–0.2% of wall on the real corpora. Abstract replay would optimize a phase that costs approximately nothing on the corpora treated as ground truth. Not built. | +| 20 | **Row 5's family on `GenerateWords`** (generation, ~100% synthesis share vs parsing's ~5%/~0.3%) | Row 20 was unmeasured; generation's share is ~100% so row 5's fold ratios would apply at face value if they transferred. `UseSynthesisFoldMemo` had never been wired into `GenerateWords` (only into the parse-then-synthesize path) — wired in under the same untraced/`MaxDegreeOfParallelism==1` restriction to make this measurable at all | Hits are no longer uniformly zero: 3 of 23 lookup-reaching fixtures clear the 5% gate (diacritic-segments 33.3%, suffixing-vowel-harmony 28.6%, suffixing-evidential-adjacency-chain 37.7%; a 4th sits just under it at 4.8%), reopening row 5 narrowly for generation. But the reopened case recloses immediately: the A/B on the best fixture shows the wall-clock "win" sitting inside its own noise floor, and the pooled hit rate is dominated by `deep-optional-affix-nesting` (925 of 1,613 lookups) at hits=0, reproducing row 5's exact 4.5%-slower key-construction cost on the identical fixture. Parity: 0 divergences, 1,236 generations, 32/33 fixtures (33rd throws identically both sides). Best case: 130 hits / 5 reps (37.7% hit rate), **7.8%** wall-clock reduction inside a **22.2%** off-arm spread — not a result. 19/23 lookup-reaching fixtures at exactly **0%**; pooled hit rate **2.67%** (43/1,613), skewed by the one large zero-hit fixture. | + +## 2026-10-21 search-cost round + +Full record: [hermitcrab-2026-10-21-search-cost-round.md](hermitcrab-2026-10-21-search-cost-round.md). +The unit is HC-Rust parser work (summed per-word `work_steps`, deterministic), so "N% less work to +parse the words". Steps are secondary, and ms are labelled. C# figures are in C#'s own counters. +Rows 24 and 25 are **HC-Rust port bugs: C# was already exact, and no C# change follows from them.** + +| # | Optimization | Expected | Outcome | Number | +| --- | --- | --- | --- | --- | +| 24 | **Analysis-side affix literals keep `StrRep` identity on wide featureless tables** (HC-Rust only; PanGloss divergence 090 follow-up) | Running HC-Rust on C#'s own export of a private featureless project made 18 lookups on its slowest word, against 1,655 on HC-Rust's own snapshot (C#: 49). That points at a loading difference, not search | **KEPT in HC-Rust; no C# change.** Above 64 character definitions HC-Rust lost the literal's identity, so an enclitic matched any final segment. C# gives featureless segments a `StrRep` `EqualTo` constraint (`CharacterDefinitionTable.cs:68`) | 7 spot words: **98.5% less work** (2,365,602 -> 35,913); worst variant **99.8% less** (50,778,583 -> 83,677). Steps on full lists fell 46-55x. Parses IDENTICAL on three private lists and on Amharic, Sena and Indonesian | +| 25 | **Re-inserted segment-class node keeps its members on featureless grammars** (HC-Rust only; PanGloss entry 103, `414ac6b5`) | A deletion-rule variant of the same project hit 1 s timeouts. A profile showed analysis at 0.1-0.3% of work, synthesis at ~90%, and zero lexically dead branches | **KEPT in HC-Rust; no C# change.** HC-Rust's re-inserted node matched any segment, and its root trie ignored the member set. C# unifies against the class's `StrRep` union | Held list (7,206 words): **94.7% less work** (303,966,311 -> 16,220,190). The 16 words at the 1 s limit: **99.9% less**. IDENTICAL everywhere; byte-identical work where it cannot fire | +| 26 | **Shared trailing slot trie in the template battery** (research candidate 2; PanGloss 095; Machine **PR #546**) | Templates with identical trailing slots (rules, order, optionality) run the same slot batch on the same word. Sena's trie-exact ceiling is 448,573 batches, 11.3% of work | **KEPT and ported.** Control factoring within one battery call, with no cache. In C# it covers sequential, untraced walks without an alternative limit. It composes with row 2 | HC-Rust: Sena **11.5% less work** (31.6% fewer steps, 448,573 batches avoided = ceiling); Mbugwe 18.1% less (1,364-word partial); Aweti 0.35%; 0 on one-template grammars. C# batches: Sena sample 47,072 -> 32,287 (31.4%); Mbugwe sample 72,467 -> 58,486 (19.3%). C# tests: a 2,904-combination differential, 126/126 HermitCrab tests, conformance 58/60 equal to the unchanged base `aab237b2` with memo on | +| 27 | **Prune analysis branches that cannot reach a root** (research candidate 1; census `d1690c13`) | The hindsight lexically-dead ceiling is 16-75% of parser work | **CLOSED on the prunable share.** Most dead work is leaves, which cost as much to prove dead as to run, or nodes that may still compound (row 10's disqualifier, 36-46% on Amharic, Sena and Mbugwe). Root-reachable synthesis failure is untouched: 86-98% of synthesis work where measured | The plain dead-internal bucket, the only one a sound summary could remove at a profit, is **0-5% on 6 of 8 lists**. The exceptions are v1 at 12.2% and Aweti at 39.8%. **Reopening condition:** a shadow summary on Aweti/v1 deciding a meaningful share of that bucket | +| 28 | **Per-parse distinct-shape lookup dedupe / fixed-cap root-result cache** (research reports R4, R5) | "97,575 lookups over 56 shapes" | **CLOSED.** That figure was trace mode, which disables merging. It is a memo regardless | Untraced, 1.13-1.49 calls per distinct (stratum, shape). Repeat-lookup work is **0.5-3.3%** of parser work (10.4% only on row 25's pre-fix bug). Dead-candidate lookups are 30-75% of lookup work, but lookup is 1-12% of parser work (35.7% on the deletion variant before fix B) | +| 29 | **Rule-level memo per battery call** | Repeated slot-rule applications inside one battery | **CLOSED.** It is no bigger than row 26 and it is a memo | Sena repeats: 33.7M work units (rule-level) vs 32.9M (slot-level) | +| 30 | **Hoist `segs_of` / borrow segments in `Transduce::new`; bitset for the root-trie active set** (HC-Rust) | Fewer allocations and a less quadratic trie walk | **NOT BUILT.** Time-only, with 0 work units by construction, so outside the owner metric | ~5.1M avoidable segment-vector builds per private-v1 held run (5.48M slot-rule attempts vs 365K slot batches). `Vec::contains` on the trie active set is quadratic | + +**Lesson of the round:** two of its three large reductions (rows 24 and 25) were port bugs, found +by running the port on the oracle's own grammar export and comparing lookup counts. Do that before +optimizing the algorithm. + +## 2026-10-27 synthesis-to-analysis round (finished) + +Full record: [hermitcrab-2026-10-27-synthesis-to-analysis-round.md](hermitcrab-2026-10-27-synthesis-to-analysis-round.md). +It answers John Maxwell's question about which synthesis tests can move into analysis. The unit is +C# synthesis steps (rule-level trace events) and analysis events, counted by the test-only harness +on `research/failure-reason-stats` @ `af97348b`, never ms. Rows 32-36 were built and measured on unpushed branches; row 37 was not run. +Ranking covers every grammar type HermitCrab can represent, not only the five samples, and each +experiment carries a synthetic stress fixture. + +| # | Optimization | Expected | Outcome | Number | +| --- | --- | --- | --- | --- | +| 31 | **Failure-reason census** (measurement, not a change) | Maxwell's four candidates (excluded co-occurrence, disjunctive allomorph, environments, excluded MPR) would be where synthesis work dies | **MEASURED; the samples refute it.** On the five samples they are almost never terminal. The work dies on template paths and surface mismatches. Synthesis is only 1.0% (Aweti) to 9.0% (Mbugwe sample) of total work, so the dominant cost is analysis-side | Real grammars: ApplicableTemplatesNotApplied **43.5%** of synthesis steps, SurfaceFormMismatch **25.6%**, unattributed 22.1%, Maxwell's four **<0.1%**. Traced, so merging is off. Harness `research/failure-reason-stats` @ `af97348b`. See the [round record](hermitcrab-2026-10-27-synthesis-to-analysis-round.md#results-of-the-round) | +| 32 | **E1: stop the synthesis allomorph loop after the recorded allomorph** | Allomorphs past the one analysis recorded are applied and then discarded | **KEEP; candidate for its own PR.** Sound, parses identical. Unpushed branch `perf/hc-allomorph-loop-cutoff` @ `d21af37e` | Synthesis steps: Mbugwe **-8.43%**, Sena **-7.13%**, stress fixtures **-40.5% / -44.3%**. Share of total work (analysis plus synthesis): Mbugwe 0.35%, Sena 0.13%, Aweti 0.007%, stress 8.5% / 9.5% | +| 33 | **E2: replay MPR features and stem names along the trail at lookup** (covers excluded MPR) | Inflection-class gates decided before synthesis | **KEEP; candidate PR, combine with E3** (same post-lookup hook). Shadow census: no success rejected. Unpushed branch `perf/hc-trail-mpr-replay` @ `4fbbfb14` (product `84f76cdf`) | Synthesis steps: Aweti **-56.9%**, Mbugwe **-11.4%**, stress **-43.7% / -88.7%**. Total work: Aweti 0.58%, Mbugwe 1.03%, stress 6.45% / 31.85%. Replay checks (set operations) are reported beside, not netted | +| 34 | **E3: replay the template and final-state outcome at lookup** | Shape-free template-path deaths decided before synthesis | **KEEP, without memoisation.** The brief's kill rule (replay states above steps saved) fired on every corpus but compares int operations with pattern matches, so the owner overrode the literal PARK. No memo: the owner caps added memory at a few MB. Next: cost the replay in comparable units and cut the walk itself (skip it when no slot is required, stop at the first violation). Unpushed branch `perf/hc-template-replay` @ `6edc4375` | Synthesis steps: Aweti **-67.9%**, Amharic **-58.3%**, Mbugwe **-12.1%**. Total work: Aweti 0.69%, Mbugwe 1.09%, Amharic 8.58%; stress fixtures unmeasured (no analysis counts). Replay states: Mbugwe 133,606 against 15,814 steps saved **Superseded: E3 was later dropped on cost, see row 38.** | +| 35 | **E4: excluded co-occurrence during analysis** (Maxwell) | Anti-monotone; prunes analysis subtrees | **PARK.** Sound, and the only experiment that cuts analysis work, but 0% on the five real grammars. Land only as a cheap safety net. Unpushed branch `perf/hc-excluded-cooccurrence` @ `02873608` (product `b07fc4d6`) | Total work: real grammars **0%**, conformance 0.23%, brief's fixture 3.2% (kill under 25% fires), dense fixtures **31.3% / 19.8%**. Zero-width "others" are not invisible for affix-process allomorphs | +| 36 | **E5: required-environment input prefilter** (Maxwell) | Linear left-then-right scan per allomorph | **DEAD END as specified.** The soundness scan disables it on 5 of 5 real grammars and 145 of 150 conformance runs. Forced on it is net negative on two of three fixtures. Revival needs digraph-safe segmentation, handling of non-uniform feature sets and over-specific synthesized nodes, and a cheaper prefilter. Unpushed branch `perf/hc-env-input-prefilter` @ `9d55dade` (product at `1af79e20`) | Real grammars **0%**. Forced on: Sena -5.7% net, Aweti -0.4% net. Stress fixtures net 65.2% / -3.8% / -37.4% | +| 37 | **E6: intermediate-shape check against SurfaceFormMismatch** | Stop a candidate at the first stratum boundary where it diverges | RESEARCH ONLY; not run; no soundness argument yet | SurfaceFormMismatch is 25.6% of real synthesis steps. Census only, killed by any success that fails it | + +## 2026-10-27 analysis-side round + +Full record: [hermitcrab-2026-10-27-analysis-round.md](hermitcrab-2026-10-27-analysis-round.md). +It follows the synthesis-to-analysis round, whose conclusion was that the dominant cost is analysis. The unit is +analysis events plus synthesis steps from the census harness (`research/analysis-cost-census` @ `38e9622e`, master +base `c2454760`), never ms. Where a check does work of its own, the table gives the figure **net** of that work in +one unit, and says which unit. Issues #561-#568 and PRs #571, #572, #574, #575 are in sillsdev/machine. + +| # | Optimization | Expected | Outcome | Number | +| --- | --- | --- | --- | --- | +| 38 | **Earlier-round PRs: E1 (PR #560), MPR and stem-name replay (PR #570); E3 template replay dropped** (rows 32-34) | E1 and E2 were KEEP candidates; E3 was KEEP without memoisation, pending a replay cost in comparable units | **E1 and E2 became PRs #560 and #570. E3 was dropped on cost** (the owner rule forbids the memo that would have cut the replay walk). The code is retained at `perf/hc-lookup-replay-templates` @ `9efd059c` | Row 34's "KEEP" is superseded by this row. Synthesis is 1.0-9.0% of total work on real grammars, so none of the three could save more than about 1% of it | +| 39 | **R1: required-slot look-ahead in the template walk** (issue #561; **PR #575**) | Mbugwe 15-35% of events (est.), Sena 3-10%, stress fixture at least 50% | **KEEP.** Sound: 0 shadow violations on every grammar, parses equal per word. The plan's kill rule (check unifications more than 3x events saved) fired on Mbugwe at 3.19x and was **waived**: it compared unification calls with walk and trace events, which are unlike units. Replaced by net unification work in one unit. Overlaps PR #546, see row 43 | Analysis events, less work: Mbugwe **6.6%**, Sena **30.8%**, Aweti **7.6%**, Amharic **8.9%**, conformance 1.6%, Indonesian 0%, stress **74.6%**. Net in unification calls over the whole parse, check included: Sena **1.56% less**, Aweti **4.50% less**, stress **37.91% less**, Mbugwe 0.33% less and conformance 0.14% less (both break-even), Indonesian 0, **Amharic 0.63% MORE** (411,073 to 413,674 calls, 24 words) | +| 40 | **R2: root-substring prune at rule outputs, gated by static restoration flags** (issue #562; **PR #572**; reopens row 27) | Sena 8-17%, the other four real grammars 0 (flags off), stress fixture at least 40% | **KEEP, with a probe skip added.** Row 27's reopening condition (a shadow summary deciding a meaningful share) is met on Sena and the stress fixture, 0 violations. The prune is off on four of five real grammars (dropped part that can hold segments, reduplication, deeper phonology). A static gate on root count was **rejected as overfitting toy grammars** | Net of probes (1 probe = 1 event), less work: Sena **15.3%** (gross 17.7%), stress **66.5%** (gross 67.7%), Aweti, Amharic, Indonesian, Mbugwe **0 (flag off)**, conformance **15.3% MORE** (7,237 probes buy 92 events). The probe skip (keep when the output has at least as many segments as its input) removed 18 of 7,255 probes | +| 41 | **T1: lean trace, nodes keep shape text and no `Word`** (issue #563; **PR #571**; scored on memory) | Retained trace memory down 90-97% on Mbugwe yɔɔna (1,032 MiB) | **KEEP; lands after PR #534** (it rewrites the same callbacks). Opt-in `TraceManager.RetainWords = false`; default behaviour unchanged. Steps identical by construction and verified | Retained trace memory, lean vs full, sum over words: Mbugwe **93.8% less**, Aweti 93.6%, Sena 92.0%, Amharic 98.8%, Indonesian 98.7%, conformance 97.4%, stress 97.5%. yɔɔna 1,035.0 to 22.6 MiB. Parse counts, node counts, census rows and conformance verdicts identical | +| 42 | **R5: edge-segment prefilter port** (row 22's archive; issue #564; **PR #574**) | Scored 0 by the plan: a prefilter removes no step, it makes each step cheaper | **KEEP.** The plan's 0 was the wrong score: census steps do not move, and the work per step falls. The first write-up was gross and left out the prefilter's own unifications; the net table charges each one as one arc check | **Net matcher work** (traversals plus arc checks plus prefilter unifications), census, less work: Aweti **47.5%**, Mbugwe **19.6%**, Sena **19.6%**, Amharic **12.8%**, Indonesian **6.5%** (gross 6.9%), stress 77.7%, conformance 23.1%. Analysis events and synthesis steps **unchanged** on all 1,796 completed pairs per mode | +| 43 | **R1 on PR #546's shared-slot walk** (issue #566) | R1 and #546 compose; the residual after both is a ceiling only | **KEEP, stacked on #546 and #575.** Draft PR #577 (commit `bbb81a00`, Closes #566). Reviewed LAND. Shadow: 0 violations. The two compose in events, but on Mbugwe and Sena the check's own unifications cost more than the walk they save, so #546 alone keeps fewer unification calls there. | Events plus synthesis steps vs master, less work: Mbugwe **32.15%**, Sena **55.64%**, Aweti **20.06%**, Amharic **9.10%**. Unification calls vs master: Mbugwe **15.19% less**, Sena **4.81% less**, Aweti **12.18% less**, Amharic **0.63% MORE**. Against #546 alone: Mbugwe **1.07% MORE** and Sena **1.40% MORE** unification calls. Reopen condition for dropping one of the two (unchanged): R1 saves under 3% of slot events on the #546 walk where it saved over 5% on master. PanGloss port: sillsdev/PanGloss #66 (for #546 and #566) | +| 44 | **R3: ordered and edge-anchored required-slot look-ahead (R1 v2)** (issue #567) | Closes the part of R1's ceiling that single-segment affixes (`-a`, `a-`) leave: Mbugwe reaches only 10% of its no-lookup ceiling | **KEEP. Draft PR #576, stacked on #575.** Shadow: 0 violations on every grammar, parses equal per word. Mbugwe gains little over R1 in unification calls, because most of its required slots are not testable; the next step is an R1 purity extension with its own soundness argument, not more ordering. PanGloss port: sillsdev/PanGloss #65 (for #576) | Events plus synthesis steps vs master, less work: Sena **40.9%**, Mbugwe **7.3%**, Aweti **11.0%**, Amharic **11.9%**. Unification calls vs master, less: Amharic **2.75% less** (R1 alone was 0.63% MORE), Sena **5.22% less**, Aweti **9.27% less**, Mbugwe **1.83% less** | +| 45 | **R4: subsequence fallback and insertion classes for R2** (issue #568) | Ceilings: Mbugwe 38.5%, Indonesian 29.6%, Aweti 14.8%, Amharic 13.6% | **DEAD END.** Sound, but the sound version fires on none of the four target grammars except Indonesian, where it is a net loss. No PR. Code at `perf/hc-root-subsequence-prune`, order-free tier at `71cba028` | Net (probe = 1 step, each trie arc = 1 step): Mbugwe **0**, Aweti **0**, Amharic **0**; Indonesian **182.9% MORE work** with the order-free tier (gross 30.8% less, 631 probes, 16,779 arcs); conformance 19.1% more. The ceilings are unsound: the plain subsequence test violates on **334,508** events (Mbugwe), **50,187** (Aweti), 2,264 (Amharic), 1,105 (Indonesian) | +| 46 | **R6: keep equivalent-analysis merging on while tracing** (issue #565) | Events down 0.2-2.4%; lookups under tracing down by 1.67x (Mbugwe), 2.19x (Aweti) | **NOT BUILT.** It changes how Try-a-Word shows folded duplicates, which is an owner product decision, and it is low in steps | none | + +**Lessons of the round** (detail in the round doc): + +1. **Count a check's own cost in the same unit as what it saves.** R1, R2 and R5 each first reported gross + numbers, and each changed sign or size once its own work was counted. R1's 3x kill rule was waived because it + compared unlike units. +2. **Census steps miss work that cheapens each step.** R5 scored 0 in steps and saves 6.5-47.5% of matcher + work. Pick the unit that the change acts on, and report steps beside it. +3. **The `truncate-morphotactic*` conformance fixtures are nondeterministic on master.** Their output differs + run to run with no change. Exclude them from every parity comparison, and say so. +4. **No memoization** (owner rule). It is why E3's replay was dropped rather than cached, and why shared work + beyond #546 is reported as a ceiling only. +5. **Porting a change can expose a divergence in the other implementation.** The PanGloss port of machine #548 + found and fixed a real Rust divergence in alpha-environment selection (PanGloss PR #13). +6. **A ceiling measured with the flags ignored can be mostly unsound.** R4's census ceilings were mostly + restoration violations; the sound share was zero. + +--- + +## The lesson that generalises + +Four independent measurements, four boundaries, same collapse: + +| measurement | key used | apparent | sound | +| --- | --- | --- | --- | +| synthesis-input dedupe (#6) | order-insensitive | 9,774x | 15–40% | +| fold-entry census | trail position only | 6,476x | not established | +| fold-step sharing (#5) | trail position only | 3.22x / 8.10x | **hits = 0** | +| FSA traversal dedupe (#23) | state + position | 93.95% collapsible | **0.28%** register-identical | + +**The redundancy in HermitCrab's synthesis is apparent, not real. The trail is what makes each +step distinct, and every measurement showing large shareable work is measuring a key that omits +it.** The thing that makes sharing visible is the thing that makes sharing wrong. + +That closes a *family*: packed parse forests **of the merge-by-key kind**, fold-step sharing and +synthesis-input dedupe all require distinct derivations to converge on a genuinely identical +state. In this engine they do not converge -- the same fact as the rules being non-order-invariant, +seen from the other side. This argument did not itself close row 19 because a contexted-constraint forest +packs without merging states; row 19 was later measured independently and closed by its die-point costs. + +## Where the time actually is + +Measured with seven exclusive buckets summing to wall. Amharic is a single run (`unaccounted` +0.1%). **Sena is spliced from two runs -- see the note in its row.** + +| grammar | breakdown | +| --- | --- | +| **Sena** heavy words | battery **51.4%**, cascade 18.4%, all synthesis ~5% -- *pre-`ExpandAlternatives` run (143,303 ms, unaccounted 20.1%)*; `ExpandAlternatives` **20.3%**, unaccounted 1.5% -- *later run (112,713 ms)*. **Two runs: not co-summable.** One re-run would put all Sena shares on one denominator | +| **Amharic** | analysis cascade **~95%** of analysis, which is 99.5% of wall; **all synthesis 0.3%** | + +**Every optimization in this ledger that worked reduced how many nodes are visited. None touched +what a node costs.** Amharic spends **~170 ms per analysis state**. (The *measured* state-count floor -- 2,555 +expansions against 2,546 states -- is a Sena heavy probe. For Amharic, near-floor behaviour is +**inferred** from R ~ 1, which compares the full key against the narrowed key, not against a +semantic floor.) + +### Inside one node (Amharic, 28 words, both probe layers live) + +| bucket | `anCascade` 81,707 ms | `anBattery` 7,368 ms | +| --- | --- | --- | +| clone / freeze / allocation | **46.7%** | 41.9% | +| **remainder (unattributed)** | **37.4%** | 15.4% | +| matcher traversal | 12.5% | **39.3%** | +| FeatureStruct algebra | 3.4% | 3.5% | +| memo-key construction | 0.0% | 0.0% | + +Censuses: 27,385 rule attempts — guards reject 19.5% (application-count 6.3%, unifiability 13.2%), +**80.5% reach matching**. Of 25,178 allomorph pattern matches, **1,938 succeed (7.7%)**, 23,240 +fail (**92.3%**). + +**The headline: only ~16% of the Amharic analysis cascade is *identified* as linguistic +computation** -- matching 12.5% plus feature unification 3.4%. Clone/freeze is **46.7%** and +**37.4% is unattributed**, which is unknown rather than known-non-linguistic. The defensible +statement is that at most a sixth of the cascade has been shown to decide linguistic questions; +of the rest, half is measured (clone) and half is still open. + +**Gate outcomes, written before the run.** "Any bucket >=50% becomes the sole build target" — clone +is 46.7%, does not fire. "Pattern matching dominates *and* >=80% of attempts fail -> build a +prefilter" — 92.3% do fail, but matching is 12.5%, so it does not dominate; does not fire. +**"Remainder >=30% means the decomposition is incomplete and that is the result" — 37.4%, FIRES.** + +So there is no declared build target yet. The next probe splits the 37.4%: `RuleCascade` / +enumerator orchestration, trail dictionary operations, LINQ allocation, memo lookups, GC. + +**A caution on the 46.7% before anyone acts on it.** `Word.Clone` being half the cascade does not by +itself mean cloning is waste — the cascade forks a `Word` per rule application precisely so it can +explore alternatives without mutating shared state. The deciding question was what fraction of +clones are *discarded unused*: cloned, rejected by a guard or a failed match, thrown away. If most +are discarded, the lever is **not cloning until a rule commits**; if most survive, the lever is +**making a clone cheaper** (copy-on-write shape, which the RUSTIFY branch already explored). The +gate was >=70% discarded for the defer-clone build. **Measured at 66.2% (row 17b), it misses the +gate, so the main build target is the cheaper clone.** Row 17b counts only +`AnalysisAffixTemplateRule`'s eager per-template clone, the site the retained deferred template +clone changes; how that change's 1.054x relates to the 66.2% was not measured. + +## One durable code constraint + +It is not expressible in code, and it bites silently. + +**HermitCrab's morphological rules are not order-invariant.** Same pending-rule multiset, + different application order, different synthesis output — 2 cases on Sena, 12 independently on + Indonesian. This bounds every packed-readout scheme. It is the assumption + Maxwell and Kaplan (CL 19(4):571–590) rely on and this engine does not satisfy. + +## Calibration record — predictions vs outcomes + +Written before the measurements, scored after. Kept because the misses are informative. + +| prediction | outcome | +| --- | --- | +| Sena key collapse 2–4x | **1.12x** — wrong, and it was the number that decided the project | +| Indonesian key collapse 1.0–1.3x | 1.17x — right | +| Amharic "most likely to disappoint" | best static classification of the three (31 of 36 rules) — wrong | +| Fold sharing ~50% wall ceiling | **0.96x realised** — wrong; the ceiling was measured with a key that would lose parses | +| Amharic synthesis-bound at ~160 ms/run | forward synthesis is **14 ms** — wrong; arithmetic on an unmeasured denominator | + +## Method rules earned here + +1. **Counting is not timing.** Three published findings in this work were retracted, all arithmetic + on an unmeasured denominator: "synthesis is the bottleneck" (candidate counts), "Amharic is + synthesis-bound" (30 s / 186 inputs), "fixtures only look synthesis-heavy because they are small" + (divided by `synForward`, which excludes the buckets the work runs in). +2. **A ratio measured with an incomplete key is not a ratio.** See the table above, three times. +3. **Warm-up dominates small fixtures.** A first A/B here showed 1.53x purely from JIT; the off-arm + alone varied 42.7% between its own first and second sample. Discard warm-up, take min of N, + interleave arms, and print the off-arm spread as a noise floor beside every speedup. +4. **Sub-50 ms fixtures cannot carry a timing claim.** One 10 ms fixture's share moved 1.3% -> 14.5% + between two runs of identical code. +5. **Three grammars minimum, reported unpooled.** Maxwell & Kaplan measured a 100x swing between two + variants of one grammar. A pooled average across fixtures of different size has had to be + retracted twice in this project. +6. **Min-of-N is only valid when the arm's spread is small relative to the effect.** Row 18's + first run read 2.385x with a 210.9% spread on one arm and 13.7% on the other; serial and clean + it is **1.015x**. Min-of-N picks each arm's luckiest sample independently, so a high-spread arm + wins spuriously. Print the spread beside every speedup and refuse a number whose spread exceeds + it. +7. **Never dispatch timing-sensitive probes concurrently.** Three at once put four testhosts on the + machine and invalidated the bucket percentages in rows 17 and 22 as well as row 18's headline. + Counts survive contention; shares and speedups do not. +8. **A shared bounded cache without eviction is not a smaller version of the optimization** — it + competes with itself for a fixed budget and can land strictly worse than no sharing. Row 18's + shared scope produced *negative* cross-word hits. +9. **Never aggregate the test logger's console output.** It prints every line twice, the second + copy space-prefixed; a `^\s*` grep doubles every total. This produced a 2x error in row 20's + first write-up. +10. **Search completeness is never traded for speed.** HermitCrab is the permanent fallback engine + behind the FST work. +11. **A bucket reporting exactly `0.00` is broken until proven otherwise.** `NodeCostProbe.Enabled` + was declared and read in `Matcher.cs` but set by nothing — two gates in two assemblies, one + wired. Both lower-layer buckets read `0.00ms`, which looks like "measured and negligible" + rather than "never ran", and their time silently inflated the remainder. Wire cross-assembly + probe gates from a single place. +12. **Always carry an explicit remainder column.** It caught the wall-split misattribution and the + dead probe gate. Four of the five retractions in this work would have been caught at the point + of measurement by a remainder that did not sum. + +--- + +## Addendum, September 2026: the round that followed, measured by counts + +Branch `perf/hc-defer-template-clone` (on upstream master, after the merged memoization PR 456) re-ran +this ledger's open questions with deterministic step counters instead of wall time. Its records live +with the code: `docs/hermitcrab-perf-2026-09-counts.md` and one file per change under +`docs/optimizations/`. Corrections to this ledger that fell out of it: + +- **Row 3 ("shape sharing shipped") was not true on master.** No reachable commit carried it; master's + `Word.Clone` deep-copied the Shape and every node feature structure. The follow-up independently implements + a different design: internal `CloneForEngine` copy-on-write with un-share at mutation sites, while public + `Word.Clone` remains a deep mutable clone. A shared shape is frozen, so a missed mutation site throws. +- **Row 22's ~10% cap for an analysis-rule prefilter was wrong by an order of magnitude.** It was computed + from matcher *self-time*. Counted directly, an edge-segment prefilter (patterns are anchored at both ends + and matched by unification, so the first/last literal segment must unify with the shape's first/last + segment) removes **87% of FST traversals** and 54% of arc checks on 45 Sena words, with identical analysis + signatures on every Sena word (morph counts only on Sena; see the disposition doc) and all 464 + conformance-fixture words. +- **"memo-key construction 0.0%" (row 21) was wrong at the profile level.** Hashing each new word's + syntactic feature structure for `AnalysisStateKey` is ~15% of CPU on the heaviest Sena word; the key + itself is cheap but freezing a fresh clone of a nested feature structure is not. +- **Row 16's complete-key result still stands.** An experimental `ExpandAlternatives` memo/hoist did not + reduce clones and was removed. The final branch has no alternatives-expansion change relative to the + pinned production baseline; the intermediate 75% counter reduction is not a shipped result. +- **The ledger never cited John Maxwell's work**: PR 491 (final-template prune in analysis) and the + 2025 `investigate_linear_traverse` branch. Both were tested. The prune was inert on every local corpus + because each has a partial morpheme, and the traversal measured negative by counts (instances used 2.3x) + because the current traverser already pools instances. Both implementations and their toggles were removed; + their rejected-lead records remain. The prune verdict was later superseded: measured with the partiality guard + lifted, the prune is 8-33x on Mbugwe and Sena with identical analysis sets, and PR #491 merged to master as + `b240b7b5` on 2026-09-28 (see the addendum in `rejected-optimizations/final-template-prune.md`). +- **Wall time on the measurement machine moved 2x between windows with identical binaries** (a session + transition in between). Every verdict in this ledger that rests on a wall-time delta under ~20% should + be read with that in mind; the round's method note is to count steps on the sequential path. diff --git a/docs/hermitcrab-perf-2026-09-counts.md b/docs/hermitcrab-perf-2026-09-counts.md new file mode 100644 index 000000000..1063034b7 --- /dev/null +++ b/docs/hermitcrab-perf-2026-09-counts.md @@ -0,0 +1,94 @@ +# HermitCrab optimization round, September 2026: final disposition + +This is the final record for the count-driven round that followed +`hermitcrab-optimization-ledger.md`. Temporary counters were used to choose targets and were then removed; +the final branch (`perf/hc-optimization-archive`, not master) carries only product changes, focused tests, the +parity/timing harness, and these records. + +## Measurement scope + +The exploration used sequential parsing, one warm-up followed by repeated samples, and canonical analysis +signature sets as the parity gate. Deterministic work counts came from 45 Sena words (five known-heavy probes +plus the first 40 list entries) and 464 words from 33 conformance fixtures. Timing was never run concurrently. +The final branch-wide sample compares production commit `d7347226` (harness-only baseline tip `4f677735`) +with candidate code tip `7370e2dd`. + +The probe counters are intentionally absent from the final diff. They put volatile gates in hot code across +two assemblies and would create permanent maintenance cost for a completed experiment. The reconstruction +design remains in `hermitcrab-probe-design.md`. + +## Retained optimizations + +Each retained mechanism has one record with its invariant, implementation, evidence, parity coverage, +portability, and limits: + +| Optimization | Principal evidence | Record | +| --- | --- | --- | +| Deferred template clone | 1.054x on the 45-word Sena battery; `WordAnalysisSignature` parity (morph counts only on Sena) | `optimizations/deferred-template-clone.md` | +| Feature-structure traversal-state reuse | equality/freezing allocation left the hot profile; not isolated | `optimizations/feature-struct-visited-state-reuse.md` | +| Frozen flat-tree equality | removes visited sets from the proved tree case; not isolated | `optimizations/feature-struct-frozen-tree-equality.md` | +| Frozen flat-tree clone | removes the copies map from the proved tree case; not isolated | `optimizations/feature-struct-frozen-tree-clone.md` | +| Single-pass `Shape.CopyTo` mapping | removes a second enumeration/dictionary build; not isolated | `optimizations/shape-copy-single-pass-map.md` | +| Copy-on-write shape sharing | clone 36% → 9% of profile; peak working set 11 GB → 6 GB | `optimizations/shape-sharing-on-clone.md` | +| Flat FST register arrays | `CreateInstanceMDArray` 11.5% → 0.2% of profile | `optimizations/flat-register-arrays.md` | +| Analysis edge-segment prefilter | 87% fewer FST traversals and 54% fewer arc checks on 45 Sena words | `optimizations/edge-segment-prefilter.md` | +| Copy-on-write syntactic feature structures | removes repeated clone/freeze/hash work measured at ~15% of a heavy Sena profile | `optimizations/syntactic-feature-structure-sharing.md` | + +Only the deferred clone was timed alone. Profile movement and deterministic work removal justify the other +retained mechanisms; the five-language result below measures the branch as a package and must not be divided +among them. + +## Rejected or removed experiments + +| Experiment | Deciding evidence | Final state | +| --- | --- | --- | +| Two-pass nondeterministic traversal | 2.3x more traversal instances, only 1.6% fewer arc checks, 27% more clones | removed; `rejected-optimizations/two-pass-nondeterministic-traversal.md` | +| Final-template interleaving prune | zero prunes on Sena, Amharic, and Mbugwe because partial morphemes disable it | removed; `rejected-optimizations/final-template-prune.md` | +| `ExpandAlternatives` memo/hoist | word clones unchanged at 4,747,013; no retained difference from production ordering | removed; `rejected-optimizations/alternatives-memo-and-hoist.md` | +| Compounding split prefilter | only ~2% of clones and ~9% of FST traversals were in scope | not built; `rejected-optimizations/compounding-split-prefilter.md` | + +All associated production switches, alternate traversal code, and research counters were deleted. The two +retained internal toggles exist solely for correctness/performance A/B of the edge prefilter and syntactic +feature-structure sharing. + +## Five-language branch-wide sample + +Each row used three fixed word-list indices and two reverse-ordered process rounds (baseline/candidate, then +candidate/baseline). Every process parsed sequentially, discarded one warm-up, and recorded three repetitions, +for six measured samples per arm. Times are the sum of each word's minimum sample across both rounds. Baseline +and candidate used identical grammar names and word keys; `scripts/hc-bench-compare.ps1` compared error +outcomes and rejected missing keys or signature differences. All ten arm comparisons passed the legacy +`WordAnalysisSignature` check, which on these FLEx-exported grammars compares morph counts only (`Morpheme.Id` +is empty; see `hermitcrab-perf-2026-09-disposition.md`), and the saved outputs contained zero parse or load +errors. + +| Language | Word-list indices | Baseline ms | Candidate ms | Ratio | Interpretation | +| --- | --- | ---: | ---: | ---: | --- | +| Sena | 49, 52, 74 | 1,000.7 | 448.3 | 2.23x | directional: 0/3 probes clear both the 50 ms and spread rules | +| Amharic | 8, 9, 267 | 6,410.0 | 3,646.3 | **1.76x** | stable: 3/3 probes clear both rules | +| Indonesian | 0, 9, 44 | 99.0 | 56.0 | 1.77x | exploratory: 0/3 probes clear both rules | +| Mbugwe | 0, 65, 1272 | 8,979.4 | 4,901.6 | **1.83x** | stable: 3/3 probes clear both rules | +| Aweti | 2, 3, 27 | 134.2 | 54.4 | 2.47x | exploratory: 0/3 probes clear both rules | + +The ratios are deliberately unpooled and are not extrapolated to whole corpora. An initially selected heavy +Sena trio did not complete the baseline within five minutes, and an Aweti sample containing index 0 crossed +two minutes; both whole testhosts were terminated and replaced symmetrically before any comparison. This is +sampling evidence, not a claim about those excluded words. + +## Correctness gate + +The minimized code tip was merged into the conformance integration branch and run through the complete +HermitCrab conformance-inclusive project: + +- NUnit discovered and ran 604 non-explicit tests: 603 passed, zero failed. +- One symlink-security test was skipped because the Windows host lacks symlink privilege. +- Fixture manifests, DTD/schema checks, source/graph audits, 452 fixture rows, and 348 coverage items passed. +- The ordinary HermitCrab suite also passed with syntactic feature-structure sharing disabled. + +## What remains + +The edge prefilter leaves roughly 44% of remaining traversals successful, so another edge check has little +headroom. Compounding construction is too small on the measured heavy word. Alternatives and packed-readout +schemes remain bounded by the ordered pending trail. The next credible large change is a persistent/shared +trail representation that can avoid memo-replay and alternative-materialization clones without merging +linguistically distinct derivations; it is a data-structure redesign, not a safe incremental tweak. diff --git a/docs/hermitcrab-perf-2026-09-disposition.md b/docs/hermitcrab-perf-2026-09-disposition.md new file mode 100644 index 000000000..c9a805d81 --- /dev/null +++ b/docs/hermitcrab-perf-2026-09-disposition.md @@ -0,0 +1,143 @@ +# HermitCrab optimization round, September 2026: final disposition of the branch + +**Date:** 2026-09-03. **Branch:** `perf/hc-optimization-pr` @ `179708d2`, renamed `perf/hc-optimization-archive`. +Origin now holds that branch as one squashed commit, `3a0a3316`; that is the authoritative tip. **Decision:** do not +merge the branch. Isolate one mechanism onto master as its own small PR; archive the rest with its records. Re-focus +on sillsdev/machine PR #491 and on grammar hygiene, which are where the large returns are. + +## Why the package is not merged + +- 896 source lines across 28 files; about a third inside `SIL.Machine.FeatureModel.FeatureStruct`, the rest in FST + traversal internals, `Word`, `Shape`, and the analysis rules. Not reviewable as one change. +- Only one mechanism (deferred template clone, 1.054x) was ever timed alone. The package figure (1.76x Amharic, + 1.83x Mbugwe) cannot be attributed to its parts (`hermitcrab-perf-2026-09-counts.md`). +- The parity gate that blessed the package compared morph counts, not analyses: `MorpherTests.WordAnalysisSignature` + joins `Morpheme.Id`, which comes from `` and is empty for every FLEx export used. On Sena, Mbugwe + and Amharic the signature therefore degenerates to a morph count per analysis, so every earlier "exact parity" + claim on those grammars verified counts, not analyses. Anything re-measured on them needs a + gloss + allomorph-index + syntactic-feature-structure signature instead. +- Meanwhile PR #491's mechanism, measured with the guard lifted (`AlwaysEnforceFinalTemplates = true`), is + 8–33x on Mbugwe and Sena with identical analysis sets, and 298x on the single heaviest Sena word; Amharic + barely moves because its per-word cost is not in the template interleaving. The branch is worth keeping only + where #491 cannot fire (Amharic-shaped grammars), and only in its most reviewable piece. + +## Disposition per mechanism + +| Mechanism | Lines / files | Evidence on the branch | Disposition | +| --- | --- | --- | --- | +| Analysis edge-segment prefilter | ~120 in 2 rule files + 9 in `Morpher` + tests | 87% fewer FST traversals on 45 Sena words (deterministic counts); one invariant; internal toggle | **Isolated onto master** as `perf/hc-edge-prefilter` (`be7f241d`), see below — **that standalone branch and its own archive tag were later deleted, but the mechanism is preserved in the whole-round archive branch `perf/hc-optimization-archive`** | +| Copy-on-write `Shape` sharing | `Word`, `Shape`, every clone-then-mutate site | clone 36% → 9% of profile; peak working set 11 GB → 6 GB | Archived. Aliasing rules are subtle; re-open only if memory becomes the constraint. Commit `dfd1be94`, `e18ca86d` | +| Copy-on-write syntactic `FeatureStruct` sharing | `Word`, `Morpher`, rules | ~15% of a heavy Sena profile; never isolated | Archived. `4a6a47ed`, `7f9fdef3`, `da35ad4e`, `f08fefa7` | +| `FeatureStruct` equality/clone/freeze fast paths, visited-set pooling | 312 lines in core FeatureModel | "not isolated" | Archived. `325f8419`, `424a3c23`, `b23a7c99`, `00ece2ba`, `2f4dabe0` | +| Flat FST register arrays | FST internals | `CreateInstanceMDArray` 11.5% → 0.2% of profile | Archived. `b1692a7a` | +| Deferred template clone | `AnalysisAffixTemplateRule` | 1.054x alone | Archived. `57703d6b`, `24d75c82` | +| `ExpandAlternatives` memo/hoist | `Morpher` | no retained difference | Already removed on the branch. `d8886b0c`, `f4195be1` | +| Two-pass nondeterministic traversal | FST | negative | Already removed. `c3366fcc` | +| Final-template prune (own version) | analysis rules | zero prunes under the guard | Already removed; superseded by #491 work. `7e6ec59d` | +| PerfBench / compare script / ParseCounters | tests, scripts | harness | Archived. Its signature is degenerate on FLEx exports; any revival must switch to a gloss/allomorph/feature-structure signature first | +| Research records (`docs/optimizations/*`, `docs/rejected-optimizations/*`, ledger, counts, probe design, this file, the #491 re-review) | docs | — | Kept on the archive branch as the record of what was tried and why | + +## The isolated prefilter + +**Status update:** `perf/hc-edge-prefilter` was deleted after this round closed, and its own archive tag was +later deleted too, but the mechanism is not lost: it remains on the whole-round archive +branch `perf/hc-optimization-archive`, which still carries +`Morpher.EdgePrefilterEnabled`, the edge constraints in `AnalysisAffixProcessRule`, and +`AnalysisEdgePrefilterTests`. The description below is kept as a standalone record of the isolated, reviewable +form, in case that smaller diff is preferred over pulling the mechanism out of the full archive again. + +Branch `perf/hc-edge-prefilter`, worktree `.worktrees/prefilter`, one commit `be7f241d` on `origin/master` +(`d7347226`). Content: `AnalysisAffixProcessAllomorphRuleSpec` captures the left/right top-level `Constraint` +feature structures at compile time; `AnalysisAffixProcessRule.Apply` rejects an allomorph before its matcher runs +when the shape's first/last Segment node provably cannot unify with them, using the matcher's own `IsUnifiable` +call and `UseDefaults`; optional edge nodes and non-Constraint edges always pass; tracing emits the same +`MorphologicalRuleNotUnapplied` the matcher would have. `Morpher.EdgePrefilterEnabled` (internal, default true) is +the same-binary A/B switch. The research counters were not carried over. `AnalysisEdgePrefilterTests` (final branch +version, no counter dependency) came with it. HermitCrab suite: 104/104. + +### Standalone measurement (same binary, toggle off = master behavior, toggle on; sequential, memo on) + +Signature: gloss + allomorph index + syntactic feature structure, compared as multisets. Counters: throwaway +increments at the two nondeterministic `Traverse` entries and at `CheckInputMatch` (not committed). + +| Grammar | Words | Wall ms off → on | FST traversals off → on | Arc checks off → on | Parity | +| --- | ---: | ---: | ---: | ---: | --- | +| Sena | 42 (49,52,74,0–39; one invalid line skipped) | 71,296 → 69,900 (−2%) | 8,760,382 → 1,270,122 (−85%) | 12,618,566 → 6,004,919 (−52%) | identical 42/42 | +| Mbugwe | 22 (0,65,1272,1–19) | 110,609 → 96,293 (−13%) | 28,054,872 → 11,419,808 (−59%) | 36,811,510 → 25,960,854 (−29%) | identical 22/22 | +| Amharic | 10 (8,9,267,5,10,12,15,20,25,30) | 111,617 → 103,403 (−7%) | 145,050 → 129,802 (−10%) | 16,705,704 → 15,777,072 (−6%) | identical 10/10 | + +**Reading.** The count reduction reproduces (Sena −85% traversals, matching the branch's −87%), and parity holds +under a real signature. But the traversals removed are the cheap ones: they would have died at their first arc +check, so wall time moves only 2–13%. The branch's package figure (1.76x Amharic, 1.83x Mbugwe) therefore came +mostly from the allocation, clone, and GC work, not from this mechanism. Standalone, the prefilter is correct, +small, and reviewable, but it is **not a large return**. Recommendation at the time: keep `perf/hc-edge-prefilter` +ready as an optional small PR (2–13% on Bantu grammars, where it stacks with #491's prune on the surviving +branches), and do not spend review budget on it ahead of #491. The standalone single-commit branch is deleted +(see status note above); a PR now starts from the mechanism preserved in the whole-round archive +(branch `perf/hc-optimization-archive`) rather than resuming +that branch. + +Raw logs: session scratchpad `pf-sena.log`, `pf-mbugwe.log`, `pf-amharic.log`. + +## Branch and worktree inventory for the round (2026-08-20 to 2026-09-03) + +Every branch below is local unless marked. Worktrees for archived branches were removed on 2026-09-03; the +branches remain. + +| Branch | Tip | What it is | State | +| --- | --- | --- | --- | +| `perf/hc-optimization-archive` (was `perf/hc-optimization-pr`) | `3a0a3316` (squashed) | the whole round, code + records | archived, unmerged; worktree `.worktrees/ledger` | +| `perf/hc-defer-template-clone` | `4a6a47ed` | working branch of the round | fully contained in the archive; worktree removed | +| `perf/hc-engine-alloc` | `f22f88a8` | allocation/two-pass side branch | fully contained in the archive; worktree removed | +| `perf/hc-conformance-check` | `0799a977` | archive merged into the conformance integration branch for the 603-test gate | verification only; worktree `.worktrees/hc-conf` can go | +| `perf/hc-edge-prefilter` | `be7f241d` | the one isolated mechanism on `origin/master` | optional small PR at the time; **branch and its own archive tag have both since been deleted, but the mechanism is preserved on branch `perf/hc-optimization-archive`** | +| `pr-491` | `96c97f76` | jtmaxwell3's PR head at review time | merged to master as `b240b7b5` on 2026-09-28; worktree `.worktrees/pr491` holds uncommitted throwaway counters + `FinalTemplateUpperBound` harness | +| `docs/hc-optimization-ledger` (remote, PR #490) | the PR head | the ledger, rows 1-30 | open docs PR | +| `feature/forest-memo`, `feature/synthesis-fold-probes`, `feature/synthesis-fold-sharing`, `feature/per-node-cost` | — | evidence branches of the ledger round | see `hermitcrab-optimization-ledger.md` | +| `investigate-memoization-rule-order` | `4f5aeeb4` | despite the name, conformance-suite commits | belongs to the conformance line, not this round | + +Records in this directory: `hermitcrab-optimization-ledger.md` (rows 1-30), `hermitcrab-perf-2026-09-counts.md` +(count round), `hermitcrab-probe-design.md`, `optimizations/*` and `rejected-optimizations/*` (one record per +mechanism), and this file. The round's execution checklist has been folded into this record; it is not kept as +a separate plan document. + +Four further records of the round were deliberately left out of this set, because they concern another author's +open PR and a sibling project rather than this round's own mechanisms. They are described here so that nothing +depends on reading them, and they remain on the `perf/hc-optimization-archive` branch: + +- `pr491-final-template-prune-upper-bound.md` — the measurement record behind the 8–33x figures above: PR #491's + prune with its partiality guard lifted, per-grammar wall time, affix-rule applications and template entries, + plus the analysis-set parity check and the signature caveat quoted earlier. It also proves the guard + over-broad, since partial *lexical entries* can never rescue a pruned branch (only partial rules can, and only + at or below the template's stratum). +- `pr491-review-draft.md` — the review comment drafted from that record: the win with the guard lifted, a + duplicate-parse bug when enforcement is active (an all-slots-skipped clone differing from its input only by + `FinalTemplateState` survives the `ApplyTemplates` equality filter), the rules-only per-stratum guard, and + state transitions that run even while the prune is disabled and cost memo hits. +- `pangloss-final-template-prune-handoff.md` — the same prune handed to PanGloss (the Rust port) for + implementation there. +- `pangloss-hc-rust-speedups-handoff-2026-09-02.md` — speedup leads handed the other way, from PanGloss into + this round. Archived; its section 4 is superseded. + +Privacy: word forms and glosses from the private grammars appear only as word-list indices and rule ids in these +records. The raw A/B logs (which contain word forms) live only in the session scratchpad and are not to be +committed. The main checkout's `.git/info/exclude` now lists the local grammar backups, `Mbugwe.xml`, the Aweti +word list, and the rulestats HTML report so they cannot be staged by accident. + +## What to do next, in order + +1. ~~Fix and merge PR #491~~ — merged on 2026-09-28 as `b240b7b5`. On master the duplicate parse is fixed (a + final template that would follow a non-template now returns no output instead of marking its clone), and + the affix-process rules set `FinalTemplateState` only while the prune is enabled. The guard, + `Morpher.IsPartial`, is still grammar-wide and still counts partial lexical entries. +2. Classify the unclassified affixes in the FLEx projects (Mbugwe 1, Amharic 1, Sena 6); that turns #491's default + prune on for Mbugwe and Amharic. Sena stays partial because of its 25 partial lexical entries, so classifying + its affixes alone does not enable the prune there. PR sillsdev/machine#553 shows that ignoring partial lexical + entries is safe and makes the guard ignore them, so once #553 merges, Sena gets the prune without reclassifying + its entries (tracked in sillsdev/machine#551). + Add the finding to the grammar advisor. +3. ~~Open the prefilter PR from `perf/hc-edge-prefilter`~~ — that branch and its own archive tag are both + deleted; a PR now starts from the mechanism preserved on branch + `perf/hc-optimization-archive` rather than resuming the deleted branch. +4. Port the prune to PanGloss per the handoff, `docs/pangloss-final-template-prune-handoff.md` on branch + `perf/hc-optimization-archive`. diff --git a/docs/hermitcrab-probe-design.md b/docs/hermitcrab-probe-design.md new file mode 100644 index 000000000..9d4129084 --- /dev/null +++ b/docs/hermitcrab-probe-design.md @@ -0,0 +1,231 @@ +# HermitCrab probe design — enough to rebuild it + +Companion to `hermitcrab-optimization-ledger.md`. That document records *what was tried and why it +failed*. This one records *how it was measured*, in enough detail to reconstruct the instrumentation +and the keys from scratch. + +**Why this exists:** the seven probe branches that produced the ledger's numbers were deleted after +the round closed. Research scaffolding in two product assemblies is not worth maintaining — the +probes gate on `volatile bool` reads inside `Matcher.cs`, a hot inner loop in `SIL.Machine` used by +every consumer. So the code is gone and this is the design record. Anything here can be rebuilt in a +day; the expensive part was learning what to measure and which measurements lie. + +--- + +## 1. Instrumentation architecture + +Two probe classes, in two assemblies, with **two independent gates**. + +`SynthesisProbe` in `SIL.Machine.Morphology.HermitCrab`, and `NodeCostProbe` in `SIL.Machine`. Each +has its own `internal static volatile bool Enabled`, default `false`. **Both must be set.** Setting +only the HermitCrab one makes every lower-layer bucket read exactly `0.00ms`, which looks like +"measured and negligible" rather than "never ran" — this cost a round (ledger method rule: a bucket +reporting exactly zero is broken until proven otherwise). Wire both from one place in the harness. + +Everything is insertion-only: a `RecordDie(...)` or `Add*Ticks(...)` call placed immediately before +an existing `return` / `continue`, never a change to control flow. Where a site read +`else if (IsTracing) { trace }`, it becomes `else { if (IsTracing) { trace } Record(...) }` — +behaviour-preserving. Counters are `Interlocked`; timers are `Stopwatch.GetTimestamp()` deltas +accumulated with `Interlocked.Add`, with `Reset*` methods so a harness reads per-word deltas. + +### The wall-time split + +**Seven mutually exclusive top-level slices that sum to wall** (six measured plus the remainder), +measured around `Morpher.ParseWord`: + +`lookup` (brackets `LexicalLookup`, materialised with `.ToList()` so the bracket covers only the +lookup itself) · `synCascade` · `synBattery` · `synForward` · `synExpand` (brackets +`Word.ExpandAlternatives()` in `SynthesizeSequential`) · `anTotal` (brackets the whole +`_analysisRule.Apply(input)` call) · `unaccounted` (defined as the remainder, never tracked). + +`synForward` is recorded **net of** `synCascade` and `synBattery`: capture both counters before the +`_synthesisRule.Apply` call and subtract their deltas after. This matters enormously and caused a +25x published error — the work a synthesis fold memo would share runs *inside* the cascade and +battery brackets, so the share to divide by is `synCascade + synBattery + synForward`, not +`synForward` alone. Template slot rules compile to the same `SynthesisAffixProcessRule` classes via +`RuleBatch` (`SynthesisAffixTemplateRule` ctor), which is why they land in `synBattery`. + +Nested *inside* `anTotal`, as a breakdown rather than additional slices: `anCascade` (the +`_mrulesRule.Apply` call in `AnalysisStratumRule.ApplyMorphologicalRules`), `anBattery` +(`ApplyTemplateBattery`), `anPhono` (`_prulesRule.Apply`). + +**Always print the remainder as its own column.** It caught the misattributed split, the dead probe +gate, and would have caught four of five retractions at the point of measurement. + +### The per-node decomposition + +Inside `anCascade` and `anBattery`, sub-buckets: `match` · `feature` · `clone` · `key` · `memo` · +`trail` · `equality` · explicit remainder. + +`match` and `feature` come from the `SIL.Machine` layer. To avoid double counting, the HermitCrab +side reads `NodeCostProbe.MatcherTicks` / `FeatureTicks` **before and after** the cascade call and +records the delta — so the lower-layer buckets are slices *inside* the HermitCrab bucket, not +alongside it. Matcher self-time has nested work subtracted at the source (inside `Matcher.cs`); +feature work needs a re-entrancy depth guard or recursive `Unify` calls double-count. + +Measured outcome, so a rebuild can sanity-check itself: on Amharic `anCascade`, clone **46.7%**, +remainder **37.4%**, match 12.5%, feature 3.4%, key **0.0%**, and memo / trail / equality each +**<0.2%**. + +### Censuses + +Rule attempts, split by which guard rejected them (`guardSelector`, +`guardApplicationCount`, `guardUnifiability`, `reachedMatching`); allomorph pattern-match +attempts vs matches; clone created vs discarded vs kept. + +Die points, as an enum with a counter array **and a parallel tick array** — timing each individual +guard check, not the enclosing `Apply()`. Count and cost diverge: `RuleNotApplicableOrPatternMismatch` +was **100.0% of events** and about **0.2% of time**, because each rejection is one list index plus one +reference compare (~29 ns). Never report a die-point histogram by count alone. + +Die points, classified for the abstract-replay question — **shape-free**: synthesis-side +`MaxApplicationCount`, syntactic-feature unification, MPR features, realizational +subsumption/blocking, co-occurrence and obligatory features in `IsWordValid`. **Shape-dependent**: +allomorph pattern match / rule-not-applicable, allomorph environments, final `IsMatch`. +`LexicalLookupMiss` is neither — it precedes any synthesis step. + +--- + +## 2. The keys + +### `AnalysisStateKey` (as it stood before #491) + +`(Shape, Stratum, SyntacticFeatureStruct, RealizationalFeatureStruct, NonHeadCount, per-rule +un-application count multiset)`. Order-independent by design — no analysis-side rule reads trail +order, which is the redundancy it collapses. The multiset is XOR-combined, not `*31`-rolled, so it +hashes commutatively. + +This field list predates #491, which added `FinalTemplateState` to the key; #522 then removed +`AnalysisStateKey` along with the analysis memo. Master's `AnalysisStratumRule.AnalysisMergeKey` carries the +fields above plus `FinalTemplateState`. A rebuilt or narrowed key that omits `FinalTemplateState` merges states +that #491's prune treats differently. + +**Freeze-on-read in the constructor is mandatory.** `AnalysisAffixTemplateRule.Apply` reassigns +`SyntacticFeatureStruct` to a fresh unfrozen clone *after* the owning `Word` is frozen, and that +setter has no `CheckFrozen()` guard. + +### The narrowed variant (row 4) + +Filter the count multiset to rules that are **not** provably shrinking. Two requirements learned the +hard way: build the retain-map **once per grammar**, not per key (key construction is on the hottest +path); and filter during hashing and equality rather than by materialising a filtered dictionary, so +the narrow key allocates no more than the full one. **Absent from the map means retain** — a rule the +classifier walk does not cover must be counted, never silently dropped. + +A `retainedCount` field is needed so equality can compare sizes without enumerating. + +### `SynthesisStateKey` (row 5) — and the trap + +Everything a synthesis step reads: shape+annotations, syntactic FS, realizational FS, MPR set, root +allomorph, disjunctive allomorph indices, applied-rule counts, `IsPartial`, +`IsLastAppliedRuleFinal`, stratum, **and the ordered remaining trail** — +`Word.MorphologicalRuleTrail` sliced to `[0, PendingTrailPosition]`. + +**The trail content is the whole game.** A key with `PendingTrailPosition` (an integer index) and no +trail content is fine for *measuring* and catastrophic for *skipping work*: two candidates at the +same index with different pending sequences compare equal, and merging them discards one +derivation's continuation. Every large apparent redundancy in this engine was measured with such a +key, and each collapsed when the content was added — 9,774x → 15–40%, 61-of-395,026 → 351,414-of-395,026, +3.22x/8.10x → `hits = 0`. + +**`Word.ValueEquals` is not a starting point.** It omits `_syntacticFS`, MPR features and the +disjunctive allomorph indices, all of which synthesis reads. Inheriting from it produces a silently +inflated ratio. + +Two more properties a real memo needs: stored results are **sets**, not values (realizational rules +are trail-exempt — `SynthesisRealizationalAffixProcessRule` has no `IsMorphologicalRuleApplicable` +gate — and several allomorphs of one rule can legitimately match one input before the disjunctive +break); and stored outputs embed the producing candidate's trail, so replay must **re-anchor**, the +mirror of `Word.ReplayOnto`. + +### `RuleLengthClassifier` (row 4's static half) + +Verdicts: `Shrinking` / `NonShrinking` / `Unknown`, where `Unknown` is treated identically to +`NonShrinking` by the key and kept distinct only so a census can tell "this grammar has zero +morphemes" from "this grammar has constructs we cannot analyse". + +An `AffixProcessAllomorph` shrinks iff **every** allomorph (a) inserts at least one *segment* +(`InsertSegments` counting only `HCFeatureSystem.Segment` nodes — a boundary-only insertion is +conservatively NonShrinking, since `InsertSegments.GenerateAnalysisLhs` omits boundaries from the +analysis pattern; `InsertSimpleContext` counts as 1), (b) has **every Lhs part captured** by a Rhs +`CopyFromInput` or `ModifyFromInput`, and (c) copies no part twice. + +The direction trap, which is easy to invert: on the analysis side rules are *un*-applied, so a rule +that inserts on synthesis **removes** on analysis, and a rule that truncates on synthesis +**untruncates** — an Lhs part with no Rhs copy makes the un-applied word **grow**. The authority is +`AnalysisMorphologicalTransform.GenerateShape`, which walks the Lhs parts and either copies the +captured span or calls `Untruncate` to regenerate the part from its pattern. Reduplication (a part +copied twice) and `CompoundingRule` are `Unknown`; conservatism only costs pruning opportunity. + +Measured census, for a rebuild to check against: Sena 19 of 27 rules shrinking, Indonesian 10 of 15, +Amharic 31 of 36. + +--- + +## 3. Harness design + +`[TestFixture]`, `[Explicit]`, environment-driven, writing only `TestContext` lines and no files — +this repo never commits real grammars, and derived corpus data must not leak into a committed path. +**Never print rule, stratum, morpheme or lexical-entry names, or word lists.** Counts, ratios and +timings only. Conformance fixture IDs are committed and safe. + +Env: `HC_MEMO_GRAMMAR`, `HC_MEMO_WORDS`, `HC_MEMO_MAX_WORDS`, `HC_PROBE_WORDS` (explicit +comma-separated list), `HC_PROBE_FIXTURES_ROOT`, `HC_MEMO_TIMEOUT_MS`. + +Breadth comes from `SIL.Machine.Morphology.HermitCrab.Conformance.Fixture.DiscoverAll(root)` over +`conformance/{languages,edge-cases}` — 33 grammars, each `grammar.xml` + `words.yaml` with +hand-derived expectations. Depth comes from the three real corpora. **Amharic: never exceed the first +28 words** — zero-based index 28 is pathological and burns hours. + +Run the memoized path: `new Morpher(traceManager, language, maxDegreeOfParallelism: 1)`. Tracing +disables the memo, so a traced run measures nothing. + +### A/B methodology, all of it earned by being wrong first + +Parity is the gate and speed is only reported. Acceptance is **analysis-set equality on canonical +morpheme-signature sets** (`MorpherTests.WordAnalysisSignature`, sorted) — never object or byte +equality, because a replayed `Word` is legitimately not field-for-field identical. That signature joins +`Morpheme.Id`, which is empty in every FLEx export used, so on Sena, Mbugwe and Amharic it compares morph counts +only. A rebuilt gate for those grammars needs a gloss + allomorph-index + syntactic-feature-structure signature, +compared as multisets (`hermitcrab-perf-2026-09-disposition.md`). + +Timing: **discard a warm-up rep per arm, then take min of N interleaved samples, and print the +off-arm spread beside every speedup.** Refuse any speedup smaller than its own spread. Three +failures forced each part of this — a first A/B read 1.53x that was entirely JIT; another read +2.385x with a 210.9% spread on one arm that was 1.015x serial; and adding one arm's warm-up while +discarding the other's manufactured a uniform ~2.4x across every fixture including ones with zero +shareable steps. + +**Never dispatch timing-sensitive probes concurrently.** Three at once put four testhosts on the +machine and invalidated three rows' bucket percentages. Check `tasklist | grep -c testhost` before +any timing run and report the count alongside the numbers. Counts survive contention; shares and +speedups do not. + +**Never aggregate the test logger's console output.** It prints every line twice, the second copy +space-prefixed, so a `^\s*` grep doubles every total — this published a 2x error. + +Sub-50 ms fixtures cannot carry a timing claim: one 10 ms fixture's share moved 1.3% → 14.5% +between two runs of identical code. Flag them (`reliable = wallMs >= 50`) rather than filtering them +out silently. + +### One concurrency hazard worth knowing + +A per-word timeout wrapper built on `Task` has no cooperative cancellation, so an abandoned word's +task keeps mutating engine state after the harness moves on. Harmless with a per-word +`AnalysisScope`; under a **shared** scope it collided with the next word on the same +`Memo`/`InProgress` collections and produced `IndexOutOfRangeException` inside +`MemoizedCombinationRuleCascade`. A shared-arm timeout must abort the run, not continue. Also note +`Task.Result` throws `AggregateException`, not the bare exception type. + +### Final reusable harness + +The retained `PerfBench` lives on branch `perf/hc-optimization-archive`, not on master, and is narrower than the +historical probes above. It accepts `HC_BENCH_GRAMMAR`, `HC_BENCH_WORD_LIST`, `HC_BENCH_REPS`, `HC_BENCH_OUT`, +and `HC_BENCH_LABEL`; it records timings, errors, resource observations, memo diagnostics, and canonical result +signatures. It does not retain the completed round's hot-loop counters. + +Timeouts are now the caller's responsibility: start one `dotnet test` process for an arm and terminate the +whole process if its budget expires. Do not restore an in-process `Task` timeout—the abandoned parse continues +running, contaminates later samples, and can race shared engine state. `scripts/hc-bench-compare.ps1`, also on +that branch only, compares grammar and word keys symmetrically before comparing errors, signatures, timing, or +spread. diff --git a/docs/hermitcrab-search-asymptotics-research.md b/docs/hermitcrab-search-asymptotics-research.md new file mode 100644 index 000000000..6c1874377 --- /dev/null +++ b/docs/hermitcrab-search-asymptotics-research.md @@ -0,0 +1,370 @@ +# HermitCrab analysis search — asymptotic research + +**Disposition: count ceilings before building.** The first two candidates below fit the two open work +items without a search-sized memo. Partial-order reduction has a narrow correctness route and poor prior +coverage. Exact finite-state compilation is a larger, restricted-subgrammar project. None is a measured +optimization, and none establishes a polynomial parser for arbitrary FieldWorks grammars. + +This integrates the six supplied reports: **R1** partial-order reduction and rewriting; **R2** packed forests +and subset DP; **R3** finite-state methods; **R4** search and symbolic sets; **R5** enumeration and graphs; +**R6** abstract interpretation and morphology prior art. The reports are research inputs, not independent +measurements. The [ledger](hermitcrab-optimization-ledger.md), its +[September disposition](hermitcrab-perf-2026-09-disposition.md), and the +[probe design](hermitcrab-probe-design.md) remain the measurement record. + +## The problem in one page + +The motivating case is **a grammar with 16 extra affix processes in one Unordered stratum**. Analysis +starts at the surface, unapplies phonology, then interleaves morphological-rule unapplication with affix +templates. Every candidate can stop for lexical lookup; candidates from intermediate strata also enter +the lookup/synthesis pipeline. A found lexical entry supplies its root allomorphs, each candidate's +alternatives are expanded, and forward synthesis must both pass validity checks and regenerate the surface +form [C1, C4, C5]. + +There are three different sources of growth: + +- **Rule orders.** The combination cascade restarts its rule loop at zero after every successful + unapplication. With `k` enabled, single-use rules, there can be `k!` full orders and + `sum(j=0..k) k!/(k-j)!` ordered prefixes. Multiple application is enabled; ordinary affix/compound + unapplication checks each rule's limit rather than a cascade-wide used-rule mask. C# realizational + unapplication has no such count guard, so a finite count-vector bound needs an eligibility proof for + the rule kinds included [C1, C2]. Allomorph branches and compounding add further choices. The 16 added + processes are not necessarily the total `k`. +- **Optional sites.** Inverse deletion inserts optional nodes into one phonological shape; it does not + immediately allocate every subset shape. Later matching still has consume/skip choices. The Rust root + trie already merges equal active trie positions. General morphological matching also returns captures + and variable bindings, so its branches cannot be merged on position alone [C6, C7]. Optionality can + therefore multiply matcher work even while the shape representation stays compact. +- **Alternatives.** An analysis-state key deliberately omits trail order. The stratum merge keeps + alternatives, and expansion grafts subsequent changes onto their source histories. Forward synthesis + reads the pending rule in order. A shared shape or rule multiset is not a shared synthesis continuation + [C3–C5]. One final analysis does not imply that only one trail needs checking. + +All numbers here are **supplied by the common brief, not reproduced in this job**: + +| Aggregate | Reported value | +| --- | --- | +| Baseline parse time | approximately 15–60 ms per input in both implementations | +| Time after the 16 additions | 17–22 s for one expensive input in Rust | +| Lexical lookups / distinct shapes | 97,575 / 56, from trace mode | +| Distinct canonical states | 10,079; they differ in shape or per-rule counts | +| Template-battery runs / distinct input states | 43,641 / 4,988; approximately 2.9 s | +| Lookup + alternative expansion + synthesis | approximately 6.5 s | +| Alternatives synthesized / distinct final analyses | 367,278 / 1 | +| Compounding bound in the case | `MaxStemCount = 2` | +| Deletion-rule variants | 200–1,600 inputs reach a 1 s per-input timeout | + +These are ceilings to investigate, not interchangeable denominators. Tracing disables the ordinary +analysis merge and C# memoization [C3, C4]; the trace lookup count cannot establish an untraced duplicate +rate. Nor do 56 shapes establish 56 lookup keys across strata. The apparent battery repetition is at most +38,653 calls (88.6%) under the brief's identity, before checking that identity and the cost of avoiding it. +The 367,278-to-one ratio proves neither order invariance nor a synthesis-sharing opportunity. + +## Verified code anchors + +Line numbers below were read directly, not copied from the reports. **C#** means the requested main +checkout, commit `a4b29742b6274a01c7398ca3e800a19fa6d2c9aa`; paths are relative to +`src/SIL.Machine.Morphology.HermitCrab/` unless qualified. The six central files (`AnalysisStratumRule`, +`AnalysisStateKey`, `MemoizedCombinationRuleCascade`, `AnalysisAffixTemplateRule`, `Morpher`, `Word`) are +text-identical in this PR worktree's starting commit `5bbfa018bdbd5228c3f1aa73db951a4f01d167d3`, ignoring +line endings. The C# anchors are a snapshot at `a4b29742`, before #491 (`b240b7b5`) merged; they were not +re-read at current master. **Rust** paths are relative to the supplied checkout's `rust/crates/`, with +branch tip `fa3bfb186852baaaf1ed28fcc0ba4602b2c19e96` at inspection. This identifies inspected files; it is +not a claim that either external checkout was clean. + +| Anchor / fact | C# file:line | Rust file:line | +| --- | --- | --- | +| **C1 — cascade and template interleaving** | `AnalysisStratumRule.cs:24–61,128–137,174–244`; base combination loop: `src/SIL.Machine/Rules/CombinationRuleCascade.cs:25–51` | `pg-rules/src/cascade.rs:219–265`; `pg-rules/src/stratum.rs:811–888`; strata/results: `pg-parse/src/morpher.rs:701–765` | +| **C2 — application/compound guards and feature effects** | `MorphologicalRules/AnalysisAffixProcessRule.cs:39–63`; `MorphologicalRules/AnalysisCompoundingRule.cs:39–77,129–133`; realizational exception: `MorphologicalRules/AnalysisRealizationalAffixProcessRule.cs:40–58` | `pg-rules/src/stratum.rs:707–724,792–805` | +| **C3 — existing sharing and actual keys** | `AnalysisStateKey.cs:31–36,60–85,94–104`; merge: `AnalysisStratumRule.cs:130–167`; inner memo: `MemoizedCombinationRuleCascade.cs:39–94`; battery memo: `AnalysisStratumRule.cs:196–218` | key fields: `pg-rules/src/analysis_state_key.rs:22–47`; key construction: `pg-rules/src/stratum.rs:664–694`; ordered output key: `pg-rules/src/word.rs:309–321,410–433`; stratum fold: `pg-rules/src/stratum.rs:1180–1239` | +| **C4 — lexical lookup, expansion, acceptance, fallback** | `Morpher.cs:167–201,346–367,417–445,634–665`; expansion: `Word.cs:445–486` | `pg-parse/src/morpher.rs:459–499,765–826,933–997`; expansion: `pg-rules/src/word.rs:487–548` | +| **C5 — ordered trails are observable** | `Word.cs:202–215,253–259,339–347,371–382,627–646` | `pg-rules/src/word.rs:318–319`; pending-rule test and consumption: `pg-rules/src/stratum.rs:1546–1571,1606–1609` | +| **C6 — optional nodes and root-trie sharing** | `PhonologicalRules/NarrowAnalysisRewriteRuleSpec.cs:40–59`; optional insertion argument: `src/SIL.Machine/Annotations/Shape.cs:160–174` | deletion sites/insertion: `pg-rules/src/rewrite.rs:2536–2581`; root trie consume/skip/dedup: `pg-parse/src/root_trie.rs:227–277` | +| **C7 — general matcher work carries captures** | `MorphologicalRules/AnalysisAffixProcessRule.cs:24–33`; capture-dependent reconstruction: `MorphologicalRules/AnalysisMorphologicalTransform.cs:61–100`; optional forks/registers/bindings: `src/SIL.Machine/FiniteState/TraversalMethodBase.cs:248–308` | template slot results and their semantic deduplication: `pg-rules/src/stratum.rs:1057–1075` | +| **C8 — template admission and slot semantics** | `AnalysisAffixTemplateRule.cs:18–73`; non-disjunctive union: `src/SIL.Machine/Rules/RuleBatch.cs:52–61`, configured at `AnalysisStratumRule.cs:27–30` | `pg-rules/src/stratum.rs:894–962,990–1075` | +| **C9 — proposer is followed by exact-engine confirmation** | exact confirmation target: `Morpher.cs:346–367` | current emitter: `pg-foma/src/emit.rs:2615–2675,2704–2745,2768–2808`; confirmation filters/reparse: `pg-foma-runtime/src/confirm.rs:498–517` | + +In particular, at `a4b29742` the C# key contains shape, stratum, syntactic and realizational features, +non-head count, and exact per-rule unapplication counts. It contains neither final-template state nor a +separate morphology history; #491 later added final-template state to the key, so current master lacks only +the morphology history. Rust adds both, saturates counts at each rule's maximum, and clears its analysis +final-template state at the stratum fold (`pg-rules/src/stratum.rs:1196–1200`). The same key name does not +imply the same equivalence relation [C3]. Neither key is a synthesis key. This inspection verifies these +fields and call sites; it does not certify a new key as complete for every continuation. + +There is a second relevant implementation difference: Rust's common dispatch applies the count gate and +pushes a pending-rule entry for every morphological rule kind. C# realizational unapplication has no count +gate and records counts without adding a pending-trail entry [C2, C5]. These are verified bookkeeping +differences, not a demonstrated output divergence here. Any proof using bounded counts or trail identity +must address the C# semantics explicitly; copying Rust's control vector is insufficient. + +## Ranked candidates + +Rank is the order in which to spend research effort under the memory constraint, not a promised speedup. +The reference labels point to the bibliography below; **every literature citation is from memory and was +not checked against a publication or online**. + +| Rank / technique | Home field + reference | Mapping in C# / Rust | Exact-parity condition; check and fallback | Expected asymptotic change | Cheapest ceiling experiment | Difference from closed rows | +| --- | --- | --- | --- | --- | --- | --- | +| **1. Grammar-compiled residual viability**; includes lexical goal propagation, abstract interpretation, and residual lexical coaccessibility | Program analysis / demand-driven databases / automata; [L1, L2, L5] | Before recursive expansion in C1/C2, with summaries of templates C8, optional nodes C6, and lexical targets C4; Rust `combination_rec` and `apply_one_mrule` | **Static:** sound transfer summaries for all remaining operations and reachable strata/roots, including compounds. **Runtime:** prune only a proved-empty abstract continuation. Unsupported operation, budget exhaustion, guessing/overlay not covered: unknown, ordinary search | Fewer explored prefixes; a certified precedence restriction can change `k!` orders to `L` viable linear extensions. No universal worst-case improvement | Shadow checks at C1/C2: count provably dead branches and descendant rule/template/lookup work, unpooled; count summary size and query work | Rows 10/11 reopen only with residual transformations **and compounding** covered; root-reachable synthesis failures remain. This is neither row 8's bare-root length bound nor row 19's feature-only synthesis replay | +| **2. Compile the template battery into a shared-prefix control trie** | Grammar factoring / parsing / finite-state compilation; [L3, L5]. A specialization of the reports' compiled-control ideas, not a reported measurement | C8: replace repeated reverse slot walks inside C1's battery. Rust `run_template_batch`, `template_unapply_slots`, `apply_slot_batch` | **Static:** share only identical slot-batch operations and optionality, with per-template admission and terminal labels retained. **Runtime:** identical input to a shared prefix; preserve all branch outcomes and existing dedup semantics. Trace, unsupported effects, or control-size budget: original battery | For `t` templates sharing prefix work `W_p`: `t·W_p` becomes `W_p`, plus divergent branches and terminal bookkeeping. For a deterministic `p`-slot prefix, this is `t·p` to `p`. No reduction of unrelated rule permutations | Compile-only census of distinct reversed slot prefixes; at C8 count attempts/outcomes at common prefixes, admitted templates, skip-only terminals, clones, and maximum DFS depth. Separate per-battery savings from avoiding battery calls | No `(Word,state)` result cache: sharing is in the grammar's control trie. Does not repeat rows 1/2/18 or the alternatives memo/hoist. Deferred cloning and the final-template filter already have separate records | +| **3. Trace-class POR with complete ordering recovery**; critical-pair analysis supplies certificates | Concurrency / term rewriting; [L4] | C1 unordered recursion; class readout at C4/C5. Same boundaries in Rust `combination_rec` and the expansion/synthesis consumer | **Static:** strong independence with enabledness and branch/effect correspondence, not just equal final shapes. **Runtime:** stack-local sleep sets; retain every class's ordered histories and all stopping/template points. Uncertified pair is dependent; unhandled rule kind uses ordinary search | Ordered-prefix exploration becomes trace-class exploration; fully independent single-use, single-outcome rules give `2^k` classes and `O(k·2^k)` transition attempts. Recovery/synthesis can still require `k!` trails | Count certified pairs **where enabled**, classes, and prefix/template work avoided; separately count recovered trails and synthesis. Earlier eligibility was only 0.00–0.05% of attempts in the supplied research context | Recovers the orders discarded by the rejected commuting-order canonicalization. Skipping class members without a full synthesis/output-invariance proof repeats row 6; merging synthesis steps repeats row 5 | +| **4. Exact finite-state compilation of a verified subgrammar** | Computational morphology / transducers; [L5] | Compile C1/C2/C6/C8 and the lexicon/forward checks C4 into a labelled relation; Rust emitter C9 is a starting point for coverage inspection, not an exact replacement | **Static:** exact regular encoding of features, annotations, root/non-head identity, all ordered outputs, copying and rule limits; unsupported construct or compile-size overflow rejects eligibility. **Runtime:** budget checked before allocating input-product storage; ordinary parser on refusal | Control alone: `prod(i)(m_i+1)` states, at most `k` outgoing choices per state; for single use `2^k` states / `O(k·2^k)` arcs. Full relation and output enumeration may still be exponential/factorial | Coverage audit and compiled states/arcs first; then accepted labelled paths, confirmation work actually removed, distinct complete analyses, and bytes. Compare against 367,278 trails, not just one output | Shares a **grammar-compiled** relation without erasing order or adding a per-search chart. A permissive proposer that still reparses has not removed the cascade | + +## 1. Residual viability: prune only a proved-empty continuation + +Let `S` be a concrete HermitCrab internal `Word` plus cascade/stratum control. Let `Reach(S)` include +every possible stopping candidate after the remaining inverse operations, including lookup in intermediate +strata and every permitted compound split. Choose a finite abstract domain, an abstraction `alpha`, and +transfers satisfying `alpha(T(S)) <= T#(alpha(S))` for **sets** of results. Compute a conservative closure +of the remaining transfers. If that closure has empty intersection with an over-approximation of all +compatible lexical targets, no concrete continuation can find a root. Removing that branch therefore +removes no successful synthesis and no reported analysis. This follows by induction over each concrete +continuation; it needs no assumption that rule order commutes. + +The target set must include everything ordinary lookup could supply. Compound non-head lookup is already +inside unapplication [C2]; it cannot be ignored or treated as a single final root. Root guessing must +either have its own sound target abstraction or retain the original candidates for the fallback branch +[C4]. Rust supplied-root overlays also affect targets (`pg-parse/src/morpher.rs:459–499`). A finite list of +ordinary root spellings is insufficient in either situation. + +Start with a grammar-compiled coarse domain: mandatory/optional segment-length intervals, safe segment +classes, and only feature constraints whose transfer is proved. Inverse truncation can **grow** a shape: +`MorphologicalRules/AnalysisMorphologicalTransform.cs:54–80,108–132` reconstructs uncaptured input parts. +Optional nodes require both skip and consume possibilities [C6]. An unsupported transform maps to top, +never to failure. Staged +refinement is useful only if every stage remains an over-approximation; an inconclusive expensive stage +returns unknown. This is the safe part of R6's abstract interpretation, R4's magic sets, R3's lazy product, +and R5's extension-oracle proposals. They are one pruning family, not four independent wins. + +R4's precedence idea is a special case with a stricter obligation. To install `r before s`, prove that +**every successful complete path containing both occurrences** respects that order, including paths with +intervening enabling rules/templates. Failure of the adjacent two-step sequence `s;r` is insufficient. +Rules are optional: `s` without `r` must remain allowed. A runtime implementation can forbid a later `r` +after `s` only under that complete-path certificate; it cannot require `r` before admitting `s` altogether. +Begin with single-use rules, no certified edges for unknown constructs, and no inference from prefix/suffix +direction alone. A cycle says that the affected combination is impossible, not that individual rules +should be deleted. + +This must remain grammar-sized: use finite transfer tables and stack-local query work, not a table keyed +by every visited `S`. A lazy lexical product that needs an unbounded visited-state set fails the memory +constraint; use a statically bounded abstraction or return unknown. Count dead **subtrees and their work** +without double-counting descendants. The ledger's 23.5% pooled dead-step result and zero on one expensive +input are reasons to stop if the stronger oracle still mostly accepts. No lexical oracle can reject a +root-reachable path merely because later synthesis will fail. + +## 2. Make the template battery cheaper by factoring its control + +The code provides a useful, smaller target than a whole-search chart. C# builds a non-disjunctive battery +and non-disjunctive batches for individual slots. Each template walks slots from last to first; an optional +slot permits the unchanged input to continue, while a required slot stops that skip path [C8]. Templates +are therefore finite control programs over **set-valued** morphological transformations. + +Compile a trie of those reversed control programs. Share a prefix only when its slot operation has the +same ordered rule identities, application role, optionality, and relevant compiled settings. Keep terminal +template identities and their entry predicates. At each battery invocation, evaluate each template's +selector/required-feature predicate on the **original input**, as today; a shared edge is eligible only +for the admitted descendant templates. Preserve per-template finality policy in Rust. Do not move entry +predicates to a transformed intermediate input [C8]. + +For an admitted template, the original slot path has an identical path through the trie. Conversely, +every trie terminal has an admitted original template. At a shared edge the inputs and operations are +identical, so applying the operation once yields exactly the result set each original execution would +have produced. Induction over prefix length preserves each template's ordered rule trail and complete +internal `Word` effects. Branches split at differing slots; required slots retain no skip edge, optional +slots retain both routes, and all-skipped terminals remain. The union of terminal results is consequently +the original battery result set. This is control factoring, with no identification of two independently +arrived search states and no commutation assumption. + +Two implementation boundaries matter. First, retain the existing slot/battery deduplication and representative +semantics: C# uses `Word.ValueEquals`, which omits syntactic features; Rust explicitly generalizes syntactic +features on collisions [C5, C8]. A new global semantic-key merge is not justified by the trie proof. The +first eligible fragment should exclude differing duplicate-result representatives unless their equivalence +is certified. Second, keep template-specific effects at terminals; if an effect changes a subsequent slot +operation, it belongs in the prefix identity or prevents sharing. Tracing follows the ordinary battery, +because template-specific trace events observe separate executions. + +The new trie has at most one node per original slot-prefix position: `O(B)` grammar storage for `B` total +slots, plus rule lists and terminal membership. Evaluate it depth-first with only the current internal +`Word`, branch iterator, and admitted-template information on the stack. No completed `(trie node, Word)` +results are retained for later arrivals. Live traversal storage is bounded by grammar depth and the current +rule-result enumeration, not total states visited. Existing result accumulation is still required. If +implementation instead introduces a growing active-state chart, it has left this proposal. + +This can reduce the factor due to duplicated template control. It does **not** turn the 43,641 battery calls +into 4,988 calls; that would need different search structure or the ruled-out memo. No supplied report +measures common slot prefixes, so a compile-only prefix census is the first gate. This specialization +combines R2's structural sharing and the reports' finite-state compilation ideas at a grammar-sized +boundary. It is a derived proposal, not an experiment any report already ran. + +## 3. Partial-order reduction: preserve the trace class, then recover its orders + +R1 supplies the essential correction to canonicalization. A class contains paths related by swapping +adjacent independent **occurrences**, not by sorting a multiset and deleting its other orders. With repeated +applications, occurrence identity includes its count/position; nondeterministic allomorph outcomes must +also have a proved correspondence. Templates, compounding, and unsupported operations initially act as +dependent barriers. + +For every certified pair and reachable eligible state, require (1) either transition preserves the other's +enabledness and limits; (2) both two-step result sets have a branch-by-branch bijection; (3) matched endpoints +agree on everything subsequent unapplication, templates and lookup can read; and (4) occurrence effects can +be replayed in each recovered order with all trail, non-head and feature history preserved. Disjoint-looking +string edits, critical-pair joinability, or local confluence alone prove none of these for the complete +internal `Word`. C2 and C7 show why feature effects and captured reconstruction are part of the obligation. +Mark uncertain pairs dependent; a conservative restricted certificate is preferable to runtime speculation. + +Under these conditions, adjacent swaps neither lose a path nor invent an impossible one. Their transitive +closure partitions executions into trace classes. A sleep-set traversal can explore one representative +of each class, **including classes of every stopping prefix**; the dependency ordering of occurrences +retains its other linearizations. Template opportunities at intermediate prefixes must remain represented, +not just the terminal class. Lazy enumeration of each class's linear extensions, with exact occurrence +effects restored, reconstructs the original ordered synthesis inputs. Running ordinary synthesis on each +restores precisely the original analysis set. This is the completeness argument missing from merely +keeping the representative. + +Store the grammar's conservative pair matrix (`O(k^2)`) and current path/sleep/linearization stacks. The +occurrence bound `D` must come from actual grammar limits; a current class can use `O(D^2)` ordering data. +Replace that class's explicit histories or stream them to the existing consumer; do not add a search-wide +class registry or DPOR backtracking graph. If the certificate or stack budget cannot be satisfied, use +the ordinary traversal. A streaming consumer is an architectural change to C4, not a drop-in set key. + +For fully independent, single-use rules with one outcome per occurrence there is one class per subset, +hence `2^k` classes rather than factorially many ordered prefixes. Allomorph branching can add further +classes. **That bound is on analysis traversal, not end-to-end parsing.** If +forward synthesis or output identity distinguishes class members, all relevant orders still need exact +validation/readout. Skipping their synthesis needs a separate proof of identical complete acceptance and +reported analysis, for every root and allomorph. C5 and ledger rows 5/6 rule out assuming it globally. +R6's stronger output-invariance condition is right for **discarding** orders; R1's recovery construction +is right for avoiding repeated pre-synthesis work while preserving them. The earlier 0.00–0.05% eligibility +is a warning, not evidence that better certificates exist. + +## 4. Finite-state compilation: exact only for a certified fragment + +An exact compiler would encode a relation from surface input to complete accepted analysis labels. It +must represent the inverse/forward rule behavior, lexical alternatives, template paths, stratum control, +features, annotations and non-heads that affect validity, with every observable ordered identity retained. +Compile bounded rule-use counts as a control vector: there are `prod(i)(m_i+1)` vectors and at most `k` +choices out of one vector. The vector alone describes permission to try rules, not their semantic outputs. + +If each primitive's encoding is exact, relational union preserves all branches and composition preserves +all legal sequences. Intersecting with the input and lexical/validity constraints then preserves exactly +the accepting paths. This gives a structural induction proof for the compiled fragment. Output labels +must carry sufficient allomorph/morpheme order, root position, non-head provenance and feature information +to reconstruct the same reported analyses. If compilation is only an over-approximation, it may reject +proved-impossible branches as in candidate 1, but its surviving paths still require ordinary confirmation. + +The compiler must refuse constructs with no established regular encoding. In particular arbitrary copying +of an unbounded captured string is not an ordinary finite-state relation; finite alphabets alone do not +fix that. Bounded copying, finite feature domains and bounded compound/rule counts need explicit encodings, +not assumptions. Unknown annotation or constraint behavior also rejects exact eligibility. Compile-time +state/arc/byte limits bound grammar storage, with fallback if composition expands too far. + +For an accepted finite relation with `Q` states and `E` arcs and an input of length `n`, recognition can +use a grammar/input-bounded product, rather than materializing each complete trail. Epsilon behavior must +be handled exactly, and productive zero-input cycles must be excluded or proved bounded. An `O(nQ)` +reachability/coaccessibility structure is acceptable only with its byte bound checked before allocation; +otherwise use ordinary parsing. Recognition sharing does not bound distinct labelled outputs or the +cost of eliminating duplicate output identities. Both the compiled relation and its readout can still +be exponential; `2^k` is the **control-state** bound, not the number of analyses. + +R3 is right that a compact proposer plus exact reparse does not solve the confirmation cascade. Its +blanket `O(k^2)` description of the present proposer is too broad: the actual emitter has a subset-mask +path for eligible unordered underlying-token zones, a dedicated-level fallback, and a legacy every-rule +per-level path [C9]. The comment in `pg-foma/src/unordered.rs:44–61` describes the latter, not all current +execution. No claim about the proposer's universal completeness or exact parity is established here. + +## Where the six reports disagreed + +| Issue | Resolution after reading code and the ledger | +| --- | --- | +| **Is the cascade already subset DP?** R2/R4/R5 emphasize a new inner chart; R6 says that would repeat memoization | R2 is right about **Rust**: C1 uses recursive enumeration and C3 merges later. R6 is right about **C#**: the sequential inner cascade and battery already replay state-keyed results. A count vector is not a sufficient semantic key. A capped chart still grows with search and is outside the hard constraint | +| **What is in `AnalysisStateKey`?** The brief/R2 describe more than the older probe-design key | R2 matches Rust's morphology-history/finality fields. Applying that description to C# is wrong; C3 lists the actual different keys. Neither warrants synthesis merging | +| **Every rule is bounded by its application count**, assumed in the brief and count-vector proposals | Correct for Rust's shared gate and C# ordinary affix/compound rules; C# realizational unapplication is an exception and does not extend the pending trail [C2, C5]. The general bound is conditional, not established for every rule kind | +| **Can commuting orders be reduced?** R1 permits recoverable classes; R6 requires invariant output | Both conditions have a different scope. Recovery can preserve every path; discarding an order requires R6's stronger condition. This resolves the ledger's parse-signature objection, but does not establish a synthesis saving | +| **Packed forest/ZDD/product graph is the best lead** in R2/R4/R5, versus R6's memory objection | R5 is right that an exact product needs full synthesis state and ordered continuation/provenance; one final analysis supplies no small-state bound. R6 is right under this job's constraint. Generic product charts are not recommended. A ZDD encodes sets; ordered trails need position/occurrence encoding, whose size none of the reports bounds | +| **Optional deletion is already compact**, emphasized by R3/R6 | Correct for shape construction and Rust trie active states [C6]. It does not establish cheap general morphological matching: C7 needs captures/bindings, and ledger row 23 closes position-only deduplication | +| **The proposer is `O(k^2)`**, R3 | Verified only for the legacy branch. C9's executed mode dispatch and mask loop take precedence over the older descriptive comment | +| **Simple static precedence could remove impossible orders**, R4 | Conditional only: adjacent-pair failure does not rule out an intervening enabler, and an optional predecessor cannot become mandatory. Candidate 1 states the stronger complete-path certificate | +| **A 64-key root cache bounds walks by `min(U,64)`**, R5; R4 also proposes bounded reuse | The raw search boundary is reusable under fixed grammar/overlay [C4], but the bound is wrong when `U > 64`: uncached keys are searched on every revisit. A byte cap must also cover shapes and result lists. Under the no-memo instruction this is not recommended | + +## Considered and not recommended + +- **Count-vector charts, generic packed analysis forests, ZDD trail tables, analysis-DAG × synthesis-state + products:** search-growing tables; rows 1/4/5/6/16 already test the necessary convergence, and a fixed cap + with restart does not make these a different allowed technique. +- **Packed synthesis readout keyed only by trail position or current shape:** omits ordered continuation + and root-dependent state; rows 5/6/16 close it. `O(Q+T+A)` only helps after proving `Q` and provenance + readout small, which no report does. +- **State/position-only matcher deduplication or two-pass capture reconstruction:** row 23 and the + [two-pass record](rejected-optimizations/two-pass-nondeterministic-traversal.md) already settle these; + the safe group-free fragment is existing prior work, not a new research result. +- **Fixed-cap exact root-result cache:** useful only for raw lookup, still memoization, and neither + 56 shapes nor 64 entries bounds all `(stratum, shape)` result bytes or repeated misses. +- **Root-only lexical gates, edge prefilters, current-length bounds, feature-only abstract replay, and + nogood subsumption:** rows 8–11/14/19/22 already cover them; candidate 1 survives only with its stronger + residual relation and a new measured dead-subtree ceiling. +- **Commuting-order canonicalization without recovery, symmetry/orbit counts:** distinct rule/morpheme + identities and ordered trails remain observable; no output-fixing grammar automorphism was established. +- **Meet-in-the-middle:** large midpoint frontiers violate the memory constraint; unordered rule splits + do not automatically turn factorial work into two square-root searches. +- **A*, k-best/k-shortest paths, reverse-search enumeration alone:** all valid analyses are required; + rearranging or ranking trail enumeration does not eliminate failed trails without a sound extension oracle. +- **Generic Earley/CKY, scalar semirings, equality saturation/e-graphs:** no proved semantics-preserving + span/equivalence boundary for these transforms; adding full transformation state/provenance restores + the unresolved search size. +- **Bimachines, two-level morphology, compile-replace, or parser names alone:** constrained morphology + formalisms supply candidate 4's tools, not a correctness proof for arbitrary HermitCrab behavior. + +## Implications for the two work items + +**Lexicon-driven pruning during unapplication.** Start with candidate 1 in shadow mode at C1/C2. The +deliverable is a per-input census of proved-dead subtrees, saved descendant work, unknown cases and +abstraction cost. Include remaining morphology, template transitions, intermediate-stratum lexical goals, +optional sites and compound non-head roots. Disable pruning where guessing/overlays are uncovered. Close +it if the result merely reproduces row 10's no-op or row 11's weak root-only ceiling. Exact product +coaccessibility is a possible stronger oracle only after a grammar-sized representation exists; building +a whole search graph to ask whether that graph has a successful leaf is not an allowed shortcut. + +**Making the affix-template battery cheaper without memoization.** Start with candidate 2's static +prefix census and admission/slot work counts. Its target is repeated control **within one invocation**. +Candidate 1 can also avoid an invocation whose entire continuation is proved dead; candidate 3 might +avoid repeated prefix work if independence coverage is real. Neither the 43,641/4,988 ratio nor the +historical battery memo authorizes reinstating a cache. Keep deferred cloning, safe edge dispatch and +the final-template policy distinct from this research: their evidence and limits already have records. +In particular, the [final-template record](rejected-optimizations/final-template-prune.md) has an addendum +reopening a narrower guard; its old zero-prune result is not a new lead to rediscover here. + +For either item, compare complete analysis sets with ordered morpheme/allomorph identities, root position, +features and non-head provenance. Do not use the legacy count-degenerate signature called out in the +September disposition. Tests support a proof; a finite corpus cannot prove parity for every export. +Tracing and stateful selectors use the existing path unless their invocation/event semantics are preserved. +Configured alternative/work limits also use the existing path unless the same limit behavior is reproduced +[C1, C4]; a resource-limited optimized query returns unknown and falls back, never a partial exact result. + +## Evidence limits and references + +**Not verified:** the brief's timings/counters, the earlier POR eligibility percentage, exact applicability +of any proposed certificate/compiler to the motivating grammar, common template-prefix coverage, a small +packed product/ZDD bound, and any end-to-end speedup. No real-grammar benchmark or new optimization was +run. Historical measurements are attributed to the supplied brief or the existing records, not fresh +results. Code inspection verifies the listed control/data facts; it does not establish universal parity +of the current Rust port or the current FST proposer. All future experiments need the stronger output +signature and ordinary-parser fallback. + +All entries below are **[memory; bibliographic details and applicability not independently verified]**. +No network or publication lookup was available. They identify home fields, not proofs of these proposals. + +- **L1.** P. Cousot and R. Cousot (1977), “Abstract Interpretation: A Unified Lattice Model for Static + Analysis of Programs by Construction or Approximation of Fixpoints.” +- **L2.** C. Beeri and R. Ramakrishnan (1991), “On the Power of Magic.” +- **L3.** B. Billot and B. Lang (1989), “The Structure of Shared Forests in Ambiguous Parsing.” Structural + sharing background; the grammar-prefix specialization here does not adopt a runtime parse chart. +- **L4.** A. Mazurkiewicz (1987), “Trace Theory”; P. Godefroid (1996), *Partial-Order Methods for the + Verification of Concurrent Systems*; F. Baader and T. Nipkow (1998), *Term Rewriting and All That*. +- **L5.** R. Kaplan and M. Kay (1994), “Regular Models of Phonological Rule Systems”; M. Mohri (1997), + “Finite-State Transducers in Language and Speech Processing”; K. Beesley and L. Karttunen (2003), + *Finite State Morphology*. + +The reports also invoke Held–Karp sequencing DP, Tomita-style shared forests, Minato's ZDDs, graph +coaccessibility, and PC-KIMMO/AMPLE/XAmple. These are useful vocabulary for the considered designs; +none supplies the missing state-equivalence, memory, or arbitrary-HermitCrab encoding proof. diff --git a/docs/optimizations/deferred-template-clone.md b/docs/optimizations/deferred-template-clone.md new file mode 100644 index 000000000..ebd7d24be --- /dev/null +++ b/docs/optimizations/deferred-template-clone.md @@ -0,0 +1,36 @@ +# Deferred per-template clone in `AnalysisAffixTemplateRule` + +**Status:** retained on `perf/hc-optimization-archive`, not merged. **Portable:** the observation is; the +saving is allocation, so the Rust port gains little unless it also clones eagerly. + +## The observation + +`AnalysisAffixTemplateRule.Apply` made an internal engine clone and froze it once per template before trying any +slot, then ran the slot DFS on the clone. The ledger measured 66% of those clones discarded unused +(2,505 created, 1,659 discarded, Amharic) and closed the "defer" option because it missed a 70% gate by +four points, choosing "make a clone cheaper" instead. + +Reading the code shows the clone had exactly one load-bearing job. The input is already frozen at every +call site; slot rules never mutate their input (`ApplyRhs` clones `match.Input`); intermediate `Source` +pointers are overwritten by the stratum. The only path that emits the input object itself is the +all-slots-skipped terminal, and `Apply` then mutates every emitted word's `SyntacticFeatureStruct`, so +that one path needs a private copy. + +## The change + +Run the DFS on the frozen input directly (untraced path). At the two emission points that can carry the +original input through unmodified (`ApplySlots` terminal, `ParallelApplySlots` push), emit a frozen +clone when the word is reference-equal to the input. The traced path keeps the eager clone because the +trace manager stamps `CurrentTrace` onto the words it is handed. A naive "no clone anywhere" version was +tried first and failed the new tests immediately (`FeatureStruct.Add` threw on a frozen structure). + +## Measured and verified + +Alternating wall-time A/B against the pinned baseline, 45 Sena words, min of two interleaved rounds: +1.054x overall, heavy words 1.02x to 1.08x; `WordAnalysisSignature` parity on all words and both rounds (on +Sena that signature compares morph counts only; see `hermitcrab-perf-2026-09-disposition.md`). Smaller than the +ledger's framing suggested: the eager clone was not where the heavy words' time went. + +Focused template tests cover required/optional slots, traced execution, and both sequential and parallel +paths. The complete conformance-inclusive HermitCrab run at the final code tip passed 603 tests with one +expected host-privilege skip. diff --git a/docs/optimizations/edge-segment-prefilter.md b/docs/optimizations/edge-segment-prefilter.md new file mode 100644 index 000000000..048bfa802 --- /dev/null +++ b/docs/optimizations/edge-segment-prefilter.md @@ -0,0 +1,65 @@ +# Edge-segment prefilter for analysis affix rules + +**Status:** retained on `perf/hc-optimization-archive`, not merged (`AnalysisAffixProcessRule`; internal +same-binary toggle `Morpher.EdgePrefilterEnabled`). **Portable:** yes; it is a property of the pattern +semantics, not of the C# engine. + +## The observation + +Every analysis affix pattern is compiled with `AnchoredToStart = true`, `AnchoredToEnd = true`, +`MatchingMethod = Unification`, and a filter that shows the matcher only Segment-type annotations. The +pattern is built from the allomorph's Lhs/Rhs: copied stem parts become groups with quantifiers, and each +inserted segment becomes a single `Constraint` node carrying that segment's feature structure +(`InsertSegments.GenerateAnalysisLhs`). A `Constraint` compiles to one mandatory arc with no epsilon +alternative (`Constraint.GenerateNfa`). + +So when the pattern's first top-level node is a `Constraint`, the traversal's first step must unify it +with the shape's first segment node, or the traversal fails; likewise the last `Constraint` and the last +segment node. The only way an annotation can be skipped is when it is marked `Optional`. + +On the heaviest Sena word, 3,564,736 pattern-match attempts produced 170,863 matches (95% failed), and +each failed attempt still ran a full nondeterministic traversal: 4,017,574 traversal instances, +9,111,459 arc checks. + +## The change + +At compile time, `AnalysisAffixProcessAllomorphRuleSpec` records `LeftEdgeConstraint` / +`RightEdgeConstraint`: the feature structure of the pattern's first / last child when that child is a +bare `Constraint` (null when it is a group, quantifier or alternation). At run time, before each +allomorph's `MultiplePatternRule.Apply`, `AnalysisAffixProcessRule.Apply` finds the shape's first and +last Segment-type nodes (skipping boundaries exactly as the matcher's filter does) and rejects the +allomorph if `edgeNode.FeatureStruct.IsUnifiable(edgeConstraint)` fails, using the same unification call +and `UseDefaults` setting the matcher itself uses. An `Optional` edge node disables the check on that +side. When tracing, the same `MorphologicalRuleNotUnapplied` event is emitted that the matcher's failure +would have produced, so traces are unchanged. + +## Measured (deterministic counts, sequential path, one parse per word, same binary) + +| 45 Sena words | prefilter off | prefilter on | +| --- | --- | --- | +| FST traversals | 12,376,416 | 1,617,662 | +| FST arc checks | 34,170,777 | 15,583,242 | +| traversal instances used | 15,642,175 | 2,429,643 | +| pattern-match attempts (counted before the check) | 11,979,916 | 11,979,916 | + +Parity: identical `WordAnalysisSignature` sets on all 45 Sena words (morph counts only on Sena; see +`hermitcrab-perf-2026-09-disposition.md`) and on all 464 conformance-fixture words +(33 grammars). Fixture traversals fell 21% (31,278 to 24,741); their patterns fail less often. + +`AnalysisEdgePrefilterTests` covers left/right constraints, boundary skipping, optional nodes, and the +disabled path. The broader HermitCrab and conformance suites cover integration behavior; the final +conformance-inclusive run passed 603 tests with one expected host-privilege skip. + +## Why the ledger's estimate was wrong + +The ledger (row 22) capped a prefilter at ~10% using the matcher's *self-time* share. The traversal's cost +is mostly booked elsewhere (instance and register allocation, feature unification, GC), so self-time +understated it by a large factor. By counts, the prefilter removes 87% of traversals outright. + +## Limits + +After the change, ~44% of the remaining traversals succeed, so a second-segment check has little left to +reject. The rule loop still runs ~110 attempts per cascade state; depending on which safe edge constraints +exist and whether the first rejects, an attempt performs zero, one, or two prefilter unifications before any +remaining match. Indexing allomorphs by edge segment could avoid both checks and matching work, but its +headroom was not established and it was not attempted. diff --git a/docs/optimizations/feature-struct-frozen-tree-clone.md b/docs/optimizations/feature-struct-frozen-tree-clone.md new file mode 100644 index 000000000..3677c7750 --- /dev/null +++ b/docs/optimizations/feature-struct-frozen-tree-clone.md @@ -0,0 +1,28 @@ +# Reuse or skip the feature-structure clone map + +**Status:** retained on `perf/hc-optimization-archive`, not merged (`FeatureStruct.Clone`). +**Portable:** yes; the specific pool is .NET-oriented, while the tree proof is general. + +## Invariant + +A frozen structure marked as a flat tree has no aliases or cycles to preserve below its root. Its top-level +clone therefore cannot encounter the same value twice, so the old-to-new copies map has no semantic work to +do. Recursive cloning inside another graph must still use the caller's map even when the child itself looks +tree-shaped, because the parent graph may refer to that child elsewhere. + +## Implementation + +A top-level clone skips the map for a proven frozen flat tree. General clones borrow a per-thread map, clear +it in `finally`, and fall back to an independent map for reentrant calls. The internal recursive constructor +always honors the caller-provided map. + +## Evidence + +This removes dictionary creation and lookup from the common clone case. It travelled with the broader clone +allocation work and was not timed independently, so the record deliberately makes no isolated speed claim. + +## Correctness coverage and limits + +`FeatureStructCloneTests` covers alias preservation, cycles, reentrant cloning, and the distinction between a +top-level tree and a tree embedded in a shared graph. The complete conformance-inclusive run passed 603 tests +with one expected host-privilege skip. diff --git a/docs/optimizations/feature-struct-frozen-tree-equality.md b/docs/optimizations/feature-struct-frozen-tree-equality.md new file mode 100644 index 000000000..6f34aa04d --- /dev/null +++ b/docs/optimizations/feature-struct-frozen-tree-equality.md @@ -0,0 +1,29 @@ +# Allocation-free equality for frozen flat feature structures + +**Status:** retained on `perf/hc-optimization-archive`, not merged (`FeatureStruct.ValueEquals`). +**Portable:** yes; the proof depends on immutability and tree shape, not the CLR. + +## Invariant + +During `Freeze`, a feature structure is marked as a flat tree only when every reachable value is a +`SimpleFeatureValue`, no value instance repeats, and no nested `FeatureStruct` is present. Once frozen, that +fact cannot change. Comparing two such structures can never trigger the general algorithm's reentrancy or +cycle guard, so a feature-count/key/value comparison is equivalent to the general visited-pair walk. + +## Implementation + +`ValueEquals` uses the direct leaf comparison only when both operands are frozen and carry the tree proof. +Every other shape—including shared leaves, nested structures, and cycles—uses the general algorithm. A +test-only slow-path entry point lets randomized tests compare both implementations on the same inputs. + +## Evidence + +The optimization removes all visited-set work from the dominant frozen, flat case. It was observed in the +same profile that motivated traversal-state reuse, but was not isolated from the other feature-structure +changes; no standalone wall-time ratio is claimed. + +## Correctness coverage and limits + +`FeatureStructTreeFastPathTests` cross-checks randomized flat structures against the slow path and exercises +false comparisons and structures that must not qualify. `FeatureStructValueEqualsFreezeTests` covers shared +and cyclic graphs. The full conformance-inclusive run passed 603 tests with one expected host skip. diff --git a/docs/optimizations/feature-struct-visited-state-reuse.md b/docs/optimizations/feature-struct-visited-state-reuse.md new file mode 100644 index 000000000..f7c2b63ee --- /dev/null +++ b/docs/optimizations/feature-struct-visited-state-reuse.md @@ -0,0 +1,29 @@ +# Reuse feature-structure traversal state + +**Status:** retained on `perf/hc-optimization-archive`, not merged (`FeatureStruct.ValueEquals` and +`Freeze`). **Portable:** the allocation-avoidance principle is portable; the `[ThreadStatic]` pool is +.NET-specific. + +## Invariant + +Equality and freezing need visited sets only after a recursive value is reached. A top-level +operation owns its traversal state until it returns. A nested top-level call cannot borrow that same state, +so it falls back to fresh collections. + +## Implementation + +The recursive APIs accept nullable state and allocate it only when traversal requires it. Ordinary top-level +calls borrow pre-sized sets and dictionaries from a per-thread pool, clear them in `finally`, and release the +pool. The pool has an `InUse` guard so reentrant calls retain the old independent-state semantics. + +## Evidence + +The heaviest Sena profile put feature-structure equality/freezing at 13.7% of wall time, with `HashSet` +resizing a major allocation source. After this change, `ValueEqualsImpl` and those resizes disappeared from +the top of the profile. This mechanism was not timed in isolation, so no standalone speedup is claimed. + +## Correctness coverage and limits + +`FeatureStructPooledVisitedSetsTests` covers reuse, clearing after success and failure, and reentrancy. Cyclic +and shared graphs continue down the general path. The complete conformance-inclusive HermitCrab run at the +final code tip passed 603 tests with one expected host-privilege skip. diff --git a/docs/optimizations/flat-register-arrays.md b/docs/optimizations/flat-register-arrays.md new file mode 100644 index 000000000..553e7eb99 --- /dev/null +++ b/docs/optimizations/flat-register-arrays.md @@ -0,0 +1,26 @@ +# Flat register arrays in the FST traversal + +**Status:** retained on `perf/hc-optimization-archive`, not merged (`SIL.Machine.FiniteState`: +`Register[]` indexed by `RegisterArray.Idx(r, k)` replaces `Register[,]` internally; the +public `FstResult.Registers` stays two-dimensional and is materialized once per accepting state). +**Portable:** no; a .NET-specific allocation path. + +## The observation + +11.5% of the heaviest Sena word's CPU profile was inside `System.Array.CreateInstanceMDArray`, the slow +generic allocator for multi-dimensional arrays, called from `Matcher.AllMatches` (8.6%) and traversal +setup (2.3%): one `new Register[n, 2]` per match call and per traversal instance. + +## The change + +Mechanical: flat arrays of length `registerCount * 2`, a two-constant index helper, `Array.Copy` on the +flat struct array, and `RegistersEqualityComparer` comparing the flat layout. No behaviour change. + +## Measured + +`CreateInstanceMDArray` 11.5% → 0.2% of the profile. Counts unchanged by construction. Together with the +other allocation changes it contributed to the heaviest word's wall time; not measured in isolation. + +`RegistersEqualityComparerTests` covers equality and hashing of the flattened layout, while the existing FSA +and FST suites cover accepting results and register materialization. The complete conformance-inclusive run +passed 603 tests with one expected host-privilege skip. diff --git a/docs/optimizations/shape-copy-single-pass-map.md b/docs/optimizations/shape-copy-single-pass-map.md new file mode 100644 index 000000000..900515da2 --- /dev/null +++ b/docs/optimizations/shape-copy-single-pass-map.md @@ -0,0 +1,28 @@ +# Build `Shape.CopyTo`'s node map in the copy pass + +**Status:** retained on `perf/hc-optimization-archive`, not merged (`Shape.CopyTo`). +**Portable:** yes, when a port has the same node-remapping requirement. + +## Invariant + +Annotations refer to nodes in the source shape and must be remapped to the corresponding nodes in the copy. +The source and destination nodes are already visited in matching order while cloning, so that loop can record +the mapping directly. No second `Zip`/`ToDictionary` traversal is required. An empty source range needs no +map; every nonempty range builds one because annotations are queried only after nodes have been copied. + +## Implementation + +`Shape.CopyTo` allocates the map on the first node of a nonempty range and fills it as each node is cloned. +Annotation ranges are then copied through that map exactly as before. + +## Evidence + +The change removes one enumeration and one LINQ-built dictionary from each annotated shape copy. It was not +timed independently from shape copy-on-write and feature-structure clone work, so no standalone ratio is +claimed. + +## Correctness coverage and limits + +`ShapeCopyToTests` covers empty ranges, leaf and spanning annotations, node/range remapping, and independence +of the copy. +The complete conformance-inclusive run passed 603 tests with one expected host-privilege skip. diff --git a/docs/optimizations/shape-sharing-on-clone.md b/docs/optimizations/shape-sharing-on-clone.md new file mode 100644 index 000000000..a5ffd563f --- /dev/null +++ b/docs/optimizations/shape-sharing-on-clone.md @@ -0,0 +1,44 @@ +# Copy-on-write Shape sharing in `Word.Clone` + +**Status:** retained on `perf/hc-optimization-archive`, not merged (`Word.CloneForEngine`, `EnsureOwnShape`, +`ResetShape`, `Shape.CreateEmptyLike`). **Portable:** partly. The *observation* (three of four clone sites +never use the copied shape) is engine-independent; the mechanism (shared reference to an immutable +structure) is how a Rust port would naturally write it already. + +## The observation + +A CPU profile of the heaviest Sena word (60 s sampled, memoized path) put `Word` cloning at 36% of wall, +of which `Shape.CopyTo` was 28%: every clone deep-copied every `ShapeNode`, every node's feature +structure (`SimpleFeatureValue.CloneImpl` 22%), every annotation, and built a node-mapping dictionary. +The callers: `Word.ReplayOnto` 15% (memo replay; never mutates the shape), `Word.ExpandAlternatives` 9% +(immediately *replaces* `_shape`), `AnalysisAffixProcessAllomorphRuleSpec.ApplyRhs` 5% (`GenerateShape` +immediately `Clear()`s the copy), non-head clones 2.5%, `LexicalLookup` (the `RootAllomorph` setter +replaces `_shape`). + +## The change + +- Internal engine clone: if the source shape is frozen, share the reference and mark `_shapeShared`; otherwise + deep-copy it. Public `Word.Clone()` always returns a deep, independently mutable shape. +- `EnsureOwnShape()`: clone once, on demand, before in-place mutation. Called in `AnalysisStratumRule` and + `SynthesisStratumRule` before phonological rules are applied to a fresh clone (rewrite specs insert + nodes and hold node references obtained by matching, so the un-share must precede the match). +- `ResetShape()`: replace with a new empty shape of the same configuration. Used by the four + `GenerateShape`/rebuild sites (analysis affix, analysis compounding subrule, synthesis affix, synthesis + compounding) instead of clone-then-clear. +- `ExpandAlternatives` marks the adopted shape as shared; the `RootAllomorph` setter and constructors mark + their fresh clones as owned. + +The safety net is that a shared shape is frozen: any mutation through a missed site throws +(`Shape.CheckFrozen`, frozen node feature structures; a missing guard on `Annotation.Range` was added). +During development the first full test run threw at exactly the two stratum sites, which is how they +were found; no other site surfaced across 122 HermitCrab tests, 464 fixture words and 45 Sena words. + +## Measured + +Profile after the change: `Word` clone 36% → 9% of wall, `Shape.CopyTo` 28.6% → 13% (the remainder is +the stratum's `EnsureOwnShape` and part copying inside `GenerateShape`), `ReplayOnto` 21.7% → 6.5%. Peak +working set on the heaviest word 11 GB → 6 GB. Wall-time attribution is unreliable on this machine (see +the round document); counts are unchanged by construction. + +`WordShapeSharingTests` covers public-clone independence, copy-on-write, reset paths, and missed-mutation +guards. The final conformance-inclusive run passed 603 tests with one expected host-privilege skip. diff --git a/docs/optimizations/syntactic-feature-structure-sharing.md b/docs/optimizations/syntactic-feature-structure-sharing.md new file mode 100644 index 000000000..9f99d4570 --- /dev/null +++ b/docs/optimizations/syntactic-feature-structure-sharing.md @@ -0,0 +1,33 @@ +# Copy-on-write syntactic feature structures in engine clones + +**Status:** retained on `perf/hc-optimization-archive`, not merged (`Word.CloneForEngine`, +`EnsureOwnSyntacticFeatureStruct`). **Portable:** partly; immutable sharing is general, while the ownership +plumbing is specific to this engine. + +## Invariant + +HermitCrab freezes a `Word` before it becomes an input to the analysis or synthesis graph. Internal clones may +therefore share that frozen word's syntactic feature structure until a rule needs to mutate it. The public +`Word.Clone()` contract remains a deep, mutable clone and never exposes copy-on-write sharing to callers. + +## Implementation + +Internal engine sites use `CloneForEngine()`. A per-`Morpher` setting is copied onto new words and controls +whether those clones share a frozen syntactic feature structure. The three in-place analysis mutation sites +call `EnsureOwnSyntacticFeatureStruct()` before modification; assignments of a fresh structure need no guard. +The opt-out is internal and captured per `Morpher`, preventing one parser or test from changing another +existing parser's behavior. + +## Evidence + +The heaviest Sena profile attributed roughly 15% of CPU to freezing and hashing a newly cloned syntactic +feature structure while constructing analysis keys. Sharing removes that repeated deep clone/freeze on the +engine path. The final five-language measurement is branch-wide rather than an isolated toggle result, so no +standalone speedup is assigned here. + +## Correctness coverage and limits + +`WordSyntacticFsSharingTests` covers copy-on-write, nested/shared structures, public-clone mutability, and +isolation between morphers. The broader HermitCrab suite exercises integration behavior and passed both with +sharing enabled and with `HC_SHARE_SYNTACTIC_FS=false`; the full conformance-inclusive run passed 603 tests +with one expected host skip. Sharing is intentionally unavailable for mutable source words. diff --git a/docs/rejected-optimizations/alternatives-memo-and-hoist.md b/docs/rejected-optimizations/alternatives-memo-and-hoist.md new file mode 100644 index 000000000..419cba7ef --- /dev/null +++ b/docs/rejected-optimizations/alternatives-memo-and-hoist.md @@ -0,0 +1,15 @@ +# `ExpandAlternatives` memoization and hoisting + +**Status:** rejected and removed from production. + +The experiment memoized `Source` recursion during `Word.ExpandAlternatives` and computed an analysis word's +expansion once outside the per-root-candidate loop. It was exact, but on the 45-word Sena battery it left word +clones unchanged at 4,747,013: deeper strata did not branch, so it saved only small temporary list work. + +A later change moved work back behind successful lexical lookup, but after the experimental memo/hoist was +removed the final product path is the same ordering as the pinned production baseline: each lexical candidate +expands its synthesis word. There is therefore no retained alternatives optimization in this branch, and the +earlier 75% “alternatives expanded” reduction must not be attributed to the final diff. + +The broader deduplication avenue remains closed by the complete-key census in the main ledger: once pending +trail content is included, only 11% of alternatives duplicate, for a 2.28% wall-time ceiling on Sena. diff --git a/docs/rejected-optimizations/compounding-split-prefilter.md b/docs/rejected-optimizations/compounding-split-prefilter.md new file mode 100644 index 000000000..067fbd07d --- /dev/null +++ b/docs/rejected-optimizations/compounding-split-prefilter.md @@ -0,0 +1,11 @@ +# Compounding split prefilter + +**Status:** rejected before implementation; temporary counters removed. + +The candidate was to query lexical reachability before materializing compounding splits. A census on the +heaviest Sena probe counted 35,176 splits, 5,524 root candidates, and 1,121 outputs. That work represented +about 2% of word clones and 9% of FST traversals—too small a ceiling for an additional indexing and correctness +surface, especially because real compounding prevents the simpler grammar-wide lexical gate. + +No production change was built. Reopen only if a different grammar shows compounding split construction as a +dominant measured bucket and supplies a sound, stratum-aware reachability predicate. diff --git a/docs/rejected-optimizations/final-template-prune.md b/docs/rejected-optimizations/final-template-prune.md new file mode 100644 index 000000000..a6f0002b5 --- /dev/null +++ b/docs/rejected-optimizations/final-template-prune.md @@ -0,0 +1,25 @@ +# Final-template interleaving prune + +**Status:** rejected and removed from production. **Portable:** the proof is portable where its guard holds. + +The experiment mirrored synthesis's rule that a non-template affix process normally cannot follow a final +template. Analysis could skip the inverse interleaving when every relevant template was final and the grammar +contained no partial morphemes. + +The available real grammars defeated the useful guard: Sena has 25 partial lexical entries and six partial +rules; Amharic and Mbugwe each have a partial rule. The prune fired zero times on all three. Synthetic tests +proved that the rule could fire safely, but that establishes correctness, not value. Carrying production +branches, state, toggles, and counters for a zero-effect optimization violated the final minimum-surface goal, +so all of them were removed. + +Reopen only for a corpus with the reported pathological final-template interleavings and no partial +morphemes, or with a proved narrower per-state partiality condition. Measure real prunes before shipping. + +## Addendum 2026-09-03 +Superseded in part. The same prune, measured with the partiality guard lifted, is 8-33x on Mbugwe and Sena with +identical analysis sets (298x on the single heaviest Sena word). The guard, not the mechanism, is what made it +inert: partial *lexical entries* can never rescue a pruned branch, because synthesis applies no template to a +partial root and then fails the candidate with `PartialParse`, so a rules-only, per-stratum guard is strictly +stronger and still sound. Reopen on that basis rather than on the corpus counts above. The full measurement +record is `docs/pr491-final-template-prune-upper-bound.md` on the `perf/hc-optimization-archive` branch, which +is outside this docs set. diff --git a/docs/rejected-optimizations/state-position-traversal-dedup.md b/docs/rejected-optimizations/state-position-traversal-dedup.md new file mode 100644 index 000000000..40a9fb66a --- /dev/null +++ b/docs/rejected-optimizations/state-position-traversal-dedup.md @@ -0,0 +1,95 @@ +# `(State, AnnotationIndex)` traversal deduplication + +**Status:** rejected in its general form; a narrowed form is sound and was measured effective. + +The idea, proposed independently as PR #511: inside `Traverse`, skip any newly produced traversal +instance whose `(State, AnnotationIndex)` pair was already pushed, on the grounds that the surviving +instance differs only in its registers and "registers only record the match, they do not filter the +traversal." + +## The motivating pathology is real + +The cost driver is not the state count. `TraversalMethodBase.Advance` forks a copy of the instance for +every `Optional` annotation at the next offset, so instance count grows exponentially in the number of +optional annotations along the input and is independent of `|States|`. A synthetic microbenchmark on a +**2-state** FSA measured 2,097,150 instances popped at 20 optional annotations (~2^(k+1)); at k>=24 it +exceeds a 5,000,000 budget. The originally reported "200,000 traversals on a 4-state fsa" is reproduced +and exceeded on fewer states. With the skip in place, the same cells are bounded at `N * |States|` +(measured ratio 0.5-1.0x), turning non-terminating cells into instant ones. + +## Why the general form is unsound + +A differential fuzz — 20,000 random pattern/input pairs, fixed seed, identical generator compiled against +both arms, comparing success, range, every group range and every variable binding — found **112 cases +where `Match()` differs from production**, while `AllMatches().First()` was identical in all 20,000. That +control pins the divergence to exactly the path the skip changes. + +- **108 cases: variable bindings.** Instances carry live `VariableBindings`, which filter later arcs + through `CheckInputMatch`; the key omits them. `Matcher.Compile` determinizes only variable-free + patterns, so every alpha-variable rule runs the nondeterministic method where the key is blindest. + Minimal case: pattern `high=$v0+` anchored both ends over 5 annotations, production matches `[0,5)`, + the skip returns no match at all. +- **4 cases: capture registers, on the deterministic path.** All are alternation plus capture groups and + all shorten the match *range*, e.g. `(g0(back=back+)|g1(high=high+back=back+))` going from `[0,4)` to + `[0,1)`. So restricting to the deterministic method is not by itself a sufficient guard. + +Both committed suites stay green under the unnarrowed change (826/0/3 and 98/0/0), so the existing tests +cannot see any of this. + +## The narrowed form that is sound + +Gate the skip on the deterministic method **and** the absence of capture groups, and leave the +nondeterministic method untouched: + +```csharp +bool dedupeByState = !allMatches && Fst.GroupNames.Count() <= 1; +``` + +Measured: fuzz divergences **112 -> 0**; the group-free microbenchmark sweep cell-for-cell identical to +the unnarrowed change across all 18 cells, so the full bound is preserved; grouped patterns fall back to +production behaviour exactly; both suites unchanged. `GroupNames.Count()` should be hoisted out of +`Traverse` — it is O(1) via the `ICollection` fast path but is evaluated once per call. + +## Where it cannot be extended, and why + +Two censuses attributed every FSA traversal call to its constructing matcher and counted, per category, +how many pushes a `(State, AnnotationIndex)` key would have collapsed — and critically, whether the +collapsed instance's registers were identical to the surviving one's. + +*Environment matchers* (the subject of issue #515) read only `.Success` and `.VariableBindings` from their +matches, never a range or group, so they looked like the ideal target. They are 0.20% of traversal +instances on Amharic and 1.18% on Mbugwe. Removing every redundant environment push would be worth 0.12% +and 0.004% of total pushes respectively. Not built. + +*Analysis-side morphological rules* hold nearly all the volume and all the apparent opportunity: +2,801,478 of Amharic's 2,808,088 pushes, 93.95% collapsible; Mbugwe 14.44%. Labelling the remaining +construction sites showed the entire figure is exactly `AnalysisAffixProcessRule` and +`AnalysisCompoundingRule` — both built with `AllSubmatches = true` and calling `AllMatches` precisely to +enumerate every distinct morph-boundary placement. Register comparison at each would-be collapse: + +| matcher | identical registers | differing registers | +| --- | --- | --- | +| Amharic `analysis-affix` | 7,446 (0.29%) | 2,573,312 (99.71%) | +| Amharic `analysis-compound` | 16 (0.03%) | 50,920 (99.97%) | +| Mbugwe `analysis-affix` | 3,022 (5.49%) | 52,056 (94.51%) | +| Mbugwe `analysis-compound` | 1,886 (2.93%) | 62,396 (97.07%) | + +The genuinely free slice is 0.28% of collapses on Amharic and 4.11% on Mbugwe — at most 0.6% of total +traversal instances, reachable only by adding a per-instance register-equality guard. `RootAllomorphTrie`'s +raw FST collapses 0% despite ~62k pushes, which is correct rather than a missed opportunity: a prefix trie +is already maximally state-shared at construction. + +Adding `VariableBindings` to the key costs nothing on this evidence: `collapsedWithBindings == collapsed` +on every row of both grammars, with a distinct-signature counter confirming the signature function +discriminates. The information that differs lives in the registers, which is exactly what `AllSubmatches` +consumers read. + +## Relationship to `two-pass-nondeterministic-traversal.md` + +That experiment merged on state, position **and bindings** and then rebuilt captures, and was measured +negative largely on the rebuild cost. This one merges on state and position only and keeps the first +instance's captures, which is cheaper and unsound rather than expensive and sound. The two are not +alternatives to the same design; they are the two ways of getting the key wrong in opposite directions. + +**Reopen only** with a grammar where the identical-registers split looks materially different from the +four rows above, or for the narrowed group-free form, which needs no further evidence to be safe. diff --git a/docs/rejected-optimizations/two-pass-nondeterministic-traversal.md b/docs/rejected-optimizations/two-pass-nondeterministic-traversal.md new file mode 100644 index 000000000..344a6bd0e --- /dev/null +++ b/docs/rejected-optimizations/two-pass-nondeterministic-traversal.md @@ -0,0 +1,18 @@ +# Two-pass nondeterministic traversal + +**Status:** measured negative and removed from production. + +This port of the `investigate_linear_traverse` approach first merged traversal instances by state, position, +and variable bindings, then rebuilt captures. HermitCrab's existing traversal already pools instances, and +its patterns do not have the repeated-group shape needed to repay the extra bookkeeping. + +On the 45-word Sena battery the alternative used 2.3 times as many traversal instances, reduced arc checks by +only 1.6%, produced more duplicate matches, and increased word clones by 27%. Conformance fixtures pointed in +the same direction. Toggle hooks, alternate algorithms, and counters were removed; only this result remains. + +Reopen only with a corpus whose automata have substantial equivalent active-state convergence and a census +showing that capture reconstruction is cheaper than the current pooled traversal. + +**See also** `state-position-traversal-dedup.md`, which drops bindings from the key instead of keeping them +and skips capture reconstruction entirely. It is cheaper than this experiment and unsound for the same +reason this one was expensive: the captures are load-bearing for `AllSubmatches` callers.