Skip to content

HermitCrab: keep final-template pruning on when only lexical entries are partial - #553

Open
johnml1135 wants to merge 4 commits into
masterfrom
fix-551-partial-entry-prune
Open

johnml1135 wants to merge 4 commits into
masterfrom
fix-551-partial-entry-prune

Conversation

@johnml1135

@johnml1135 johnml1135 commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Start here: src/SIL.Machine.Morphology.HermitCrab/Morpher.cs, HasPartialRules (and its doc comment): the contract the other edits follow.

Status: open since 2026-10-23. The reviewer's reply on the IsPartial removal (FieldWorks does not read it, nothing further needed) is in. Two review notes on test coverage (F5, F6: the analysis-side AlwaysEnforceFinalTemplates override, and the PruneDisagreeingCopiesTests false case) are still open for the author. CI has no red checks (the claude-review job was skipped).

Closes #551. The final-template prune from #491 (merged) is no longer skipped when only lexical entries are partial. Morpher.HasPartialRules is true only if a partial morpheme is not a LexEntry, and AnalysisAffixTemplateRule and AnalysisAffixProcessRule read it in place of Morpher.IsPartial, which is removed.

Reviewers arrive asking whether a partial root could license a derivation the prune rejects. It cannot: SynthesisAffixTemplatesRule skips every template when the root allomorph's morpheme is partial. A partial root with a non-partial LexFamily relative can still take a template, with the relative as root, but that word is non-partial, so a following non-partial rule is blocked either way. Partial rules are different: they make the word partial after the root, so they keep disabling the prune.

Where to look:

  • Morpher.HasPartialRules: the !(morpheme is LexEntry) filter over GetPartialMorphemes().
  • AffixTemplateTests.PartialLexicalEntryDoesNotDisableEarlyPruningOfFinalTemplate: prune fires at trace depth 2 with a partial entry and no partial rule, sab and sabv parse, sabd and sabdv do not; with a partial rule sagdv still parses as 32 PAST NOM. It fails with the old guard (depth 6 instead of 2).
  • Public API: Morpher.IsPartial is gone and nothing in the library read it; FieldWorks does not.

Not here: no fixture exercises ChooseInflectionalStem substitution (traced by reading only); lexical-pattern guesses were already excluded from the partial-morpheme set.

Verification: ./local_check.sh --agent-strict passed before the rebase onto master. After the rebase and the IsPartial removal only AffixTemplateTests and PruneDisagreeingCopiesTests were rerun (46/46). Not rerun: the full strict gate on the final head.

Next: author to decide on F5 and F6, then approve.


Reading this a year from now -- start here

Prune is the early rejection, during analysis, of a final template followed by a non-partial rule. #551 suspected that ignoring partial entries was unsound. The argument that it is sound is under "Decisions, and why"; the conformance evidence is under "Evidence". #480 (open) carries the fixture.

Decisions, and why
  • A word is partial from its root (Word copies entry.IsPartial) or from a partial rule. Synthesis lets a non-partial rule follow a final template when the input word is partial (SynthesisAffixProcessRule.cs:67). Templates never apply to a partial root, so a partial entry cannot reach that case.
  • ChooseInflectionalStem runs before the partial-root check, so the nuance above holds; the HasPartialRules doc comment states it.
  • IsPartial was removed because nothing reads it once both analysis guards use HasPartialRules. The three test preconditions that used it assert GetPartialMorphemes() directly.
Surprising findings
  • A reviewer traced one more path: a blockable template slot rule can call Word.CheckBlocking, which swaps in a partial family relative that a non-partial rule may follow. The output should match analysing the relative directly as a root, so no parse is expected to be lost. Unverified.
Deferred
  • F5: a partial rule with AlwaysEnforceFinalTemplates on, to cover the analysis-side override (for example at the end of the new test).
  • F6: a partial rule in the PruneDisagreeingCopiesTests fixture, so the false case tests copy-pruning with the final-template prune off.
Evidence
  • Fixture conformance/edge-cases/final-template-partial-root on Add a conformance suite for HermitCrab, with its adequacy argument #480 (integrate-conformance-framework at fa8d9b2) has partial roots in a lower stratum and in the template's own stratum. Every final-template string on a partial root has no analysis, with non-partial controls.
  • In a scratch copy with the same guard, all 142 conformance runs over 132 fixtures (including pathological) passed, and the prune fired on the new fixture. These counts are from the earlier run and were not recomputed.
  • ./local_check.sh --agent-strict passed before the rebase; AffixTemplateTests and PruneDisagreeingCopiesTests 46/46 after.

🤖 Generated with Claude Code


This change is Reviewable

@codecov-commenter

codecov-commenter commented Oct 23, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.73%. Comparing base (c245476) to head (69edb79).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #553      +/-   ##
==========================================
+ Coverage   74.57%   74.73%   +0.16%     
==========================================
  Files         453      453              
  Lines       38342    38342              
  Branches     5284     5284              
==========================================
+ Hits        28593    28656      +63     
+ Misses       8583     8532      -51     
+ Partials     1166     1154      -12     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread src/SIL.Machine.Morphology.HermitCrab/Morpher.cs Outdated
Comment thread tests/SIL.Machine.Morphology.HermitCrab.Tests/AffixTemplateTests.cs Outdated
@claude

This comment has been minimized.

@johnml1135
johnml1135 force-pushed the fix-551-partial-entry-prune branch from 57a3b7e to f1ac77e Compare October 26, 2026 11:49
johnml1135 added a commit that referenced this pull request Oct 26, 2026
PR #553 shows ignoring partial lexical entries in the final-template prune
guard is safe, so Sena gets the prune once it merges.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread src/SIL.Machine.Morphology.HermitCrab/Morpher.cs Outdated
@claude

This comment has been minimized.

@jtmaxwell3

Copy link
Copy Markdown
Collaborator

I think that you can remove Morpher.IsPartial in the code and in the comments, since it is superceded by Morpher.HasPartialRules.

johnml1135 and others added 3 commits October 27, 2026 09:56
The analysis-side final-template prune (491) was switched off whenever Morpher.IsPartial
was true, and IsPartial counts partial lexical entries. Grammars with partial entries but
no partial rules therefore never pruned.

A partial entry cannot make the prune lose a parse. A word takes its partiality from its
root, but SynthesisAffixTemplatesRule applies no template to a word whose root morpheme is
partial, so a derivation that unapplies a final template can never end at a partial root.
Partial rules differ: they make a word partial after the template has applied, which lets
a non-partial rule follow the final template, so they still disable the prune.

Add Morpher.HasPartialRules and read it in the two analysis guards. IsPartial keeps its
meaning. Lexical-pattern guesses were already excluded from IsPartial and pruned.

Closes #551

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The summary claimed a derivation that unapplies a final template can never end
at a partial root. SynthesisAffixTemplatesRule.Apply calls ChooseInflectionalStem
before the root check, so a partial root can take a template through a
non-partial family relative. State the contract and why the prune stays sound.

The test left the partial "sab" entry in the shared Language; remove it in a
finally block.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Nothing reads it now that the analysis-side final-template checks use
HasPartialRules. The three test preconditions assert GetPartialMorphemes()
directly.

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
@johnml1135
johnml1135 force-pushed the fix-551-partial-entry-prune branch from f1ac77e to b507a89 Compare October 27, 2026 17:57
@johnml1135

Copy link
Copy Markdown
Collaborator Author

I think that you can remove Morpher.IsPartial in the code and in the comments, since it is superceded by Morpher.HasPartialRules.

Done in b507a89: I removed Morpher.IsPartial, its assignment and the stale comment in Morpher.cs. Nothing read it after the analysis-side checks moved to HasPartialRules. The three test preconditions that used it now assert GetPartialMorphemes() directly. csharpier passes, and AffixTemplateTests and PruneDisagreeingCopiesTests pass (46/46). The branch is also rebased onto master.

Comment thread src/SIL.Machine.Morphology.HermitCrab/Morpher.cs
Comment thread tests/SIL.Machine.Morphology.HermitCrab.Tests/AffixTemplateTests.cs
@claude

This comment has been minimized.

The removed reasoning claimed a substituted family relative is always
non-partial, which blocking can contradict. The contract alone is
accurate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/// <summary>
/// A Morpher is partial if any of the elements are partial.
/// True if any morpheme other than a <see cref="LexEntry"/> is partial. Analysis may prune a final
/// template that follows a non-template rule only when this is false.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI: F7. The summary says analysis prunes "only when this is false", but it also prunes when this is true and AlwaysEnforceFinalTemplates is set (AnalysisAffixTemplateRule.cs:47, AnalysisAffixProcessRule.cs:83). Consider "only when this is false or AlwaysEnforceFinalTemplates is set".

@claude

claude Bot commented Oct 29, 2026

Copy link
Copy Markdown
Contributor
  1. Verdict: approve. The only change since b507a89 is the doc-only trim of the HasPartialRules summary in 69edb79, and it fixes F3.
  2. Most important: F7, src/SIL.Machine.Morphology.HermitCrab/Morpher.cs:102. The trimmed summary leaves out the AlwaysEnforceFinalTemplates override. This is a documentation nit, not a behavior defect.
  3. Counts: 0 Critical, 0 Important, 1 Low new. No findings are open apart from F7.
  4. Ran: /code-review high 553 over the whole PR. Every result either repeated an earlier thread (F4, F6) or was on code unchanged since b507a89 and not Critical. gh pr checks 553: Build on ubuntu-22.04 and windows-latest and claude-review were pending, and Comment hygiene passed (advisory).
  5. Not verified: that the CheckBlocking path from F3 loses no parse. No test covers a partial root inside a LexFamily.

Public API: Morpher.IsPartial removed, Morpher.HasPartialRules added. Grammars whose only partial morphemes are lexical entries now get the final-template prune during analysis. Target framework, package, and sillsdev/machine.py parity: none verified.

Findings: F1 addressed, F2 addressed, F3 addressed, F4 withdrawn (FieldWorks does not read IsPartial, per jtmaxwell3), F5 accepted (deferred to #507), F6 accepted (deferred to #507), F7 new.

Reviewed at 69edb79

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HermitCrab: partial lexical entries switch off final-template pruning for the whole grammar

3 participants