Skip to content

Fix line wrapping, spacing, and bidi ordering in shaping - #292

Open
egonelbre wants to merge 18 commits into
go-text:mainfrom
egonelbre:fix/shaping
Open

egonelbre wants to merge 18 commits into
go-text:mainfrom
egonelbre:fix/shaping

Conversation

@egonelbre

@egonelbre egonelbre commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Sixteen commits, one fix each. Grouped by area.

Bidi ordering:

  • L2 reversal swapped the visual indices of line[i] and line[end-1-i]. That only works while the indices are still in logical order. Once a higher level has reversed an inner sequence the swap scrambles them. Levels {0,2,2,1,1,1,0}, an LTR paragraph with an RTL span holding two LTR runs, came out with the LTR runs swapped and the RTL runs shuffled. The two "bidi example" test cases with nested levels expected that output and now expect what UAX #9 specifies.
  • L1 was never applied. A space at the end of an RTL run in an LTR paragraph was drawn at the visual start of the run, in the middle of the line. The wrapper now cuts trailing whitespace off an opposing run into its own Output at the paragraph level, and measures the run without it when deciding whether the word fits. No-break spaces stay inside the run.
  • bidi closed a run only when the direction changed, so level 0 text followed by level 2 text inside an LRE shared one run and one reported level. A run now ends on any level change. Callers see more runs, and kerning no longer applies across an LRE or LRI boundary. Pango and ICU itemize the same way.

Line wrapping:

  • A run set that does not cover the whole paragraph panicked on a nil mapping. The last run now ends the text.
  • The single-run fast path skipped trailing whitespace trimming that every other path did.
  • cutRun and post-processing mutated the caller's glyph slice, so wrapping the same runs twice used stale advances. The slice is copied only when it changes.
  • Vertical advances are negative and were summed as such, so vertical text never overflowed a line. Widths are now measured as magnitudes.
  • Whitespace detection used zero glyph width, which also matched every glyph shaped without extents. The source rune now decides when extents were skipped.
  • Segmenter passed the caller's Face through to every run instead of selecting one, when the input had a face set.

Spacing:

  • Letter spacing treated glyph index 0 as the logical start of the run. In RTL runs it is the logical end, so the boundary trim hit the wrong glyph.
  • Vertical text shrank under positive word and letter spacing because vertical advances are negative.

Shaper:

  • The harfbuzz font cache was keyed on the Font, so two Faces with different variation coordinates shared one shaped result. It is keyed on the Face now.
  • The cache evicted one entry per insert. After SetFontCacheSize shrank it, it stayed over the limit. A negative size no longer panics.
  • Size was ceiled to whole pixels before scaling, so 12.5 and 13 shaped identically and Output.Size disagreed with the metrics. Tests that relied on the ceiling pass integer sizes now.
  • Runes and cluster counts used the unclamped run range while the glyphs used the clamped one.
  • Two unused cluster mapping implementations are gone. Only tests referenced them.

@egonelbre

Copy link
Copy Markdown
Contributor Author

PS: Here I'm less confident in my own review of the code as I do not have the necessary in-depth knowledge.

@benoitkugler

Copy link
Copy Markdown
Contributor

Hi,
Could you extract commits 2d8451e and 995f73d in a separate PR ? These are high priority bug fixes I would like to merge as soon as possible.

@egonelbre

Copy link
Copy Markdown
Contributor Author

@benoitkugler done #296

@whereswaldon whereswaldon left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been able to review through 563ce7f so far. I'm out of time for today, so I'll circle back when I can.

Comment thread shaping/shaping.go Resolved
Comment thread shaping/lru.go Resolved

@whereswaldon whereswaldon left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a nice set of improvements, thank you for submitting them. It took a long time to go through it all, and I have a small number of remaining questions and requests.

Comment thread shaping/input_test.go
}
}

func TestSplitByFaceIgnoresInputFace(t *testing.T) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it desirable to ignore the input face?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clarified in code documentation.

Comment thread shaping/wrapping.go Outdated
Comment on lines +899 to +901
for a, b := 0, len(run.Glyphs)-1; a < b; a, b = a+1, b-1 {
run.Glyphs[a], run.Glyphs[b] = run.Glyphs[b], run.Glyphs[a]
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not slices.Reverse here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code initially was based on Go 1.19 which didn't have slices.Reverse. Updated it to use slices.Reverse.

Comment thread shaping/wrapping.go
Comment on lines +891 to +893
copy(split, line[:i])
split[i] = prefix
copy(split[i+2:], line[i+1:])

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On an initial read, I couldn't figure out why we never wrote to split[i+1]. After taking a step back, I realized that we modify i at the end of this block, and then the loop-end line[i] = run is writing into that index in the slice. This is sufficiently subtle and non-local to warrant a comment, in my opinion.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added a comment

Comment thread shaping/wrapping_test.go Outdated
Comment on lines +3839 to +3844
case run.Runes.Offset <= 4 && run.Runes.Offset+run.Runes.Count > 4:
before = run.VisualIndex
case run.Runes.Offset <= 8 && run.Runes.Offset+run.Runes.Count > 8:
digits = run.VisualIndex
case run.Runes.Offset <= 12 && run.Runes.Offset+run.Runes.Count > 12:
after = run.VisualIndex

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using these inequality ranges feels unnecessarily complex here. The text should segment at multiples of four runes in this sample text, and the runs should be stored in logical order within the line. Why not just say:

before = lines[0].Runs[1].VisualIndex
digits = lines[0].Runs[2].VisualIndex
after = lines[0].Runs[3].VisualIndex

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Only tests referenced mapRunesToClusterIndices v1/v2 and
mapRuneToClusterIndex. Rename v3 to mapRunesToClusterIndices.
A Face carries the variation coordinates. Two Faces on one Font with
different coords shaped with the same cached harfbuzz.Font.
Size is already in 26.6 units, the same scaleShift the shaper removes
from the harfbuzz positions. Sizes 12.5 and 13 now shape differently,
and Output.Size matches the metrics. Tests using Size 72 and 36 relied
on the ceiling to 2 and 1 px. They now pass those sizes directly.
Glyphs are in visual order, so with a TowardTopLeft progression index 0
is the logical end of the run. AddLetterSpacing and
trimStartLetterSpacing assumed index 0 was the start. They now read the
run progression to find the logical start and end.
cutRun and postProcessLine modified the caller Output.Glyphs in place,
so wrapping the same runs again produced stale advances. Copy the glyph
slice only when it changes.
WrapParagraph skipped trailing whitespace trimming for a single run that
fits. Prepare and WrapNextLine trimmed it.
processBreakOption discarded the Peek ok flag and indexed a nil mapping.
The last available run now ends the text, checked against the usual
width and truncation rules.
UAX go-text#9 L1 resets whitespace at the end of each line to the paragraph
embedding level, so it sits at the line end whatever direction the run
containing it has. The wrapper left such whitespace inside the run, so
a space at the end of an RTL run in an LTR paragraph was drawn at the
visual start of the run, in the middle of the line.

postProcessLine now applies L1 at each wrapped line boundary
(resetTrailingBidiWhitespace). Trailing whitespace inside a run that
opposes the paragraph direction is cut off into its own Output, which
takes the paragraph direction and base level, with its glyphs reversed
into a copy. A wrapped Line can therefore hold more Outputs than the
caller fed in. Clusters that mix whitespace with visible characters
stay whole.

advanceSpaceAware measures an opposing run without its trailing L1
whitespace, so the space that L1 moves to the line end no longer
decides whether the word before it fits. Only spaces L1 covers count;
a no-break space stays inside the run and keeps its width.

A run with Level 0 and a direction opposing the paragraph was built by
hand rather than by Segmenter, and is left alone.
@whereswaldon

Copy link
Copy Markdown
Member

@egonelbre In the future, please don't force-push or re-push the work already reviewed when you address the review comments. It means I have to re-review all of that code because I can't know that it hasn't changed in-place from the version I already reviewed. I know this was likely a tool doing it for you, but please see if you can prevent it in the future.

@whereswaldon whereswaldon left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, these look good to me know. @benoitkugler @andydotxyz , could we get a second review?

@egonelbre

egonelbre commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor Author

@whereswaldon of course, no problem. Also, in this case it was manual.

Different projects have different ways of working. I'm mostly used to Go (the project) approach, which prefers linear history and reviewing one commit at a time. It does use Gerrit, which makes that workflow significantly easier. Unfortunately, Github isn't so kind to linear history and doesn't provide out of the box reviewing and tracing through per-commit history. Also, personally I prefer linear history such that every single commit stays correct and is buildable.

Either way, I'm not trying to convince you, I just don't know what approach a project uses unless it's communicated somewhere upfront. I guess based on how the git history roughly looks and will get it wrong sometimes.

PS: there is git range-diff origin/main...b27cba7 origin/main...2b364f0 that allows diffing ranges.

@andydotxyz

Copy link
Copy Markdown
Contributor

Thank you, these look good to me know. @benoitkugler @andydotxyz , could we get a second review?

Sorry I've been out of action for a while and trying to catch up - but big PRs will take a while to get to, if Benoit can it will be faster :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants