Skip to content

Migrate main AE forest table from reactable+plotly to lt - #169

Open
yihui wants to merge 43 commits into
mainfrom
migrate-main-table-to-lt
Open

yihui wants to merge 43 commits into
mainfrom
migrate-main-table-to-lt

Conversation

@yihui

@yihui yihui commented Oct 4, 2026 •

Copy link
Copy Markdown
Collaborator

Follow-up to #168 (which moved the drill-down DETAIL table to lt). This migrates the main interactive AE forest table — and its two inline figures — from reactable + vendored Plotly to the lt package.

What changed

The main table, the per-arm proportion dot plot, and the risk-difference error bar are now a single lt table made interactive with lt_interactive(), replacing:

  • the reactable core table (reactable2() wrapper),
  • the two Plotly inline figures (sparkline_point_js() + vendored plotly-min.js),
  • the crosstalk incidence slider and parameter dropdown.

R

  • R/lt_table.R (new) — format_lt_forestly() builds the structural lt table: per-arm n/(%) columns under treatment-group spanners, lt_dotplot() proportion figure (one staggered dot per arm), lt_errorbar() risk-difference figure (point + 95% CI, zero reference line, favor-direction axis), column widths, hidden helper columns (parameter, hide_prop, hide_n, CI bounds), and the show/hide menu spec. Interaction-free so it can be tested in isolation. The two inline figures draw into dedicated lt columns via into=, so no duplicate value arrays travel in the JSON.
  • R/format_ae_forestly.R — drops the reactable/plotly column specs; exposes figure metadata (ranges, colors, headers, per-arm groupings, widths) for lt instead.
  • R/ae_forestly.R — layers lt_interactive() + native row-detail drill-down + the forestly controls on the structural table. Keeps the public width argument (now sizing lt's container; see Table sizing) and the historically misspelled dowload_button.
  • R/html_dependency.R — drops the plotly/crosstalk/search-filter dependencies; adds the forestly-widgets (table CSS) and forestly-ae-drilldown (listing CSS) stylesheet dependencies. No JavaScript.
  • R/utils.R — carries %||%, get_label, assign_label (previously in reactable2.R).
  • Deletes R/reactable2.R, R/search_filter.R, R/sparkline_point_js.R and the inst/js plotly/crosstalk/sparkline assets.

Controls

  • The AE-criteria dropdown, incidence slider, and CSV download are all lt's own. The dropdown and slider are lt typed filters (lt_interactive(filter = ...)) bound to the hidden helper columns (parameter, hide_prop/hide_n), which lt renders and wires automatically; the download is lt's built-in current-view CSV export. forestly ships no widget JavaScript of its own — only CSS.

Table sizing

  • The forestly-ae container is sized in inst/css/forestly-widgets.css: the table takes its natural width, is centered, and is capped just under the viewport (100vw minus a fixed slack for the scrollbar/gutter), so it breaks out of a narrow parent yet scrolls inside its own container instead of forcing a page-level horizontal scrollbar.
  • ae_forestly(width=) is retained: a number (pixels) or any CSS value fixes the container width via lt_wrap(style=), while NULL (default) keeps the natural width. Either way the viewport cap still holds, so a wider table scrolls inside itself rather than widening the page.

Drill-down detail

  • The embedded listing store is deduplicated across parameters and keyed client-side by AE term; in SOC-display mode (components = "soc") it also ships and keys a per-record SOC so a SOC row expands. The forestly-ae class is placed on lt's own container via lt_wrap().

Dependencies

  • Drop crosstalk, reactable, reactR; require lt (>= 0.4.43) — for the inline figures (lt_dotplot/lt_errorbar), the typed filters, the dedicated plot columns (into=), and lt_wrap() (container class + user width).

Tests & docs

  • Rewrite test-ae_forestly.R and test-format_ae_forestly.R for the lt output; add test-lt_table.R; delete test-sparkline_point_js.R and its snapshot.
  • Refresh design-pattern.Rmd and CLAUDE.md; add a NEWS entry (0.1.7).

🤖 Generated with Claude Code

…-up)

The main interactive AE forest table, its inline proportion dot plot, and its
risk-difference error bar are now a single `lt` table (yihui/lt) made
interactive with `lt_interactive()`, replacing the reactable core, the vendored
Plotly figures, and the crosstalk slider/dropdown. The drill-down listings keep
rendering lazily on expand (as in #168).

R:
- R/lt_table.R: new format_lt_forestly() builds the structural lt table
  (per-arm n/(%) columns under treatment-group spanners, lt_dotplot proportion
  figure, lt_errorbar risk-difference figure, column widths, hidden helper
  columns parameter/hide_prop/hide_n/CI bounds, and the show/hide menu spec).
- R/format_ae_forestly.R: drop the reactable/plotly column specs; expose figure
  metadata (ranges, colors, headers, per-arm groupings, widths) for lt instead.
- R/ae_forestly.R: layer lt_interactive() + native row-detail drill-down + the
  forestly controls on the structural table. Argument names unchanged
  (including the historically misspelled `dowload_button`).
- R/html_dependency.R: drop plotly/crosstalk/search-filter deps; add the
  forestly-widgets JS+CSS dependency.
- R/utils.R: carry %||%, get_label, assign_label (previously in reactable2.R).
- Delete R/reactable2.R, R/search_filter.R, R/sparkline_point_js.R and the
  inst/js plotly/crosstalk/sparkline assets.

Controls (inst/js/forestly-widgets.js, inst/css/forestly-widgets.css):
- Dependency-free parameter dropdown, dual-thumb incidence slider, and CSV
  download that drive the table through lt's el._lt.filter() controller,
  filtering on the hidden helper columns.

Deps: drop crosstalk, reactable, reactR; require lt (>= 0.4.21).

Tests/docs: rewrite test-ae_forestly.R and test-format_ae_forestly.R for the lt
output; add test-lt_table.R; delete test-sparkline_point_js.R and its snapshot;
refresh design-pattern.Rmd and CLAUDE.md; add NEWS entry.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@yihui
yihui marked this pull request as draft October 4, 2026 23:51
yihui and others added 11 commits October 4, 2026 20:38
… axes

- Draw the per-arm proportion dots on a single track instead of staggering them
  onto separate rows; the forest-plot vignettes don't jitter dots vertically.
- Keep each AE table cell on one line, truncating overflow with an ellipsis
  rather than wrapping; hovering a cell reveals its full text. Figure cells (an
  SVG that must not be clipped) and the drill-down detail are exempt.
- Fix the proportion dot plot and risk-difference error-bar axes being drawn one
  column to the right of their plots when the SOC column is offered as a
  hide/show toggle (display_soc_toggle = TRUE). The axes live in the table's
  plot-footer row, and lt's column-visibility menu did not hide a toggled-off
  column there, leaving a stray cell that shifted the axes. Fixed in lt and now
  requires lt (>= 0.4.22).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace forestly's hand-built parameter dropdown and incidence slider with
lt's declarative typed filters, declared through lt_interactive(filter = ...)
in ae_forestly() and bound to the table's hidden helper columns (parameter,
hide_prop/hide_n). lt renders them automatically (a funnel under a visible
column's header, a chip in the control bar for a hidden column), so the
corresponding bespoke HTML controls and their CSS/JS drop out. The CSV
download remains the only forestly-owned widget, reading the table's current
view via lt's el._lt contract.

Bump the lt requirement to 0.4.23 (the typed-filter API) and refresh NEWS and
CLAUDE.md to describe the new filter wiring.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lt now provides a CSV download button directly through
lt_interactive(download = ...), so ae_forestly() no longer needs its own
download control. Wire dowload_button to lt's download (named "ae-forest.csv"),
and remove the bespoke button div, its JavaScript (inst/js/forestly-widgets.js),
and its CSS. forestly now ships no widget JavaScript of its own; the
forestly-widgets dependency carries only the table's cell-truncation styles.

Bump the lt requirement to 0.4.28 (where download landed).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reorder the main table so the proportion dot plot and risk-difference
error bar come right after the term/SOC columns, ahead of the per-arm
n/(%) and numeric difference columns, matching the forest-plot
convention of showing the figures first.

Stagger the dot plot's per-arm dots onto separate tracks (as the
original forest plot did) so near-equal proportions stay distinguishable,
and drop the error bar's color/legend when only a single comparison is
drawn, since the 'vs. <ref>' spanner already names it.

Extend the one-line/ellipsis cell styling to header cells too, so a long
group label over a narrow n/(%) column no longer wraps into a tall
header row, while figure (svg) cells stay exempt.
When a filtered dataset yields no computable risk differences (or
proportions) -- e.g. a comparison arm with no analysis subjects -- the
sliced matrix passed to range() is all-NA. With na.rm = TRUE, range()
then warns 'no non-missing arguments to min/max; returning Inf/-Inf' and
returns a degenerate c(Inf, -Inf) range that breaks the dot-plot and
error-bar axis scaling.

Replace the four range() calls with a safe_range() helper that drops
non-finite values and falls back to c(0, 0) when none remain, so the
figures still render without warnings.
apply(..., max, na.rm = TRUE) over a row with no non-missing values
warns and returns -Inf. Use a safe_max() helper that returns 0 for such
a row, so the hidden helper columns the slider filters on stay finite.
safe_max() and safe_range() differed only by the aggregator and the
all-NA fallback, so collapse them into agg_finite(x, f, empty). Also
hoist the per-arm prop/n slices (prop_grp/n_grp) that were sliced in
several places.
Consolidate the reactable/Plotly-to-lt migration into a single entry
(the main table and drill-down listings are no longer separate items),
drop the reactable-internal optimizations the migration itself removed,
require lt (>= 0.5), and move the dowload_button spelling note out to an
issue (#173).
Close #173.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment thread R/ae_forestly.R Outdated
Comment thread R/ae_forestly.R Outdated
A checklist in the control bar (lt's LT.ui.chip + popover + checklist, in the
el._lt.chips group) chooses which treatment arm(s) show in each expanded AE
listing; the detail callback filters its records by the checked set and
el._lt.resetDetail re-renders open details. The picker offers the arms present
in the listing (not aggregate forest columns). Requires lt (>= 0.4.37).

Close #174
Close #175

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@yihui
yihui force-pushed the migrate-main-table-to-lt branch from 3f01567 to 23ce6a4 Compare October 8, 2026 05:32
yihui and others added 11 commits October 8, 2026 13:41
Set xfun.tojson.dict = 0.5 and xfun.tojson.pretty = FALSE once near the
top of ae_forestly() (scoped, restored on exit) so both JSON payloads the
widget embeds -- the drill-down listing store and the main table lt
serializes in format() -- are dictionary-encoded and whitespace-free.
Repetitive columns (repeated labels, formatted numbers with few levels,
the per-row parameter string, the duplicated figure-input columns) now
ship as compact codes plus uniques, sharply shrinking a large forest
table's HTML. The listing store no longer needs the dict/pretty args
inline. Requires the xfun that defaults these args from the options.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ae_forestly() now sets xfun.tojson.pretty = FALSE, so the serialized lt
spec and the drill-down group list ship without the pretty-print spaces
these assertions matched. Match the compact form ("type":"select",
["Placebo","Low Dose","High Dose"], etc.).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The drill-down treatment-group picker built its popover with
`LT.ui.popover(doc, label, build)` and then wrapped the returned funnel
in `LT.ui.chip()`. Without lt's `host` argument the popover's click
target is only the bare funnel button, so clicking the chip's label text
("Treatment group") did nothing -- the popover never opened.

Build the chip first and pass `chip.el` as the popover's `host`, then
drop the funnel wrap into the chip, matching how lt's own typed-filter
chips wire themselves. The whole chip is now the click target.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The drill-down listing shipped, per main-table row, a precomputed array
of record indices (run- or delta-encoded). On a large table that index
dwarfed everything else -- ~72MB of a 139MB dummy output -- and relied
on the records being contiguous in the store to compress at all.

Ship no per-row pointers. Add two dict-encoded key columns (`param`,
`soc`) to the shared record store and have the detail callback build a
`parameter + "\r" + upper(term or SOC) -> record indices` map once on
first expand, then resolve each row by its own `parameter`/`name`. This
removes the whole index (down to a few MB of keys), drops the
map/run/delta encoding in R, and no longer depends on store order.

Also drop a redundant as.factor() on the key columns: xfun::tojson()
dict-encodes any atomic vector and coerces factors to character anyway.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The slider range was computed in R: the max rounded up to the nearest 10 and
forced to at least 100 for the proportion filter, with a hardcoded step of 1.
For percentages well under 100 (the usual case) this stretched the track to
0-100 and, with step 1, snapped decimal incidences to the endpoints -- dragging
the slider past 1 emptied the table.

Drop the R-side range logic and pass only `type = "range"` plus the label; lt
now derives nice, data-bracketing min/max and a sensible step from the bound
column itself. A user-supplied `filter_range` still overrides min/max.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ae_forestly() wrapped the lt table in a fixed-width div that also carried the
forestly-ae class (for cell styles) and an overflow-x scroll container. All
three are lt's own job now: lt_width() sizes the table, lt_class() puts the
forestly-ae class on lt's container, and lt's `.lt-wrap` already scrolls
horizontally. The wrapper div is dropped; the table is emitted straight into
the tagList. Requires lt (>= 0.4.40) for lt_class().

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ae_forestly(width=) hard-set the table container's width in pixels (default
1400). Replace it with a CSS rule on the `forestly-ae` container: the table
takes its natural width, is centered, and is capped just under the viewport
(100vw minus a fixed slack for the scrollbar/gutter) so a wider table breaks
out of a narrow parent yet still scrolls inside its own container instead of
adding a page-level horizontal scrollbar.

With this the table fits any page on its own, so an explicit width is rarely
needed; the argument is removed rather than kept as a seldom-used override. It
was added this development cycle and never shipped to CRAN, so no deprecation.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The one-line-per-cell rule (clip with ellipsis, reveal full text on hover)
was also matching the lt control-bar row, so the filter chips and their
popovers were clipped and only appeared while the row was hovered. Scope the
rule to skip the control-bar row (tr.lti-head) so the chips stay visible.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Remove the sort of the AE listing (toupper + order + reindex over ~950k
rows). The client-side detail callback builds its own
param+term/SOC -> record-index map, so records need no particular order
and carry no per-row index; the sort was leftover from an earlier
contiguous-run index and cost ~16s on large inputs for no effect.

Build the embedded drill-down record store lazily. The payload is
dict-encoded JS (not plain JSON), so it can't ride as a deferred
<script type="application/json">. Wrapping it in a getStore() closure
lets V8 only pre-parse it at load; the tens-of-MB of array literals
compile and allocate on first expand instead of when the spec's IIFE
runs, keeping them off the initial-paint path. The control-bar chip
still registers eagerly on mount.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The hide_prop/hide_n helper columns were built with a per-row apply()
calling agg_finite() once per AE term -- a hotspot on large tables.
Replace with a column-wise pmax() (non-finite cells -> -Inf so they are
skipped; an all-non-finite row resets to the empty value), which is
numerically identical and avoids the per-row R call overhead.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
yihui and others added 7 commits October 9, 2026 09:56
* Dedupe drill-down store across parameters (~36% smaller HTML)

The drill-down store embedded the stacked per-parameter AE listing verbatim.
Because the parameters overlap -- an AE meeting several criteria (e.g. "any",
"serious" and "drug-related") is one physical record repeated once per matching
parameter -- the listing is ~3x its distinct rows (953k vs 300k on an 8-param,
300k-event example). Every column was serialized at the full stacked length,
so both the xfun::tojson dictionaries and the per-row code arrays carried the
3x overhead.

Store each distinct record once (`records`, plus its match-key `soc`) and add
`members`, mapping each parameter to the indices of the records in it. The
client builds the same `parameter + "\r" + upper(term or SOC) -> record indices`
lookup, now iterating `members` and keying each record by its own event and SOC,
so a term row and a SOC row still both resolve. The listing rendered for every
(parameter, term/SOC) row is unchanged.

On the 300k-event example the store drops 46.8 MB -> 20.9 MB and the whole HTML
71.0 MB -> 45.1 MB. The store is already built lazily (#190), so this is purely
a file-size / parse win, not a change to first paint.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* Address review: drop as.factor, use match(), trim comment

- `ref`: `match(rec_key, rec_key[u_first])` instead of `as.integer(factor(...))`
  -- same first-appearance 0-based index without building a factor.
- Drop `as.factor()` on the record columns and `soc`: xfun::tojson(dict=) now
  dictionary-encodes plain character vectors identically, so the coercion is
  no longer needed.
- Consolidate the 3-paragraph store comment into one.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
`format_lt_forestly()` fed `lt_dotplot()` / `lt_errorbar()` private copies of
every value and CI column (`prop_fig_1..ng`, `diff_fig_1..nd`, `diff_lo_*`,
`diff_up_*`) so the originals could stay visible as numbers -- the plots "use
up" (hide) the columns they draw. But a plot draws its SVG only into the first
value column's cell and reads the rest by reference, hidden or not. So each
figure needs just one private anchor copy to draw into; the other proportions
and comparison diffs read from the visible numeric columns (`hide = FALSE`) and
the error bar's CI bounds from the already-shipped hidden `lower_*`/`upper_*`.

This drops `prop_fig_2..ng`, `diff_lo_*` and `diff_up_*` from the payload
entirely. On the 595,918-row profiling table the serialized main table shrinks
24.2 MB -> 20.7 MB (-14.7%); the figures are identical (same values feed the
same dots and error bars). Saves (ng - 1) + 2*nd columns.

Column names are passed to lt as plain character vectors (the error-bar
value/lower/upper triple as a length-3 vector), not as.formula().

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
lt 0.4.42 adds into= to lt_dotplot()/lt_errorbar(): the plot draws into a
runtime-synthesized column instead of clobbering the first value column,
so callers no longer need a throwaway anchor copy. Drop the prop_fig_1 /
diff_fig_1 copy columns (and the vestigial prop_fig/diff_fig NA
placeholders) and point each figure at its own into= column, positioned
after SOC with lt_move().

Two value arrays no longer travel in the JSON payload (lt ships only the
new column name; the runtime materializes it). Data columns 19 -> 17.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A main-table row's name is an AE term by default but a SOC when
components="soc". The detail lookup keys records by term and, in SOC
mode, also by SOC so a SOC row resolves. Ship the per-record soc array
and the SOC key only then; the common term-only table omits both to
keep the payload small.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lt renamed lt_class() to the more general lt_wrap(); tag the table
container the same way. Require lt (>= 0.4.43).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ae_forestly(width=) fixes the table container width (a number is pixels,
or any CSS value) via lt_wrap(); NULL (default) keeps the natural width.
Either way the width is capped at the viewport, so a wide table scrolls
inside the table rather than adding a whole-page scrollbar. Restores the
public width argument dropped in the reactable->lt migration.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
utils.R was created in #167 (2026); an earlier commit on this
branch wrongly reset it to 2023. Also remove a fabricated #190
reference from a test comment.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@yihui
yihui marked this pull request as ready for review October 10, 2026 04:40
@yihui
yihui requested a review from LittleBeannie October 10, 2026 04:40
Comment thread R/lt_table.R
Comment thread R/lt_table.R Outdated
@LittleBeannie

Copy link
Copy Markdown
Collaborator

This is a very substantial commit covering a lot of great updates! I have a few comments and suggestions:

Comment 1: When users hover over the eye icon, the tooltip currently says "Columns". Can we relabel it to "Select columns to be displayed in the AE forest plot"?
image

Comment 2: Should we disable the advanced search for "AE Criteria" and restrict it to a single-choice dropdown list?
image

Comment 3: For the "Treatment group" filter, can we relabel it to "Select arms to include in the drill-down AE table"?
image

Comment 4: Can we add a tooltip when the user hovers over "AE proportion (%)" and "Risk difference (%) + 95% CI vs. Placebo"?
image

Comment 5: Can we add a search bar for "SOC Name", "n", and "(%)"?
image

yihui and others added 2 commits October 12, 2026 10:11
Pass an unnamed `TRUE` default to lt_interactive(filter=) so every
column gets a filter box, then specialize: AE Criteria stays a
single-select dropdown, SOC Name a dropdown, and the numeric n/(%)/
difference columns get range sliders (range_cols from
format_lt_forestly()).

Relabel the drill-down treatment-group picker to "Arms in drill-down
tables", which says what it scopes.

Requires lt (>= 0.4.44) for the empty-column filter skip.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@yihui

yihui commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator Author

Thanks for the review! Addressed each:

1. Eye-icon tooltip — fixed in lt (yihui/lt@5164a9a): the hover text is now "Show or hide columns".

2. AE Criteria expression box — left as-is. Suppressing it just for this one dropdown would mean a special case for a single filter, which didn't seem worth the added complexity. Let me know if you'd rather I disable it. I think most users (statisticians) know that multiple choices don't make sense here, right?

3. Treatment-group filter label — relabeled to "Arms in drill-down tables".

4. Plot tooltips — the dot plot and error bars already carry hover tooltips, so no change needed. Let me know if you can't see tooltips when hovering over dots or bars.

image

5. Filters on all columns — done. Every column now has a filter: dropdowns for AE Criteria and SOC Name, range sliders for the numeric columns.

The clip-on-hover rule (one line per cell, full text on hover) exempted only
the control-bar row (.lti-head), not the per-column filter row (.lti-filters).
A funnel popover under a column header lives in that row, so it got
overflow:hidden while unhovered and vanished when the mouse left the cell.
Exempt .lti-filters too, as .lti-head already is.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@yihui yihui linked an issue Oct 12, 2026 that may be closed by this pull request
R color names like "grey60"/"grey40" and palette indices are not valid
CSS colors, so the browser-rendered dot plot / error bar silently fell
back to black. Convert the `color` vector via col2rgb()/rgb() before it
reaches lt, so any R color spec renders correctly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@LittleBeannie

LittleBeannie commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

2. AE Criteria expression box — Please disable the following red search bar, as the reviewers (not only statistcians) can be very creative and they may want multiple choice and ended with wrong results!
image

4. Plot tooltips - This is invisable from my end... I am using the example in ./vignettes/customize-color.Rmd
5. Filters on all columns - As shown in the following screenshot, I can do more than the controls provided by the slide bar. I am wondering if we should remove the slide bar for simplicity?
image

One more quick comment: When I drill down to the AE listing table, the table width becames immediately larger than the one without drill down. Can we keep the same width for an easy review?

yihui and others added 2 commits October 13, 2026 12:11
The AE-criteria dropdown is single-choice: a custom filter expression
would never apply, so pass `box = FALSE` (lt >= 0.4.45) to render the
dropdown without the expression box.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Expanding a row no longer forces the entire embedded listing to resolve.
Two changes to the detail callback and its payload:

- Per-parameter key map: the term/SOC -> record-index map is now built
  for only the expanded row's parameter (its `members[param]` slice) on
  that parameter's first expand, instead of scanning every record of
  every parameter up front.

- Lazy column decode: each record column ships as data -- dictionary
  `{u, c}` when repetitive, else plain `{p}` -- instead of xfun's
  `[codes].map(i => u[i])` form that rebuilds the full string column on
  load. `colGet` decodes only the rows of the detail actually opened.

On a 300k-AE benchmark this cuts the one-time resolve on first expand
(string materialization + full map scan) from ~630ms to ~46ms, with no
change in payload size.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@yihui

yihui commented Oct 13, 2026 •

Copy link
Copy Markdown
Collaborator Author

Thanks! All three are fixed:

  • The AE criteria filter is now a simple dropdown — the free-text box is gone.

    image
  • If you type a custom filter the slider can't represent, the slider now hides itself so it never shows a mismatched value.

    image
  • Expanding or collapsing a drill-down table no longer changes the main table's width dramatically. It may still jump around a little bit (if a certain drill-down table to be expanded is wider than others), but I don't think it will bother users much.

yihui and others added 3 commits October 13, 2026 17:59
The embedded listing store carried its bulk integers as JSON number
arrays. Two changes make it smaller and faster to parse:

- Dictionary codes (and the per-record data they index) now travel as a
  base64 little-endian typed array `{b, w, n}` -- ~1-2 bytes per value
  instead of digits+commas -- and the client turns each into a TypedArray
  (far faster than parsing a big number array). The store is parsed with
  JSON.parse rather than evaluated as an object literal.

- Per-parameter record membership is one bitmask per distinct record
  (`pmask`, bit i = parameter i), replacing the per-parameter index lists
  that repeated a record's index once per matching criterion. The client
  scans the masks for a parameter's bit on first expand.

On a 300k-AE benchmark: HTML 38->27 MB, load 6.5->4.7 s, and the first
page's detail build (parse + sizing) 885->232 ms.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Scope the clip-on-hover rule to body data cells only (header, footer and
drill-down detail cells now wrap normally), and add a background highlight on
the hovered data row. Collapse the stylesheet into one nested block.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fold the drill-down listing CSS into forestly-widgets.css and drop the
separate html_dependency_ae_drilldown() so the main table and its
drill-down ship as a single stylesheet dependency. Move the surviving
html_dependency_forestly_widgets() alongside its sole caller in
ae_forestly.R and delete R/html_dependency.R.

Rescope the drill-down rules under .forestly-ae .lti-detail (the old
.forestly-ae-drilldown class is gone), share bold headers and zebra
rows with the main table, and shrink the detail type.

Build the on-demand drill-down store with xfun::tojson(dict = 0)
directly instead of toggling the global option.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@yihui
yihui force-pushed the migrate-main-table-to-lt branch from eeedf66 to 2ac9b85 Compare October 14, 2026 04:48
yihui and others added 3 commits October 14, 2026 00:58
Remove tests that grep generated JS source for internal identifiers
(getStore, s.records, LT.ui.*, tbl._lt.resetDetail), assert lt spec
serialization fields ("type":"select", "wrap":{"class":...},
"download", width px), inspect htmlDependency wiring (dep names,
lt-interactive.js in scripts, stylesheet paths), or check for the
absence of long-deleted widgets. These break on harmless refactors
without catching real regressions.

What remains tests observable behavior: the table renders without
error, public options surface their documented effect in the output
(filter headers, dropped incidence slider), and the drill-down picker
offers the listing's own arms rather than the aggregate "Total".

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
forestly now owns the {u, c} shape of each dict-encoded store column and
uses xfun only to base64-pack the codes (pack_vec) and decode them
client-side (unpack, via xfun::json_unpack); it no longer injects
xfun::json_reviver or hardcodes xfun's internal dict keys. pack_vec
passes dict = 0 so a packed column is always a bare width-tagged object
unpack() can read (a dict-encoded mask would otherwise break it).

Also ship one row per distinct record (u_row) in the store columns, so
they line up with pmask and the per-record indices the client looks up.

Requires xfun (>= 0.61.7).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
xfun dropped the list(mode, threshold) form of pack (now numeric-only)
and moved the json/js switch to a separate mode argument whose default
flipped to "js". Set xfun.tojson.pack = 100 (threshold) and
xfun.tojson.mode = "js" via options for the main table, and pass
mode = "json" explicitly on the two calls that must stay valid JSON
(the base64 packer and the JSON.parse'd store). Also drop the now
redundant pretty = FALSE on store_json since the global option covers it.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Correct the misspelled dowload_button argument in ae_forestly()? Allow numerical filtering for percentages

2 participants