Skip to content

feat(precompute): update each open pane's summary state per sample - #830

Draft
zzylol wants to merge 1 commit into
feat/concurrent-enginefrom
feat/per-sample-ingestion
Draft

zzylol wants to merge 1 commit into
feat/concurrent-enginefrom
feat/per-sample-ingestion

Conversation

@zzylol

@zzylol zzylol commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

Stack 3/4. Base: #829.

Why

Pane builds at ingestion buffered every sample, then turned each into a row of Values with its labels and JSON series identity, grouped rows by byte-vector keys and built states. That cost 10.5 µs per sample, against ASAPQuery's 2.3 µs, which updates each pane's accumulator in place per sample. User decision: ingestion updates summary state directly per sample, as the DAG's ingestion side compiled per sample (design doc, "Ingestion: the DAG run per sample"). There is no hand-written query path.

What

A stored definition whose build is Scan → TimeShift* → TimeRange(pane) → SummaryAgg runs per sample:

  • Per series, once: the stateless prefix. The executor's RawSampleSelector turns the series' first sample into its input row, which applies the Scan's label matchers. The row's group columns give its group: for a per-series SummaryAgg, every column but the value and the time; for by (...), its keys. The result is cached by label set.
  • Per sample: the open pane's state for that group is updated in place with the updater the executor's batch SummaryAgg uses (asap_executor::factory::create_planner_accumulator), with the same value and timestamp, in arrival order. Counters keep the batch build's rule that times strictly increase within a group.
  • Per group, once: the output row's layout. The definition's compiled build runs over the group's first row, and its one output row is kept as the group's template. At pane close, each group's row is the template with the pane's timestamp and the group's state. So the stored pane has exactly the DAG's output schema and the batch build's row order.

A build of another shape keeps the batch build from buffered samples:

  • a row filter;
  • a keyed or shared-grid family;
  • operators between the TimeRange and the SummaryAgg;
  • Scan matchers that read the value or time.

Each worker logs which definitions run per sample. The sharded part merge of the previous PR is unchanged.

Before this PR

Each pane's samples were buffered per worker and built at close with the executor's batch build, which made one row per sample.

After this PR

No per-pane batch is built from buffered samples for these shapes: in the examples' no-raw plans, all 23 ingestion-time outputs (counter rate and sum over 10 s panes, KLL over 60 s and 600 s panes). Only the group template is built, once per group.

Verification

  • cargo test --workspace passes, including e2e_examples (9), range, kept.
  • New per_sample::per_sample_panes_equal_the_batch_build. For Example 1 (rate, sum), Example 3b (KLL, 1-minute panes) and Example 4b (KLL, 10-minute panes), on 1 and 4 shards, every pane built per sample equals the executor's batch build of the same samples, row by row. Exact accumulator states are compared by their serialized bytes. KLL states are compared by their quantiles at every percent (exact below k). KLL seeds its compaction randomly per sketch, so two batch builds of the same samples are not byte-identical either. A mutation that drops every other sample makes both tests fail.
  • New per_sample::per_sample_panes_apply_label_matchers.

Benefit harness

Same setup as #828.

s10

Run Prometheus core-s Data plane core-s Data plane peak RSS GiB rate p50/p99 ms topk p50/p99 ms p99[5m] p50/p99 ms p99[60m] p50/p99 ms forwarded
Prometheus alone 19.3 - - (Prometheus 0.09) 9.9 / 17.8 10.2 / 20.5 20.2 / 35.4 114.0 / 144.8 -
pr2 20.4 43.8 0.47 41.6 / 57.9 39.8 / 52.2 136.4 / 154.7 858.3 / 979.9 0
pr3 20.4 43.9 0.48 41.7 / 54.5 40.4 / 50.9 136.5 / 151.1 847.7 / 1004.6 0

s10-noraw

Run Prometheus core-s Data plane core-s Data plane peak RSS GiB rate p50/p99 ms topk p50/p99 ms p99[5m] p50/p99 ms p99[60m] p50/p99 ms forwarded
Prometheus alone 18.8 - - (Prometheus 0.09) 9.7 / 19.1 9.9 / 19.2 20.1 / 38.1 94.3 / 141.9 -
pr2 9.9 35.0 0.30 22.8 / 143.2 19.5 / 29.9 70.9 / 112.8 97.1 / 133.1 325
pr3 10.1 31.5 0.31 26.2 / 45.0 19.8 / 26.6 72.7 / 87.2 110.6 / 136.8 325

s50

Run Prometheus core-s Data plane core-s Data plane peak RSS GiB rate p50/p99 ms topk p50/p99 ms p99[5m] p50/p99 ms p99[60m] p50/p99 ms forwarded
Prometheus alone 90.0 - - (Prometheus 0.14) 44.4 / 75.8 45.2 / 71.8 98.0 / 135.5 348.7 / 645.5 -
pr2 96.6 174.0 1.73 163.3 / 221.9 157.9 / 217.4 582.6 / 645.4 3853.3 / 4901.5 0
pr3 98.0 173.0 1.75 166.0 / 223.2 158.1 / 207.4 582.5 / 651.9 4036.9 / 4685.9 0

s50-noraw

Run Prometheus core-s Data plane core-s Data plane peak RSS GiB rate p50/p99 ms topk p50/p99 ms p99[5m] p50/p99 ms p99[60m] p50/p99 ms forwarded
Prometheus alone 87.5 - - (Prometheus 0.14) 44.8 / 73.5 44.9 / 75.9 97.0 / 142.6 349.1 / 666.8 -
pr2 45.3 145.3 1.10 81.5 / 566.6 63.7 / 115.0 308.2 / 389.7 445.6 / 578.3 325
pr3 46.1 130.0 0.98 106.5 / 285.2 64.0 / 214.4 310.5 / 876.1 452.5 / 533.2 325

With raw data retained the selected plan builds nothing at ingestion time (12 outputs kept across evaluations), so those runs are unchanged within noise. Without raw data, all 23 ingestion-time outputs run per sample. The remaining data-plane CPU is mostly query-time evaluation in the executor's row engine (design doc, "Execution engine"), which this PR does not change. "Prometheus alone" is the baseline of the pr2 run.

Planner gaps found

  • No per-sample build API. The executor exposes the per-sample updater (create_planner_accumulator) but not the lowering of a SummaryAgg node to its input columns (value, time, groups), nor its output assembly (scope_timestamp is pub(crate)). The backend reads the groups from the plan's SummaryAgg payload (the same rule as the executor's lowering), and gets the layout by running the compiled build once per group. The parity test guards the copy of the rule. Smallest API proposal, in asap-executor:
    pub struct IngestionPaneBuild { /* compiled from a stored output */ }
    impl IngestionPaneBuild {
        pub fn compile(dag: &PhysicalASAPDAG, output: &StoredOutput) -> Result<Option<Self>, Error>;
        /// The stateless prefix: the series' group, or None if the Scan drops it.
        pub fn group_of(&self, labels: &BTreeMap<String, String>) -> Result<Option<GroupKey>, Error>;
        pub fn new_state(&self) -> Box<dyn AccumulatorUpdater>;
        pub fn update(&self, state: &mut dyn AccumulatorUpdater, timestamp_ms: i64, value: f64) -> Result<(), String>;
        /// The pane's rows, as the batch SummaryAgg emits them.
        pub fn finish(&self, scope: Scope, states: Vec<(GroupKey, Box<dyn AccumulatorUpdater>)>) -> Result<Batch, Error>;
    }
  • KLL is not reproducible. asap_sketchlib seeds each KLL's compaction coin randomly, so no two builds of the same samples are byte-identical. A seed derived from the definition and the pane would make parity byte-exact.

The ingestion side of a stored definition, Scan -> TimeShift* ->
TimeRange(pane) -> SummaryAgg, runs per sample instead of over each pane's
buffered rows:

- per series, once: the executor's RawSampleSelector turns the series'
  first sample into its input row (applying the Scan's matchers), and the
  row's group columns give its group; cached by label set;
- per sample: the open pane's state for that group is updated in place
  with the updater the executor's batch build uses for the family
  (create_planner_accumulator), with the same value and timestamp;
- per group, once: the definition's compiled build runs over the group's
  first row, and its output row is the template each closed pane's row is
  made from (pane timestamp and state replaced).

A build of another shape (row filter, keyed or shared-grid family,
operators between the range and the summary, matchers reading the value or
time) keeps the batch build. A parity test checks every family the
examples store (counter rate, sum, KLL at two pane widths) against the
executor's batch build of the same samples: exact states byte for byte,
KLL by quantile (KLL seeds its compaction randomly per sketch).
@zzylol zzylol added enhancement New feature or request area: data plane Runtime ingestion, storage, and query-serving data paths area: summary execution Warm-summary binding, readout, composition, and result decoding integration: ASAPPlanner ASAPPlanner API, IR, candidate, and selected-plan integration labels Oct 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: data plane Runtime ingestion, storage, and query-serving data paths area: summary execution Warm-summary binding, readout, composition, and result decoding enhancement New feature or request integration: ASAPPlanner ASAPPlanner API, IR, candidate, and selected-plan integration

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant