Repository navigation
Conversation
This was referenced Oct 7, 2026
…puts Each MatAtIngestion output the query side reads (the executor's materialization frontier) reads one tumbling pane of raw samples. Outputs that differ only in which pane they read share one SummaryDefinition, identified by a SHA-256 of their sub-DAG with node ids blanked and offset TimeShifts skipped. Each output binds to its definition with an offset: at evaluation time T it reads the pane ending at T - offset. SummaryStore keeps built panes by definition and end. A plan with a node kept across evaluations is rejected: the backend declares it cannot keep one. #509 Example 4b on a deployment without raw data: its six 10-minute KLL panes become one definition with offsets 0..50 min.
zzylol
force-pushed
the
feat/physical-dag-handoff
branch
from
October 18, 2026 15:40
77f23de to
440851b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack 2/5. Base: #818. Next: #820.
Why
The data plane loads the plan but doesn't know which outputs it must store. Each
MatAtIngestionoutput the query side reads needs one stored summary definition that the precompute engine (#820) fills and the query engine (#821) reads.What
Sds::from_dagturns the plan's ingestion-time frontier (asap_executor::physical_planner::frontier_from_timing) into stored summary definitions:TimeRange(width) ← TimeShift(offset)* ← Scan: a pane ofwidthending atT − offset. Any other shape is rejected asUnsupportedShape. The control plane expands eachPaneChain(refactor!: plan with ASAPPlanner main's stage pipeline and drop the old execution stack #818), so every pane of a chain has this shape.TimeShifttakes its input's identity, so the newest pane (no shift) and the older panes (shifted) share one. In Example 4b without raw data, the six 10-minute KLL panes become one definition with offsets 0, 10, …, 50 min.MatAtQueryKeptAcrossEvaluationsnode is rejected (KeptAcrossEvaluations). The backend declaresquery_time_results_retained: false, so the Planner does not select one.SummaryStorekeeps panes by(definition, end)and evicts them by end time.examples/planner-layering-example3b.json(Pattern B) and-example4b.json(p99 over 60 min every 10 min, 1,000 series sampled every second) in the current workload format.How
One pass over the frontier: walk single inputs down to the
TimeRange(pane width), then sum offsetTimeShifts down to theScan(offset); hash the output's sub-DAG, memoized per node; group outputs by hash. The definition's builder is its lowest-numbered output.Key code interfaces added / modified
Before this PR
The data plane installed the plan without knowing what to store.
After this PR
Example 4b on a deployment without raw data: one definition, six outputs reading it at offsets 0..50 min.
Usage
Verification
data_plane/tests/sds.rs(7 tests): sharing and offsets, plans without definitions, identity independent of node numbering, identity depends on the source, rejected shapes, rejectedMatAtQueryKeptAcrossEvaluations, store lookup and eviction.cargo test -p control_plane -p data_plane, fmt, clippy on this PR's tree.