From 349ace8e0b4a5cbcfe18f163b787f0842b4dddbc Mon Sep 17 00:00:00 2001 From: zzylol <50204836+zzylol@users.noreply.github.com> Date: Fri, 16 Oct 2026 17:29:24 +0000 Subject: [PATCH 01/10] docs(user-guides): describe the current devtools, Stage Viewer tabs and stage document run-a-query.md lists the binaries that exist (stage_pipeline, show_logical_dag, the corpus tools) instead of show_pre_asap_ir, show_post_asap_ir and dag_export. stage-viewer.md matches the viewer on main: tabs 1, 3a and 3b, Examples 2 and 4 from the command line with numbers regenerated by stage_pipeline, the editor's real fields, every stage_pipeline option including the cost models, and no deployment-inputs panel. The viewer README shows cost_per_second, usage and front. --- README.md | 3 +- docs/README.md | 3 +- docs/user_guide_docs/run-a-query.md | 124 ++++++++++----------------- docs/user_guide_docs/stage-viewer.md | 123 +++++++++++++------------- tools/dag-viewer/README.md | 22 +++-- 5 files changed, 129 insertions(+), 146 deletions(-) diff --git a/README.md b/README.md index 3c21bd7d1..b426de3b8 100644 --- a/README.md +++ b/README.md @@ -1,11 +1,12 @@ # ASAPPlanner -ASAPPlanner turns SQL, PromQL, and MetricsQL query workloads into legal candidate plans that may use Approximate Streaming Analytics Primitives (ASAPs), such as sketches and exact summaries. It normalizes language-specific queries into a shared representation, then enumerates and ranks semantically equivalent alternatives. Downstream systems choose, deploy, and execute a physical plan. +ASAPPlanner turns SQL, PromQL, and MetricsQL query workloads into legal candidate plans that may use Approximate Streaming Analytics Primitives (ASAPs), such as sketches and exact summaries. It normalizes language-specific queries into a shared representation, enumerates semantically equivalent alternatives, checks each against the queries' accuracy and latency requirements, and selects a physical plan under the deployment's cost model. Downstream systems deploy and execute that plan. ## Start here - New to the repository? Read the [planner pipeline](docs/design_docs/concepts/planner-pipeline.md), then the [glossary](docs/design_docs/concepts/glossary.md). - Want to run a query? Follow [Run and inspect a query](docs/user_guide_docs/run-a-query.md). +- Want to see how a workload is planned, stage by stage? Use the [Stage Viewer](docs/user_guide_docs/stage-viewer.md). - Embedding Planner? Use the [library API guide](docs/develop_docs/library-api.md). - Extending Planner? Start with the [ASAP-aware mapping architecture](docs/develop_docs/asap-aware-mapping-architecture.md), then [extend ASAP-aware mapping](docs/develop_docs/extend-asap-aware-mapping.md). - Evaluating a design? Browse the [design documentation](docs/design_docs/README.md), [developer documentation](docs/develop_docs/README.md), and [user guides](docs/user_guide_docs/README.md). diff --git a/docs/README.md b/docs/README.md index c4fc3ea7f..a2c2c3f5b 100644 --- a/docs/README.md +++ b/docs/README.md @@ -19,7 +19,8 @@ Start with [ASAPPlanner input, output, and workflows](design_docs/architecture/i for the integration boundary, nested inputs, and choice of planning workflow. Use [Public library functions and examples](develop_docs/library-api.md) for -frontend lowering, workload search, ranking, selection and DAG assembly. +frontend lowering, planning a workload through the stages to a selected plan, +and the legacy workload search, ranking, selection and DAG assembly. ## Extend the planner diff --git a/docs/user_guide_docs/run-a-query.md b/docs/user_guide_docs/run-a-query.md index 1f172a81c..882d94da7 100644 --- a/docs/user_guide_docs/run-a-query.md +++ b/docs/user_guide_docs/run-a-query.md @@ -1,12 +1,11 @@ # ASAPPlanner CLI user guide -Use the `asap-devtools` commands to inspect query IR, export DAGs, and inspect -corpus coverage. These commands do not deploy or execute a physical plan. +Use the `asap-devtools` commands to plan a workload through the planning +stages, inspect the logical DAG a query lowers into, and inspect corpus +coverage. These commands do not deploy or execute a physical plan. To develop an application using the Rust library, start with [Library API: definitions, options, and examples](../develop_docs/library-api.md). -That guide explains how to choose strategies and models, rank candidates, and -assemble selected DAGs. ## Choose a command @@ -15,19 +14,37 @@ tool on first use. | Command (`cargo run -p asap-devtools --bin … -- …`) | Input / options | Result | | --- | --- | --- | -| `show_pre_asap_ir --sample-interval-ms 1000 queries.txt` | File path, or stdin when omitted | Prints canonical Pre-ASAP IR | -| `show_post_asap_ir --sample-interval-ms 1000 queries.txt` | Same query file format | Prints all sketch-strategy Post-ASAP candidates using a fixed approximate target, in cost-model order | -| `dag_export --sample-interval-ms 1000 --promql ""` | One PromQL expression | Exports a query DAG for inspection | -| `dag_export --sql ""` | One SQL expression using the tool's catalog | Exports a query DAG for inspection | +| `stage_pipeline --promql "" --out plan.json` | One or more `--promql` / `--sql` queries, or `--example ` | Plans the workload through Stages 0–3 and writes the stage document the [Stage Viewer](stage-viewer.md) shows | +| `show_logical_dag --sample-interval-ms 1000 queries.txt` | File path, or stdin when omitted | Prints each query's logical DAG (Pre-ASAP IR) | | `analyze_corpora --corpora --sample-interval-ms 1000 --out-dir ` | Repository PromQL corpora, output directory | Writes successful/error IR dumps and summary reports | | `analyze_corpora --sql-corpora --out-dir ` | Repository SQL corpora, output directory | Writes SQL corpus reports | | `variant_coverage --sample-interval-ms 1000` | Repository corpora | Reports Pre-ASAP IR variant coverage | -| `sketch_coverage --sample-interval-ms 1000 --epsilon 0.01` | Repository corpora; epsilon defaults to `0.01` | Reports sketch/reuse opportunities among successfully lowered queries | +| `sketch_coverage --sample-interval-ms 1000 --epsilon 0.01` | Repository corpora; epsilon defaults to `0.01` | Reports which summary candidates Pass 1 offers for each successfully lowered query | -## Inspect a query from the command line +## Plan a workload -Run these commands from the repository root. They are inspection tools; their -demonstration defaults are not the configuration of your downstream deployment. +`stage_pipeline` runs the query-language frontends (Stage 0), logical +ASAP-aware optimization (Stage 1), physical ASAP-aware optimization +(Stage 2) and plan selection (Stage 3), and writes one JSON document: + +```sh +cargo run -p asap-devtools --bin stage_pipeline -- \ + --promql "topk by (job) (10, sum_over_time(http_requests_total[1m]))" \ + --epsilon 0.01 --delta 0.001 --out plan.json +``` + +The document's `stage3_plan_selection.selected` names the selected plan, and +`stage3_plan_selection.costs` holds each shown plan's `cost_per_second`. +For this query, `selected` is `P1`, the exact plan. The Count-Min top-k +plan (`P2`) is rejected because the command declares no metric type, so the +samples are not proven non-negative, and the CountSketch top-k plan (`P3`) +because its output has no guarantee; `stage3_plan_selection.rejected` gives +each reason. +Open the document in the [Stage Viewer](stage-viewer.md) to see every +stage, or run a built-in example of the design document, such as +`--example planner-layering-1`. The Stage Viewer guide lists every option. + +## Inspect a query's logical DAG ### Create a query file @@ -40,26 +57,30 @@ promql> quantile(0.99, rate(http_requests_total[5m])) sql> SELECT service, COUNT(*) FROM metrics GROUP BY service ``` -Blank lines and lines beginning with `#` are ignored. The two file/stdin tools -accept `sql>` and `promql>`; MetricsQL is available through the library frontend. +Blank lines and lines beginning with `#` are ignored. `show_logical_dag` +accepts `sql>` and `promql>`; MetricsQL is available through the library frontend. SQL examples use the fixed catalog `metrics(ts: Timestamp, service: Utf8, region: Utf8, latency: Float64, bytes: Int64)`. -For your own schema, provide a `SqlCatalog` through the library API. +For your own schema, provide a `SqlCatalog` through the library API, or +declare tables with `stage_pipeline --table`. -### Show the Pre-ASAP IR +### Show the logical DAG Run: ```sh -cargo run -p asap-devtools --bin show_pre_asap_ir -- --sample-interval-ms 1000 queries.txt +cargo run -p asap-devtools --bin show_logical_dag -- --sample-interval-ms 1000 queries.txt ``` You can also provide the queries through stdin: ```sh -cargo run -p asap-devtools --bin show_pre_asap_ir -- --sample-interval-ms 1000 < queries.txt +cargo run -p asap-devtools --bin show_logical_dag -- --sample-interval-ms 1000 < queries.txt ``` +Each input line is followed by its debug IR or an `ERR:` message. The tool +never picks a sketch: it shows what a query means, with exact operators only. + To dump and compare every PromQL corpus, run: ```sh @@ -78,62 +99,8 @@ The corresponding SQL corpus analysis is: cargo run -p asap-devtools --bin analyze_corpora -- --sql-corpora --out-dir artifacts/sql_pre_asap ``` -### Inspect Post-ASAP IR candidates - -Run: - -```sh -cargo run -p asap-devtools --bin show_post_asap_ir -- --sample-interval-ms 1000 queries.txt -``` - -Or through stdin: - -```sh -cargo run -p asap-devtools --bin show_post_asap_ir -- --sample-interval-ms 1000 < queries.txt -``` - -`show_post_asap_ir` uses an approximation target of ε = 0.01 and prints every -available binding from the sketch strategy for each query, numbered in cost-model -order. If no candidate is available, it prints the pre-ASAP fallback as candidate -1. It does not show the complete ranked workload candidate set or choose a -deployment. Its SQL examples use a fixed demonstration catalog, not -your database schema. Use the -[library workflow](../develop_docs/library-api.md) to retain workload alternatives -and provide your own models. - -The default strategy generates DDSketch quantile-ratio candidates even when no -input-domain evidence is available. Such candidates have `guarantee: None`: -they do not claim a certified end-to-end accuracy bound. They also remain -visible in a target-aware `CandidateLogicalASAPDAGs` so the downstream backend can decide -whether to select them using its own evidence. Planner's automatic -`global_selection` skips them; their presence alone does not show that they -meet the requested target. - -Each input line is followed by its debug IR or an `ERR:` message. Pre-ASAP and -Post-ASAP output use the same node format: Post-ASAP output adds summary nodes -(state, readouts) and keeps the original exact operators wherever no summary -replaces them. An -approximate target permits approximation; it does not guarantee a legal or -certified sketch. The tool prints plans, not query results. - ## More inspection commands -### Export a query DAG - -Export pre-ASAP IR for SQL or PromQL queries for use with the interactive DAG viewer: - -```sh -cargo run -p asap-devtools --bin dag_export -- --sql "" -``` - -or: - -```sh -cargo run -p asap-devtools --bin dag_export -- --sample-interval-ms 1000 --promql "" -``` - -See [`tools/dag-viewer/RUNNING.md`](../../tools/dag-viewer/RUNNING.md) for instructions on running the DAG viewer. - ### Check IR variant coverage Parse the query corpora in the repository and report which pre-ASAP IR variants are exercised: @@ -142,15 +109,15 @@ Parse the query corpora in the repository and report which pre-ASAP IR variants cargo run -p asap-devtools --bin variant_coverage -- --sample-interval-ms 1000 ``` -### Check sketch-replacement coverage +### Check summary coverage -Parse the same query corpora, lower them with an approximate `AccuracyTarget`, and report what fraction of each corpus's successfully-lowered queries got a genuine sketch alternative (`SketchApproximation`, e.g. KLL vs. DDSketch) and/or a cross-query common-subexpression-reuse candidate (`CommonSubexpressionReuse`): +Parse the same query corpora, lower them with an approximate `AccuracyTarget`, run Pass 1 of logical ASAP-aware optimization over each query, and report which summary candidates Pass 1 offers, plus the fraction of each corpus's successfully lowered queries with at least one: ```sh cargo run -p asap-devtools --bin sketch_coverage -- --sample-interval-ms 1000 --epsilon 0.01 ``` -`--epsilon` is optional (defaults to `0.01`). See the binary's own doc comment for exactly how "coverage" is defined and attributed back to each query. +`--epsilon` is optional (defaults to `0.01`). Without an approximate target, Pass 1 offers no summary candidate. See the binary's own doc comment for exactly how coverage is counted. ## Additional examples @@ -175,6 +142,7 @@ cargo run -p asap-devtools --example canonical_examples - [Post-ASAP IR reference](../design_docs/concepts/post-asap-ir.md) PromQL commands require `--sample-interval-ms`, the nonzero sample interval in -milliseconds declared for every metric a query reads. The examples use one -second; supply the interval for your data. The mixed-input `show_pre_asap_ir` and -`show_post_asap_ir` tools require this option even for SQL-only input files. +milliseconds declared for every metric a query reads (`stage_pipeline` +defaults it to 15000). The examples use one second; supply the interval for +your data. `show_logical_dag` requires this option even for SQL-only input +files. diff --git a/docs/user_guide_docs/stage-viewer.md b/docs/user_guide_docs/stage-viewer.md index 956bc7c64..c44b01664 100644 --- a/docs/user_guide_docs/stage-viewer.md +++ b/docs/user_guide_docs/stage-viewer.md @@ -3,7 +3,7 @@ The Stage Viewer shows, in a browser, what ASAPPlanner does with one workload at each of its planning stages, and why it selects the plan it selects. Use it to check a plan by hand, to compare the alternatives the planner considered, -and to see how the deployment's capabilities and the query requirements +and to see how the deployment's cost model and the query requirements change the choice. The stages are those of @@ -13,20 +13,12 @@ The stages are those of | --- | --- | | 0. Frontends | The logical DAG the queries lower into, one root per query | | 1. Logical ASAP-aware optimization | The logical ASAP candidates: each choice of exact or approximate computation per query (Pass 1), with shared inputs, shared summaries and tumbling panes (Pass 2) | -| 2. Physical ASAP-aware optimization | The physical candidates of each logical candidate, which differ in what runs at ingestion time and what runs at query time | -| 3. Plan selection | Each physical candidate checked for accuracy, latency and deployment capabilities, priced per second, and the cheapest selected | +| 2. Physical ASAP-aware optimization | The physical candidates of each logical candidate, which differ in what is materialized: state maintained at ingestion time, a node built once per evaluation at query time and read by several queries, or nothing | +| 3. Plan selection | Each physical candidate checked for accuracy and latency, compared under the deployment's cost model (by default priced per second), and one selected | The viewer reads one JSON document per workload, written by the `stage_pipeline` command. It does not run the plans. -> **Based on [PR #574](https://github.com/ProjectASAP/ASAPPlanner/pull/574)** -> (branch `stack/509-viewer-stages`), which redesigns the viewer, together with -> the PRs it is stacked on: #613 (`stage_pipeline` examples 2 and 4b, and -> pricing every plan), #610 (deployment inputs in the document) and #609 -> (deployment capabilities). Until they are merged, check out -> `stack/509-viewer-stages` to use the viewer as described here; `main` still -> has the earlier Pre/Post-ASAP viewer. - ## Start the viewer Run from the repository root, with Rust/Cargo and Python 3 installed: @@ -69,14 +61,20 @@ plan wins. | Tab | Workload | What to look for | | --- | --- | --- | | 1 · dashboard panels | Two PromQL panels over 1M counter series every 10 s: an exact per-job rate, and the approximate top-10 series per job | The exact query stays exact; the CMS+heap and CountSketch+heap top-k plans are rejected with their reasons, so the all-exact plan wins; the plans that share the two queries' scan and range selector tie with it, and the independent plan is kept on enumeration order. Pass 2 also builds Q1's rate and Q2's option in six 10-s tumbling panes, each alone or both (27 logical plans); rebuilt at every refresh, the panes cost more than the whole window. Stage 2 adds 30 plans that maintain Q1's rate panes, Q2's panes or both at ingestion time (57 physical plans); continuously ingesting PromQL data is taken to arrive in time order, so the rate panes may be maintained. Keeping seven panes of 1M per-series states costs more than rebuilding them (12.94 against 8.54 per second for either query's exact panes), so P1, all exact, is still selected at 7.94 | -| 2 · SQL flow statistics | Distinct sources, entropy and L2 of per-source counts over the last minute of flows | All 82 plans compose and are priced; none is pruned. The SQL frontend recognizes Q2 as entropy and Q3 (its counts cast to DOUBLE) as L2, each with an exact or UnivMon option and an exact population count for the empty-input NULL. 16 plans come from Pass 1. Pass 2 adds the plans that share identical nodes and, by the summary-capability rule, the plans in which one UnivMon serves two of the queries or all three, sized with each parameter's largest value among them. 70 plans are invalid: HLL or UnivMon for Q1 (50) because its bound has "failure probability None" (both need the source's distinct bound, which this example does not declare), and UnivMon for Q2 (20) because entropy has no certified bound. 12 are valid: Q1 exact or KMV, Q2 exact, and Q3 exact or UnivMon, each alone or sharing identical nodes. Q3's UnivMon keeps its L2 guarantee through the exact population count joined beside it and the `CASE` that decides the empty-input NULL on that count. Theta is not offered: `asap_sketchlib` has no Theta sketch. P1 (all exact) is selected at 6.08 per second; KMV on Q1 costs the same to three decimals, UnivMon on Q3 costs 8.08, and the shared plans cost 7.44 or more, because a shared node that is not materialized is charged once per query that reads it | -| 3a · historical p99 batch | Five p99 reports over 1–5 years, run once | All 972 plans are priced: 243 Pass 1 plans, each with 3 shared-input plans. Each query builds its own sketch at query time. P485 is selected: KLL for all five queries. DDSketch in place of KLL ties, and the tie goes to enumeration order. The all-exact plan costs about 16% more | +| 3a · historical p99 batch | Five p99 reports over 1–5 years, run once | All 972 plans are priced or rejected: 243 Pass 1 plans, each with 3 shared-input plans. Each query builds its own sketch at query time. P485 is selected: KLL for all five queries. 844 plans are invalid: every plan with DDSketch, because each query's target is a rank error and DDSketch bounds a relative error. The all-exact plan costs about 16% more | | 3b · live p99 panel | p99 over the last 5 min every minute, 1M series | P2, KLL, is selected at 2.100 per second; exact over the window's raw samples costs 2.400. Pass 2 adds KLL and DDSketch in five 1-min tumbling panes (5 plans in all): rebuilt at every evaluation, the KLL panes cost 2.183, the whole build plus merging the panes. DDSketch misses the rank target. Memory a query holds only while it runs is not charged. Stage 2 adds the KLL and DDSketch panes maintained at ingestion time (7 plans in all): maintained, the KLL panes keep six panes of 1M per-series KLLs, about 107 GB, for 13,326.5 per second, so P2 still wins. Every priced plan meets the 200 ms latency bound: the whole-window KLL takes 126 ms per evaluation, the panes rebuilt at query time 131 ms and the maintained panes 7 ms | -| 4a · monthly p99 reports | Example 3a repeated monthly | The same plan, with its cost amortized over a month | -| 4b · panes that pay off | p99 over the last hour every 10 min, 1k series sampled every second, raw data not kept by the deployment | Maintaining six 10-min KLL panes at ingestion time wins | To link to one example, add its name after `#`, for example -`http://127.0.0.1:8000/#example4b`. +`http://127.0.0.1:8000/#example3b`. + +The other examples of the design document have no tab. Write their +documents with `stage_pipeline` (see [From the command line](#from-the-command-line)) +and open them with **Open stage document…**: + +| Example | Command options | What to look for | +| --- | --- | --- | +| 2 · SQL flow statistics | `--example planner-layering-2` | Distinct sources, entropy and L2 of per-source counts over the last minute of flows. All 82 logical plans compose; none is pruned. The SQL frontend recognizes Q2 as entropy and Q3 (its counts cast to DOUBLE) as L2, each with an exact or UnivMon option and an exact population count for the empty-input NULL. 16 plans come from Pass 1. Pass 2 adds the plans that share identical nodes and, by the summary-capability rule, the plans in which one UnivMon serves two of the queries or all three, sized with each parameter's largest value among them. Stage 2 adds 102 plans in which a node that two queries read is built once per evaluation at query time and read by both (`query time, shared: …` in the label), 184 physical plans in all. 164 plans are invalid: HLL or UnivMon for Q1 (120) because its bound has "failure probability None" (both need the source's distinct bound, which this example does not declare), and UnivMon for Q2 (44) because entropy has no certified bound. 20 are valid: Q1 exact or KMV, Q2 exact, and Q3 exact or UnivMon, each alone, sharing identical nodes, or with the shared node built once at query time. Q3's UnivMon keeps its L2 guarantee through the exact population count joined beside it and the `CASE` that decides the empty-input NULL on that count. Theta is not offered: `asap_sketchlib` has no Theta sketch. P1 (all exact) is selected at 6.08 per second; KMV on Q1 costs the same to three decimals, and UnivMon on Q3 costs 8.08. The shared plans cost 7.44 or more when the shared node is not materialized, because it is then charged once per query that reads it, and 7.04 when it is built once at query time | +| 4 · one workload, three cost models | `--example planner-layering-3b` with `--cost-model` | Example 3b's live p99 panel under each cost model. Cost model 1 (`usage-over-time`, the default) selects P2, the whole-window KLL, at 2.100 per second. Cost model 2 (`provisioned-capacity`) selects P1, exact, at 1024.0 per second against 3221.0 for P2, because it prices peak memory, and the KLL states hold about 17.8 GB at their full capacity. Cost model 3 (`cluster-capacity --cluster-memory-bytes 256000000000 --cluster-disk-bytes 1000000000000`) selects by `--cluster-objective`: `lowest-latency` selects P3-m1, the KLL panes maintained at ingestion time (7 ms per evaluation); `least-memory` selects P1; `pareto-front` lists P1, P2 and P3-m1 and selects P3-m1. With 64 GB of memory (`--cluster-memory-bytes 64000000000`), the plans with KLL panes are over the limit and invalid, `lowest-latency` selects P2, and the front is P1 and P2. Under cost model 3 the plans have no price | The workload statistics in the examples (series counts, ingestion rates) are illustrative. Read the costs as a ranking, not as a prediction of real @@ -97,29 +95,19 @@ planner must meet: - **latency**: the maximum time an evaluation may take at query time; - **every N s**: how often a repeating query runs. -The line under the queries counts the candidates at each stage, for example -`1 logical DAG → 88 logical ASAP → 112 physical → 1 selected (40 invalid, 71 -valid but costlier)`. - -### Deployment inputs - -What the deployment told the planner, which Stage 3 uses to accept or reject -plans: - -| Row | Meaning | -| --- | --- | -| Exact aggregates | Exact computations the executor can run (Sum, Count, Min, Max, Increase, Rate) | -| Sketches → estimates | Each sketch the executor can build, with the estimates it can be read for, for example `Kll → Quantile`. A plan that needs anything else is invalid. | -| Ingestion-time maintenance | Whether the deployment can keep state up to date as data arrives | -| Keep query-time results across evaluations | Whether results built at query time can be kept for the next evaluation | -| Memory budget | The most state the deployment can keep, or none | -| Raw data | Whether the deployment keeps raw samples anyway. If it does not, plans that read raw data at query time pay to keep the samples they read. | -| Cost model | The cost model, its unit, and its calibration constants, such as the cost of one CPU operation and of keeping one byte for one second | -| Accuracy model | The model that decides whether a sketch meets a query's accuracy target | +The line under the queries counts the candidates at each stage, for example, +in Example 1, `Candidates per stage: 1 logical DAG → 27 logical ASAP → 57 +physical → 1 selected (38 invalid, 18 valid but costlier)`. ### Stage 3 · plans by cost -Every physical plan, cheapest first, with its total cost per second. +The selected plan first, then the other valid plans cheapest first, each +with its cost per second; then the invalid plans with their reasons, the +plans Stage 2 pruned, and the Stage 1 candidates that did not compose. +Under cost model 3 the plans have no price: the list shows the selected +plan and marks the other valid ones `valid · costlier` without a reason; +the document's `stage3_plan_selection.rejected` holds the reasons (for +example `slower: latency 144.0 ms, …`). A node that is not materialized is recomputed for each query that reads it, so a node shared by two queries is charged twice; a materialized node, at ingestion or at query time, is charged once, and the nodes below it once for @@ -131,33 +119,37 @@ it (plus once for each query that also reads them directly). A selected or costlier plan also says, per query, which metric its accuracy was accepted in, e.g. `q1: metric not set; accepted rank (Kll)`. - **✗ invalid**: fails a check, with the reason. Examples: an accuracy target - the sketch cannot guarantee, an evaluation over the latency bound, or a - summary the deployment cannot build. + the sketch cannot guarantee, an evaluation over the latency bound, or, + under cost model 3, a plan over the cluster's memory or disk. -Each label names the choice per query, for example `Q1 exact · Q2 CMS+heap · -shared input`, and, for physical plans, what runs at ingestion time, for -example `ingestion time: Kll ×6 panes`. +Each label names the choice per query, for example `Q1 exact (Sum acc, Rate +acc) · Q2 CMS+heap`, what Pass 2 shares or splits into panes, for example +`shared Scan (Q1, Q2)` or `Kll for Q1 in 5 × 1m panes`, and, for physical +plans, what is materialized: `ingestion time: Kll ×5 panes` for state +maintained as data arrives, `query time, shared: Count acc` for a node built +once per evaluation and read by several queries. Click a plan to show it in the three lanes. For workloads with many plans, -the document may carry only the cheapest ones. The list then says so, for -example `showing 64 of 486 plans, cheapest first; Stage 3 priced 486 and -selected among all of them`. The selection is made over all plans either -way. +the document may carry only the first ones in enumeration order, plus the +selected one. The list's header then says so, for example `972 candidates +(document shows 65)` in Example 3a. The selection is made over all plans +either way. ### The three lanes | Lane | Shows | | --- | --- | -| Stage 0 · Logical DAG | The queries as lowered by the frontends | -| Stage 1 · Logical ASAP DAG | The logical candidate of the plan you picked. The **candidate** menu shows any other. | -| Stage 2 · Physical ASAP DAG | The plan you picked, with each node's timing (ingestion time or query time) and Stage 3 cost. The **candidate** menu shows any other. | +| Stage 0 · Query-language frontends | The queries as lowered by the frontends | +| Stage 1 · Logical ASAP candidate | The logical candidate of the plan you picked. The **candidate** menu shows any other. | +| Stage 2 · Physical ASAP candidate | The plan you picked, with each node's timing (ingestion time or query time) and Stage 3 cost. The **candidate** menu shows any other. | Data flows from the bottom up. Node colors: - blue: data sources; - grey: relational operators (filter, project, aggregate, sort, …); - purple: summary operators (build a sketch or exact accumulator, merge, - estimate, finalize). + estimate, finalize); +- dashed: a standalone scalar query, such as `time()` or `1 + 2`. A thick border marks a query's result. A sketch build shows its configuration and how many instances it keeps, for example `summary: CmsWithHeap · depth 7 @@ -184,19 +176,21 @@ Click an edge to see the schema and data state it carries. ### In the page -Click **Query editor**, enter PromQL queries one per line, and click -**Plan**: +Click **Query editor**, declare the tables the SQL queries read, enter SQL +or PromQL queries, and click **Plan**: | Field | Meaning | | --- | --- | -| ε | Relative error allowed for every query. Leave it empty for exact queries. | -| δ | Failure probability allowed. Needs ε. | -| sample interval (ms) | How often each series is sampled (default 15000) | +| Input tables | One table per row: a name, one column per line (`name TYPE [NOT NULL]`) and an optional time column. Only SQL queries read them; PromQL schemas are inferred from the metrics a query reads. | +| Queries | One query per row: a name, the language (SQL or PromQL), the tables a SQL query reads, and its text. Unchecked rows are left out of the workload. | +| ε | Error allowed for every query (default 0.01). | +| metric | The error metric ε is stated in: Absolute, Relative or Rank. `default` leaves it to the planner, which accepts a plan in any metric that meets ε; a quantile query must name one. | -The editor plans the queries as one batch that runs once. It declares only -the sample interval, so the planner uses default statistics for everything -else. SQL is not available in the editor yet; use the command below with a -built-in example, or a document written elsewhere. +The editor plans the checked queries as one batch that runs once, with no +failure probability δ. It declares a 15 s sample interval for every metric +the PromQL queries read and no other data facts, so the planner uses +default statistics for everything else. For δ, another sample interval or +another cost model, use the command below. ### From the command line @@ -210,11 +204,18 @@ cargo run -p asap-devtools --bin stage_pipeline -- \ | Option | Meaning | | --- | --- | -| `--promql ` | A query; repeat for several | +| `--promql `, `--sql ` | A query; repeat and mix for several, in workload order | +| `--name ` | Names the query before it (default `q1`, `q2`, …) | +| `--table ''` | A table the SQL queries read, for example `{"name": "metrics", "columns": [{"name": "ts", "type": "timestamp"}, {"name": "service", "type": "utf8"}], "time_index": 0}`; repeat for several | | `--epsilon`, `--delta` | Accuracy for every query; without them the queries are exact | -| `--sample-interval-ms ` | Sample interval of every metric the queries read (default 15000) | -| `--example ` | A built-in example instead of `--promql`: `planner-layering-1`, `-2`, `-3a`, `-3b`, `-4a` or `-4b` | +| `--metric ` | The error metric of that accuracy: `absolute`, `absolute_stream_l1`, `absolute_stream_l2`, `relative` or `rank`. Without it, a plan is accepted in whatever metric meets ε (and δ). | +| `--sample-interval-ms ` | Sample interval of every metric the PromQL queries read (default 15000) | +| `--example ` | A built-in example instead of queries: `planner-layering-1`, `-2`, `-3a` or `-3b` | +| `--cost-model ` | How plan selection compares valid plans: `usage-over-time` (cost model 1, the default: CPU, scanned bytes and memory over time, priced per second), `provisioned-capacity` (cost model 2: peak memory, peak CPU cores and disk, priced) or `cluster-capacity` (cost model 3: no prices) | +| `--cluster-memory-bytes `, `--cluster-disk-bytes `, `--cluster-cpu-cores ` | Cost model 3 only: the cluster's limits; a plan over one is invalid. The CPU limit is optional. | +| `--cluster-objective ` | Cost model 3 only: `lowest-latency`, `least-memory` or `pareto-front` (the plans no other plan beats on latency, memory and disk; the lowest-latency one is selected) | | `--max-displayed ` | How many plans the document shows (default 64), in enumeration order, plus the selected plan. The planner still optimizes and prices every plan, so the selection does not depend on it. | +| `--max-combinations ` | The most candidates a run plans (default 10000), counted at Stage 1 and again at Stage 2; a larger workload is refused with exit status 3 and no document | | `--out ` | Where to write the document | Open the document with **Open stage document…**, by dropping the file on the diff --git a/tools/dag-viewer/README.md b/tools/dag-viewer/README.md index 0c77ad224..3570f1d18 100644 --- a/tools/dag-viewer/README.md +++ b/tools/dag-viewer/README.md @@ -7,8 +7,9 @@ A browser view of what the planner does with one workload, read from an one root per query; - **Stage 1**, the logical ASAP candidates: Pass 1's per-computation choices, each followed by its Pass 2 identical-expression candidates; -- **Stage 2**, the physical candidate of each composed Stage 1 candidate, - or the reason Stage 2 pruned it; +- **Stage 2**, the physical candidates of each composed Stage 1 candidate, + one per materialization assignment (`P`, then `P-m`), or the + reason Stage 2 pruned it; - **Stage 3**, which checks every physical candidate, prices the valid ones per second and selects the cheapest. @@ -75,12 +76,17 @@ time column). The server runs `stage_pipeline` with `--sql`, `--promql`, "pruned": [{ "from_logical": "L3", "display_name": "...", "reason": "..." }] }, "stage3_plan_selection": { - "costs": { "P1": { "total": 7.9, "unit": "cost_per_second", "source": "...", - "per_node": { "": { "cost": 2.3, "multiplier": 1, "detail": "..." } } } }, + "costs": { "P1": { "cost_per_second": 7.9, "unit": "cost_per_second", "source": "...", + "usage": { "cpu_ops_per_second": 4100210.0, "scanned_bytes_per_second": 38400000.0, + "average_memory_bytes": 0.0, "peak_memory_bytes": 16481088, + "disk_bytes": 192000000, "peak_cpu_cores": 2, + "latency_per_query": [{ "compaction_ms": 0.0, "query_ms": 33.2 }] }, + "per_node": { "": { "cost_per_second": 2.3, "multiplier": 1, "detail": "..." } } } }, "accepted": { "P1": [{ "target_metric": null, "accepted_metric": "rank", "sketches": ["Kll"], "summary": "metric not set; accepted rank (Kll)" }] }, "selected": "P1", - "rejected": [{ "id": "P2", "valid": true, "reason": "..." }] + "rejected": [{ "id": "P2", "valid": true, "reason": "..." }], + "front": [] } } ``` @@ -99,6 +105,12 @@ time column). The server runs `stage_pipeline` with `--sql`, `--promql`, - `stage3_plan_selection.accepted` has, per shown valid candidate and query, the metric the target set (null when not set) and the metric Stage 3 accepted (null for an exact output). +- `costs` has, per shown valid candidate, its resource `usage` and its + `cost_per_second` under the run's cost model (`--cost-model`). A + candidate's `cost_per_second` is null under cost model 3, which has no + prices; a node's is null under cost models 2 and 3, which price or limit + the whole candidate's peaks. `front` lists the Pareto front under cost + model 3's `pareto-front` objective, and is empty otherwise. - When Pass 1 fails, Stage 1 has `error` and no candidates. When selection fails, `selected` is null, with the rejections or with `error`. From e6bc33644bd3426f7f97609be29df3fcb66cb8b6 Mon Sep 17 00:00:00 2001 From: zzylol <50204836+zzylol@users.noreply.github.com> Date: Fri, 16 Oct 2026 17:29:24 +0000 Subject: [PATCH 02/10] docs(develop): AccuracyTarget builders, AccuracyGuarantee and the stage pipeline API library-api.md gains a section on planning a workload through the stages (e2e_plan, run_stage1_to_stage3, PlanSelectionSettings and cost models 1-3) and uses the AccuracyTarget builders; its examples compile again. Other developer docs drop dag_export and use the current type names. --- .../end-to-end-accuracy-guarantees.md | 40 ++-- docs/develop_docs/library-api.md | 206 ++++++++++++++---- docs/develop_docs/offline-sketch-evidence.md | 4 + .../develop_docs/physical-compile-coverage.md | 3 +- docs/develop_docs/physical-handoff-costs.md | 8 +- docs/develop_docs/pre-asap-ir.md | 8 +- docs/develop_docs/storage-operation-costs.md | 5 +- .../target-candidate-api-migration.md | 2 +- 8 files changed, 211 insertions(+), 65 deletions(-) diff --git a/docs/develop_docs/end-to-end-accuracy-guarantees.md b/docs/develop_docs/end-to-end-accuracy-guarantees.md index fdc4c5015..639ab02a0 100644 --- a/docs/develop_docs/end-to-end-accuracy-guarantees.md +++ b/docs/develop_docs/end-to-end-accuracy-guarantees.md @@ -28,7 +28,7 @@ aggregate intent and AccuracyTarget For a single summary, the child is exact and propagation leaves the local summary guarantee as the final guarantee. For nested summaries, child and local -guarantees are combined. `AccuracyTarget` is the requirement; `ResultGuarantee` +guarantees are combined. `AccuracyTarget` is the requirement; `AccuracyGuarantee` is the evidence produced for one concrete candidate. The design document's @@ -44,7 +44,7 @@ The main implementation locations are: | Accuracy model and built-in propagation | `asap_aware_mapping::accuracy` | | Candidate construction and legality filtering | `asap_aware_mapping::replacement` | | Parameter sizing hooks | `asap_aware_mapping::cost_model` | -| Guarantee and rejection export | `asap_types::dag_export` and the `dag_export` devtool | +| Guarantee and rejection export | The `stage_pipeline` devtool's document (`crates/devtools/src/bin/stage_pipeline.rs`) | Read the sections below when changing one of those contracts. @@ -55,26 +55,27 @@ Accuracy reasoning and allocation are separate from cost modeling: ```rust trait AccuracyModel { fn local_guarantee(/* family, query, parameters */) - -> Option; + -> Option; fn propagate( &self, op: &CompositionOperator, - inputs: &[ResultGuarantee], - local: Option<&ResultGuarantee>, + inputs: &[AccuracyGuarantee], + local: Option<&AccuracyGuarantee>, stats: &PropagationStats, - ) -> Result; + ) -> Result; fn satisfies( &self, - guarantee: &ResultGuarantee, + guarantee: &AccuracyGuarantee, target: &AccuracyTarget, ) -> bool; } ``` -`AccuracyBudgetAllocator` proposes finite parameter allocations for nested -approximate layers. Every candidate must then be resized, propagated, and +`accuracy::budget::per_input_target` splits a target equally among the +approximate inputs of one composition (ε / n for an additive metric, +(1 + ε)^(1/n) − 1 for a relative one, δ / n). Every candidate must then be resized, propagated, and checked before being treated as satisfying the target. This is distinct from candidate visibility: direct DDSketch ratios lacking domain evidence remain in `CandidateLogicalASAPDAGs` with `guarantee: None` and can appear in `cost_sorted`, but automatic @@ -184,10 +185,17 @@ delta = unknown ``` Sizing inverts the magnitude. HLL's current parameter model contains precision -only; it does not encode a confidence-level budget. HLL may therefore satisfy -`Epsilon` but cannot satisfy `EpsilonDelta`. Supporting a confidence target -would require an explicitly identified estimator contract and corresponding -parameters whose semantics prove the requested failure probability. +only; it does not encode a confidence-level budget. Generic HLL may therefore +satisfy a target with only `error` set, but not one that also sets +`failure_probability`. + +Classic HLL (its linear-counting branch) has an estimator-specific confidence +bound (`accuracy/estimators/hll.rs`). Its precision is sized from (ε, δ), and +its failure probability stays symbolic, +`classic_hll_failure_probability[p=P,eps=ε](max_distinct[source])`, until +plan selection binds it with the source's +`DataWorkload.sources[source].max_distinct_per_evaluation`. A missing bound +leaves the target unmet. ### KMV and Theta @@ -280,8 +288,10 @@ accuracy-targeted candidate. Never copy the inner guarantee onto Hydra alone. ## Adding or changing an accuracy target -`Exact`, `Epsilon`, and `EpsilonDelta` are the current vocabulary, not a closed -set. If a new summary has a final-result requirement they cannot express: +`AccuracyTarget`'s fields (`error`, an optional `failure_probability`, an +optional `accuracy_metric` and an `accuracy_scope`; built with `exact()` or +`max_error(ε)` plus `.failure_probability(δ)`, `.accuracy_metric(m)` and +`.accuracy_scope(s)`) are the current vocabulary, not a closed set. If a new summary has a final-result requirement they cannot express: 1. Define the target's result semantics and compatible `AccuracyMetric`. 2. Define the parameters and evidence needed to prove satisfaction. diff --git a/docs/develop_docs/library-api.md b/docs/develop_docs/library-api.md index 9968148a7..59fa4a8f5 100644 --- a/docs/develop_docs/library-api.md +++ b/docs/develop_docs/library-api.md @@ -4,7 +4,9 @@ Audience: developers embedding ASAPPlanner or adding strategies/models. This is a compact reference for the public workflow APIs, not an exhaustive symbol reference. The [CLI guide](../user_guide_docs/run-a-query.md) covers command-line inspection; the [design overview](../design_docs/architecture/README.md) defines ownership. -ASAPPlanner's primary output is `CandidateLogicalASAPDAGs`; ranking is a view over its candidates. +ASAPPlanner's output is the plan selected by its planning stages +(`PlanOutput`, see [Plan a workload through the stages](#plan-a-workload-through-the-stages)). +The legacy search below outputs `CandidateLogicalASAPDAGs`; ranking is a view over its candidates. Downstream owns physical binding and commitment. Selection/DAG assembly helpers do not deploy a plan, and a serializable DAG is not evidence of runtime readiness. @@ -13,6 +15,7 @@ do not deploy a plan, and a serializable DAG is not evidence of runtime readines | Desired result | Calls | Example | | --- | --- | --- | | Pre-ASAP IR | Frontend `lower_*` | [Lower a query](#lower-a-query-into-pre-asap-ir) | +| Selected plan for a workload (Stages 0–3) | `asap_planner::e2e_plan` | [Plan a workload](#plan-a-workload-through-the-stages) | | All ranked candidates | `search_workload_with_targets` -> `cost_sorted` | [Generate and rank](#generate-and-rank-candidates) | | Custom optimization set | Construct `Vec>`, then search | [Strategies and models](#choose-strategies-and-models) | | Selected semantic DAG / export | `global_selection` -> `assemble_selected_dag` -> export | [Selection example](#optional-whole-plan-selection-and-dag-assembly) | @@ -23,7 +26,8 @@ artifact, while preserving the checks required by its intended consumer. ## Dependencies Inside this workspace, depend on the frontend you need, `asap-aware-mapping`, -and `asap-types`. External users can use Git dependencies pinned to a compatible +and `asap-types`, or on `asap-planner`, which wraps every frontend and the +planning stages. External users can use Git dependencies pinned to a compatible revision; use the same revision across these crates. For the example below: ```toml @@ -69,13 +73,20 @@ nonzero), whatever the arrival. A query with one fails with interval. Explicit range selectors retain their query-specified range and need no interval. Use `lower_promql_workload_with_histograms` to supply a histogram catalog for the whole workload. Each entry carries its -own accuracy requirement, with these explicit target choices: +own accuracy requirement, an `AccuracyTarget` with the fields `error` (ε), +`failure_probability` (δ, optional), `accuracy_metric` (optional) and +`accuracy_scope` (default the whole result). Build it with: -| Value | Meaning | Example | +| Builder | Meaning | Example | | --- | --- | --- | -| `AccuracyTarget::Exact` | No approximation permitted | Exact aggregation/unchanged-query alternatives only | -| `AccuracyTarget::Epsilon(e)` | An epsilon error requirement interpreted by the relevant accuracy rule | `Epsilon(0.01)` | -| `AccuracyTarget::EpsilonDelta { epsilon, delta }` | Error requirement with a failure-probability bound | `{ epsilon: 0.01, delta: 0.05 }` | +| `AccuracyTarget::exact()` | No approximation permitted (the `QueryRequirements` default) | Exact aggregation/unchanged-query alternatives only | +| `AccuracyTarget::max_error(e)` | Error at most ε, with no δ requirement and no metric set | `max_error(0.01)` | +| `.failure_probability(d)` | Also require the bound to hold with probability at least 1 − δ | `max_error(0.01).failure_probability(0.05)` | +| `.accuracy_metric(m)` | State ε in `AccuracyMetric` `m` (`Absolute { scale }`, `Relative` or `Rank`); unset accepts a guarantee in any metric | `.accuracy_metric(AccuracyMetric::Rank)` | +| `.accuracy_scope(s)` | Which output values one statement covers (`AccuracyScope`) | `.accuracy_scope(AccuracyScope::WholeResult)` | + +In JSON the target is a plain struct, for example +`{"error": 0.01, "failure_probability": 0.001, "accuracy_metric": "rank"}`. Epsilon does not mean the same error quantity for every statistic. Inspect the candidate's guarantee and its error metric; a target is a requirement, not proof @@ -133,7 +144,134 @@ async lower_sql_dialect(query: &str, catalog: &SqlCatalog, | `ElasticSQL` | Returns `UnsupportedDialect` | The catalog is required and describes your tables. For a complete schema-building -example, see [the CLI frontend example](../../crates/devtools/src/bin/show_pre_asap_ir.rs). +example, see `lineitem_catalog` in [the planner's end-to-end tests](../../crates/planner/tests/e2e_plan.rs). + +## Plan a workload through the stages + +`asap_planner::e2e_plan` runs the whole planner: the query-language frontends +(Stage 0), then the default optimization pass `Stage1ToStage3Pass`, which runs +logical ASAP-aware optimization (Stage 1), physical ASAP-aware optimization +(Stage 2) and plan selection (Stage 3) in order. + +```text +asap_planner::e2e_plan(input: UserInput<'_>) -> Result // async +UserInput::new(workload: &PlanningWorkload, frontend_specific: FrontendInput<'_>, + models: PlanningModels<'_>) -> UserInput<'_> +FrontendInput::{Sql { catalog: &SqlCatalog }, Promql { histograms: Option }, Metricsql} + +// A caller that already holds lowered roots calls the stages directly: +asap_aware_mapping::run_stage1_to_stage3( + roots: Vec<(Id, QueryRoot)>, + entries: &[QueryWorkloadEntry], // one per root: accuracy, latency bound, recurrence + data: &DataWorkload, + models: PlanningModels<'_>, +) -> Result, PerComputationCandidateError> +``` + +| Input or output | Meaning | +| --- | --- | +| `PlanningModels::builtin()` | Built-in accuracy model and no extra evidence. `with_plan_selection(&settings)` sets the deployment's `PlanSelectionSettings` | +| `PlanSelectionSettings` (`asap_aware_mapping::stage3_plan_selection`) | `cost_model`, `resource_speed` (`ms_per_cpu_op`, `ms_per_scanned_byte`) and `non_repeating_query_horizon_seconds` (default 3,600) | +| `CostModel::UsageOverTime(ResourceUnitCost)` | Cost model 1 (default, `ResourceUnitCost::illustrative()`): unit cost × usage per second; the cheapest valid candidate wins | +| `CostModel::ProvisionedCapacity(ResourceUnitCost)` | Cost model 2: unit cost × peak memory, peak CPU cores and disk; the cheapest valid candidate wins | +| `CostModel::ClusterCapacity(ClusterLimits)` | Cost model 3: memory, disk and optional CPU cores are hard limits; `ClusterObjective::{LowestLatency, LeastMemory, ParetoFront}` chooses among the candidates that fit | +| `PlanOutput.plans` | One `QueryPlan { entry_index, root }` per query, in `QueryWorkload::entries()` order | +| `PlanOutput.selection` | Stage 3's `Selection`: `selected`, `costs` (each candidate's `CandidateCost` with its `ResourceUsage` and `cost_per_second`, `None` under cost model 3), `accepted`, `rejected` (with reasons) and `front` | +| `PlanOutput::execution_timed_dag()` | The planned queries as one `PhysicalASAPDAG`, timed with the default `MaterializationAssignment` (see [Export and explain](#export-and-explain)) | + +Each candidate's `ResourceUsage` holds its usage without prices: +`cpu_ops_per_second`, `scanned_bytes_per_second`, `average_memory_bytes`, +`peak_memory_bytes`, `disk_bytes`, `peak_cpu_cores` and `latency_per_query`. +The deployment's choice of cost model reads it. + +Example (add `asap-planner` and an async runtime such as `tokio` to the +dependencies): + +```rust +use asap_planner::{e2e_plan, FrontendInput, PlanningModels, UserInput}; +use asap_types::types::AccuracyTarget; +use asap_types::workload::{ + BatchEntry, DataArrival, DataWorkload, DurationMs, MetricType, PlanningWorkload, Query, + QueryLanguage, QueryRequirements, QueryWorkload, SourceFacts, +}; + +#[tokio::main] +async fn main() -> Result<(), Box> { + let workload = PlanningWorkload { + query_workload: QueryWorkload { + language: QueryLanguage::PromQL, + query_batch: Some(vec![BatchEntry { + query: Query("topk by (job) (10, sum_over_time(http_requests_total[1m]))".into()), + requirements: QueryRequirements { + accuracy: AccuracyTarget::max_error(0.01).failure_probability(0.001), + max_latency_ms: Some(100.0), + }, + predictability: Default::default(), + invocations: 1, + execute_at: None, + }]), + repeating_queries: None, + }, + // What is known about each source the queries read. + data_workload: Some(DataWorkload { + arrival: DataArrival::AtRest, + sources: [( + "http_requests_total".to_string(), + SourceFacts { + series_count: Some(1_000_000), + metric_type: Some(MetricType::Counter), + sample_interval: Some(DurationMs(15_000)), + ..Default::default() + }, + )] + .into(), + }), + }; + let input = UserInput::new( + &workload, + FrontendInput::Promql { histograms: None }, + PlanningModels::builtin(), + ); + let output = e2e_plan(input).await?; + let selection = output.selection.as_ref().expect("Stage 3 selected a plan"); + println!( + "selected {} at {:?} per second", + selection.selected, selection.costs[&selection.selected].cost_per_second + ); + for rejection in &selection.rejected { + println!("{}: {}", rejection.id, rejection.reason); + } + let dag = output.execution_timed_dag()?; + println!("{} physical nodes", dag.nodes.len()); + Ok(()) +} +``` + +It prints `selected P1 at Some(0.000313...) per second`, then the two +rejected heap top-k candidates, `P2` and `P3`, each with the reason +`q1: the output has no guarantee, ...`. To plan under cost +model 3 instead: + +```rust +use asap_aware_mapping::stage3_plan_selection::{ + ClusterLimits, ClusterObjective, CostModel, PlanSelectionSettings, +}; + +let settings = PlanSelectionSettings { + cost_model: CostModel::ClusterCapacity(ClusterLimits { + memory_bytes: 64 << 30, + disk_bytes: 1 << 40, + cpu_cores: None, + objective: ClusterObjective::LowestLatency, + }), + ..Default::default() +}; +let models = PlanningModels::builtin().with_plan_selection(&settings); +``` + +The `stage_pipeline` devtool runs the same stages and writes every stage's +candidates as one JSON document; see the +[Stage Viewer guide](../user_guide_docs/stage-viewer.md). ## Generate and rank candidates @@ -161,7 +299,7 @@ search_workload_with_targets<'s, Id>( roots: Vec<(Id, Rc, Option)>, strategies: &[Box], accuracy_model: &dyn AccuracyModel, -) -> CandidateLogicalASAPDAGs +) -> Result, SearchError> CandidateLogicalASAPDAGs::cost_sorted(&self, cost_model: &dyn CostModel) -> Vec> @@ -205,7 +343,7 @@ use asap_aware_mapping::{ use asap_types::types::AccuracyTarget; fn main() -> Result<(), Box> { - let accuracy = AccuracyTarget::Epsilon(0.01); + let accuracy = AccuracyTarget::max_error(0.01); let workload = PlanningWorkload { query_workload: QueryWorkload { language: QueryLanguage::PromQL, @@ -234,7 +372,7 @@ fn main() -> Result<(), Box> { vec![("q1", root, Some(accuracy))], &strategies, &DefaultAccuracyModel, - ); + )?; for group in space.cost_sorted(&cost_model) { for (candidate, cost) in group.candidates.iter().zip(&group.costs) { println!("candidate={candidate:?}, reported_cost={cost:?}"); @@ -279,8 +417,8 @@ Callers do not apply `with_series_identity` themselves. Compile each with `promql_rows::compile_current_series_evaluation`; other queries keep their previous inventory. `global_selection` never commits these candidates; the backend compiles and prices them. CandidateLogicalASAPDAGs lists no placement variants: node timing -comes from a `MaterializationAssignment` (all query time until Stage 2 -materialization, #509, decides otherwise). +comes from a `MaterializationAssignment` (all query time for the legacy search; +Stage 2 materialization chooses one per physical candidate). ## Choose strategies and models @@ -330,9 +468,9 @@ replacement::default_strategies_with_evidence<'a>( | Factory | Use when | Models used | | --- | --- | --- | -| `default_strategies()` | Exploring with built-in defaults | Built-in cost/accuracy/allocation defaults | -| `default_strategies_with(&model)` | Supplying deployment-specific costing/sizing | Supplied cost model; default accuracy/allocation | -| `default_strategies_with_evidence(&model, &evidence)` | Supplying planning-time accuracy evidence as well | Supplied cost and evidence; default accuracy/allocation | +| `default_strategies()` | Exploring with built-in defaults | Built-in cost/accuracy defaults | +| `default_strategies_with(&model)` | Supplying deployment-specific costing/sizing | Supplied cost model; default accuracy | +| `default_strategies_with_evidence(&model, &evidence)` | Supplying planning-time accuracy evidence as well | Supplied cost and evidence; default accuracy | | Explicit vector | Controlling which context-free strategies are supplied | Models passed into each constructor | ### Example: supply two strategies and run search @@ -350,7 +488,7 @@ use asap_aware_mapping::{ use asap_types::types::AccuracyTarget; fn main() -> Result<(), Box> { - let accuracy = AccuracyTarget::Epsilon(0.01); + let accuracy = AccuracyTarget::max_error(0.01); let workload = PlanningWorkload { query_workload: QueryWorkload { language: QueryLanguage::PromQL, @@ -380,7 +518,7 @@ fn main() -> Result<(), Box> { ]; let space = search_workload_with_targets( vec![("q1", root, Some(accuracy))], &strategies, &DefaultAccuracyModel, - ); + )?; println!("{:#?}", space.cost_sorted(&model)); Ok(()) } @@ -401,7 +539,6 @@ Module-qualified paths below are relative to `asap_aware_mapping`. | `&dyn CostModel` | `empirical_cost::EmpiricalCostModel::new(provider)` | Offline sketch-benchmark model: ranks algorithms using matching offline measurements | | `&dyn CostModel` | `physical_plan_cost_model::PhysicalPlanCostModel::new(&provider, calibration)?` | Deployment-specific physical-plan model: compares complete physical alternatives using provider evidence and resource calibration; evidence may be offline or online | | `&dyn AccuracyModel` | `DefaultAccuracyModel` | Built-in guarantee rules and satisfaction checks | -| `&dyn AccuracyBudgetAllocator` | `EqualSplitAllocator` | Built-in allocation of composition accuracy budgets | | `&dyn AccuracyEvidenceProvider` | `NoAccuracyEvidence` | No extra planning-time statistics; evidence-dependent claims remain unavailable | | Any provider trait above | Your implementation | Supplies alternative models/evidence under the same contracts | @@ -429,18 +566,17 @@ accuracy guarantees. ```rust use asap_aware_mapping::{ - DefaultAccuracyModel, DefaultCostModel, EqualSplitAllocator, + DefaultAccuracyModel, DefaultCostModel, NoAccuracyEvidence, ReplacementStrategy, ASAPStrategies, }; fn main() { let cost = DefaultCostModel; let accuracy = DefaultAccuracyModel; - let allocation = EqualSplitAllocator; let evidence = NoAccuracyEvidence; let strategies: Vec> = vec![Box::new( ASAPStrategies::new_with_planning_inputs_and_evidence( - &cost, &accuracy, &allocation, &evidence, + &cost, &accuracy, &evidence, ), )]; // Use &strategies and &accuracy in search_workload_with_targets. @@ -454,28 +590,26 @@ Constructor definition: ASAPStrategies::new_with_planning_inputs_and_evidence( cost_model: &dyn CostModel, accuracy_model: &dyn AccuracyModel, - allocator: &dyn AccuracyBudgetAllocator, evidence: &dyn AccuracyEvidenceProvider, ) -> ASAPStrategies ``` All provider arguments are required for this constructor. They must outlive the strategy vector. `ASAPStrategies::new(&cost_model)` is the shorter -constructor using default accuracy/allocation and no extra evidence. +constructor using the default accuracy model and no extra evidence. | Extension point | What it controls | What it cannot establish alone | | --- | --- | --- | | `ReplacementStrategy` | Proposed semantic alternatives | Permission to violate query semantics or downstream support | | `CostModel` | Candidate ordering/sizing hooks and recurrence cost hooks | Correctness, measured costs without evidence, or installed runtime support | | `AccuracyModel` | Derivation, propagation and satisfaction of guarantees | A meaningful guarantee without its required assumptions/evidence | -| `AccuracyBudgetAllocator` | Local accuracy requirements proposed within composition | End-to-end correctness without subsequent validation | | `AccuracyEvidenceProvider` | Planning-time statistics used by supported strategies | Authority to change query requirements | Models may be consumed during generation as well as ranking. Construct strategies with the intended model/evidence; replacing only the final sorting model does not regenerate parameter choices. For evidence-aware defaults, use `asap_aware_mapping::replacement::default_strategies_with_evidence`. -For custom accuracy/allocation/evidence on sketches, +For custom accuracy/evidence on sketches, `ASAPStrategies::new_with_planning_inputs_and_evidence` exposes these providers. Keep each provider's evidence scope and freshness valid for the query population. @@ -483,8 +617,8 @@ Keep each provider's evidence scope and freshness valid for the query population `PlanningWorkload` holds `QueryWorkload` and optional `DataWorkload` as peer inputs. `QueryWorkload` contains the language and optional batch/repeating -entries. Entries carry requirements, predictability, recurrence and time -selection. These facts are separate: repeated queries can read data at rest. +entries. Entries carry requirements, predictability and recurrence; a +query's window comes from its text (range and offset). These facts are separate: repeated queries can read data at rest. `DataWorkload` holds `arrival` (`DataArrival`: `AtRest`, `Ingesting` or `Mixed` with `rows_per_second`, or `Unknown`) and `sources`, a map from PromQL metric or SQL table name to `SourceFacts`: `series_count`, @@ -551,7 +685,7 @@ constructs the selected semantic DAG while preserving shared nodes. | `global_selection()` | Which compatible choices should be used together, accounting for sharing and dependencies? | A coordinated selection across targets under the supplied model | Plain `global_selection()` does not decide materialization or establish -physical deployment feasibility. Stage 2 materialization (#509) will own +physical deployment feasibility. Stage 2 materialization owns materialization; downstream still owns physical commitment. | Method on `CandidateLogicalASAPDAGs` / `GlobalSelection` | Behavior | @@ -575,7 +709,7 @@ GlobalSelection::assemble_selected_dag(&self, target: &Rc) ``` For structural inspection only, this complete example selects a semantic root -and exports its inspection DAG. It performs no materialization or deployment +and prints it. It performs no materialization or deployment planning. ```rust @@ -610,12 +744,11 @@ fn main() -> Result<(), Box> { )), }; let root = lower_promql_workload(&workload)?.remove(0); - let space = search_workload(vec![("q1", root)]); + let space = search_workload(vec![("q1", root)])?; let selection = space.global_selection(&DefaultCostModel); // Search may canonicalize roots; use the root returned by CandidateLogicalASAPDAGs. - if let Some(summary) = selection.assemble_selected_dag(&space.roots[0].1)? { - let dag = asap_types::dag_export::export_summary(&summary); - println!("{dag:#?}"); + if let Some(selected) = selection.assemble_selected_dag(&space.roots[0].1)? { + println!("{selected:#?}"); } Ok(()) } @@ -625,14 +758,12 @@ fn main() -> Result<(), Box> { | Function/type | Purpose | | --- | --- | -| `asap_types::dag_export::export(&query)` | Pre-ASAP inspection dag | -| `asap_types::dag_export::export_summary(&summary)` | Post-ASAP inspection dag | | `asap_types::ir::apply_materialization_timings(&root, &assignment, &mut TimingMemo::new())` | Write execution timing into every node from a `MaterializationAssignment` (default: all query time) and validate the data-state edges; `PlanOutput::execution_timed_dag()` applies the default to a planned workload | | `asap_types::ir::physical_export::compile_physical_asap_dag(&timed_root)` | Export a timed DAG as a `PhysicalASAPDAG` (`PHYSICAL_ASAP_DAG_WIRE_VERSION`); rejects an untimed node; `compile_physical_asap_workload(&roots)` exports a batch with one root per query | | `PhysicalASAPDAGDocument::new(dag)` and `.validate()` | Versioned semantic envelope and explicit validation; constructing it alone does not validate | | `explain_replacements` / `explain_replacements_with` | Findings from default/custom-strategy search; not a complete physical feasibility report | -Choose the export matching your intended handoff: an inspection DAG is not +Choose the export matching your intended handoff: an untimed semantic DAG is not interchangeable with a versioned execution contract. Preserve cost/guarantee evidence needed downstream instead of exporting only a bare DAG. For public symbol details, build local API documentation with: @@ -644,6 +775,7 @@ cargo doc -p asap-aware-mapping -p asap-types --no-deps ## Source references - [Frontend PromQL](../../crates/frontend-promql/src/lib.rs), [SQL](../../crates/frontend-sql/src/lib.rs), [MetricsQL](../../crates/frontend-metricsql/src/lib.rs) +- [Planner facade](../../crates/planner/src/lib.rs), [planning stages](../../crates/asap-aware-mapping/src/pass/stage1_to_stage3.rs), [plan selection and cost models 1–3](../../crates/asap-aware-mapping/src/stage3_plan_selection.rs) - [Search, ranking and selection](../../crates/asap-aware-mapping/src/replacement.rs) - [Cost models](../../crates/asap-aware-mapping/src/cost_model.rs) - [Workload types](../../crates/types/src/workload.rs) diff --git a/docs/develop_docs/offline-sketch-evidence.md b/docs/develop_docs/offline-sketch-evidence.md index 4a6d0c2ca..40d1d1e37 100644 --- a/docs/develop_docs/offline-sketch-evidence.md +++ b/docs/develop_docs/offline-sketch-evidence.md @@ -102,6 +102,10 @@ parameters solely because one dataset had low observed error. ## Query-matched offline recommendations +> **Removed.** The `empirical_comparison` module that implemented this section +> has been deleted, and nothing in the planner reads `OfflineComparisonEvidence` +> now. The section is kept as the record of its contract. + `empirical_comparison::recommend_offline` consumes the companion [`OfflineComparisonEvidence` JSON format](offline-comparison-evidence.schema.json). This combines the sketch artifact with explicit query bindings and a separately diff --git a/docs/develop_docs/physical-compile-coverage.md b/docs/develop_docs/physical-compile-coverage.md index 8db8cc63e..3f39a094b 100644 --- a/docs/develop_docs/physical-compile-coverage.md +++ b/docs/develop_docs/physical-compile-coverage.md @@ -6,8 +6,7 @@ Audience: developers moving computation from ASAPQuery-backend into ## Contract Logical selection decides what to compute. A `MaterializationAssignment` sets -node timing (all query time until Stage 2 materialization, #509, decides -otherwise). `physical_planner::compile` turns a timed `PhysicalASAPDAG` into physical +node timing (Stage 2 materialization chooses it per physical candidate). `physical_planner::compile` turns a timed `PhysicalASAPDAG` into physical operator DAGs. The backend owns ingestion, panes, storage, stored-state evaluation, external exact engines, pricing/selection, and execution scheduling. diff --git a/docs/develop_docs/physical-handoff-costs.md b/docs/develop_docs/physical-handoff-costs.md index dcd95e747..ae9a6db0b 100644 --- a/docs/develop_docs/physical-handoff-costs.md +++ b/docs/develop_docs/physical-handoff-costs.md @@ -12,8 +12,8 @@ is distinct from CPU work, scanned bytes, and stored byte occupancy; it is not collapsed into the generic CPU/byte resource container. The physical-plan adapter accepts an optional `PhysicalHandoffProfile` in the -immutable `PhysicalEvidenceSnapshot`. `dag_export --planner-cost-json` accepts -the same profile in a top-level `boundaries` field. With no profile, these +immutable `PhysicalEvidenceSnapshot`. No devtool reads the profile; a caller +supplies it through the library. With no profile, these dimensions remain unestimated and the existing resource objective is preserved. The profile's `plans` list binds handoffs to complete physical alternatives. @@ -87,7 +87,7 @@ traffic is inferred from logical edges, operator buffers, or scan bytes. Unknown endpoints, mismatched payloads, absent node evidence, duplicate IDs, stale evidence, invalid coefficients, and integer overflow return typed errors; -ranking/export report the comparison as unavailable. This extends the physical +ranking reports the comparison as unavailable. This extends the physical plan adapter; summary-maintenance costing for Stage 2 materialization (#509) is a separate follow-up integration point. @@ -95,6 +95,4 @@ Verification: ```sh cargo test -p asap-aware-mapping --test physical_handoff_cost -cargo test -p asap-devtools --bin dag_export handoff_bytes_export_and_change_plan_selection -python3 -m unittest discover -s tools/dag-viewer -p 'test_render.py' ``` diff --git a/docs/develop_docs/pre-asap-ir.md b/docs/develop_docs/pre-asap-ir.md index 51d79dc2a..35ce46c3c 100644 --- a/docs/develop_docs/pre-asap-ir.md +++ b/docs/develop_docs/pre-asap-ir.md @@ -32,8 +32,9 @@ pub struct OperatorNode { pub operator: Operator, // NonASAP(NonASAPOp) | ASAP(ASAPOp) pub result_kind: OperatorResultKind, // Relation | InstantVector | RangeVector | State | Scalar pub schema: Schema, // output schema, derived at construction - pub guarantee: Option, // None until accuracy assessment establishes one + pub guarantee: Option, // None until accuracy assessment establishes one pub timing: Option, // None until a materialization assignment is applied + pub materialization: Materialization, // NotMaterialized until Stage 2 assigns one } ``` @@ -49,6 +50,9 @@ pub struct OperatorNode { - `timing` is `None` in every front-end DAG and every candidate. It is written by `ir::timing::apply_materialization_timings` (see the Post-ASAP IR document); export rejects an untimed node. +- `materialization` is `NotMaterialized` in every front-end DAG. Stage 2 writes it, with + `timing`, through `ir::timing::apply_materialization_timings` from its materialization + assignment. `OperatorNode::children()` returns the operator's inputs in field order followed by the operator nodes its scalar expressions read (see "Scalar expressions"). Every DAG traversal — @@ -295,7 +299,7 @@ first, reporting single-column cardinality as tuple cardinality. Realization is the single-column one with a wider item: a tuple becomes a `SummaryInputExpr::Tuple`, which the distinct-count sketches (HLL, KMV) hash as one value. UnivMon is withheld from a tuple — it estimates frequency -moments over a single value stream. At `AccuracyTarget::Exact` the node stays a +moments over a single value stream. At `AccuracyTarget::exact()` the node stays a logical pass-through at any arity. Each SQL argument must be a bare column, qualifier preserved so a tuple over a join resolves to the correct side; an expression argument is rejected rather than reduced over a probe column. diff --git a/docs/develop_docs/storage-operation-costs.md b/docs/develop_docs/storage-operation-costs.md index 999b79103..30e27f442 100644 --- a/docs/develop_docs/storage-operation-costs.md +++ b/docs/develop_docs/storage-operation-costs.md @@ -1,8 +1,8 @@ # Storage operation estimates The physical-plan ranking adapter accepts an optional `StorageIoProfile` in -its immutable `PhysicalEvidenceSnapshot`. `dag_export --planner-cost-json` -accepts the same profile in the document's top-level `storage_io` field. +its immutable `PhysicalEvidenceSnapshot`. No devtool reads the profile; a +caller supplies it through the library. Omitting the profile preserves the existing CPU/memory/scan-byte objective; operation counts are unestimated, not inferred to be zero. @@ -84,5 +84,4 @@ Verification: ```sh cargo test -p asap-aware-mapping --test storage_io cargo test -p asap-aware-mapping --lib storage_io -cargo test -p asap-devtools --bin dag_export storage_requests_export_and_change_plan_selection ``` diff --git a/docs/develop_docs/target-candidate-api-migration.md b/docs/develop_docs/target-candidate-api-migration.md index ea90fe328..e9c9e5436 100644 --- a/docs/develop_docs/target-candidate-api-migration.md +++ b/docs/develop_docs/target-candidate-api-migration.md @@ -25,7 +25,7 @@ counterpart) are prerequisites, not additional changes here. The workflow remains one selection call per workload followed by one assembly call per query root. The summary-maintenance lifecycle API named above was later -removed; Stage 2 materialization (#509) will own maintenance decisions. +removed; Stage 2 materialization (#509) owns maintenance decisions. ## Later: unified operator IR (operator flattening) From ec7d5117189d655aec9eb3ca0f748d50c71767e6 Mon Sep 17 00:00:00 2001 From: zzylol <50204836+zzylol@users.noreply.github.com> Date: Fri, 16 Oct 2026 17:29:25 +0000 Subject: [PATCH 03/10] docs(design): planner-layering examples use the current DataWorkload and stage results The shared and Example 2 data tables use arrival and sources[...] facts, lookback and as_of are marked as query-text columns, and Stages 2 and 3 say what is implemented (materialization options, resource usage and cost models 1-3). The acceptance docs carry the counts, costs and test statuses that stage_pipeline and the tests give at this head. --- .../planner-layering-example1-acceptance.md | 103 ++++++++------ .../planner-layering-example2-acceptance.md | 131 ++++++++++-------- .../planner-layering-example3-acceptance.md | 35 ++--- .../planner-layering-example4-acceptance.md | 44 +++--- .../design_docs/proposals/planner-layering.md | 66 ++++++--- 5 files changed, 215 insertions(+), 164 deletions(-) diff --git a/docs/design_docs/proposals/planner-layering-example1-acceptance.md b/docs/design_docs/proposals/planner-layering-example1-acceptance.md index 00c6d14b0..50153e5d9 100644 --- a/docs/design_docs/proposals/planner-layering-example1-acceptance.md +++ b/docs/design_docs/proposals/planner-layering-example1-acceptance.md @@ -17,7 +17,7 @@ whose expectation needs a feature that is not in the stack yet is |---|---|---| | Query-language frontends | 1 workload `LogicalDAG` | same | | Logical ASAP-aware optimization | Pass 1 (3) × identical-expression rule (2) × window-composition rule (3 × 3) = **54** | Pass 1 (3), each with 2 identical-expression candidates and 6 window-composition candidates (Q1's rate and Q2's option in six 10-s tumbling panes, alone or both, each also with a shared scan) = **27**. Sliding windows are not implemented. | -| Physical ASAP-aware optimization | Materialization options per window form = **156** | Physical operator implementation, every sub-DAG `NotMaterialized` = **27** (the doc's Raw plans, with panes rebuilt at every refresh). Materialization enumeration is #604. | +| Physical ASAP-aware optimization | Materialization options per window form = **156** | Physical operator implementation, plus materialization: each logical candidate with every sub-DAG `NotMaterialized` (27: the doc's Raw plans, with panes rebuilt at every refresh), and each pane chain also `MatAtIngestion(Memory)` (30 more) = **57** | | Plan selection | 1 plan | 1 plan | Every MVP candidate is one of the doc's candidates: in logical ASAP-aware @@ -27,10 +27,11 @@ query; in physical ASAP-aware optimization, the one where both queries are ## Workload -Shared data workload: `continuously_ingesting`, 15 s ingestion interval, -1,000,000 series, about 66,667 samples/s, `zipf`, volume unknown. +Shared data workload: `arrival` = `Ingesting { rows_per_second }`, about +66,667 samples/s; `sources["http_requests_total"]` has `series_count` = +1,000,000, `sample_interval` = 15 s and `metric_type` = counter. -| Query | Repeats | `lookback` | `as_of` | Accuracy | Latency | +| Query | Repeats | lookback | as_of | Accuracy | Latency | |---|---|---|---|---|---| | Q1 `sum by (job) (rate(http_requests_total[1m]))` | 10 s | 1 m | evaluation time | exact | none | | Q2 `topk by (job) (10, sum_over_time(http_requests_total[1m]))` | 10 s | 1 m | evaluation time | ε = 0.01, δ = 0.001 | ≤ 100 ms | @@ -75,11 +76,14 @@ Invariants: * Every candidate covers both queries. * Q1 never reaches a summary node. * Every target's exact option is its exact accumulator (`SummaryAgg` with an - exact family → `FinalizeExactAccumulator`): three per candidate. + exact family → `FinalizeExactAccumulator`): three per candidate with exact + Q2, two with a heap Q2 (Q1's rate and sum). Q2's heaps are whole-expression + heaps, Q2 top-*k*'s fused computation: they absorb its `sum_over_time`, + which is then not computed. * Q2's sketch families are exactly {`CmsWithHeap`, `CountSketchWithHeap`}, each per subpopulation (job). Exact Q2 has no sketch. * Only the scan and range nodes may be shared between the queries. No summary - is shared. (Today nothing is shared.) + is shared. * Nothing is pruned: the heap sketches are admitted by ε = 0.01, δ = 0.001, and Q1's exact target admits only the exact option. @@ -87,10 +91,8 @@ Ignored until their feature lands: * Hydra (`HydraCms` over all (job, series) keys, a heap per job) as a third Q2 summary (#605). -* A shared-input variant of every Q2 option next to the independent one - (Pass 2 identical-expression rule, #588). -## Physical ASAP-aware optimization: 27 candidates +## Physical ASAP-aware optimization: 57 candidates The three whole-window candidates without a shared input: @@ -105,56 +107,64 @@ accumulator, each finalized. Invariants: -* Exactly one physical candidate per logical candidate, and none pruned. - `from_logical` is a bijection onto the logical ids, so no candidate is - dropped before plan selection. +* Every logical candidate keeps its all-`NotMaterialized` physical candidate, + and none is pruned, so no candidate is dropped before plan selection. A + logical candidate with `n` pane chains has 2ⁿ physical candidates: each + chain `NotMaterialized` or `MatAtIngestion(Memory)` (ids `P-m`). + Only pane chains are placed at ingestion time; Q1's rate panes may be, + because continuously ingesting PromQL data is taken to arrive in time + order per series. * Each physical candidate keeps its logical Q2 option and input sharing. * Exact top-*k* is implemented as a sort followed by a limit: each job's series sorted by the ranked value, descending, then a `Project` that restores the logical root columns `[job, topk_10]`. * A summary Q2 is a sketch build node feeding an estimation node. There is no merge node, because there are no window summaries. -* Every node is `NotMaterialized` and runs at query time. +* In the whole-window candidates, every node is `NotMaterialized` and runs + at query time. Runtime capability check (not part of the doc): every candidate compiles in -the physical planner. Ignored on two runtime gaps: CMS + heap needs a -non-negative weight contract (#593), and the CountSketch + heap estimate's -output is not the keyed-evaluation shape the physical planner builds (#579). +the physical planner. Ignored on two runtime gaps: a candidate's CMS + heap +weights are an open requirement that only the selected plan's confirmation +answers, and the heaps' top-*k* estimate output is not the keyed-evaluation +shape the physical planner builds (#579). ## Plan selection: 1 plan | Id | Outcome | Reason | |---|---|---| -| P1 | selected | the only valid candidate | -| P2 | invalid | Count-Min needs non-negative update weights, and `sum_over_time` weights are not proven non-negative | -| P3 | invalid | a top-*k* output has no guarantee (a heap's bound is per item frequency, not top-*k* membership), and the deployment has no evidence that it meets the target | +| P1 | selected | all exact, at 7.94 per second (cost model 1) | +| P2, P3 | valid · costlier | the shared-scan and shared-scan-and-range candidates of P1; they tie with it up to rounding and lose on enumeration order | +| P10 | invalid | Count-Min + heap: a top-*k* output has no guarantee (a heap's bound is per item frequency, not top-*k* membership), and the deployment has no evidence that it meets the target | +| P19 | invalid | CountSketch + heap: the same reason | + +The 38 heap candidates (whole window or panes, rebuilt or maintained) are +invalid for that reason; the other 18 exact candidates are valid but +costlier. Count-Min's weights are the raw samples of `http_requests_total`, +which the workload declares a counter, so they are proven non-negative; +without that declaration Count-Min is also rejected on its weights. Exact +panes rebuilt at every refresh cost 8.54, and maintaining a query's exact +panes at ingestion time costs 12.94. Invariants: * One selected id. Every other candidate is listed once as rejected, with a reason, and marked invalid or valid but costlier. * Only valid candidates are priced. Each valid candidate's cost has one entry - per DAG node, and `total` is their sum. A materialized node is charged + per DAG node, and `cost_per_second` is their sum. A materialized node is charged once. A `NotMaterialized` node is charged once per consumer that reads it (its multiplier), where a consumer is a query or a materialized node and - a read stops at materialized nodes. Every node is `NotMaterialized` today; + a read stops at materialized nodes. `stage3_plan_selection_charging_helper_*` check the helper on a hand-built candidate with a materialized node. * The selected plan is no more expensive than any valid candidate. * The selected plan compiles in the physical planner. +* For each Q2 option, the shared-input candidate costs no more than its + separate counterpart. -Ignored until Pass 2 (#588): for each Q2 option, the shared-input candidate -costs no more than its separate counterpart. - -When #588's shared-input candidates land, each Q2 option has two -candidates. The tests already count only candidates whose queries share no -node (Pass 1's 3 logical and 3 physical), pick a Q2 option's candidate by -(option, no shared input), and check only that every invalid candidate is a -heap one, not that exactly two are invalid, so they need no change. - -The doc's typical winners need window forms or materialization, and its -"Raw with a shared input" needs Pass 2. Latency bounds are not checked, so -whether an exact Q2 misses 100 ms is not tested. +The doc's typical winners need summaries that plan selection can accept for +Q2, or materialization that pays off; neither happens here. The latency +check runs: P1's Q2 takes about 46 ms per evaluation, within its 100 ms. ## Where the doc and the planner differ @@ -163,17 +173,19 @@ whether an exact Q2 misses 100 ms is not tested. comes in #605. The tests follow the planner. 2. **Count-Min + heap is invalid.** The doc expects the accuracy model to judge it. Plan selection admits Count-Min only over weights proven - non-negative, which `sum_over_time` over an arbitrary metric is not - (counter declarations: #593). + non-negative; the workload declares `http_requests_total` a counter + (`sources[metric].metric_type`), which proves them. It still rejects the + candidate, because a top-*k* output has no guarantee (item 3). 3. **No summary is selectable.** The doc expects a summary for Q2 to win in most deployments. A composed top-*k* estimate has no guarantee, so without deployment evidence every heap candidate is invalid and the exact plan is selected. 4. **Physical ASAP-aware optimization is materialization in the doc's example.** Example 1 lists only window-form and materialization options. - The MVP reads "physical operator implementation" from the physical - ASAP-aware optimization section ("TopK as a sort followed by a limit"), - which gives one physical candidate per logical candidate. + The planner also reads "physical operator implementation" from the + physical ASAP-aware optimization section ("TopK as a sort followed by a + limit"), and enumerates materialization only for summary builds: each + pane chain is `NotMaterialized` or `MatAtIngestion(Memory)`. 5. **Where exact top-*k* becomes sort + limit.** The Pass 1 diagram draws exact Q2 as "sort + limit 10 per job", but the frontend emits `Aggregate{[TopK]}` and the physical ASAP-aware optimization section calls @@ -184,9 +196,14 @@ whether an exact Q2 misses 100 ms is not tested. 7. **"Shared summary charged once"** does not apply literally: the doc says no summary is shared in Example 1. Plan selection charges a `NotMaterialized` node once per consuming query and a materialized one - once; the tests check that rule. A shared input therefore saves nothing - until it is materialized (#604). + once; the tests check that rule. Stage 2 materializes only summary + builds, so a shared input saves nothing here. 8. **Latency.** The doc says an exact top 10 rebuilt at every refresh "may - miss" 100 ms. Plan selection does not check latency yet. -9. **Accuracy of Count-Min heap merges** matters only with tumbling windows, - which are out of MVP scope. + miss" 100 ms. With the illustrative calibration it takes about 46 ms, so + the exact plan meets the bound. +9. **Accuracy of Count-Min heap merges.** The heap panes are invalid like + their whole-window builds, so the approximate merge is never judged. +10. **Input sharing in Pass 2.** The doc gives each Pass 1 candidate "2 input + choices (separate or shared)". Following Algorithm 2 line 31 (one + candidate per non-empty subset of the sharing options), the planner + gives 3: separate, shared scan, and shared scan and range selector. diff --git a/docs/design_docs/proposals/planner-layering-example2-acceptance.md b/docs/design_docs/proposals/planner-layering-example2-acceptance.md index b204b457c..f69473929 100644 --- a/docs/design_docs/proposals/planner-layering-example2-acceptance.md +++ b/docs/design_docs/proposals/planner-layering-example2-acceptance.md @@ -14,36 +14,40 @@ needs a missing feature is `#[ignore]`d with the PR that delivers it. ## Workload -Three SQL dashboard panels over `flows`, every 10 s, `lookback` 1 m, `as_of` +Three SQL dashboard panels over `flows`, every 10 s, lookback 1 m, as_of evaluation time, no latency requirement. | Query | Computation | SQL (abridged) | Accuracy | |---|---|---|---| | Q1 | `Distinct(src_ip)` | `SELECT COUNT(DISTINCT src_ip) FROM flows WHERE ts >= now() - INTERVAL '1 minute'` | ε = 0.02, δ = 0.01 | | Q2 | `Entropy(src_ip)` | `SELECT -SUM(p * LN(p)) FROM (SELECT COUNT(*) * 1.0 / SUM(COUNT(*)) OVER () AS p … GROUP BY src_ip)` | ε = 0.05, δ = 0.01 | -| Q3 | `L2(src_ip)` | `SELECT SQRT(SUM(c * c)) FROM (SELECT src_ip, COUNT(*) AS c … GROUP BY src_ip)` | ε = 0.01, δ = 0.01 | - -Data workload: the shared one (`continuously_ingesting`, about 66,667 -samples/s, `zipf`, volume unknown) with `input_cardinality` = 10,000,000 and no -`data_ingestion_interval`. The catalog is `flows(ts, src_ip)`; the doc names -no other column. - -**Stand-in.** The SQL frontend lowers Q1 to a distinct count but Q2 and Q3 to -exact relational plans (the doc's own TODO; #598). The planning stages -therefore run on a PromQL stand-in with the same structure, three statistics -of one input over the same 1-min window: `distinct_over_time(flows_src_ip[1m])`, +| Q3 | `L2(src_ip)` | `SELECT SQRT(SUM(CAST(c AS DOUBLE) * CAST(c AS DOUBLE))) FROM (SELECT src_ip, COUNT(*) AS c … GROUP BY src_ip)` | ε = 0.01, δ = 0.01 | + +Data workload: the shared arrival (`Ingesting`, about 66,667 samples/s) with +`sources["flows"].series_count` = 10,000,000 and no `sample_interval` (SQL +needs none). The catalog is `flows(ts, src_ip)`; the doc names no other +column. Q3 casts its counts to DOUBLE: the doc's Int64 `c * c` can overflow, +so the SQL frontend recognizes only the Float64 product. + +**Stand-in.** The SQL frontend lowers Q1 to a distinct count and recognizes +Q2 and Q3 as `Entropy` and `L2`. Its time filter stays a scan predicate, so +most tests run the planning stages on a PromQL stand-in with the same +structure, three statistics of one input over the same 1-min window: +`distinct_over_time(flows_src_ip[1m])`, `entropy_over_time(flows_src_ip[1m])` and `l2_over_time(flows_src_ip[1m])`, -with Q1–Q3's targets and a 15-s ingestion interval (PromQL needs one). +with Q1–Q3's targets and a 15-s `sample_interval` on `flows_src_ip`. The +`stage_pipeline --example planner-layering-2` document plans the SQL +workload itself. ## Expected candidates per stage -| Stage | Doc | Today (stand-in) | +| Stage | Doc | Planner (SQL workload, `stage_pipeline --example planner-layering-2`) | |---|---|---| -| Query-language frontends | One summary-free DAG: `Distinct`, `Entropy`, `L2` over `flows.src_ip`, last 1 m | Same for the stand-in. SQL: Q1 is `aggregate:cardinality`; Q2 and Q3 are `count` per `src_ip` → window function / projection → `sum` (not recognized). | +| Query-language frontends | One summary-free DAG: `Distinct`, `Entropy`, `L2` over `flows.src_ip`, last 1 m | Same. Q1 is a distinct count; Q2 is `Entropy` and Q3 is `L2` over the per-source counts, each with an exact population count for the empty-input NULL. | | Logical ASAP-aware optimization, Pass 1 | Each computation: exact, a specialized summary, UnivMon. 3 × 3 × 3 = **27** | Q1: exact, HLL, KMV, UnivMon (no Theta: `asap_sketchlib` has no Theta sketch). Q2, Q3: exact, UnivMon (no specialized entropy or norm summary). 4 × 2 × 2 = **16** | -| Logical ASAP-aware optimization, Pass 2 | Summary-capability rule: + 1 (one UnivMon for all three, sized for ε = 0.01) + 9 (3 pairs share a UnivMon × 3 options of the third). Independent kept. **37**. Window composition left out. | Not implemented: **20**, no candidate shares a node. | -| Physical ASAP-aware optimization | Same options as Example 1: no window form → rebuilt from raw samples each refresh. | One physical candidate per logical candidate, every sub-DAG `NotMaterialized`: **20**. | -| Plan selection | One UnivMon sized for ε = 0.01 vs three separate summaries; the shared one usually wins (one update per flow record instead of three). | HLL is invalid (its bound has no failure probability for the (ε, δ) target) and so is every UnivMon (its estimates have no guarantee). Q1 on KMV with Q2, Q3 exact is valid but costlier. The all-exact plan is selected. | +| Logical ASAP-aware optimization, Pass 2 | Summary-capability rule: + 1 (one UnivMon for all three, sized for ε = 0.01) + 9 (3 pairs share a UnivMon × 3 options of the third). Independent kept. **37**. Window composition left out. | Identical-expression rule (Q2 and Q3's shared per-source count, and the shared scan) and summary-capability rule (one UnivMon for two of the queries or all three, sized with each parameter's largest value among its readers): **82** | +| Physical ASAP-aware optimization | Same options as Example 1: no window form → rebuilt from raw samples each refresh. | Each logical candidate with every sub-DAG `NotMaterialized`, plus `MatAtQuery(Memory)` for each summary build two or more queries read: **184** | +| Plan selection | One UnivMon sized for ε = 0.01 vs three separate summaries; the shared one usually wins (one update per flow record instead of three). | 164 invalid: HLL or UnivMon for Q1 (120), whose bound has failure probability None without a distinct bound for `flows`, and UnivMon for Q2 (44), whose entropy has no guarantee. 20 valid. P1, all exact, is selected at 6.08 per second (cost model 1); KMV for Q1 costs the same to three decimals; the shared per-source count costs 7.44 rebuilt for each query and 7.04 kept `MatAtQuery`; UnivMon for Q3 costs 8.08. | ## Invariants and tests @@ -53,7 +57,7 @@ with Q1–Q3's targets and a 15-s ingestion interval (PromQL needs one). |---|---|---| | The SQL workload is valid, in entry order | `workload_encodes_example2` | passes | | SQL Q1 lowers to a distinct count over `flows` | `stage0_query_language_frontends_sql_q1_lowers_to_a_distinct_count` | passes | -| SQL Q2 and Q3 lower to `FrequencyEntropy` and `FrequencyL2` | `stage0_query_language_frontends_sql_q2_q3_lower_to_entropy_and_l2` | ignored: #598 | +| SQL Q2 and Q3 lower to `FrequencyEntropy` and `FrequencyL2` | `stage0_query_language_frontends_sql_q2_q3_lower_to_entropy_and_l2` | passes | | The stand-in lowers to the three statistics over one 1-min range | `stage0_query_language_frontends_standin_lowers_to_three_statistics_of_one_input` | passes | | One summary-free DAG; the queries share no node | `stage0_query_language_frontends_one_summary_free_dag_without_sharing` | passes | @@ -61,28 +65,30 @@ with Q1–Q3's targets and a 15-s ingestion interval (PromQL needs one). | Invariant | Test | Status | |---|---|---| -| Once recognized, the SQL workload has the stand-in's Pass 1 options | `stage1_logical_asap_optimization_sql_has_the_standin_options` | ignored: #598 (ambiguity 9) | +| Once recognized, the SQL workload has the stand-in's Pass 1 options | `stage1_logical_asap_optimization_sql_has_the_standin_options` | passes | | Each statistic has an exact and a UnivMon option | `stage1_logical_asap_optimization_pass1_offers_exact_and_univmon_for_each_statistic` | passes | | Q1 has a specialized distinct-count summary (HLL or KMV) | `stage1_logical_asap_optimization_pass1_offers_a_distinct_count_summary` | passes | | Q2 and Q3 have a specialized entropy and norm summary | `stage1_logical_asap_optimization_pass1_offers_specialized_entropy_and_l2_summaries` | ignored: no PR yet | | Every combination of the queries' own options is kept, with no shared summary | `stage1_logical_asap_optimization_keeps_every_independent_combination` | passes | -| A candidate has one UnivMon build read by all three queries, feeding a distinct-count, an entropy and an L2 estimate | `stage1_logical_asap_optimization_summary_capability_adds_one_univmon_for_all_three` | ignored: #595, #596 | -| For each pair, a candidate shares one UnivMon between the two while the third takes each of its own options | `stage1_logical_asap_optimization_summary_capability_adds_pairwise_shared_univmons` | ignored: #595, #596 | -| A shared UnivMon has the parameters Pass 1 gives its strictest consumer alone | `stage1_logical_asap_optimization_shared_univmon_is_sized_for_the_strictest_consumer` | ignored: #595, #596 | -| UnivMon is sized per target: Q3's (ε = 0.01) is larger than Q2's (ε = 0.05) | `stage1_logical_asap_optimization_univmon_is_sized_per_accuracy_target` | ignored: #621 | -| Only a UnivMon is ever shared across statistics | `stage1_logical_asap_optimization_only_univmon_is_shared_across_statistics` | passes (vacuously today) | +| A candidate has one UnivMon build read by all three queries, feeding a distinct-count, an entropy and an L2 estimate | `stage1_logical_asap_optimization_summary_capability_adds_one_univmon_for_all_three` | passes | +| For each pair, a candidate shares one UnivMon between the two while the third takes each of its own options | `stage1_logical_asap_optimization_summary_capability_adds_pairwise_shared_univmons` | passes | +| A shared UnivMon takes each parameter's largest value among its consumers, each sized alone | `stage1_logical_asap_optimization_shared_univmon_takes_each_parameters_largest_own_size` | passes | +| UnivMon is sized per target: Q3's (ε = 0.01) is larger than Q2's (ε = 0.05) | `stage1_logical_asap_optimization_univmon_is_sized_per_accuracy_target` | passes | +| Only a UnivMon is ever shared across statistics | `stage1_logical_asap_optimization_only_univmon_is_shared_across_statistics` | passes | +| Stage 1 does not read the distinct bound; Q1's HLL failure probability stays symbolic in the source's bound | `stage1_logical_asap_optimization_leaves_the_hll_failure_probability_symbolic` | passes | | Candidates are valid DAGs with unique ids | `stage1_logical_asap_optimization_candidates_are_valid_and_uniquely_named` | passes | ### Physical ASAP-aware optimization | Invariant | Test | Status | |---|---|---| -| One physical candidate per logical candidate, none pruned | `stage2_physical_asap_optimization_keeps_every_logical_candidate` | passes | -| The shared UnivMon stays one build read by all three queries | `stage2_physical_asap_optimization_keeps_the_shared_univmon_as_one_build` | ignored: #595, #596 | +| Each logical candidate keeps one physical candidate with nothing materialized, plus one per other materialization assignment; none pruned | `stage2_physical_asap_optimization_keeps_every_logical_candidate` | passes | +| The shared UnivMon stays one build read by all three queries | `stage2_physical_asap_optimization_keeps_the_shared_univmon_as_one_build` | passes | -Materialization and window variants are out of scope: the doc leaves out the -window-composition Pass 2 candidates for this example, and a summary with no window -form is rebuilt at every refresh (Example 1's table). +Window-composition candidates are out of scope: the doc leaves them out for +this example, and a summary with no window form is rebuilt at every refresh +(Example 1's table) unless a shared build is kept `MatAtQuery` within one +evaluation. ### Plan selection @@ -90,59 +96,62 @@ form is rebuilt at every refresh (Example 1's table). |---|---|---| | One selected id; every other candidate rejected once with a reason | `stage3_plan_selection_selects_one_and_explains_the_rest` | passes | | The selected plan is the cheapest valid one | `stage3_plan_selection_selects_cheapest_valid` | passes | -| Each node of a valid candidate is charged once per consuming query while `NotMaterialized` (all today), once when materialized | `stage3_plan_selection_charges_each_node_per_consuming_query` | passes | -| A UnivMon estimate has a guarantee, so it is judged against its target instead of rejected for having none | `stage3_plan_selection_judges_univmon_with_an_accuracy_model` | ignored: L2 in #621; distinct count and entropy have no PR | -| One materialized UnivMon for all three costs no more than three separate UnivMons | `stage3_plan_selection_shared_univmon_costs_no_more_than_three` | ignored: #595, #596, #621, #604 | -| Selection direction: the selected plan has at most one UnivMon, because one shared UnivMon dominates several over the same input | `stage3_plan_selection_selected_plan_has_at_most_one_univmon` | passes (vacuously today: no UnivMon is valid) | +| Each node of a valid candidate is charged once per consuming query while `NotMaterialized`, once when materialized | `stage3_plan_selection_charges_each_node_per_consuming_query` | passes | +| No candidate builds a Theta sketch | `stage3_plan_selection_has_no_theta_candidate` | passes | +| A UnivMon estimate has a guarantee, so it is judged against its target instead of rejected for having none | `stage3_plan_selection_judges_univmon_with_an_accuracy_model` | ignored: needs a UnivMon entropy accuracy model; L2 and the distinct count have one | +| Given a distinct bound for the source, classic HLL's failure probability is bound and Q1's HLL is accepted; a larger bound enlarges its precision, also in a tumbling-pane chain | `stage3_plan_selection_accepts_hll_given_a_distinct_bound`, `…_enlarges_hll_for_a_larger_distinct_bound`, `…_enlarges_tumbling_pane_hll` | passes | +| KMV certifies Q1 for any failure probability, without a distinct bound | `stage3_plan_selection_accepts_kmv_for_any_failure_probability_without_a_bound` | passes | +| UnivMon's distinct count is certified given a distinct bound its layers cover; its L2 is certified from layer 0 alone, also through SQL Q3's exact population count and `CASE` | `stage3_plan_selection_accepts_univmon_distinct_given_a_distinct_bound`, `…_rejects_univmon_distinct_past_its_layers`, `…_accepts_univmon_l2` | passes | +| One materialized UnivMon for all three costs no more than three separate UnivMons | `stage3_plan_selection_shared_univmon_costs_no_more_than_three` | ignored: needs a UnivMon entropy accuracy model and keeping the shared UnivMon materialized | +| Selection direction: the selected plan has at most one UnivMon, because one shared UnivMon dominates several over the same input | `stage3_plan_selection_selected_plan_has_at_most_one_univmon` | passes | +| The selected KMV plan's Q1 runs in the executor | `stage3_plan_selection_selected_kmv_plan_executes_end_to_end` | passes | The doc's "the shared candidate usually wins" depends on the cost model, so it is tested only as the dominance above: one build sized for the strictest consumer costs no more than three builds that include one of that size. Whether it also beats the exact plans is the model's call. The dominance needs the -shared UnivMon materialized (#604): plan selection charges a `NotMaterialized` +shared UnivMon materialized: plan selection charges a `NotMaterialized` node once per consuming query, so unmaterialized it costs three builds of the strictest size. ## Ambiguities and conflicts with the planner -1. **SQL recognition.** Q2 and Q3 lower to exact relational plans, so the doc's - workload has no `Entropy` or `L2` intent (#598). Hence the stand-in. The - stand-in's statistics are per series (`PerEntity`), the doc's over the - whole column; with one `flows_src_ip` series they coincide. +1. **SQL recognition.** The SQL frontend recognizes Q2 and Q3 only in the + forms above (Q3 with a Float64 product). The stand-in's statistics are per + series (`PerEntity`), the doc's over the whole column; with one + `flows_src_ip` series they coincide. 2. **Pass 1 options.** The doc gives each computation three options. The - planner gives Q1 five (three specialized distinct-count summaries) and Q2, + planner gives Q1 four (exact and three distinct-count summaries) and Q2, Q3 two (no specialized entropy or norm summary). The tests check presence, not counts. 3. **Candidate count.** The doc's 37 excludes the identical-expression rule's - shared-input variant (Example 1 counts it). Neither Pass 2 rule exists yet - (#588, #595), so today there are 20. Tests do not pin a count. + shared-input candidates (Example 1 counts them). The planner applies both + rules and gives 82. Tests do not pin a count. 4. **Independent UnivMons on a shared input.** The doc keeps independent candidates. Once Pass 2 shares the input, identical UnivMon builds may merge; the tests require the independent combinations only without a shared summary. -5. **Sizing.** "Sized for the strictest requirement, ε = 0.01" presumes - UnivMon sizing per ε; UnivMon has a fixed shape today (#621). +5. **Sizing.** "Sized for the strictest requirement, ε = 0.01" becomes: each + consumer is sized alone, and the shared build takes each parameter's + largest value among them. 6. **Accuracy.** Plan selection accepts an output whose guarantee meets the - target or for which the deployment has evidence. No UnivMon estimate has - a guarantee, so every UnivMon candidate is invalid; #621 certifies L2 - only. HLL is rejected against an (ε, δ) target because its bound has no - failure probability; the doc treats a distinct summary as valid. + target or for which the deployment has evidence. UnivMon's L2 and + distinct count are certified (the distinct count only given a distinct + bound for the source); its entropy is not, so every UnivMon candidate for + Q2 is invalid. HLL's and UnivMon's distinct-count failure probability + stays unknown without the deployment's bound on distinct items per + evaluation (`sources["flows"].max_distinct_per_evaluation`), which this + example does not declare; KMV needs none. The doc treats a distinct + summary as valid. 7. **Pairwise count.** "The third keeps any of its own 3 options" includes its own UnivMon, sized for itself. The test requires the third's full Pass 1 option set. -8. **Data workload.** Only `input_cardinality` and `data_ingestion_interval` - change, so the ingestion rate stays about 66,667/s with 10,000,000 keys. -9. **SQL candidates compose.** On `flows(ts, src_ip)`, every logical - candidate of the SQL workload composes, including the CMS and CountSketch - candidates for Q2's and Q3's `COUNT(*) … GROUP BY src_ip`. Until #598 - recognizes the Entropy and L2 idioms, those per-source counts are what - Pass 1 sees, so the SQL options differ from the stand-in's. +8. **Data workload.** Only the `flows` source's facts change (its + `series_count`, and no `sample_interval`), so the ingestion rate stays + about 66,667/s with 10,000,000 keys. ## Devtool -No `--example planner-layering-2` yet. It needs: - -* SQL `Entropy` and `L2` recognition (#598), so the demo shows the doc's - workload rather than exact relational plans; -* a SQL lowering path with the `flows` catalog in `stage_pipeline` (devtool - only; it lowers PromQL only today). +`stage_pipeline --example planner-layering-2` plans the SQL workload with the +`flows(ts, src_ip)` catalog. The stage viewer has no tab for it; open the +document with **Open stage document…**. diff --git a/docs/design_docs/proposals/planner-layering-example3-acceptance.md b/docs/design_docs/proposals/planner-layering-example3-acceptance.md index 11a78d372..41ae5c459 100644 --- a/docs/design_docs/proposals/planner-layering-example3-acceptance.md +++ b/docs/design_docs/proposals/planner-layering-example3-acceptance.md @@ -20,12 +20,12 @@ that needs a missing feature is `#[ignore]`d with the PR that delivers it. ## Workload -Shared data workload, except Pattern A's `arrival` is `mixed`. +Shared data workload, except Pattern A's `arrival` is `Mixed`. **Pattern A.** One `query_batch`, `invocations: 1` at T, `ad_hoc`, ε = 0.005, δ = 0.01. Tests use T = 2026-01-01T00:00:00Z and a 365-day year. -| Query | `lookback` | `as_of` | +| Query | lookback | as_of | |---|---|---| | q1 `quantile_over_time(0.99, latency_ms[5y])` | 5 y | T | | q2 `quantile_over_time(0.99, latency_ms[1y])` | 1 y | T | @@ -34,7 +34,7 @@ Shared data workload, except Pattern A's `arrival` is `mixed`. | q5 `quantile_over_time(0.99, latency_ms[3y] offset 2y)` | 3 y | T − 2 y | **Pattern B.** `quantile_over_time(0.99, latency_ms[5m])`, every 1 min, -`lookback` 5 m, `as_of` evaluation time, ε = 0.01, δ = 0.01, ≤ 200 ms. +lookback 5 m, as_of evaluation time, ε = 0.01, δ = 0.01, ≤ 200 ms. ## Expected candidates per stage @@ -43,8 +43,8 @@ Shared data workload, except Pattern A's `arrival` is `mixed`. | Query-language frontends | A: five scan → range → quantile chains. B: one. | Same; q3–q5 also have a `time_shift`. | | Logical ASAP-aware optimization, Pass 1 | A: exact or KLL over its own interval per query, 2⁵ = 32. B: exact, KLL. | A: exact, KLL (k = 547), DDSketch per query, 3⁵ = **243**. B: exact, KLL (k = 269), DDSketch, **3**. | | Logical ASAP-aware optimization, Pass 2 | A: window composition adds one EH of KLLs over [T − 5 y, T], one merge + p99 estimate per query, plus a candidate for every grouping of two or more queries onto shared window summaries ("hundreds"). B: each Pass 1 option × {none, sliding L = 5 min s = 1 min (5 active windows), 1-min tumbling (merge 5)} = 6. Independent kept. | Identical-expression rule: each Pass 1 candidate also gets its shared-input candidates (A: 3 per Pass 1 candidate; B: none). Window-composition rule, tumbling panes only: B's KLL and DDSketch each also come as 1-min tumbling panes (a `PaneChain` of 5, merged before the p99 estimate), **5** in all; A is run once, so it gets none. No sliding windows or EHs. | -| Physical ASAP-aware optimization | See Example 4. | One physical candidate per logical candidate, every sub-DAG `NotMaterialized`. | -| Plan selection | Not stated for Example 3. | Every candidate is valid. A: KLL for all five queries (P485); DDSketch in place of KLL ties, and the tie goes to enumeration order. B: KLL (P2, 2.100 per second); the 1-min KLL panes, rebuilt at every evaluation since nothing is materialized, cost 2.183 (the same rows and updates plus merging 5 panes per series); exact costs 2.400; DDSketch, whole or in panes, misses the rank target. | +| Physical ASAP-aware optimization | See Example 4. | Each logical candidate with every sub-DAG `NotMaterialized` (A: 972; B: 5), plus, for B, each pane chain maintained `MatAtIngestion(Memory)` (B: 7 in all). | +| Plan selection | Not stated for Example 3. | A: 128 valid, 844 invalid: every candidate with a DDSketch misses the rank target (its bound is relative). KLL for all five queries (P485) is selected at 39,828.80 per second (cost model 1); all exact costs 46,252.80, about 16% more. B: KLL (P2, 2.100 per second); the 1-min KLL panes rebuilt at every evaluation cost 2.183 (the same rows and updates plus merging 5 panes per series); maintained at ingestion time they keep six panes of 1,000,000 per-series KLLs for 13,326.53; exact costs 2.400; DDSketch, whole or in panes, misses the rank target. Every valid B plan meets 200 ms: 126 ms for the whole-window KLL, 131 ms for the rebuilt panes, 7 ms for the maintained panes, 144 ms exact. | ## Invariants and tests @@ -57,13 +57,13 @@ Shared data workload, except Pattern A's `arrival` is `mixed`. | One summary-free frontend DAG; the queries share no node | `stage0_query_language_frontends_a_one_summary_free_dag_without_sharing` | passes | | Each query has an exact and a KLL option | `stage1_logical_asap_optimization_a_pass1_offers_exact_and_kll_per_query` | passes | | Five independent KLLs, each read by one query, all the same size | `stage1_logical_asap_optimization_a_keeps_five_independent_klls` | passes | -| The identical-expression rule shares only raw input (scan, range, shift) and keeps the unshared variant | `stage1_logical_asap_optimization_a_identical_expression_rule_shares_only_raw_input` | ignored: #588 | +| The identical-expression rule shares only raw input (scan, range, shift) and keeps the unshared candidate | `stage1_logical_asap_optimization_a_identical_expression_rule_shares_only_raw_input` | passes | | One EH of KLLs over ≥ 5 y is read by all five queries; each has its own merge feeding a p99 estimate | `stage1_logical_asap_optimization_a_window_composition_adds_one_eh_for_all_five` | ignored: #628, #634 | | The EH candidate coexists with the five independent KLLs | `stage1_logical_asap_optimization_a_keeps_independent_and_shared_window_summaries` | ignored: #628, #634 | | Every pair of queries shares a window summary in some candidate | `stage1_logical_asap_optimization_a_window_composition_groups_every_pair` | ignored: #628, #634 | | One selected id, the rest explained; the selected plan is the cheapest valid one | `stage3_plan_selection_a_selects_cheapest_valid` | passes | -| Each node of a valid candidate is charged once per consuming query while `NotMaterialized` (all today), once when materialized | `stage3_plan_selection_a_charges_each_node_per_consuming_query` | passes | -| For the same per-computation choices, sharing the scan costs no more than separate scans | `stage3_plan_selection_a_shared_scan_is_not_costlier` | ignored: #588, #632 (see ambiguity 7) | +| Each node of a valid candidate is charged once per consuming query while `NotMaterialized`, once when materialized | `stage3_plan_selection_a_charges_each_node_per_consuming_query` | passes | +| For the same per-computation choices, sharing the scan costs no more than separate scans | `stage3_plan_selection_a_shared_scan_is_not_costlier` | ignored: #632 (see ambiguity 7) | ### Pattern B @@ -89,8 +89,9 @@ and the `SummaryEstimate`. ## Ambiguities and conflicts with the planner 1. **Pass 1 families.** The doc offers exact and KLL; the planner also offers - DDSketch. Tests check presence of exact and KLL, and require window forms - for every summary option. + DDSketch, which plan selection rejects against the rank targets. Tests + check presence of exact and KLL, and require window forms for every + summary option. 2. **Exact in window forms.** Pattern B counts 2 × 3 = 6, so the exact option also gets sliding and tumbling forms. Tumbling needs a mergeable exact state for quantiles (the retained values), which the planner has no @@ -105,8 +106,8 @@ and the `SummaryEstimate`. them. 5. **EH accuracy.** A boundary inside an EH bucket is approximate. Whether the accuracy model charges that against ε = 0.005 is not stated; not tested. -6. **`as_of` and `offset`.** The table gives `as_of` = T − 1 y and the query - text has `offset 1y`. We read `as_of` as the window's end (the result of +6. **as_of and `offset`.** The table gives as_of = T − 1 y and the query + text has `offset 1y`. We read as_of as the window's end (the result of the offset at T), not as a second shift. The frontend lowers the offset from the text. 7. **Plan selection statistics.** A scan reads the window above it, through @@ -114,10 +115,10 @@ and the `SummaryEstimate`. scan keeps one year of rows. A shared scan that is not materialized is still charged once per query at the longest window it serves, so sharing it costs more than separate scans, and the invariant test waits on #632. -8. **Identical-expression sharing.** Not mentioned by the doc. Once - implemented (#588), the planner can share the `latency_ms` scan and the - 2-y time shift read by q4 and q5 (the shift sits below the range). +8. **Identical-expression sharing.** Not mentioned by the doc. The planner + shares the `latency_ms` scan and the 2-y time shift read by q4 and q5 + (the shift sits below the range). 9. **Additional sliding forms.** L shorter than W (merged) is allowed by the Pass 2 rules but not listed for Pattern B. Tests allow extra forms. -10. **Latency.** Pattern B's ≤ 200 ms is not checked by any test; plan - selection does not check latency yet. +10. **Latency.** Plan selection checks Pattern B's ≤ 200 ms; every valid + candidate meets it (see the table above). diff --git a/docs/design_docs/proposals/planner-layering-example4-acceptance.md b/docs/design_docs/proposals/planner-layering-example4-acceptance.md index 66edad16f..37d59390b 100644 --- a/docs/design_docs/proposals/planner-layering-example4-acceptance.md +++ b/docs/design_docs/proposals/planner-layering-example4-acceptance.md @@ -19,10 +19,10 @@ with Pattern A varied as the doc does: | Variant | Recurrence | Predictability | `arrival` | |---|---|---|---| -| A, as given | batch, `invocations: 1` at T | `ad_hoc` | `mixed` | -| A, monthly | repeating every 30 days, window ending at each run | `Predictable { known_at: T }` | `mixed` | -| A, at rest | batch, `invocations: 1` at T | `ad_hoc` | `at_rest` (no ingestion rate) | -| B | repeating every 1 min | `Predictable` | `continuously_ingesting` | +| A, as given | batch, `invocations: 1` at T | `ad_hoc` | `Mixed { rows_per_second }` | +| A, monthly | repeating every 30 days, window ending at each run | `Predictable { known_at: T }` | `Mixed { rows_per_second }` | +| A, at rest | batch, `invocations: 1` at T | `ad_hoc` | `AtRest` (no ingestion rate) | +| B | repeating every 1 min | `Predictable` | `Ingesting { rows_per_second }` | ## Expected physical candidates @@ -53,8 +53,9 @@ separate option. ASAP-aware optimization enumerates `NotMaterialized`, `MatAtIngestion` and `MatAtQuery` (state reused within one evaluation) for every summary build. B1 and B2 exist. B3 (panes built at query time and kept across -evaluations) comes with G9 (#625), and Pattern A's shared EH with #628 and -#634, so A1–A3 do not exist yet. +evaluations) needs query-time state kept across evaluations, which is not +on main yet, and Pattern A's shared EH needs window segments shared across +queries (#628, #634), so A1–A3 do not exist yet. ## Invariants and tests @@ -75,7 +76,7 @@ it. |---|---|---| | Every node upstream of an ingestion-time node also runs at ingestion time (all variants) | `stage2_physical_asap_optimization_ingestion_time_upstream_is_ingestion_time` | passes: B1 and the monthly variant's maintained panes | | With data at rest, no node runs at ingestion time | `stage2_physical_asap_optimization_at_rest_runs_nothing_at_ingestion_time` | passes | -| A materialized output covers its consumers: retention ≥ max over its readers of `lookback` + offset | `stage2_physical_asap_optimization_materialized_output_covers_its_consumers` | passes: B1 keeps 6 min ≥ 5 min | +| A materialized output covers its consumers: retention ≥ max over its readers of lookback + offset | `stage2_physical_asap_optimization_materialized_output_covers_its_consumers` | passes: B1 keeps 6 min ≥ 5 min | ### Pattern A @@ -92,19 +93,22 @@ it. | Invariant | Test | Status | |---|---|---| -| The tumbling KLL candidate yields exactly B1, B2, B3 | `stage2_physical_asap_optimization_b_tumbling_kll_has_three_materialization_options` | ignored: B3 (G9, #625) | +| The tumbling KLL candidate yields exactly B1, B2, B3 | `stage2_physical_asap_optimization_b_tumbling_kll_has_three_materialization_options` | ignored: B3 (query-time state kept across evaluations) | | Until B3: the tumbling KLL candidate yields B1 and B2, both priced within the 200 ms latency bound (implementer test) | `stage2_physical_asap_optimization_b_tumbling_kll_has_b1_and_b2` | passes | | B1 builds at ingestion time; its merge and estimate run at query time | `stage2_physical_asap_optimization_b_b1_builds_at_ingestion_and_merges_at_query_time` | passes | -| B3 runs nothing at ingestion time | `stage2_physical_asap_optimization_b_b3_keeps_query_time_windows` | ignored: B3 (G9, #625) | +| B3 runs nothing at ingestion time | `stage2_physical_asap_optimization_b_b3_keeps_query_time_windows` | ignored: B3 (query-time state kept across evaluations) | | The sliding KLL yields exactly ingestion time and query time, kept | `stage2_physical_asap_optimization_b_sliding_kll_has_two_materialization_options` | ignored: sliding windows (no PR yet) | -| B2 costs at least B1 and B3 | `stage3_plan_selection_b_rebuilding_every_window_costs_most` | ignored: B3 (G9, #625) | -| With the built-in models, B1 is the cheapest of the three | `stage3_plan_selection_b_prefers_ingestion_time_tumbling_windows` | ignored: the built-in model contradicts it. B1 keeps 6 panes of 1M per-series KLLs (about 107 GB) for 13,326.5 cost/s; B2 rebuilds them for 2.183, and the whole-window KLL (P2, 2.100) is selected | +| B2 costs at least B1 and B3 | `stage3_plan_selection_b_rebuilding_every_window_costs_most` | ignored: B3 (query-time state kept across evaluations) | +| With the built-in models, B1 is the cheapest of the three | `stage3_plan_selection_b_prefers_ingestion_time_tumbling_windows` | ignored: cost model 1 contradicts it. B1 keeps 6 panes of 1M per-series KLLs (about 107 GB) for 13,326.5 cost/s; B2 rebuilds them for 2.183, and the whole-window KLL (P2, 2.100) is selected. Needs B3 and a rewrite relative to the cost model | +| Each cost model chooses what its deployment pays for: cost model 1 the whole-window KLL; cost model 2 the exact plan (smallest peak memory); cost model 3 with lowest latency B1 in 256 GB, and the whole-window KLL in 64 GB, where B1 is invalid on peak memory | `stage3_plan_selection_b_cost_models_choose_differently` | passes | +| Under cost model 3, the Pareto front over peak memory, disk and latency is {exact, whole-window KLL, B1} and B1 is selected; B2 is dominated by the whole-window KLL; least memory selects the exact plan | `stage3_plan_selection_b_pareto_front_over_memory_disk_and_latency` | passes | ### Selection direction, relative to the cost model * A3 > A1 and B2 ≥ B1, B3 hold under any model that charges each build: A3 and B2 do strictly more of the same work. -* A1 ≤ A2 as given, and B1 ≤ B3, depend on the model. The doc's reasons are +* A1 ≤ A2 as given, and B1 ≤ B3, depend on the cost model (cost model 1, 2 + or 3). The doc's reasons are that A2 maintains years of history for one batch, and that B3 adds the newest build to every evaluation and needs raw data at query time. The tests assert the doc's direction under the built-in models; if the model @@ -132,7 +136,7 @@ it. repeating entry whose window ends at each run, known at T. 4. **Retention unit.** "Kept 5 min" is read as the event-time span the stored state covers. For A1, which lives only for the batch, that is 5 y of - history. The test requires retention ≥ each reader's `lookback` + offset. + history. The test requires retention ≥ each reader's lookback + offset. 5. **Backfill.** A2 backfills old data once. Whether that one-time cost is charged to the plan, and how it is amortized, is not stated; not tested. 6. **Pruning A2 at rest.** "A2 is not generated" is read as physical @@ -149,12 +153,12 @@ it. ## Devtool -No `--example planner-layering-4`. Pattern A and B already run as -`planner-layering-3a` and `3b`, and until physical ASAP-aware optimization -materializes something, Example 4's variants plan identically. A useful id -needs: +No `--example planner-layering-4`. Pattern A and B run as +`planner-layering-3a` and `3b`, and Pattern B's materialization choices +(B1, B2) appear in the `3b` document; `--cost-model` selects cost model 1, 2 +or 3 (with `--cluster-memory-bytes`, `--cluster-disk-bytes` and +`--cluster-objective` for cost model 3). A useful id still needs: -* Pass 2 window composition and materialization enumeration (#601, #604); -* the workload's recurrence passed to physical ASAP-aware optimization; * the variant (as given, monthly, at rest) selectable in `stage_pipeline`; -* the viewer showing each node's materialization and retention. +* the viewer showing each materialized node's retention (it shows each + node's materialization already). diff --git a/docs/design_docs/proposals/planner-layering.md b/docs/design_docs/proposals/planner-layering.md index 6da1ca611..f576aae29 100644 --- a/docs/design_docs/proposals/planner-layering.md +++ b/docs/design_docs/proposals/planner-layering.md @@ -149,7 +149,7 @@ ASAPPlanner relies on the assumptions below. ### ASAPPlanner System Overview: Planning Stages -ASAPPlanner takes a [query workload](https://github.com/ProjectASAP/ASAPPlanner/blob/main/crates/types/src/workload.rs), a [data workload](https://github.com/ProjectASAP/ASAPPlanner/blob/main/crates/types/src/workload.rs#L531) and the deployment's +ASAPPlanner takes a [query workload](https://github.com/ProjectASAP/ASAPPlanner/blob/main/crates/types/src/workload.rs), a [data workload](https://github.com/ProjectASAP/ASAPPlanner/blob/main/crates/types/src/workload.rs#L384) and the deployment's inputs (TODO: define this data structure, issue [#525](https://github.com/ProjectASAP/ASAPPlanner/issues/525)), and returns one optimal physical plan. It decides what is computed, how it is computed, and which plan is best. The deployment only supplies data and query inputs and executes the plan: it provides its empirical cost model, empirical accuracy model and capabilities, but never optimize queries. @@ -162,7 +162,7 @@ flowchart TB subgraph INPUTS["Inputs"] direction LR QW["Query workload
(PromQL / SQL / MetricsQL,
query recurrence,
accuracy requirements,
latency requirements)"]:::input - DW["+ Data workload
(streaming vs. data at rest,
data distribution,
cardinality)"]:::input + DW["+ Data workload
(streaming vs. data at rest,
per-source series counts,
sample intervals)"]:::input DI["+ Deployment inputs
(cost model,
accuracy model,
deployment capabilities)"]:::input end @@ -775,6 +775,13 @@ the `DataWorkload`. Typical outcomes: A shared summary is materialized once for all its consumers. See Example 4. +The planner implements a subset of these options today. Stage 2 enumerates +materialization only for summary builds (`SummaryAgg` and `PaneChain`); each +gets `NotMaterialized`, `MatAtIngestion(Memory)` when it is eligible to run +at ingestion time, and `MatAtQuery(Memory)` when two or more queries read +it. A `MatAtQuery` state is reused within one evaluation only, and nothing +is placed on disk yet. + #### Physical operator implementation Physical operator implementation converts every node to physical operators, for @@ -885,6 +892,18 @@ which is what lets one shared summary beat several cheaper independent ones: the cost of a shared summary is estimated once, with the demand of all its consumers. +Selection first measures each valid candidate's resource usage, with no +prices (`ResourceUsage`: CPU operations and scanned bytes per second, average +and peak memory, disk, peak CPU cores and each query's latency). The +deployment's cost model (`CostModel`, in `PlanSelectionSettings`) then turns +usage into a choice: + +| Cost model | Deployment | How the plan is chosen | +|---|---|---| +| Cost model 1, `UsageOverTime(ResourceUnitCost)` (default) | Pay-as-you-go | Unit cost × usage per second; the cheapest wins | +| Cost model 2, `ProvisionedCapacity(ResourceUnitCost)` | Pays for provisioned capacity | Unit cost × peak memory, peak CPU cores and disk; the cheapest wins | +| Cost model 3, `ClusterCapacity(ClusterLimits)` | Its own cluster | No prices: a candidate over the cluster's memory, disk or (optionally) cores is invalid; the objective `LowestLatency`, `LeastMemory` or `ParetoFront` chooses among the rest | + ### 4. Execution Execution runs outside ASAPPlanner. The deployment runs the selected plan as @@ -896,13 +915,14 @@ To see examples in DAG Viewer, following the instructions [here](TODO: write an Each example's workload is shown as tables. Field names in code font are the fields of -[`workload.rs`](https://github.com/ProjectASAP/ASAPPlanner/blob/main/crates/types/src/workload.rs), -except `lookback` and `as_of`, which the query text states (its range and -offset). Each query reads the event-time window [`as_of` − `lookback`, `as_of`]. -`as_of` is the window's end; `lookback` is its length. An -`as_of` of "evaluation time" means no offset: the window ends whenever the -query runs, so it moves forward with each evaluation. A fixed `as_of`, such as -T − 1 y, pins the window to a historical interval. Approximate accuracy targets are `EpsilonDelta`: the answer's error +[`workload.rs`](https://github.com/ProjectASAP/ASAPPlanner/blob/main/crates/types/src/workload.rs). +The lookback and as_of columns are not fields: the query text states them (its range and +offset). Each query reads the event-time window [as_of − lookback, as_of]. +as_of is the window's end; lookback is its length. An +as_of of "evaluation time" means no offset: the window ends whenever the +query runs, so it moves forward with each evaluation. A fixed as_of, such as +T − 1 y, pins the window to a historical interval. An approximate accuracy target +(`AccuracyTarget::max_error(ε).failure_probability(δ)`) means the answer's error is at most ε with probability at least 1 − δ. | Example | Shows | @@ -922,10 +942,9 @@ Unless an example says otherwise, every example uses this data workload: | `DataWorkload` field | Meaning | Value | |---|---|---| -| `arrival` | Whether the data is at rest, still arriving, or both | `continuously_ingesting` | -| `data_ingestion_interval` | How often each series delivers one sample (the scrape interval in Prometheus). PromQL uses it as the look-back horizon of instant selectors. | 15 s | -| `ingestion_rate` | Samples arriving per second across all series | about 66,667 samples/s | -| `input_cardinality` | Number of distinct series (or keys) | 1,000,000 series | +| `arrival` | Whether the data is at rest, still arriving (`Ingesting`), or both (`Mixed`); an arriving variant carries `rows_per_second`, the samples arriving per second across all sources | `Ingesting`, about 66,667 samples/s | +| `sources[metric].sample_interval` | How often each series of the metric delivers one sample (the scrape interval in Prometheus). PromQL uses it as the look-back horizon of instant selectors. | 15 s | +| `sources[metric].series_count` | Number of distinct series (or keys) of the metric | 1,000,000 series | With 1,000,000 series each sampled every 15 s, the ingestion rate is 1,000,000 / 15 ≈ 66,667 samples/s. @@ -935,7 +954,7 @@ With 1,000,000 series each sampled every 15 s, the ingestion rate is **Query workload.** Two PromQL dashboard panels over the last minute. The first needs an exact total; the second tolerates error. -| Query | Repeats | `lookback` | `as_of` | Accuracy requirement | Latency requirement | +| Query | Repeats | lookback | as_of | Accuracy requirement | Latency requirement | |---|---|---|---|---|---| | Q1: `sum by (job) (rate(http_requests_total[1m]))` | every 10 s | 1 m | evaluation time | exact | none | | Q2: `topk by (job) (10, sum_over_time(http_requests_total[1m]))` | every 10 s | 1 m | evaluation time | ε = 0.01, δ = 0.001 | ≤ 100 ms | @@ -973,7 +992,7 @@ flowchart TB exact Sum accumulator state. The per-`job` top-*k* has three options, so Q2 has three options. The exact one keeps a per-series sum and sorts within each `job`, which is costly at one million Zipf-distributed series. - The `EpsilonDelta` target also admits a **Count-Min Sketch with a top-*k* + The approximate (ε, δ) target also admits a **Count-Min Sketch with a top-*k* heap per `job`**, and **Hydra over the whole `job` column**, where one sketch covers every (`job`, series) key and a top-*k* heap is still kept per `job`. @@ -1192,10 +1211,11 @@ FROM ( ); ``` -The SQL frontend does not yet recognize the Q2 and Q3 forms as `Entropy` and -`L2`. TODO: add this recognition to the per-language frontends. +The SQL frontend recognizes Q2 as `Entropy` and Q3 as `L2` when Q3 casts its +counts to DOUBLE (`CAST(c AS DOUBLE) * CAST(c AS DOUBLE)`): the Int64 `c * c` +above can overflow, so only the Float64 product is recognized. -| Query | Computation | Repeats | `lookback` | `as_of` | Accuracy requirement | +| Query | Computation | Repeats | lookback | as_of | Accuracy requirement | |---|---|---|---|---|---| | Q1 | `Distinct(src_ip)` | every 10 s | 1 m | evaluation time | ε = 0.02, δ = 0.01 | | Q2 | `Entropy(src_ip)` | every 10 s | 1 m | evaluation time | ε = 0.05, δ = 0.01 | @@ -1205,8 +1225,8 @@ The data workload differs from the shared one in two fields: | `DataWorkload` field | Value | |---|---| -| `input_cardinality` | 10,000,000 distinct source IPs | -| `data_ingestion_interval` | not needed for SQL | +| `sources["flows"].series_count` | 10,000,000 distinct source IPs (groups of rows) | +| `sources["flows"].sample_interval` | not needed for SQL | **Pass 1.** Rewrite rules recognize the three computations, and each gets its per-computation candidates from the Pass 1 table: exact, a specialized summary, or @@ -1313,7 +1333,7 @@ all executed together at time T. | `predictability` | `ad_hoc` | | Accuracy requirement | ε = 0.005, δ = 0.01 | -| Query | `lookback` | `as_of` | +| Query | lookback | as_of | |---|---|---| | `quantile_over_time(0.99, latency_ms[5y])` | 5 y | T | | `quantile_over_time(0.99, latency_ms[1y])` | 1 y | T | @@ -1321,7 +1341,7 @@ all executed together at time T. | `quantile_over_time(0.99, latency_ms[1y] offset 2y)` | 1 y | T − 2 y | | `quantile_over_time(0.99, latency_ms[3y] offset 2y)` | 3 y | T − 2 y | -The data workload is the shared one, except `arrival` is `mixed`: five years +The data workload is the shared one, except `arrival` is `Mixed`: five years of data at rest, plus data still arriving. * **Pass 1.** Each `quantile_over_time` gets an exact candidate and a KLL over @@ -1376,7 +1396,7 @@ flowchart LR **Pattern B: one repeating query with overlapping windows.** A real-time p99 panel over the last 5 min, refreshed every minute. -| Query | Repeats | `lookback` | `as_of` | Accuracy requirement | Latency requirement | +| Query | Repeats | lookback | as_of | Accuracy requirement | Latency requirement | |---|---|---|---|---|---| | `quantile_over_time(0.99, latency_ms[5m])` | every 1 min | 5 m | evaluation time | ε = 0.01, δ = 0.01 | ≤ 200 ms | From 6a129d1fcd87718c00f5ff81f895175e8d3df4f0 Mon Sep 17 00:00:00 2001 From: zzylol <50204836+zzylol@users.noreply.github.com> Date: Fri, 16 Oct 2026 17:29:25 +0000 Subject: [PATCH 04/10] docs(design): rename ResultGuarantee, AccuracyTarget forms and DataWorkload fields AccuracyGuarantee, the AccuracyTarget builders, per-source max_distinct_per_evaluation for classic HLL, and arrival/sources in the historical proposals. --- .../evidence-dependent-candidates.md | 6 ++-- docs/design_docs/concepts/accuracy-models.md | 34 +++++++++++-------- docs/design_docs/concepts/post-asap-ir.md | 4 +-- .../analytical-resource-cost.md | 24 ++++++------- .../end-to-end-accuracy-guarantees.md | 26 +++++++------- .../asap-aware-mapping/summary-properties.md | 6 ++-- .../proposals/asap-primitive-schema.md | 6 ++-- .../proposals/error-resource-profile.md | 10 +++--- .../design_docs/proposals/operator-sharing.md | 8 ++--- 9 files changed, 66 insertions(+), 58 deletions(-) diff --git a/docs/design_docs/architecture/evidence-dependent-candidates.md b/docs/design_docs/architecture/evidence-dependent-candidates.md index 437cfe4b0..b01cc8d21 100644 --- a/docs/design_docs/architecture/evidence-dependent-candidates.md +++ b/docs/design_docs/architecture/evidence-dependent-candidates.md @@ -10,7 +10,7 @@ not turn an unknown guarantee into a satisfied accuracy requirement. ## Candidate state Here, **certified for a target** means Planner has a fully known -`ResultGuarantee` for the candidate and `AccuracyModel::satisfies(guarantee, +`AccuracyGuarantee` for the candidate and `AccuracyModel::satisfies(guarantee, target)` returns `true`. **Uncertified** means Planner cannot make that claim: the guarantee is absent, contains unknown terms, or is known not to meet the target. An uncertified summary may still be a well-formed logical candidate; @@ -19,13 +19,13 @@ The exact path (the pre-ASAP sub-DAG kept by `retain_exact`) has an exact guaran | State | Planner representation | Consequence / next step | |---|---|---| -| Known guarantee | `ResultGuarantee` with evaluable bound and failure probability | Planner checks whether the guarantee satisfies the query's accuracy target. If this candidate is selected, the backend checks whether its implementation can realize the selected summary; it does not re-decide the accuracy target. | +| Known guarantee | `AccuracyGuarantee` with evaluable bound and failure probability | Planner checks whether the guarantee satisfies the query's accuracy target. If this candidate is selected, the backend checks whether its implementation can realize the selected summary; it does not re-decide the accuracy target. | | Missing accuracy/domain evidence | Symbolic `BoundExpr::Unknown` or `ProbabilityExpr::Unknown`, or `guarantee: None` on a constructible summary | Inspect `ReplacementSubDAG::has_missing_accuracy_evidence()`, obtain applicable evidence or apply explicit policy; do not claim certification. | | Missing cost | `CostModel::candidate_cost()` returns `None` for a `ReplacementSubDAG` (including a non-finite or negative legacy estimate) | Keep that logical summary/rewrite candidate in `CandidateLogicalASAPDAGs` for inspection; provide a comparable cost before selecting it by cost. This does not make it a deployable physical plan. | | Unknown runtime support | `ReplacementSubDAG::runtime_support_evidence(model)` returns `None` | Candidate remains visible; bind a concrete implementation and confirm support before deployment. | | Known invalid evidence or impossible semantics | No candidate; where supported, a `RejectedCandidate` records the error | Do not deploy. | -`ResultGuarantee::has_unknown()` detects symbolic gaps. The candidate-level +`AccuracyGuarantee::has_unknown()` detects symbolic gaps. The candidate-level helper also covers summaries with no guarantee model, including DDSketch-ratio `None`. A root `AccuracyTarget` rejects a fully known guarantee that misses the target, but does not prune a candidate solely because a required statistic diff --git a/docs/design_docs/concepts/accuracy-models.md b/docs/design_docs/concepts/accuracy-models.md index baeadbe9e..8d7595462 100644 --- a/docs/design_docs/concepts/accuracy-models.md +++ b/docs/design_docs/concepts/accuracy-models.md @@ -31,15 +31,16 @@ These three concepts have different roles: `AccuracyTarget` is the authoritative requirement type: -- `Exact` requires zero error and zero failure probability. -- `Epsilon` constrains the error magnitude in the relevant metric; it does not specify a failure-probability budget. -- `EpsilonDelta` constrains both error magnitude and failure probability. +- `AccuracyTarget::exact()` requires zero error and zero failure probability. +- `AccuracyTarget::max_error(ε)` constrains the error magnitude in the relevant metric; it does not specify a failure-probability budget. +- `.failure_probability(δ)` adds a failure-probability bound, so the target constrains both error magnitude and failure probability. +- `.accuracy_metric(m)` optionally fixes the metric ε is stated in; when it is unset, a guarantee in any metric that meets ε and δ is accepted. The target at a query root applies to the entire returned result. An aggregate's own target guides its local candidate construction. Nested local targets do not replace the root requirement: the composed result must satisfy it as well. -A `ResultGuarantee` contains an error metric, an error-bound expression, a +An `AccuracyGuarantee` contains an error metric, an error-bound expression, a failure-probability expression and provenance. The metric is essential: | Metric | Quantity bounded | @@ -195,8 +196,8 @@ configuration or candidate is needed. and propagation evidence. The evidence must cover the population to which the claimed guarantee applies: sources, filters, grouping, evaluation windows and all merged panes. Evidence for a narrower population cannot silently certify -a wider one. The workload-backed provider checks freshness before exposing -its supported data characteristics. +a wider one. Data-workload facts are declared: the planner has no clock, so the caller +drops facts that no longer hold before planning. Source contracts are assertions that the source or deployment must establish and enforce. They are not inferred from observed cardinality or sampled value @@ -212,14 +213,19 @@ an observation into a guarantee. ### Example: bounded Classic HLL -A deployment supplies `EstimatorContract::ClassicHll` for the complete aggregate -expression. It asserts the classic estimator, independent uniform bucket -hashing and an enforced maximum distinct population per evaluation, including -all merged panes. Planner combines this contract with the query or allocated -local target, selects a supported precision, derives the guarantee and uses -the normal propagation and selection checks. - -The current model supports maxima from 1 to 4096 and precisions from 4 to 18, +A deployment declares `DataWorkload.sources[source].max_distinct_per_evaluation` +for the source the distinct count reads. It asserts an enforced maximum +distinct population per evaluation, including all merged panes; the classic +estimator and independent uniform bucket hashing are the model's +assumptions. Stage 1 sizes the precision from the query or allocated local +target and leaves the failure probability symbolic, as +`classic_hll_failure_probability[p=P,eps=ε](max_distinct[source])`. Stage 3 +binds it from the source's declared bound, enlarges the precision when the +bound needs it, and uses the normal selection checks. A bound outside the +contract's range gives a failure probability of 1; a missing bound leaves it +unknown, so the target is not met. + +The current model supports maxima from 1 to 131,072 (2^17) and precisions from 4 to 18, and certifies only configurations that remain in the linear-counting branch. It bounds collisions across every integer cardinality in the declared domain and bounds overestimation deterministically. It is not an RSE-to-normal diff --git a/docs/design_docs/concepts/post-asap-ir.md b/docs/design_docs/concepts/post-asap-ir.md index d073a1499..bfeb88f6b 100644 --- a/docs/design_docs/concepts/post-asap-ir.md +++ b/docs/design_docs/concepts/post-asap-ir.md @@ -71,7 +71,7 @@ execution implement the actual build and update operations. Not every summary fa Exact work is represented by the ordinary operators, unchanged: - A sub-DAG the planner does not rewrite keeps its `NonASAPOp` nodes. Plan - assembly marks such a sub-DAG with an exact `ResultGuarantee` + assembly marks such a sub-DAG with an exact `AccuracyGuarantee` (`asap_aware_mapping::replacement::retain_exact`); a sub-DAG with no ASAP operator and no guarantee is a logical rewrite candidate that has not been assessed yet (`is_logical_rewrite`). @@ -90,7 +90,7 @@ Exact work is represented by the ordinary operators, unchanged: `Limit` (both with the same `partition_by`) ranks and selects the joined rows. Completeness evidence belongs to pruning, not ranking. -Every `OperatorNode` carries its schema and an optional `ResultGuarantee`. +Every `OperatorNode` carries its schema and an optional `AccuracyGuarantee`. State and query values have different contracts; see [Schema and physical data for ASAP primitives](../proposals/asap-primitive-schema.md). Exact operations over approximate evaluations still require composed accuracy guarantees. See the diff --git a/docs/design_docs/proposals/asap-aware-mapping/analytical-resource-cost.md b/docs/design_docs/proposals/asap-aware-mapping/analytical-resource-cost.md index 6edba7713..0564acb38 100644 --- a/docs/design_docs/proposals/asap-aware-mapping/analytical-resource-cost.md +++ b/docs/design_docs/proposals/asap-aware-mapping/analytical-resource-cost.md @@ -208,28 +208,28 @@ parameters, memory, and calibration determine the winner. ## Canonical workload and evidence sources The cost model does not define a second workload schema. It consumes the -normalized workload, lowered query IR, and freshness-aware statistics: +normalized workload, lowered query IR, and declared statistics: | Cost concept | Canonical source | |---|---| -| Source row cardinality | Fresh `DataWorkload.input_cardinality: Evidence`. | -| Arrival and update rate | `DataWorkload.arrival` and `ingestion_rate`. | +| Source row cardinality | `DataWorkload.sources[source].series_count`. | +| Arrival and update rate | `DataWorkload.arrival` and its `rows_per_second`. | | Query demand | `QueryWorkloadEntry.recurrence`, normalized over an explicit horizon. | | Event-time range | The query text (its range and offset). | | Accuracy and latency requirements | `QueryWorkloadEntry.requirements`. | | Operator shape, grouping keys, and constants such as Top-K limit and offset | Lowered query and selected physical plan. | `DataWorkload` does define whether input is streaming: its `arrival` field is -`AtRest`, `ContinuouslyIngesting`, `Mixed`, or `Unknown`, and a continuous -arrival rate comes from fresh `ingestion_rate` evidence. These facts describe +`AtRest`, `Ingesting`, `Mixed`, or `Unknown`, and the arriving variants +carry the arrival rate as `rows_per_second`. These facts describe how source data arrives. They do not choose materialization or a window framework: `Incremental` describes how a selected summary state is updated, while tumbling, sliding, and exponential histogram describe how that state is organized over time. -Evidence is read through `Evidence::value_at(planning_time)`. Stale, -future, or improperly time-bounded evidence remains unknown. Costing follows -the same freshness rule as accuracy checking. +Every data-workload fact is declared. The planner has no clock, so the +caller drops facts that no longer hold before planning; an undeclared fact +remains unknown. Costing follows the same rule as accuracy checking. The current workload schema does not yet contain every physical statistic. The missing facts have explicit ownership: @@ -308,7 +308,7 @@ It is not an independent workload axis. A repeated rate without a horizon cannot produce a finite total cost. An at-rest estimate must not be reused for `unknown`, `mixed`, or -`continuously_ingesting` data. Callers fail closed instead of pretending that +`ingesting` data. Callers fail closed instead of pretending that incremental updates are a one-time snapshot build. The at-rest summary alternative scans the selected source snapshot once and @@ -839,8 +839,8 @@ Cost evidence cannot replace accuracy evidence. Examples include: - approximate Top-K requires a lower bound for the kth selected item, an upper bound for excluded items, and their union-bounded failure probability; -- distribution-sensitive sizing or error propagation consumes fresh - `DataWorkload.distribution` evidence; +- distribution-sensitive sizing or error propagation needs distribution + evidence, which `DataWorkload` does not carry; - a shared sketch layout requires a proven composition bound. Window candidates therefore contain both physical-resource evidence and a @@ -866,7 +866,7 @@ EH Universal/GSum, arbitrary sub-window: Here `suffix_rows` is the cardinality from the query's left endpoint through the maintained window's right endpoint, and `query_rows` is the requested sub-window cardinality. Both must be positive and `suffix_rows >= query_rows`. -The failure probability is carried separately in `ResultGuarantee`. The EH+KLL +The failure probability is carried separately in `AccuracyGuarantee`. The EH+KLL rule is accepted only for a KLL summary; the current Universal/GSum rule is accepted only for exact `Count` or `Sum` accumulator states. An EH assignment with an exact marker, a mismatched family, invalid parameters, or an unregistered diff --git a/docs/design_docs/proposals/asap-aware-mapping/end-to-end-accuracy-guarantees.md b/docs/design_docs/proposals/asap-aware-mapping/end-to-end-accuracy-guarantees.md index e1546e1b6..a272f8905 100644 --- a/docs/design_docs/proposals/asap-aware-mapping/end-to-end-accuracy-guarantees.md +++ b/docs/design_docs/proposals/asap-aware-mapping/end-to-end-accuracy-guarantees.md @@ -115,7 +115,7 @@ candidate can be proved legal. ### End-to-end example Consider two compatible approximate operations and a final requirement -`EpsilonDelta { epsilon: 0.10, delta: 0.02 }`. +`AccuracyTarget::max_error(0.10).failure_probability(0.02)`. 1. The allocator proposes local requirements of `(0.05, 0.01)` for each approximate operation. @@ -169,8 +169,8 @@ defines a standing planner contract rather than a time-bounded project. The design must: 1. Represent each summary's final-result correctness requirement with an - `AccuracyTarget`. The existing `Exact`, `Epsilon`, and `EpsilonDelta` - variants remain valid where their semantics match, but they are not a + `AccuracyTarget`. The existing `exact()`, `max_error(ε)`, and + `max_error(ε).failure_probability(δ)` forms remain valid where their semantics match, but they are not a closed set: add or refine target variants when a new summary requires a correctness contract that those variants cannot express faithfully. 2. Preserve the meaning of each error metric instead of treating every bound @@ -200,10 +200,10 @@ AccuracyTarget describes what the query requires committed summary parameters + optional evidence - determine each operation's local ResultGuarantee + determine each operation's local AccuracyGuarantee CompositionOperator + AccuracyModel - combine local and child guarantees into the final ResultGuarantee + combine local and child guarantees into the final AccuracyGuarantee AccuracyModel::satisfies accepts or rejects the candidate before CostModel ranking @@ -216,7 +216,7 @@ Four components have distinct responsibilities: | `AccuracyTarget` | State the required correctness of the final query result | Describe evidence or choose parameters | | `AccuracyBudgetAllocator` | Propose local requirements for approximate layers | Prove that the combined result is legal | | `AccuracyModel` | Derive, compose, and check guarantees | Rank candidates by cost | -| `ResultGuarantee` | Record the bound, failure probability, metric, and evidence for one candidate result | Express what the query requested | +| `AccuracyGuarantee` | Record the bound, failure probability, metric, and evidence for one candidate result | Express what the query requested | Candidate generation first enumerates an implementation, commits its concrete parameters, and derives a local guarantee from those same parameters. If the @@ -230,10 +230,10 @@ parameter-configuration modes, planner/runtime boundary, and explanation data. ### Guarantee IR -Finalized post-ASAP values may carry a `ResultGuarantee`: +Finalized post-ASAP values may carry an `AccuracyGuarantee`: ```rust -struct ResultGuarantee { +struct AccuracyGuarantee { metric: AccuracyMetric, bound: BoundExpr, failure_probability: ProbabilityExpr, @@ -382,9 +382,9 @@ combined implementations without creating a second legality pipeline. ### Accuracy targets and allocation -`AccuracyTarget::Exact` accepts only a zero bound and zero failure probability. -`AccuracyTarget::Epsilon` checks the evaluated magnitude. An -`AccuracyTarget::EpsilonDelta` requires both an evaluated magnitude no greater +`AccuracyTarget::exact()` accepts only a zero bound and zero failure probability. +`AccuracyTarget::max_error(ε)` checks the evaluated magnitude. A +`max_error(ε).failure_probability(δ)` target requires both an evaluated magnitude no greater than epsilon and an evaluated failure probability no greater than delta. These variants are the currently supported target vocabulary, not a permanent @@ -399,7 +399,7 @@ epsilon-style numeric error contract. Any new or revised target must define: The planner must fail closed until the corresponding guarantee model, propagation rules, and final satisfaction check exist. It must not force a new -summary into `Exact`, `Epsilon`, or `EpsilonDelta` merely to reuse the existing +summary into `exact()`, `max_error(ε)`, or `max_error(ε).failure_probability(δ)` merely to reuse the existing API. The initial allocator uses conservative finite choices, including equal splits @@ -450,7 +450,7 @@ uses the same candidate space and legality checks as optimization. Three concepts are necessary: -- `ResultGuarantee` is the authoritative description of final-result error; +- `AccuracyGuarantee` is the authoritative description of final-result error; reusing `AccuracyTarget` would conflate a request with evidence that the request was met. - `AccuracyModel` separates correctness rules from `CostModel`; embedding diff --git a/docs/design_docs/proposals/asap-aware-mapping/summary-properties.md b/docs/design_docs/proposals/asap-aware-mapping/summary-properties.md index da2ce90ff..888f103ea 100644 --- a/docs/design_docs/proposals/asap-aware-mapping/summary-properties.md +++ b/docs/design_docs/proposals/asap-aware-mapping/summary-properties.md @@ -114,7 +114,7 @@ candidate generation ``` Every finalized post-ASAP value carries a machine-readable -`ResultGuarantee`: a typed error metric, symbolic bound and probability +`AccuracyGuarantee`: a typed error metric, symbolic bound and probability expressions, and provenance. Exact values have zero error; a sketch readout's guarantee is derived from the sizing formula that produced its parameters. @@ -128,7 +128,9 @@ semantics: - KLL, KMV, and Theta expose identified 99%-confidence contracts derived from committed parameters. HLL retains its RSE magnitude but has unknown failure probability because its current parameters encode precision, not a - confidence-level budget; it therefore cannot satisfy `EpsilonDelta`. + confidence-level budget; it therefore cannot satisfy a target with a + failure probability unless the source declares + `max_distinct_per_evaluation`, which the classic HLL contract binds. - TopK membership is certified only when the widened confidence interval of the kth selected item is strictly above every excluded item's widened interval. Missing or overlapping interval evidence fails closed. diff --git a/docs/design_docs/proposals/asap-primitive-schema.md b/docs/design_docs/proposals/asap-primitive-schema.md index 06a23bfd1..c83ce0066 100644 --- a/docs/design_docs/proposals/asap-primitive-schema.md +++ b/docs/design_docs/proposals/asap-primitive-schema.md @@ -870,7 +870,7 @@ OperatorNode §6.1 │ └── fields[i].dtype: FieldDataType ◀───────┘ §6.3 (the same type: the output │ └── Sketch(SketchKind, GroupingStrategy) `state` field has type `family`) │ └── category, algorithm, params §6.4 -├── guarantee: ResultGuarantee §6.5 (accuracy of an evaluation) +├── guarantee: AccuracyGuarantee §6.5 (accuracy of an evaluation) └── coverage() → SummaryCoverage §6.6 └── definition: OperatorNode (again a node, so the same structure) ``` @@ -904,7 +904,7 @@ pub struct OperatorNode { pub schema: Schema, /// The accuracy statement of the output, once known. `None` does not /// mean exact. - pub guarantee: Option, + pub guarantee: Option, /// When the node runs: `IngestionTime` or `QueryTime`. `None` until /// planning assigns it. pub timing: Option, @@ -1325,7 +1325,7 @@ pub struct CurrentSeriesInput { } ``` -A finalized value's accuracy statement is `ResultGuarantee` (`post_asap/guarantee.rs`): `metric` (which error is measured, e.g. rank error), `bound` (the error bound), `failure_probability` (the chance the bound does not hold) and `provenance` (which estimates the bound came from). It is attached to evaluation and finalized nodes, and to `MaintainPopulation` nodes (whose population is exact); never to sketch or accumulator state. +A finalized value's accuracy statement is `AccuracyGuarantee` (`post_asap/guarantee.rs`): a `spec` (`AccuracySpec`) with `accuracy_metric` (which error is measured, e.g. rank error), `error` (the error bound), `failure_probability` (the chance the bound does not hold) and `accuracy_scope` (which output values one statement covers), plus `provenance` (which estimates the bound came from). It is attached to evaluation and finalized nodes, and to `MaintainPopulation` nodes (whose population is exact); never to sketch or accumulator state. ### 6.6 Summary coverage (`crates/types/src/ir/summary_coverage.rs`) diff --git a/docs/design_docs/proposals/error-resource-profile.md b/docs/design_docs/proposals/error-resource-profile.md index d4ccd4a04..829f5cd95 100644 --- a/docs/design_docs/proposals/error-resource-profile.md +++ b/docs/design_docs/proposals/error-resource-profile.md @@ -1,16 +1,16 @@ # Error–Resource Profile (ERP) -> Status: implemented ERP v1 with proposed extensions. The -> [ERP module](../../../crates/asap-aware-mapping/src/erp.rs) consumes discrete -> empirical profiles; benchmark observations are not worst-case accuracy proofs. +> Status: ERP v1 with proposed extensions. The planner's ERP module, which +> consumed discrete empirical profiles, has been removed; no planning stage +> reads ERP profiles today. Benchmark observations are not worst-case accuracy proofs. > See the status, supported selection modes and remaining work below. ## Status ERP v1 is a discrete, shape- and distribution-conditioned profile exchanged between `sketch-bench` and ASAPPlanner. It is an empirical planning input, not a proof -of a worst-case sketch guarantee. The implementation lives in -`asap-aware-mapping::erp`; `approxbench erp` exports the producer artifact. +of a worst-case sketch guarantee. The planner-side consumer +(`asap-aware-mapping::erp`) has been removed; `approxbench erp` exports the producer artifact. ## Motivation diff --git a/docs/design_docs/proposals/operator-sharing.md b/docs/design_docs/proposals/operator-sharing.md index 8bd8d499a..fbe03cc0c 100644 --- a/docs/design_docs/proposals/operator-sharing.md +++ b/docs/design_docs/proposals/operator-sharing.md @@ -61,7 +61,7 @@ struct OperatorNode { operator: Operator, result_kind: OperatorResultKind, schema: Schema, - guarantee: Option, + guarantee: Option, timing: Option, } @@ -345,10 +345,10 @@ enum ExecutionTiming { |---|---|---| | `operator` | Operation category, parameters and dependencies | `Operator`, `NonASAPOp` and `ASAPOp` in §1 | | `result_kind`, `schema` | The output category and fields, including identity/time metadata | Derived from `operator` and its actual inputs, then retained on the resolved node (§2.1) | -| `guarantee` | An established result-accuracy guarantee, when available | Existing `ResultGuarantee` and composition rules (§2.2); `None` never means exact | +| `guarantee` | An established result-accuracy guarantee, when available | Existing `AccuracyGuarantee` and composition rules (§2.2); `None` never means exact | | `timing` | The assigned ingestion/query execution phase | Physical planning under #509 (§2.3); `None` means not assigned | -`ResultGuarantee` retains its existing definition. `Operator`, `OperatorNode`, +`AccuracyGuarantee` retains its existing definition. `Operator`, `OperatorNode`, `OperatorResultKind` and the common node layout are proposed; `Schema` is unified as specified below. This is a resolved-plan interface: name resolution must finish before producing these concrete `ColumnId`/`Schema` nodes. @@ -381,7 +381,7 @@ actual inputs, including producers referenced by scalar expressions. Reading an approximate result through `PromqlScalarFromVector` or a SQL scalar subquery does not make it exact; unknown accuracy must not be treated as exactness. -The common node reuses `guarantee: Option` from `SummaryNode`. +The common node reuses `guarantee: Option` from `SummaryNode`. `Some` records an established guarantee; `None` covers an unassessed or unknown result, or state whose accuracy is only established at evaluation. Exactness must be explicitly established using the existing model. This proposal introduces no new From ba08683c29836a370ecd78a723aacc6805e75f2a Mon Sep 17 00:00:00 2001 From: zzylol <50204836+zzylol@users.noreply.github.com> Date: Fri, 16 Oct 2026 17:32:21 +0000 Subject: [PATCH 05/10] docs(design): list the rollup and exact top-k reuse rules in the Pass 2 rule table --- docs/design_docs/proposals/planner-layering.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/design_docs/proposals/planner-layering.md b/docs/design_docs/proposals/planner-layering.md index f576aae29..bb8e5befa 100644 --- a/docs/design_docs/proposals/planner-layering.md +++ b/docs/design_docs/proposals/planner-layering.md @@ -541,6 +541,8 @@ their overlapping windows. | Identical-expression rule | The input and computation semantics are identical. | One common computation node serving multiple consumers. | | Summary-capability rule | The computations have the same summary input data and the same window, and one summary supports all requested computations and their accuracy requirements. | One summary build node feeding several estimation nodes, e.g. UnivMon → distinct count, entropy, L2 norm. | | Window-composition rule | The computations have the same summary input data, and one window summary can answer the requested windows within their accuracy requirements. | One window summary feeding per-query merge (where needed) and estimation nodes, e.g. a sliding-window or tumbling-window KLL, or an Exponential Histogram with a KLL per EH bucket. | +| Rollup rule | A coarse exact accumulator (`sum`, `min`, `max`) reads the same rows as a finer exact accumulator the workload already computes, grouped by a superset of its keys. | The finer accumulator's finalized values feeding an exact accumulator grouped by the coarse keys, e.g. `sum by (job)` from `sum by (job, instance)`. | +| Exact top-k reuse rule | Exact top-k computations read the same input and partition with different `k`. | One ranking (`Limit(k_max)` over `Sort`) that each top-k reads its first `k` rows from. | The rules compare computations by their **summary input data**: what a summary for that computation would ingest, namely the data source, the filters, and the key From 0ae736c71496af0463ff09c131b5799c4cea990d Mon Sep 17 00:00:00 2001 From: zzylol <50204836+zzylol@users.noreply.github.com> Date: Fri, 16 Oct 2026 17:47:48 +0000 Subject: [PATCH 06/10] docs: deployment capabilities, kept query-time panes and pass1_pruned The Stage Viewer guide and the viewer README describe --deployment-capabilities, the document's deployment_capabilities and pass1_pruned sections and per_node's three fields. library-api.md lists DeploymentCapabilities; the Stage 2 paragraph and the Example 4 acceptance doc describe MatAtQueryKeptAcrossEvaluations (B3). --- .../planner-layering-example4-acceptance.md | 20 ++++++++++++------- .../design_docs/proposals/planner-layering.md | 7 +++++-- docs/develop_docs/library-api.md | 3 ++- docs/user_guide_docs/stage-viewer.md | 10 +++++++--- tools/dag-viewer/README.md | 13 ++++++++++++ 5 files changed, 40 insertions(+), 13 deletions(-) diff --git a/docs/design_docs/proposals/planner-layering-example4-acceptance.md b/docs/design_docs/proposals/planner-layering-example4-acceptance.md index 37d59390b..8d1b135e3 100644 --- a/docs/design_docs/proposals/planner-layering-example4-acceptance.md +++ b/docs/design_docs/proposals/planner-layering-example4-acceptance.md @@ -53,9 +53,13 @@ separate option. ASAP-aware optimization enumerates `NotMaterialized`, `MatAtIngestion` and `MatAtQuery` (state reused within one evaluation) for every summary build. B1 and B2 exist. B3 (panes built at query time and kept across -evaluations) needs query-time state kept across evaluations, which is not -on main yet, and Pattern A's shared EH needs window segments shared across -queries (#628, #634), so A1–A3 do not exist yet. +evaluations) is offered when the deployment declares +`query_time_results_retained` in its `deployment_capabilities`: Stage 2 then +adds `MatAtQueryKeptAcrossEvaluations` for a tumbling-pane chain that builds +one new pane per evaluation. Pattern A's shared EH needs window segments shared across +queries (#628, #634), so A1–A3 do not exist yet. B3's +4 kept panes plus the newest cost 8,884.53 per second under cost model 1, +between B2 (2.183) and B1 (13,326.53). ## Invariants and tests @@ -93,12 +97,12 @@ it. | Invariant | Test | Status | |---|---|---| -| The tumbling KLL candidate yields exactly B1, B2, B3 | `stage2_physical_asap_optimization_b_tumbling_kll_has_three_materialization_options` | ignored: B3 (query-time state kept across evaluations) | +| The tumbling KLL candidate yields exactly B1, B2, B3 | `stage2_physical_asap_optimization_b_tumbling_kll_has_three_materialization_options` | passes (with `query_time_results_retained`) | | Until B3: the tumbling KLL candidate yields B1 and B2, both priced within the 200 ms latency bound (implementer test) | `stage2_physical_asap_optimization_b_tumbling_kll_has_b1_and_b2` | passes | | B1 builds at ingestion time; its merge and estimate run at query time | `stage2_physical_asap_optimization_b_b1_builds_at_ingestion_and_merges_at_query_time` | passes | -| B3 runs nothing at ingestion time | `stage2_physical_asap_optimization_b_b3_keeps_query_time_windows` | ignored: B3 (query-time state kept across evaluations) | +| B3 runs nothing at ingestion time | `stage2_physical_asap_optimization_b_b3_keeps_query_time_windows` | passes | | The sliding KLL yields exactly ingestion time and query time, kept | `stage2_physical_asap_optimization_b_sliding_kll_has_two_materialization_options` | ignored: sliding windows (no PR yet) | -| B2 costs at least B1 and B3 | `stage3_plan_selection_b_rebuilding_every_window_costs_most` | ignored: B3 (query-time state kept across evaluations) | +| B2 costs at least B1 and B3 | `stage3_plan_selection_b_rebuilding_every_window_costs_most` | ignored: cost model 1 contradicts it. With query-time results retained, B3 keeps 4 panes for 8,884.5 cost/s and B1 6 for 13,326.5, while B2 rebuilds them for 2.183. Needs a rewrite relative to the cost model | | With the built-in models, B1 is the cheapest of the three | `stage3_plan_selection_b_prefers_ingestion_time_tumbling_windows` | ignored: cost model 1 contradicts it. B1 keeps 6 panes of 1M per-series KLLs (about 107 GB) for 13,326.5 cost/s; B2 rebuilds them for 2.183, and the whole-window KLL (P2, 2.100) is selected. Needs B3 and a rewrite relative to the cost model | | Each cost model chooses what its deployment pays for: cost model 1 the whole-window KLL; cost model 2 the exact plan (smallest peak memory); cost model 3 with lowest latency B1 in 256 GB, and the whole-window KLL in 64 GB, where B1 is invalid on peak memory | `stage3_plan_selection_b_cost_models_choose_differently` | passes | | Under cost model 3, the Pareto front over peak memory, disk and latency is {exact, whole-window KLL, B1} and B1 is selected; B2 is dominated by the whole-window KLL; least memory selects the exact plan | `stage3_plan_selection_b_pareto_front_over_memory_disk_and_latency` | passes | @@ -155,7 +159,9 @@ it. No `--example planner-layering-4`. Pattern A and B run as `planner-layering-3a` and `3b`, and Pattern B's materialization choices -(B1, B2) appear in the `3b` document; `--cost-model` selects cost model 1, 2 +(B1, B2, and B3 with +`--deployment-capabilities '{"query_time_results_retained": true}'`) appear in +the `3b` document; `--cost-model` selects cost model 1, 2 or 3 (with `--cluster-memory-bytes`, `--cluster-disk-bytes` and `--cluster-objective` for cost model 3). A useful id still needs: diff --git a/docs/design_docs/proposals/planner-layering.md b/docs/design_docs/proposals/planner-layering.md index bb8e5befa..11c2cb643 100644 --- a/docs/design_docs/proposals/planner-layering.md +++ b/docs/design_docs/proposals/planner-layering.md @@ -781,8 +781,11 @@ The planner implements a subset of these options today. Stage 2 enumerates materialization only for summary builds (`SummaryAgg` and `PaneChain`); each gets `NotMaterialized`, `MatAtIngestion(Memory)` when it is eligible to run at ingestion time, and `MatAtQuery(Memory)` when two or more queries read -it. A `MatAtQuery` state is reused within one evaluation only, and nothing -is placed on disk yet. +it. A `MatAtQuery` state is reused within one evaluation only. When the +deployment's capabilities declare `query_time_results_retained`, a +tumbling-pane chain that builds exactly one new pane per evaluation also gets +`MatAtQueryKeptAcrossEvaluations(Memory)`: the panes built at query time are +kept for the next evaluations. Nothing is placed on disk yet. #### Physical operator implementation diff --git a/docs/develop_docs/library-api.md b/docs/develop_docs/library-api.md index 59fa4a8f5..9619df60b 100644 --- a/docs/develop_docs/library-api.md +++ b/docs/develop_docs/library-api.md @@ -170,7 +170,8 @@ asap_aware_mapping::run_stage1_to_stage3( | Input or output | Meaning | | --- | --- | -| `PlanningModels::builtin()` | Built-in accuracy model and no extra evidence. `with_plan_selection(&settings)` sets the deployment's `PlanSelectionSettings` | +| `PlanningModels::builtin()` | Built-in accuracy model and no extra evidence. `with_plan_selection(&settings)` sets the deployment's `PlanSelectionSettings`; `with_deployment_capabilities(&capabilities)` sets its `DeploymentCapabilities` (field `deployment_capabilities`) | +| `DeploymentCapabilities` (`asap_aware_mapping::deployment_capabilities`) | What the executor supports: `supported_summaries` (`None`, the default, supports every summary the planner builds; otherwise `SupportedSummary { family: SketchAlgorithm, readouts: Vec }` entries; Pass 1 prunes others with `PruneReason::UnsupportedByDeployment`), `raw_data_retained` (default `true`; when `false`, a candidate that scans raw data at query time is invalid, and stored data is refused with `SelectionError::InvalidDeploymentCapabilities`) and `query_time_results_retained` (default `false`; when `true`, Stage 2 offers `MatAtQueryKeptAcrossEvaluations` for tumbling panes) | | `PlanSelectionSettings` (`asap_aware_mapping::stage3_plan_selection`) | `cost_model`, `resource_speed` (`ms_per_cpu_op`, `ms_per_scanned_byte`) and `non_repeating_query_horizon_seconds` (default 3,600) | | `CostModel::UsageOverTime(ResourceUnitCost)` | Cost model 1 (default, `ResourceUnitCost::illustrative()`): unit cost × usage per second; the cheapest valid candidate wins | | `CostModel::ProvisionedCapacity(ResourceUnitCost)` | Cost model 2: unit cost × peak memory, peak CPU cores and disk; the cheapest valid candidate wins | diff --git a/docs/user_guide_docs/stage-viewer.md b/docs/user_guide_docs/stage-viewer.md index c44b01664..7b2325dc3 100644 --- a/docs/user_guide_docs/stage-viewer.md +++ b/docs/user_guide_docs/stage-viewer.md @@ -74,7 +74,7 @@ and open them with **Open stage document…**: | Example | Command options | What to look for | | --- | --- | --- | | 2 · SQL flow statistics | `--example planner-layering-2` | Distinct sources, entropy and L2 of per-source counts over the last minute of flows. All 82 logical plans compose; none is pruned. The SQL frontend recognizes Q2 as entropy and Q3 (its counts cast to DOUBLE) as L2, each with an exact or UnivMon option and an exact population count for the empty-input NULL. 16 plans come from Pass 1. Pass 2 adds the plans that share identical nodes and, by the summary-capability rule, the plans in which one UnivMon serves two of the queries or all three, sized with each parameter's largest value among them. Stage 2 adds 102 plans in which a node that two queries read is built once per evaluation at query time and read by both (`query time, shared: …` in the label), 184 physical plans in all. 164 plans are invalid: HLL or UnivMon for Q1 (120) because its bound has "failure probability None" (both need the source's distinct bound, which this example does not declare), and UnivMon for Q2 (44) because entropy has no certified bound. 20 are valid: Q1 exact or KMV, Q2 exact, and Q3 exact or UnivMon, each alone, sharing identical nodes, or with the shared node built once at query time. Q3's UnivMon keeps its L2 guarantee through the exact population count joined beside it and the `CASE` that decides the empty-input NULL on that count. Theta is not offered: `asap_sketchlib` has no Theta sketch. P1 (all exact) is selected at 6.08 per second; KMV on Q1 costs the same to three decimals, and UnivMon on Q3 costs 8.08. The shared plans cost 7.44 or more when the shared node is not materialized, because it is then charged once per query that reads it, and 7.04 when it is built once at query time | -| 4 · one workload, three cost models | `--example planner-layering-3b` with `--cost-model` | Example 3b's live p99 panel under each cost model. Cost model 1 (`usage-over-time`, the default) selects P2, the whole-window KLL, at 2.100 per second. Cost model 2 (`provisioned-capacity`) selects P1, exact, at 1024.0 per second against 3221.0 for P2, because it prices peak memory, and the KLL states hold about 17.8 GB at their full capacity. Cost model 3 (`cluster-capacity --cluster-memory-bytes 256000000000 --cluster-disk-bytes 1000000000000`) selects by `--cluster-objective`: `lowest-latency` selects P3-m1, the KLL panes maintained at ingestion time (7 ms per evaluation); `least-memory` selects P1; `pareto-front` lists P1, P2 and P3-m1 and selects P3-m1. With 64 GB of memory (`--cluster-memory-bytes 64000000000`), the plans with KLL panes are over the limit and invalid, `lowest-latency` selects P2, and the front is P1 and P2. Under cost model 3 the plans have no price | +| 4 · one workload, three cost models | `--example planner-layering-3b` with `--cost-model` | Example 3b's live p99 panel under each cost model. Cost model 1 (`usage-over-time`, the default) selects P2, the whole-window KLL, at 2.100 per second. Cost model 2 (`provisioned-capacity`) selects P1, exact, at 1024.0 per second against 3221.0 for P2, because it prices peak memory, and the KLL states hold about 17.8 GB at their full capacity. Cost model 3 (`cluster-capacity --cluster-memory-bytes 256000000000 --cluster-disk-bytes 1000000000000`) selects by `--cluster-objective`: `lowest-latency` selects P3-m1, the KLL panes maintained at ingestion time (7 ms per evaluation); `least-memory` selects P1; `pareto-front` lists P1, P2 and P3-m1 and selects P3-m1. With 64 GB of memory (`--cluster-memory-bytes 64000000000`), the plans with KLL panes are over the limit and invalid, `lowest-latency` selects P2, and the front is P1 and P2. Under cost model 3 the plans have no price. With `--deployment-capabilities '{"query_time_results_retained": true}'`, Stage 2 adds P3-m2, the KLL panes built at query time and kept across evaluations, at 8,884.53 per second under cost model 1; with `'{"raw_data_retained": false}'`, every plan that scans raw data at query time is invalid and P3-m1, the panes maintained at ingestion time, is selected at 13,326.53 | The workload statistics in the examples (series counts, ingestion rates) are illustrative. Read the costs as a ranking, not as a prediction of real @@ -119,8 +119,11 @@ it (plus once for each query that also reads them directly). A selected or costlier plan also says, per query, which metric its accuracy was accepted in, e.g. `q1: metric not set; accepted rank (Kll)`. - **✗ invalid**: fails a check, with the reason. Examples: an accuracy target - the sketch cannot guarantee, an evaluation over the latency bound, or, - under cost model 3, a plan over the cluster's memory or disk. + the sketch cannot guarantee, an evaluation over the latency bound, a + query-time scan of raw data the deployment does not retain, or, under cost + model 3, a plan over the cluster's memory or disk. A summary the deployment + cannot build is never offered: the document's + `stage1_logical_asap_optimization.pass1_pruned` lists it with its reason. Each label names the choice per query, for example `Q1 exact (Sum acc, Rate acc) · Q2 CMS+heap`, what Pass 2 shares or splits into panes, for example @@ -214,6 +217,7 @@ cargo run -p asap-devtools --bin stage_pipeline -- \ | `--cost-model ` | How plan selection compares valid plans: `usage-over-time` (cost model 1, the default: CPU, scanned bytes and memory over time, priced per second), `provisioned-capacity` (cost model 2: peak memory, peak CPU cores and disk, priced) or `cluster-capacity` (cost model 3: no prices) | | `--cluster-memory-bytes `, `--cluster-disk-bytes `, `--cluster-cpu-cores ` | Cost model 3 only: the cluster's limits; a plan over one is invalid. The CPU limit is optional. | | `--cluster-objective ` | Cost model 3 only: `lowest-latency`, `least-memory` or `pareto-front` (the plans no other plan beats on latency, memory and disk; the lowest-latency one is selected) | +| `--deployment-capabilities ''` | What the executor supports, for example `{"supported_summaries": [{"family": "Kll", "readouts": ["Quantile"]}], "raw_data_retained": true, "query_time_results_retained": true}`. `supported_summaries` (default: every summary the planner can build) lists the sketch families and the statistics each may answer; a summary outside it is not offered. With `raw_data_retained: false` (default `true`), a plan that scans raw data at query time is invalid. With `query_time_results_retained: true` (default `false`), tumbling panes built at query time may be kept for the next evaluations (`query time, kept` in the label). The document repeats the inputs under `deployment_capabilities`; the page does not show them yet. | | `--max-displayed ` | How many plans the document shows (default 64), in enumeration order, plus the selected plan. The planner still optimizes and prices every plan, so the selection does not depend on it. | | `--max-combinations ` | The most candidates a run plans (default 10000), counted at Stage 1 and again at Stage 2; a larger workload is refused with exit status 3 and no document | | `--out ` | Where to write the document | diff --git a/tools/dag-viewer/README.md b/tools/dag-viewer/README.md index 3570f1d18..435c11450 100644 --- a/tools/dag-viewer/README.md +++ b/tools/dag-viewer/README.md @@ -65,9 +65,14 @@ time column). The server runs `stage_pipeline` with `--sql`, `--promql`, "workload": { "queries": [{ "id": "q1", "language": "promql", "text": "...", "requirements": { "accuracy": { "error": 0.0, "failure_probability": 0.0, "accuracy_scope": "whole_result" }, "repeat_interval_ms": 10000 } }] }, + "deployment_capabilities": { "supported_summaries": null, "raw_data_retained": true, + "query_time_results_retained": false }, "stage0_query_language_frontends": { "dag": "" }, "stage1_logical_asap_optimization": { "combinations": 9, "capped": false, + "pass1_pruned": [{ "target": 0, "query": "Q1", + "pruned": [{ "candidate": "DDSketch", "reason": "UnsupportedByDeployment" }], + "fused_pruned": [] }], "candidates": [{ "id": "L1", "display_name": "...", "dag": "" }, { "id": "L2", "display_name": "...", "error": "..." }] }, @@ -111,6 +116,14 @@ time column). The server runs `stage_pipeline` with `--sql`, `--promql`, prices; a node's is null under cost models 2 and 3, which price or limit the whole candidate's peaks. `front` lists the Pareto front under cost model 3's `pareto-front` objective, and is empty otherwise. +- `deployment_capabilities` repeats the run's `--deployment-capabilities` + inputs with defaults filled in. `pass1_pruned` has, per Pass 1 target with + a pruned candidate, the target's index, the query it is listed under, and + each pruned candidate of the target (`pruned`) and of its fused + computation (`fused_pruned`) with the reason. The page does not show + either section yet. +- A node's `per_node` entry holds only `cost_per_second`, `multiplier` and + `detail`. - When Pass 1 fails, Stage 1 has `error` and no candidates. When selection fails, `selected` is null, with the rejections or with `error`. From 9a00db03431deef55725e177c2539a09cfac2265 Mon Sep 17 00:00:00 2001 From: zzylol <50204836+zzylol@users.noreply.github.com> Date: Fri, 16 Oct 2026 19:07:24 +0000 Subject: [PATCH 07/10] docs: measured operation costs, removed legacy cost models, split_shared_by_timing Measured sketch costs reach Stage 3 through PlanSelectionSettings::measured_operation_costs (stage_pipeline --measured-operation-costs, convert_rqe_atomic_costs); per_node gains price_source. EmpiricalCostModel, PhysicalPlanCostModel and PhysicalEvidenceSnapshot are gone, so the docs that named them say so. post-asap-ir.md uses split_shared_by_timing. --- docs/design_docs/concepts/post-asap-ir.md | 2 +- docs/develop_docs/library-api.md | 27 +++++--------------- docs/develop_docs/offline-sketch-evidence.md | 8 ++++++ docs/develop_docs/physical-handoff-costs.md | 7 ++--- docs/develop_docs/storage-operation-costs.md | 6 ++--- docs/user_guide_docs/run-a-query.md | 1 + docs/user_guide_docs/stage-viewer.md | 1 + tools/dag-viewer/README.md | 8 +++--- 8 files changed, 29 insertions(+), 31 deletions(-) diff --git a/docs/design_docs/concepts/post-asap-ir.md b/docs/design_docs/concepts/post-asap-ir.md index bfeb88f6b..936c544ea 100644 --- a/docs/design_docs/concepts/post-asap-ir.md +++ b/docs/design_docs/concepts/post-asap-ir.md @@ -148,7 +148,7 @@ The pass then validates every edge (rows or exact-accumulator state into a `SummaryAgg`, state into an evaluation, a timed `MaintainPopulation` under an `EvaluatePopulation`, no ingestion work reading a query-time value) and rejects a node reached from two consumers that need different timings; -`split_shared_by_phase` copies such a sub-DAG for one side before the +`split_shared_by_timing` copies such a sub-DAG for one side before the assignment is applied. How long a materialized output is kept (retention) is also Stage 2's decision and is not in the assignment. diff --git a/docs/develop_docs/library-api.md b/docs/develop_docs/library-api.md index 9619df60b..022fb6dc5 100644 --- a/docs/develop_docs/library-api.md +++ b/docs/develop_docs/library-api.md @@ -172,7 +172,7 @@ asap_aware_mapping::run_stage1_to_stage3( | --- | --- | | `PlanningModels::builtin()` | Built-in accuracy model and no extra evidence. `with_plan_selection(&settings)` sets the deployment's `PlanSelectionSettings`; `with_deployment_capabilities(&capabilities)` sets its `DeploymentCapabilities` (field `deployment_capabilities`) | | `DeploymentCapabilities` (`asap_aware_mapping::deployment_capabilities`) | What the executor supports: `supported_summaries` (`None`, the default, supports every summary the planner builds; otherwise `SupportedSummary { family: SketchAlgorithm, readouts: Vec }` entries; Pass 1 prunes others with `PruneReason::UnsupportedByDeployment`), `raw_data_retained` (default `true`; when `false`, a candidate that scans raw data at query time is invalid, and stored data is refused with `SelectionError::InvalidDeploymentCapabilities`) and `query_time_results_retained` (default `false`; when `true`, Stage 2 offers `MatAtQueryKeptAcrossEvaluations` for tumbling panes) | -| `PlanSelectionSettings` (`asap_aware_mapping::stage3_plan_selection`) | `cost_model`, `resource_speed` (`ms_per_cpu_op`, `ms_per_scanned_byte`) and `non_repeating_query_horizon_seconds` (default 3,600) | +| `PlanSelectionSettings` (`asap_aware_mapping::stage3_plan_selection`) | `cost_model`, `resource_speed` (`ms_per_cpu_op`, `ms_per_scanned_byte`) `non_repeating_query_horizon_seconds` (default 3,600) and `measured_operation_costs` (default empty: one `MeasuredOperationCost { sketch, operation, cpu_ns_per_operation }` per measured sketch configuration and `SummaryOperation` (`Update`, `Merge`, `Estimate`); a sketch node with an exactly matching entry is priced from it, every other node from the formulas, and each node's `NodeCost::price_source` says which) | | `CostModel::UsageOverTime(ResourceUnitCost)` | Cost model 1 (default, `ResourceUnitCost::illustrative()`): unit cost × usage per second; the cheapest valid candidate wins | | `CostModel::ProvisionedCapacity(ResourceUnitCost)` | Cost model 2: unit cost × peak memory, peak CPU cores and disk; the cheapest valid candidate wins | | `CostModel::ClusterCapacity(ClusterLimits)` | Cost model 3: memory, disk and optional CPU cores are hard limits; `ClusterObjective::{LowestLatency, LeastMemory, ParetoFront}` chooses among the candidates that fit | @@ -537,31 +537,16 @@ Module-qualified paths below are relative to `asap_aware_mapping`. | Parameter | Available value / constructor | Meaning | | --- | --- | --- | | `&dyn CostModel` | `DefaultCostModel` | Built-in ordering/sizing and structural estimates; no measured deployment guarantee | -| `&dyn CostModel` | `empirical_cost::EmpiricalCostModel::new(provider)` | Offline sketch-benchmark model: ranks algorithms using matching offline measurements | -| `&dyn CostModel` | `physical_plan_cost_model::PhysicalPlanCostModel::new(&provider, calibration)?` | Deployment-specific physical-plan model: compares complete physical alternatives using provider evidence and resource calibration; evidence may be offline or online | | `&dyn AccuracyModel` | `DefaultAccuracyModel` | Built-in guarantee rules and satisfaction checks | | `&dyn AccuracyEvidenceProvider` | `NoAccuracyEvidence` | No extra planning-time statistics; evidence-dependent claims remain unavailable | | Any provider trait above | Your implementation | Supplies alternative models/evidence under the same contracts | -### Offline measurements versus physical-plan costing +### Measured sketch costs -These models differ in scope, not simply in whether they are offline or online. - -| Model | Evidence and comparison | Missing evidence / limits | -| --- | --- | --- | -| `EmpiricalCostModel` | Offline sketch benchmarks matched to exact parameters, distribution, environment and validity interval; current algorithm ranking uses measured update CPU nanoseconds | If the measurements required for ranking are incomplete, preserves the incoming algorithm order. `estimate_cost()` still uses `DefaultCostModel` structural scores | -| `PhysicalPlanCostModel` | A downstream provider supplies a consistent evidence snapshot and complete physical alternatives; calibration converts modeled resource quantities into comparable costs | A candidate with incomplete evidence is unavailable, without structural-cost fallback. Current candidate admission also requires it to cost less than the raw alternative | - -`PhysicalPlanCostModel` does not collect online telemetry itself. Its provider -may supply offline estimates/calibration or evidence derived from online -observations. Therefore, “offline sketch-benchmark model” and “physical-plan cost -model” describe their roles more accurately than “offline model” and “online model.” - -For example, a sketch with the lowest measured update cost can rank first under -`EmpiricalCostModel`, while its complete execution plan can still cost more than -another sketch or raw execution under `PhysicalPlanCostModel`. Offline error -measurements alone do not authorize smaller sketch parameters or replace formal -accuracy guarantees. +Measured CPU time per sketch operation now reaches plan selection, not the +legacy search: put it in `PlanSelectionSettings::measured_operation_costs` +(see [Plan a workload through the stages](#plan-a-workload-through-the-stages)). +The legacy offline sketch-benchmark and physical-plan cost models are removed. ### Example: configure all sketch-strategy providers diff --git a/docs/develop_docs/offline-sketch-evidence.md b/docs/develop_docs/offline-sketch-evidence.md index 40d1d1e37..9e4bc02f1 100644 --- a/docs/develop_docs/offline-sketch-evidence.md +++ b/docs/develop_docs/offline-sketch-evidence.md @@ -1,5 +1,13 @@ # Consuming offline sketch measurements +> **Status: no planner consumer.** The planner-side reader of this artifact +> (`empirical_cost`, `EmpiricalCostModel`) has been removed. Plan selection +> reads measured CPU time per sketch operation from +> `PlanSelectionSettings::measured_operation_costs`, which +> `convert_rqe_atomic_costs` (in `asap-devtools`) builds from sketch-bench's +> `rqe_atomic_costs.json`. This document is kept as the record of the +> artifact contract. + This document is for developers integrating sketch-bench with the planner. The Rust schema is `asap_aware_mapping::empirical_cost::EvidenceArtifact`; its JSON schema version is `1`. Required artifact-level `benchmark_version` and diff --git a/docs/develop_docs/physical-handoff-costs.md b/docs/develop_docs/physical-handoff-costs.md index ae9a6db0b..037b08900 100644 --- a/docs/develop_docs/physical-handoff-costs.md +++ b/docs/develop_docs/physical-handoff-costs.md @@ -11,9 +11,10 @@ calibration, and ranking remain in the mapping crate. handoff traffic/write work is distinct from CPU work, scanned bytes, and stored byte occupancy; it is not collapsed into the generic CPU/byte resource container. -The physical-plan adapter accepts an optional `PhysicalHandoffProfile` in the -immutable `PhysicalEvidenceSnapshot`. No devtool reads the profile; a caller -supplies it through the library. With no profile, these +The `PhysicalHandoffProfile` estimator is kept in `physical_handoff_cost`, but +its consumer, the legacy physical-plan adapter and its +`PhysicalEvidenceSnapshot`, has been removed. No planning stage or devtool +reads the profile yet. With no profile, these dimensions remain unestimated and the existing resource objective is preserved. The profile's `plans` list binds handoffs to complete physical alternatives. diff --git a/docs/develop_docs/storage-operation-costs.md b/docs/develop_docs/storage-operation-costs.md index 30e27f442..c86900ca6 100644 --- a/docs/develop_docs/storage-operation-costs.md +++ b/docs/develop_docs/storage-operation-costs.md @@ -1,8 +1,8 @@ # Storage operation estimates -The physical-plan ranking adapter accepts an optional `StorageIoProfile` in -its immutable `PhysicalEvidenceSnapshot`. No devtool reads the profile; a -caller supplies it through the library. +The `StorageIoProfile` estimator is kept in `storage_io`, but its consumer, +the legacy physical-plan ranking adapter and its `PhysicalEvidenceSnapshot`, +has been removed. No planning stage or devtool reads the profile yet. Omitting the profile preserves the existing CPU/memory/scan-byte objective; operation counts are unestimated, not inferred to be zero. diff --git a/docs/user_guide_docs/run-a-query.md b/docs/user_guide_docs/run-a-query.md index 882d94da7..324aa6c2b 100644 --- a/docs/user_guide_docs/run-a-query.md +++ b/docs/user_guide_docs/run-a-query.md @@ -15,6 +15,7 @@ tool on first use. | Command (`cargo run -p asap-devtools --bin … -- …`) | Input / options | Result | | --- | --- | --- | | `stage_pipeline --promql "" --out plan.json` | One or more `--promql` / `--sql` queries, or `--example ` | Plans the workload through Stages 0–3 and writes the stage document the [Stage Viewer](stage-viewer.md) shows | +| `convert_rqe_atomic_costs ` | sketch-bench's per-operation cost table | Writes the measured table `stage_pipeline --measured-operation-costs` reads; skipped rows are listed on stderr | | `show_logical_dag --sample-interval-ms 1000 queries.txt` | File path, or stdin when omitted | Prints each query's logical DAG (Pre-ASAP IR) | | `analyze_corpora --corpora --sample-interval-ms 1000 --out-dir ` | Repository PromQL corpora, output directory | Writes successful/error IR dumps and summary reports | | `analyze_corpora --sql-corpora --out-dir ` | Repository SQL corpora, output directory | Writes SQL corpus reports | diff --git a/docs/user_guide_docs/stage-viewer.md b/docs/user_guide_docs/stage-viewer.md index 7b2325dc3..9cb0a4ee9 100644 --- a/docs/user_guide_docs/stage-viewer.md +++ b/docs/user_guide_docs/stage-viewer.md @@ -218,6 +218,7 @@ cargo run -p asap-devtools --bin stage_pipeline -- \ | `--cluster-memory-bytes `, `--cluster-disk-bytes `, `--cluster-cpu-cores ` | Cost model 3 only: the cluster's limits; a plan over one is invalid. The CPU limit is optional. | | `--cluster-objective ` | Cost model 3 only: `lowest-latency`, `least-memory` or `pareto-front` (the plans no other plan beats on latency, memory and disk; the lowest-latency one is selected) | | `--deployment-capabilities ''` | What the executor supports, for example `{"supported_summaries": [{"family": "Kll", "readouts": ["Quantile"]}], "raw_data_retained": true, "query_time_results_retained": true}`. `supported_summaries` (default: every summary the planner can build) lists the sketch families and the statistics each may answer; a summary outside it is not offered. With `raw_data_retained: false` (default `true`), a plan that scans raw data at query time is invalid. With `query_time_results_retained: true` (default `false`), tumbling panes built at query time may be kept for the next evaluations (`query time, kept` in the label). The document repeats the inputs under `deployment_capabilities`; the page does not show them yet. | +| `--measured-operation-costs ` | A JSON table of measured CPU time per sketch operation (`[{"sketch": …, "operation": "update", "cpu_ns_per_operation": …}]`, as `convert_rqe_atomic_costs` writes it). A sketch node whose parameters match an entry exactly is priced from it; each node's `price_source` in the document says `measured` or `analytical`. | | `--max-displayed ` | How many plans the document shows (default 64), in enumeration order, plus the selected plan. The planner still optimizes and prices every plan, so the selection does not depend on it. | | `--max-combinations ` | The most candidates a run plans (default 10000), counted at Stage 1 and again at Stage 2; a larger workload is refused with exit status 3 and no document | | `--out ` | Where to write the document | diff --git a/tools/dag-viewer/README.md b/tools/dag-viewer/README.md index 435c11450..4709fd490 100644 --- a/tools/dag-viewer/README.md +++ b/tools/dag-viewer/README.md @@ -86,7 +86,8 @@ time column). The server runs `stage_pipeline` with `--sql`, `--promql`, "average_memory_bytes": 0.0, "peak_memory_bytes": 16481088, "disk_bytes": 192000000, "peak_cpu_cores": 2, "latency_per_query": [{ "compaction_ms": 0.0, "query_ms": 33.2 }] }, - "per_node": { "": { "cost_per_second": 2.3, "multiplier": 1, "detail": "..." } } } }, + "per_node": { "": { "cost_per_second": 2.3, "multiplier": 1, "detail": "...", + "price_source": "analytical" } } } }, "accepted": { "P1": [{ "target_metric": null, "accepted_metric": "rank", "sketches": ["Kll"], "summary": "metric not set; accepted rank (Kll)" }] }, "selected": "P1", @@ -122,8 +123,9 @@ time column). The server runs `stage_pipeline` with `--sql`, `--promql`, each pruned candidate of the target (`pruned`) and of its fused computation (`fused_pruned`) with the reason. The page does not show either section yet. -- A node's `per_node` entry holds only `cost_per_second`, `multiplier` and - `detail`. +- A node's `per_node` entry holds `cost_per_second`, `multiplier`, + `detail` and `price_source` (`measured` when `--measured-operation-costs` + priced it, else `analytical`). - When Pass 1 fails, Stage 1 has `error` and no candidates. When selection fails, `selected` is null, with the rejections or with `error`. From 93fd9a40fb24cef7a55a0b330a4b0c83c3f1333c Mon Sep 17 00:00:00 2001 From: zzylol <50204836+zzylol@users.noreply.github.com> Date: Sat, 17 Oct 2026 20:21:43 +0000 Subject: [PATCH 08/10] docs(develop): describe the stage code after the legacy search was removed library-api.md drops the search, ranking, strategy, cost-model and selection sections and keeps lowering, planning through the stages, inputs and export; its examples compile and run. The architecture, contracts and extension guides are rewritten around Pass 1, Pass 2, Stage 2 and Stage 3. Docs about removed APIs carry a historical note, and the reading paths point at the stage docs. --- README.md | 2 +- docs/README.md | 9 +- docs/develop_docs/README.md | 7 +- .../asap-aware-mapping-architecture.md | 358 ++---- .../asap-aware-mapping-contracts.md | 504 ++------ .../end-to-end-accuracy-guarantees.md | 8 +- .../develop_docs/extend-asap-aware-mapping.md | 1132 ++--------------- docs/develop_docs/library-api.md | 460 +------ .../metrics-observability-corpora.md | 31 +- docs/develop_docs/offline-sketch-evidence.md | 4 +- .../planner-vocabulary-migration.md | 5 + docs/develop_docs/replacement-explanations.md | 7 + .../target-candidate-api-migration.md | 7 + 13 files changed, 390 insertions(+), 2144 deletions(-) diff --git a/README.md b/README.md index b426de3b8..89a9cadb7 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ ASAPPlanner turns SQL, PromQL, and MetricsQL query workloads into legal candidat - Want to run a query? Follow [Run and inspect a query](docs/user_guide_docs/run-a-query.md). - Want to see how a workload is planned, stage by stage? Use the [Stage Viewer](docs/user_guide_docs/stage-viewer.md). - Embedding Planner? Use the [library API guide](docs/develop_docs/library-api.md). -- Extending Planner? Start with the [ASAP-aware mapping architecture](docs/develop_docs/asap-aware-mapping-architecture.md), then [extend ASAP-aware mapping](docs/develop_docs/extend-asap-aware-mapping.md). +- Extending Planner? Read the [planning stages design](docs/design_docs/proposals/planner-layering.md), then the [ASAP-aware mapping architecture](docs/develop_docs/asap-aware-mapping-architecture.md), which maps each stage to its code, then [extend ASAP-aware mapping](docs/develop_docs/extend-asap-aware-mapping.md). - Evaluating a design? Browse the [design documentation](docs/design_docs/README.md), [developer documentation](docs/develop_docs/README.md), and [user guides](docs/user_guide_docs/README.md). The [documentation map](docs/README.md) gives each audience a complete reading path. diff --git a/docs/README.md b/docs/README.md index a2c2c3f5b..4ef582c9b 100644 --- a/docs/README.md +++ b/docs/README.md @@ -20,13 +20,14 @@ for the integration boundary, nested inputs, and choice of planning workflow. Use [Public library functions and examples](develop_docs/library-api.md) for frontend lowering, planning a workload through the stages to a selected plan, -and the legacy workload search, ranking, selection and DAG assembly. +and exporting it. ## Extend the planner -1. Read the [ASAP-aware mapping architecture](develop_docs/asap-aware-mapping-architecture.md). -2. Consult [mapping contracts](develop_docs/asap-aware-mapping-contracts.md). -3. Follow [Extend ASAP-aware mapping](develop_docs/extend-asap-aware-mapping.md). +1. Read the [planning stages design](design_docs/proposals/planner-layering.md). +2. Read the [ASAP-aware mapping architecture](develop_docs/asap-aware-mapping-architecture.md), which maps each stage to its code. +3. Consult [mapping contracts](develop_docs/asap-aware-mapping-contracts.md). +4. Follow [Extend ASAP-aware mapping](develop_docs/extend-asap-aware-mapping.md). ## Understand a design diff --git a/docs/develop_docs/README.md b/docs/develop_docs/README.md index 817722adf..a49707790 100644 --- a/docs/develop_docs/README.md +++ b/docs/develop_docs/README.md @@ -14,7 +14,10 @@ formats, evidence, and verification workflows. - [Offline sketch evidence](offline-sketch-evidence.md) - [Metrics-observability corpora](metrics-observability-corpora.md) - [Physical handoff cost references](physical-handoff-costs.md), [storage operations](storage-operation-costs.md) -- [Replacement explanations](replacement-explanations.md) - [Physical compile coverage for deployment computation](physical-compile-coverage.md) -- [Planner vocabulary migration (#427)](planner-vocabulary-migration.md) +Historical records of removed APIs: + +- [Planner vocabulary migration](planner-vocabulary-migration.md) +- [Target candidate API migration](target-candidate-api-migration.md) +- [Replacement explanations](replacement-explanations.md) diff --git a/docs/develop_docs/asap-aware-mapping-architecture.md b/docs/develop_docs/asap-aware-mapping-architecture.md index eaf3d879a..45efadf4b 100644 --- a/docs/develop_docs/asap-aware-mapping-architecture.md +++ b/docs/develop_docs/asap-aware-mapping-architecture.md @@ -1,265 +1,101 @@ # ASAP-Aware Mapping architecture -This document explains the architecture of ASAP-aware mapping: the planning -layer that turns logical query operations into alternative `Realization` -values built from ASAP primitives, such as exact summaries and approximate -sketches. Here, a **realization** is one candidate physical form of -one logical operation—not a selected workload plan or a deployed executable. -Read it to understand how strategies, realizations, and costing interact. - -For procedural work—adding a `ReplacementStrategy`, changing a `CostModel`, or -writing the expected tests—use [Extend ASAP-aware mapping](extend-asap-aware-mapping.md). - -For the higher-level motivation and replacement-plan-search design, see the -[ASAP-aware mapping overview](../design_docs/architecture/asap-aware-mapping.md). Current interfaces are -defined in [mapping contracts](asap-aware-mapping-contracts.md). - -Names such as `MyStrategy`, `MyCostModel`, and `PreferDDSketch` are -illustrative; they do not ship with this crate. Samples that use real public -types and functions follow the APIs exported by `asap-aware-mapping`. - -If you only need to find the right extension point, start with the [extension map](extend-asap-aware-mapping.md#7-current-extension-map). If you are implementing a strategy, read this mental model, the [mapping contracts](asap-aware-mapping-contracts.md), and the [extension guide](extend-asap-aware-mapping.md). - ---- - -## Code architecture - -## 1. Mental model - -ASAP-aware mapping has two different jobs that should remain separate: - -1. **Generate valid alternatives.** -2. **Rank alternatives and optionally coordinate compatible selections.** - -`ReplacementStrategy` is responsible for the first job. - -`CostModel` supplies preferences and cost evidence for the second; search and -selection APIs apply those decisions while preserving legality. - -A strategy should answer: - -> Does this transformation apply here, and if so, what are all semantically valid replacements? - -A cost model should answer: - -> Given valid choices, which choices are preferable, and how should they be parameterized? - -Do not put cost-based pruning into a `ReplacementStrategy`. A strategy must enumerate every valid alternative, even when the default cost model clearly prefers one. See [Rule 2](extend-asap-aware-mapping.md#rule-2-enumerate-do-not-rank). - ---- - -## 2. Architecture overview - -The diagram below follows a workload of one or more query roots through target discovery, candidate generation, ranking, reporting, and downstream visualization. Section 3 focuses on the replacement-strategy path. - -Terminology used in the diagram: - -- A **workload** is the set of named queries planned together. A **query root** - is the top-level `Rc` (the unified operator IR) for one of - those queries. **Pre-ASAP** means a DAG that contains only ordinary - `NonASAPOp` operators, before the planner realizes an operation with ASAP - primitives; **post-ASAP** means the same IR after some nodes became `ASAPOp` - summary operators. -- A **DAG** (directed acyclic graph) represents query operators whose sub-DAGs - may be shared. See [sub-DAG sharing and ASAP-aware CSE](../design_docs/proposals/planner-layering.md#pass-2-asap-aware-common-subexpression-elimination) - for the sharing rules. Rust's `Rc` (reference-counted pointer) records - shared node identity. -- A **target** is one replaceable site. A **candidate** is one valid alternative - for it. `Replacement::SubDAG` is a replacement sub-DAG: either a constructed - post-ASAP summary (it contains an `ASAPOp`, e.g. an exact accumulator or an - approximate sketch) or a logical rewrite with no ASAP operator - (`is_logical_rewrite` tells them apart). - `Replacement::ExactComposition` refers to a child target whose realization - must remain undecided until compatible selection. A **sketch** - is a compact data structure that trades exactness for bounded error. A - query's **accuracy target** states the allowed error and failure probability. - A candidate's **rationale** is its human-readable explanation. -- `CandidateLogicalASAPDAGs` is a compact candidate space with one - `TargetSubDAGCandidates` per target instead of one full plan per combination of choices. - A `node_hash` is a structural fingerprint used to narrow explanation lookup; - exact structural equality is still checked afterward. - -```mermaid -flowchart TB - classDef input fill:#e8f1ff,stroke:#4b78b8,color:#172b4d - classDef generate fill:#e7f7ef,stroke:#31835e,color:#173f2d - classDef store fill:#fff6dd,stroke:#b78922,color:#513d0c - classDef choose fill:#fcebdc,stroke:#c46a25,color:#572d0c - classDef report fill:#f2eafe,stroke:#7950b3,color:#34204f - - subgraph DISCOVERY[1. Discover every replaceable site] - WL["Input workload
one or more named pre-ASAP OperatorNode roots"]:::input - SEARCH["search_workload_with
run CSE once, then visit every node in every root DAG"]:::generate - TARGET["TargetSubDAG
one candidate site plus the number of workload locations
that reference the same Rc<OperatorNode>"]:::generate - WL -->|"roots"| SEARCH -->|"one target per distinct node"| TARGET - end - - subgraph GENERATION[2. Generate all legal alternatives at each site] - STRATEGY["ReplacementStrategy
when a target matches, enumerate every legal replacement;
implementations generate but do not choose"]:::generate - CAND["ReplacementSubDAG candidates
each contains a Subtree (summary or logical rewrite) or ExactComposition
plus typed provenance and rationale;
no alternative is removed solely on cost"]:::store - TARGET -->|"try every registered strategy"| STRATEGY --> CAND - CM(["CostModel
orders candidates and supplies
deployment-specific parameters"]):::choose - CM -. "rank and parameterize; accuracy checks remain required" .-> STRATEGY - end - - subgraph SEARCHSPACE[3. Store the workload-wide search space] - SPACE["CandidateLogicalASAPDAGs
one TargetSubDAGCandidates per target; each candidate set keeps
all candidates, including dependent compositions"]:::store - CAND -->|"deduplicate by target and candidate identity"| SPACE - end - - subgraph RANKING[Optional ranked view] - SORT["CandidateLogicalASAPDAGs::cost_sorted
use the CostModel to order each candidate set
and cost every candidate"]:::choose - RANKED["RankedTargetSubDAGCandidates
the same candidates in preferred order,
with costs aligned by index"]:::choose - SPACE --> SORT -->|"reorder only; preserve every candidate"| RANKED - end - - subgraph REPORTING[Optional reporting view] - EXPLAIN["explain_replacements
select reportable candidates, copy their rationale,
and add kind, location, target, and node_hash"]:::report - EXPORT["dag_export
narrow by node_hash, then confirm structural equality"]:::report - VIEWER["dag-viewer
show a badge and explanation beside that node"]:::report - SPACE -->|"reporting view; no new planner decision"| EXPLAIN --> EXPORT --> VIEWER - end +Audience: developers changing the planner. This document maps the planning +stages of the [design](../design_docs/proposals/planner-layering.md) to the +code. For the public types and their rules, read the +[contracts](asap-aware-mapping-contracts.md); to add behavior, follow +[Extend ASAP-aware mapping](extend-asap-aware-mapping.md). + +## Crates + +| Crate | Role | +| --- | --- | +| `asap-types` (`crates/types`) | The IR (`ir`: `OperatorNode`, `NonASAPOp`, `ASAPOp`, timing, export), workload types (`workload`), accuracy types (`types`: `AccuracyTarget`, `AccuracyMetric`), summary types (`post_asap`: `SketchAlgorithm`, `FamilyCapability`, `AccuracyGuarantee`) | +| `asap-frontend-sql`, `asap-frontend-promql`, `asap-frontend-metricsql` (with `asap-frontend-common`) | Stage 0, the query-language frontends: lower each query into the logical DAG (`QueryRoot`s of `NonASAPOp` nodes, no summaries, no timing) | +| `asap-aware-mapping` (`crates/asap-aware-mapping`) | Stages 1–3 | +| `asap-planner` (`crates/planner`) | The facade: `e2e_plan` runs Stage 0, then the optimization pass (`Stage1ToStage3Pass` by default) | +| `asap-physical-operators` | The executor's operators and summary kernels over `asap_sketchlib`; it runs a `PhysicalASAPDAG` | +| `asap-devtools` | `stage_pipeline` and the other inspection tools, the design-doc example workloads (`planner_layering_examples`) and the candidate labels | + +Dependencies point one way: the frontends and `asap-aware-mapping` depend on +`asap-types`, never on each other; `asap-planner` depends on all of them. + +## The stages in `asap-aware-mapping` + +```text +QueryRoots (Stage 0) + │ per_computation_candidates::generate_per_computation_candidates Stage 1, Pass 1 + ▼ +PerComputationCandidates ──without_unsupported_summaries(deployment)──┐ + │ pass::stage1_to_stage3::stage1_logical_asap_optimization │ Stage 1, enumeration + │ compose_logical_candidate per choice, then asap_aware_cse rules │ + Pass 2 + ▼ │ +Stage1Candidate / LogicalCandidate (each with its Pass2Candidate) ◄──┘ + │ stage2_physical_asap_optimization::optimize_physical_asap Stage 2 + ▼ +PhysicalCandidate (one per materialization assignment) + │ stage3_plan_selection::select_plan Stage 3 + ▼ +Selection (selected id and plan, costs, accepted, rejected, front) ``` -The generic `ReplacementStrategy` box is the extension point. The default -registry supplies summary realization, Hydra grouping, shared-sub-DAG, -average-rewrite and exact-composition strategies. Section 3.3 describes the -registries and the workload-derived roll-up rule. - ---- - -## 3. How the current pieces fit together - -The planner repeats one operation throughout the workload: find a target, ask -each registered strategy for every valid replacement, and store those -replacements as alternatives for that target. Ranking happens only after the -complete alternative set has been built. - -### 3.1 Discover targets across the workload - -Use `search_workload` or `search_workload_with` for normal planner search. The -search performs these steps: - -1. Run CSE once to merge structurally identical sub-DAGs that may legally be - shared. -2. Walk the complete DAG beneath every query root, including nodes below - unshared parents. -3. Construct one `TargetSubDAG` per distinct node, with the node's measured - `consumer_count`. -4. Run every registered `ReplacementStrategy` against each target to a - **fixpoint**: repeat until no new candidates are discovered, subject to - `MAX_SEARCH_ITERATIONS`. Search also prepares compatible compositions. - `search_workload_with_targets` applies explicit per-root accuracy targets - before ranking. - -The discovery and strategy-invocation path is: - -```mermaid -flowchart LR - classDef workload fill:#e7f7ef,stroke:#31835e,color:#173f2d - classDef common fill:#fff6dd,stroke:#b78922,color:#513d0c - - ROOTS["Input
one or more named OperatorNode roots"]:::workload - ROOTS --> CSE["Canonicalize sharing
merge structurally identical, legally shareable sub-DAGs"]:::workload - CSE --> WALK["Discover sites
walk the complete DAG, including nodes below unshared parents"]:::workload - WALK --> T["Build TargetSubDAG
retain the sub-DAG's Rc identity and measured consumer_count"]:::workload - T --> MATCH - MATCH["matches(target)
cheaply decide whether this strategy has alternatives"]:::common - MATCH -->|"true"| REPLACE["propose(target)
construct supported legal alternatives;
retain structured accuracy rejections"]:::common - MATCH -->|"false"| NONE["No candidates
continue with the next strategy"]:::common - REPLACE --> OUT["Candidate list for this strategy and target
each ReplacementSubDAG carries the replacement and rationale"]:::common -``` - -`consumer_count` is workload information, not an estimate of runtime -executions. It matters to strategies such as `SharedSubDAGStrategy`, which -only has a share-versus-recompute choice when a target has multiple consumers. - -### 3.2 Generate candidates through `ReplacementStrategy` - -For each target, the planner first calls `matches(target)`. A matching strategy -then supplies accepted candidates and structured rejections through -`propose(target)`. Its default wraps `replacements(target)`; strategies with -accuracy checks can override it. - -Each returned `ReplacementSubDAG` contains: - -- a `Replacement`: a constructed summary, logical rewrite or dependent exact - composition; -- the proposing strategy name and typed provenance; and -- the rationale for offering that replacement. - -The containing `TargetSubDAGCandidates` records the target. Legality includes required schema, -capability and accuracy checks; supported algorithm applicability alone is not -a result certificate. - -The complete `replacements()` result is the candidate set produced by one -strategy for one target. A strategy may order or parameterize candidates with -help from a `CostModel`, but it must not remove a valid candidate because of -cost. - -### 3.3 Current concrete strategies - -The default context-free registry contains five `ReplacementStrategy` implementations: - -- `ASAPStrategies` matches supported aggregate and binary shapes. Its - `replacements(target)` method constructs every legal post-ASAP summary sub-DAG, - including applicable sketch, exact-accumulator, and pass-through - realizations. Candidates are sized and ordered for the target's accuracy - requirement; candidates without a sufficient guarantee are rejected before - costing. -- `SharedSubDAGStrategy` uses `consumer_count` to identify shared targets. It - emits both build-once-and-share and recompute-independently rewrites when a - target has multiple consumers. -- `HydraGroupingStrategy` proposes eligible shared multi-subpopulation layouts. -- `AvgToSumOverCountStrategy` proposes supported average rewrites. -- `ExactCompositionStrategy` preserves child-target references for compatible - composition selection. - -`default_strategies_with` uses `SemanticEquivalentRewriteStrategy` in its rewrite -slot. The evidence-aware registry supplies the accuracy evidence provider to -summary and Hydra construction. Search derives `RollupStrategy` after CSE from -the actual sibling set. See the -[registry definitions](../../crates/asap-aware-mapping/src/replacement.rs). - -The important rule is: - -> Strategies should reuse existing decision and implementation logic where possible instead of reimplementing it. - -These strategies expose their alternatives through the same -`ReplacementSubDAG` interface, so search and reporting do not need -strategy-specific discovery logic. - -### 3.4 Store and rank the complete search space - -Workload search deduplicates candidates into a `CandidateLogicalASAPDAGs`. Each distinct -target has one `TargetSubDAGCandidates` containing retained alternatives and -rejection reasons. This -compact representation preserves independent choices without enumerating a flat -list of `2^N` complete plans for `N` replaceable targets. - -`CandidateLogicalASAPDAGs::cost_sorted` ranks each target's existing candidates with the -supplied `CostModel`. It returns the same candidates in preferred order, with -costs aligned by index; ranking does not select or remove a candidate. - -### 3.5 Single-target use and final selection - -`TargetSubDAG::new(&root)` creates a target for one isolated node and sets -`consumer_count` to `1`. It is useful for tests and focused tooling, but it does -not discover targets or provide workload-level sharing information. Use -`search_workload` or `search_workload_with` whenever accurate consumer counts -matter. - -A single-target inspection caller may take the first -candidate with `.into_iter().next()` and handle the empty case according to its -execution policy. Constructing all candidates before taking the first costs -more than constructing only the preferred candidate, but it keeps the strategy -contract consistent and preserves the full choice set for other callers. - -`CandidateLogicalASAPDAGs::global_selection` optionally coordinates cross-target sharing and -composition choices. `GlobalSelection::assemble_selected_dag` constructs the selected -semantic DAG. These APIs do not decide materialization or establish physical -deployment feasibility. Recurrence-aware variants require the corresponding -workload and evidence inputs; downstream owns physical commitment and execution. -See the [library workflow](library-api.md#optional-whole-plan-selection-and-dag-assembly). - ---- +`pass::run_stage1_to_stage3` runs all of it over lowered roots and returns +every stage's candidates (`Stage1ToStage3Run`); `Stage1ToStage3Pass` is the +same pipeline behind the `OptimizationPass` trait that `asap-planner` calls. + +| Concern | Where it lives | +| --- | --- | +| Pass 1: which computations a query has (`ComputationTarget`), how each can be computed (`ComputationCandidate`: `Exact`, `ExactAccumulator`, `Summary`), fused computations (`FusedComputation`), pruning (`PruneReason`) and composing one choice into a logical DAG (`compose_logical_candidate`) | `per_computation_candidates.rs` | +| Which summary families serve an intent (`summary_candidates`), the intent's accuracy target and (ε, δ) budget, default sizing | `realization.rs` | +| Pass 2 rules: identical-expression, summary-capability, window-composition (tumbling panes), rollup and exact top-k reuse; each produces `*Candidate` parts and `*Pass2Candidate` whole-workload plans | `asap_aware_cse.rs` | +| Enumeration of Stage 1 candidates (the Cartesian product of Pass 1 choices, each followed by its Pass 2 candidates) | `pass/stage1_to_stage3.rs` | +| What each summary family can merge and answer (`FamilyCapability::of`: `merge_inputs`, `readouts`) | `asap-types`, `post_asap/family_capability.rs` | +| Stage 2: materialization options per summary build (`NotMaterialized`, `MatAtIngestion`, `MatAtQuery`, `MatAtQueryKeptAcrossEvaluations`), ingestion-time eligibility, physical operator implementation | `stage2_physical_asap_optimization.rs` | +| Timing and data-state edge rules applied from a `MaterializationAssignment` | `asap-types`, `ir/timing.rs` and `post_asap/execution_data_state.rs` | +| Stage 3: accuracy check, latency check, deployment-capability checks, resource usage (`ResourceUsage`), cost models 1–3 (`CostModel`), measured operation costs | `stage3_plan_selection.rs`, `stage3_plan_selection/measured_operation_costs.rs` | +| The analytical per-operator resource model Stage 3 measures with | `analytical_cost.rs`, `physical_operator_statistics.rs` | +| The accuracy algebra: per-family guarantees, composition, budget split | `accuracy/` (`AccuracyModel`, `DefaultAccuracyModel`, `estimators/`, `composition.rs`, `budget.rs`) | +| What the executor supports (`DeploymentCapabilities`) | `deployment_capabilities.rs` | +| The inputs every stage reads (`PlanningModels`: accuracy model, evidence, plan selection settings, deployment capabilities) and the `OptimizationPass` trait | `pass/mod.rs` | + +Not yet wired into the stages: `maintained_population.rs` (maintained +populations for instant top-k, quantile and sum), `pane_sharing.rs`, +`storage_io.rs`, `physical_handoff_cost.rs` and `query_physical_lowering.rs`. + +## Algorithm in one pass + +1. **Pass 1** walks each root, finds every aggregate computation, and gives it + one exact option plus the summary families of `summary_candidates` that the + accuracy target permits and the deployment supports. A whole-expression + top-k or a nested sum also gets a fused computation that absorbs the + target beneath it. +2. **Enumeration** takes one choice per target and composes the logical DAG. + For each composed DAG, the Pass 2 rules find what can be shared, split into + panes or reused, and each subset of their candidates becomes another + Stage 1 candidate. +3. **Stage 2** gives each summary build its materialization options and emits + one physical candidate per assignment (the all-`NotMaterialized` one + first), skipping assignments that put a build at ingestion time over one + that is not. +4. **Stage 3** checks each physical candidate: every query's root guarantee + against its target (binding the statistics Stage 1 left symbolic from the + `DataWorkload`), every query's latency against `max_latency_ms`, raw-data + scans against `raw_data_retained`, and, under cost model 3, the cluster's + limits. It measures the valid ones and selects one under the cost model; + ties keep enumeration order. + +No stage caps the candidates it plans; `stage_pipeline` caps only what it +displays and refuses workloads over `--max-combinations`. + +## Verification + +- Unit tests sit beside each module (`cargo nextest run -p asap-aware-mapping + -E 'test()'`). +- `crates/asap-aware-mapping/tests/per_computation_candidates.rs` covers + Pass 1. +- `crates/integration-tests/tests/planner_layering_example{1,2,3,4}.rs` pin + the design document's examples through every stage; + `reuse_rules.rs`, `deployment_capabilities.rs`, `materialization_e2e.rs` + and `stage1_logical_asap_optimization.rs` cover single rules end to end. +- `crates/devtools/tests/stage_pipeline.rs` checks the stage document and the + committed viewer example. diff --git a/docs/develop_docs/asap-aware-mapping-contracts.md b/docs/develop_docs/asap-aware-mapping-contracts.md index 02630e91f..22669b7cc 100644 --- a/docs/develop_docs/asap-aware-mapping-contracts.md +++ b/docs/develop_docs/asap-aware-mapping-contracts.md @@ -1,392 +1,116 @@ # ASAP-Aware Mapping contracts -This reference defines the current public concepts and interface contracts used -by ASAP-aware mapping. Read the [architecture](asap-aware-mapping-architecture.md) -first; use the [extension guide](extend-asap-aware-mapping.md) when changing one. - -## Interfaces and definitions - -## 1. Glossary - -### `TargetSubDAG` - -A pre-ASAP `OperatorNode` that a strategy may replace. - -```rust -pub struct TargetSubDAG<'a> { - pub root: &'a Rc, - pub consumer_count: usize, -} -``` - -`root` is the actual `Rc` from the workload. - -`consumer_count` counts structural references, not runtime executions. It is the number of places in the workload DAG that point to this exact `Rc` node. - -For example, consider two top-level queries: - -- `sum by (service) (rate(m[5m]))` -- `avg by (service) (rate(m[5m]))` - -After `share_common_sub_dags` merges their identical `rate(m[5m])` sub-DAGs, both query trees point to the same `Rc`. That node's `consumer_count` is `2`, regardless of how often either query executes. - -Use: - -```rust -let target = TargetSubDAG::new(&root); -``` - -when you are inspecting a node in isolation. - -Use: - -```rust -let target = TargetSubDAG::with_consumer_count(&root, count); -``` - -when the caller already knows the real number of consumers. - -`TargetSubDAG::new` assumes one consumer. - ---- - -### `Replacement` - -The actual object that substitutes the target. - -There are currently two forms: - -```rust -pub enum Replacement { - SubDAG(Rc), - ExactComposition(ExactComposition), -} -``` - -Use `Replacement::SubDAG` for a replacement sub-DAG. It is one of: - -- a constructed post-ASAP summary plan: the sub-DAG contains an `ASAPOp` - (`SummaryAgg`, `SummaryEstimate`, ...); -- a logical rewrite: only `NonASAPOp` nodes and no guarantee yet. - -`is_logical_rewrite(&node)` tells the two apart. A kept pre-ASAP sub-DAG -(`retain_exact`) has no ASAP operator but carries an exact guarantee, so it -counts as a bound decision, not a rewrite. - -Use `Replacement::ExactComposition` when an exact operation refers to a child -target whose realization must remain undecided. Selection coordinates the -parent/child pair; DAG assembly constructs and validates the composed DAG. -See [exact_composition.rs](../../crates/asap-aware-mapping/src/exact_composition.rs). - -Examples: - -```text -Quantile(...) - -> SummaryEstimate(SummaryAgg(KLL)) -``` - -is a summary `Subtree`; KLL (Karnin–Lang–Liberty) is a quantile-sketch algorithm. - -```text -compute independently - vs. -reuse an already shared logical sub-DAG -``` - -is represented as two logical-rewrite `Subtree`s. - ---- - -### `ReplacementSubDAG` - -One candidate replacement plus an explanation. - -```rust -pub struct ReplacementSubDAG { - pub replacement: Replacement, - pub strategy: &'static str, - pub provenance: ReplacementProvenance, - pub rationale: String, -} -``` - -The `rationale` is for debugging, reporting, and explaining planner choices. It is human-readable text, not a machine-readable protocol. - -Every candidate should carry a useful rationale. Search records the strategy -name, and selection uses typed provenance instead of inferring candidate roles -from the rationale or pointer identity. - ---- - -### `ReplacementStrategy` - -The main extension point for adding a new optimization or replacement source. - -```rust -// Required methods; the trait also provides default name and propose methods. -pub trait ReplacementStrategy { - fn matches(&self, target: &TargetSubDAG<'_>) -> bool; - - fn replacements( - &self, - target: &TargetSubDAG<'_>, - ) -> Vec; -} -``` - -The two required methods have intentionally different responsibilities. -`name` supplies the diagnostic strategy name. `propose` returns accepted -candidates and structured accuracy rejections; its default wraps `replacements`. -Search calls `propose`, so rejected candidates remain available for explanation. - -`matches` answers: - -> Does this strategy have anything to offer for this target? - -`replacements` answers: - -> What are all semantically valid alternatives for this target? - -`replacements` must be **exhaustive and not cost-filtered**. When its output has -a preferred order, that ordering must come from the supplied `CostModel`; the -strategy must still return every supported legal candidate. Required accuracy, -schema and capability checks can reject an otherwise applicable algorithm; -exhaustiveness is not a promise of all theoretically possible plans. - ---- - -### `Realization` - -One valid realization of an `AggIntent`, the pre-ASAP description of what an -aggregation must compute without committing to a physical summary algorithm. -A realization may be an approximate sketch, an exact mergeable -accumulator, or a pass-through that keeps the original operation instead of -building a summary. `realizations_for_intent` enumerates these concrete -realizations; `ASAPStrategies::replacements()` constructs each one as -a `ReplacementSubDAG`. It returns all -candidates in preferred order without selecting a winner. At workload scale, -`search_workload`/`search_workload_with` preserve all supported legal alternatives -across every `TargetSubDAG`. Optional planner APIs coordinate compatible semantic selections; physical -commitment and placement remain downstream deployment decisions. - -This guide uses the Cascades/Volcano terminology: - -- An **implementation rule** maps a logical operation to a candidate physical - realization. For example, a quantile `AggIntent` may have KLL and DDSketch - `Realization` values. -- A **transformation rule** maps a logical operation to another logical - operation. In this crate, that kind of candidate is a logical-rewrite - `Replacement::SubDAG`. -- A **replacement candidate** packages either kind of result as a - `ReplacementSubDAG` for search. `CandidateLogicalASAPDAGs` stores and ranks these candidates. -- **Physical commitment and placement** happen downstream. An `Realization` - therefore does not mean that the planner has committed the workload to that - choice. - -The concrete flow is: - -```text -AggIntent - -> realizations_for_intent(): enumerate Realization values - -> ASAPStrategies: construct ReplacementSubDAG candidates - -> CandidateLogicalASAPDAGs: store and rank candidates - -> downstream deployment: select and place a final choice -``` - -`ReplacementStrategy` enumerates supported legal candidates. The caller can -consume ranked candidate sets or use coordinated selection and DAG assembly; -see [code architecture §3](asap-aware-mapping-architecture.md#3-how-the-current-pieces-fit-together). - ---- - -### `CostModel` - -`CostModel` covers every deployment-specific numeric or configuration decision—not only which candidate is cheapest. For example, sketch sizing trades memory and update cost for accuracy, so it belongs here too. - -The crate cannot hardcode real deployment costs: `asap-aware-mapping` uses `asap-types` and pinned `asap_sketchlib` mapping -bounds, but does not execute workloads or own deployment measurements. Most hooks therefore provide the crate's built-in static behavior as a default. Override only the decisions your deployment needs to change. - -| Hook | Use it to | Default? | -|---|---|---| -| `rank_candidates` | Order valid sketch algorithms | No | -| `size_params` | Convert an accuracy target into sketch parameters | Yes | -| `realize_extension` | Map a custom intent to a realization | Yes | -| `evaluation_extension` | Query a custom extension summary | Panics until paired with a custom realization | -| `cse_recompute_cost` | Estimate independent recomputation | Yes | -| `cse_shared_maintenance_cost` | Estimate shared maintenance | Yes | -| `cse_share_decision` | Choose sharing or recomputation | Yes | -| `estimate_cost` | Attach a comparable numeric cost to a replacement | Returns `NaN` (IEEE “not a number”); `DefaultCostModel` provides real values | - -- **`rank_candidates`** — order the sketch candidates for one `AggIntent`, best first. This is the only required hook. - - ```rust - fn rank_candidates(&self, intent: &AggIntent, candidates: &[SketchAlgorithm]) -> Vec; - ``` - -- **`size_params`** — choose parameters, such as sketch capacity, for an already-selected `SketchAlgorithm` and accuracy target `(eps, delta)`, where `eps` is the tolerated error and `delta` is the tolerated probability of exceeding that error. It is separate from ranking so a deployment can customize sizing without changing algorithm preference. The trait provides a default implementation. - - ```rust - fn size_params(&self, kind: SketchAlgorithm, intent: &AggIntent, eps: f64, delta: f64) -> SketchParams; - ``` - -- **`realize_extension`** — map a deployment-defined `AggIntent::Extension` to a post-ASAP `Realization`. The default is `Realization::PassThrough`. - - Use `AggIntent::Extension { ext_kind, payload }` for intent shapes that only your deployment needs. Core treats both fields as opaque. For example, a deployment can tag an approximate-frequency intent with `ext_kind: "frequency"` and recognize it in `realize_extension`: - - ```rust - fn realize_extension(&self, ext_kind: &str, _payload: &serde_json::Value) -> Realization { - if ext_kind == "frequency" { - Realization::Sketch(SketchKind::new( - SketchAlgorithm::CountSketch, - SketchParams::CountSketch { width: 1024, depth: 5 }, - )) - } else { - Realization::PassThrough // fall back to the default for anything else - } - } - ``` - - Return `Realization::PassThrough` for unrecognized extension kinds. Do not panic. - - ```rust - fn realize_extension(&self, ext_kind: &str, payload: &serde_json::Value) -> Realization; - ``` - -- **`evaluation_extension`** — define how queries read an extension summary that `realize_extension` mapped to a `Sketch`. The two hooks are a pair: realization defines what is maintained; evaluation defines how it is queried. Override both for the same `ext_kind`. The default evaluation panics to prevent a silent wrong answer. - - ```rust - fn evaluation_extension(&self, ext_kind: &str, payload: &serde_json::Value, col: &ColumnRef) -> SketchStatistic; - ``` - -- **`cse_recompute_cost`** — estimate the one-time cost of recomputing a CSE candidate's sub-DAG independently at a single consumer. Default: `default_cse_recompute_cost`, a structural-size proxy. - - ```rust - fn cse_recompute_cost(&self, candidate: &CseCandidate) -> Cost; - ``` - -- **`cse_shared_maintenance_cost`** — estimate the cost of maintaining one shared summary continuously for the life of the workload. Default: `default_cse_shared_maintenance_cost`, a per-family weight table. - - ```rust - fn cse_shared_maintenance_cost(&self, candidate: &CseCandidate) -> Cost; - ``` - - Both hooks return `Cost`, currently a unitless `f64` newtype. The wrapper allows the type to grow later—for example, to separate CPU, memory, and network costs—without changing every hook signature. - -- **`cse_share_decision`** — choose between one shared summary and independent recomputation at each consumer. By default, it shares when maintenance cost is no greater than total recomputation cost. Override the two cost inputs first; override this decision hook only when you need a different policy. - - ```rust - fn cse_share_decision(&self, candidate: &CseCandidate) -> ShareDecision; - ``` - -- **`estimate_cost`** — attach a comparable numeric cost to an already-constructed replacement. `CandidateLogicalASAPDAGs::cost_sorted` calls it for every candidate and keeps the returned values aligned with the ranked candidates. The trait default returns `f64::NAN` deliberately; override it when a custom model's callers need displayable or otherwise consumable numeric costs. `DefaultCostModel` provides real values derived from its CSE cost hooks. - - ```rust - fn estimate_cost( - &self, - candidate: &ReplacementSubDAG, - target: &TargetSubDAG<'_>, - ) -> f64; - ``` - -A custom cost model does not necessarily need to override every hook. The current tests include a model that overrides only `rank_candidates`, relying on defaults for the rest. Such a minimal model inherits `estimate_cost`'s `NaN` placeholder; it must also override `estimate_cost` if consumers require numeric costs. - ---- - -### `CandidateLogicalASAPDAGs` / `TargetSubDAGCandidates` / `RankedTargetSubDAGCandidates` — the whole-workload view - -`ReplacementStrategy` answers "what are the candidates for this one target?" `CandidateLogicalASAPDAGs` answers the same question for every target in a whole workload at once, without enumerating `2^N` fully-copied plans for `N` independently-choosable sites. - -```rust -// replacement.rs - -// One TargetSubDAGCandidates per distinct TargetSubDAG in the whole workload — -// never a flat list of fully assembled plans. -pub struct TargetSubDAGCandidates { - pub target: Rc, - pub consumer_count: usize, - pub candidates: Vec, // accepted alternatives, unranked - pub rejected: Vec, // failed accuracy checks -} - -pub struct RankedTargetSubDAGCandidates<'a> { - pub target: &'a Rc, - pub consumer_count: usize, - pub candidates: Vec<&'a ReplacementSubDAG>, // same candidates, ranked - pub costs: Vec, // costs[i] <-> candidates[i] -} -``` - -`search_workload(roots)` runs the shared-sub-DAG pass once, discovers every target across every root's whole DAG (not just root-level sharing — a `SharedSubDAGStrategy` candidate three levels under an unshared `Filter` is exactly as real a site as a shared whole root), and asks every registered strategy to a fixpoint. Two logically different candidates at two different targets are never copied into two separate plans — they're two entries in two different `TargetSubDAGCandidates`s, sharing every other node in the workload by construction. - -`CandidateLogicalASAPDAGs::cost_sorted(cost_model)` is the one ranking step: for each candidate set, it dispatches by candidate shape — the `SharedSubDAGStrategy` share/recompute pair (recognized by `ReplacementProvenance::CseShare`/`CseRecompute`) goes through `CostModel::cse_share_decision`; a set with a Hydra shared-grid alternative goes through `CostModel::grouping_state_cost`; a set whose candidates all realize sketches (a `ASAPStrategies` choice) goes through `CostModel::rank_candidates`; and any other mixed set is ordered by `CostModel::candidate_cost`. Every candidate gets a numeric cost aligned index-for-index in `costs`. Count in, count out—nothing is dropped to produce a ranking. Legality checks -may already have removed proposals before this boundary. In particular, -`search_workload_with_targets` checks explicit per-root targets, while retaining -direct DDSketch ratios with missing domain evidence and no root guarantee for -backend inspection. Automatic selection skips those unproven ratios. Use -`global_selection` for coordinated sharing/composition choices and -`GlobalSelection::assemble_selected_dag` for the resulting semantic DAG; neither deploys it. - ---- - -### Family, category, algorithm, and parameters - -A summary's identity has four levels: family (`FieldDataType` variant), sketch -category (`SketchCategory`), algorithm (`SketchAlgorithm`), and the validated -committed choice (`SketchKind`). The levels and their validation are specified in -[Schema and physical data for ASAP primitives](../design_docs/proposals/asap-primitive-schema.md#3-proposed-schema-design). - -Where this matters in practice: `CostModel::rank_candidates`, `CostModel::size_params`, and `ASAPStrategies::replacements` operate at the **algorithm** level. `summary_candidates(intent)` returns a list of `SketchAlgorithm`s (`[Kll, DDSketch]` for a `Quantile` intent), never a bare `SketchKind` with nothing chosen underneath it. `SketchKind` appears after an algorithm has been selected and sized—on `Realization::Sketch(SketchKind)` and `FieldDataType::Sketch(SketchKind, GroupingStrategy)`. - ---- - -### `Matcher` - -`Matcher` is a smaller, separate extension point: - -```rust -pub trait Matcher { - fn is_satisfied_by(&self, required: &Realization, available: &Realization) -> bool; -} -``` - -`Matcher` does not decide how to build a summary. It asks whether an existing summary can satisfy a required realization without building anything new. This is similar to a database reusing a materialized view or index. - -For example, a deployment may have a DDSketch for `latency` while a request -was planned with KLL. A shared quantile query category is insufficient to prove -substitutability: rank-error and relative-value guarantees differ. The deployment -must check the required guarantee, parameters, grouping, coverage and storage -compatibility before reusing state. `Matcher::is_satisfied_by(required, available)` lets the deployment make that decision: `required` is what the query needs, and `available` is what the inventory already contains. - -The crate provides no default `Matcher` implementation because the answer depends on deployment-specific inventory and storage rules. If you need one, implement the complete trait for your deployment. - ---- - -## 2. Replacement explanations (`explanation.rs`) - -`explanation::explain_replacements`/`explain_replacements_with` answer a different question than everything above: not "what could this target become" (`ReplacementStrategy::replacements`) but "why does the replacement already discovered for this target exist, and where." It is a **reporting view over `CandidateLogicalASAPDAGs`**, not a second search or a second rule engine — this crate's *explanation of a replacement*, not an applicability classifier deciding admissibility from scratch. - -### The rule - -> A `TargetSubDAG` is worth explaining exactly when its `CandidateLogicalASAPDAGs` candidate list contains something beyond the trivial, no-op realization. - -Concretely, `explanation.rs` reports three candidate kinds from each `TargetSubDAGCandidates`: - -- `ExplanationKind::SketchApproximation` — the set contains a summary `Replacement::SubDAG` that realizes `FieldDataType::Sketch(..)`, not just an exact/pass-through candidate. -- `ExplanationKind::CommonSubexpressionReuse` — `consumer_count >= 2` and the set contains `SharedSubDAGStrategy`'s "build once and share" candidate (the `Replacement::SubDAG` whose `Rc` is the set's `target`). - -- `ExplanationKind::ExactComposition` — the candidate set contains an exact operation - composed with a child target whose realization remains a coordinated choice. - -Each `ReplacementExplanation::reason` is copied verbatim from the matching candidate's own `ReplacementSubDAG::rationale`. Nothing in `explanation.rs` re-explains why a candidate is valid; that explanation already exists exactly once, on the candidate itself. - -`ReplacementExplanation` carries both `node_hash` and `target`. A downstream consumer first compares `node_hash` with an exported `DAGNode::hash` to narrow the search, then compares the exact `target` node with the exported node's in-process `DAGNode::source_node`. This preserves the hash's role as a fast filter while making the final association collision-safe; `location` remains human-readable presentation text rather than a machine identifier. - -### Why there is no `ExplanationRule` trait - -Explanations are derived from candidates already present in `CandidateLogicalASAPDAGs`. A new candidate kind therefore requires an `impl ReplacementStrategy` wired into `default_strategies`/`default_strategies_with`; a second explanation-specific trait would duplicate registration and could drift from the actual search space. Custom callers supply strategies through `explain_replacements_with`, using the same extension point exposed by `search_workload_with`. - -### How it derives `location` text - -`CandidateLogicalASAPDAGs`/`TargetSubDAGCandidates` track `Rc` pointer identity, not human-readable breadcrumbs. `ReplacementExplanation::location` provides prose such as `root "dash_a" > lhs` so reporting consumers can identify the relevant part of the query without interpreting pointer identity. Location derivation does not make replacement or costing decisions. - ---- +Audience: developers changing the planner. This reference states the rules +the stage code relies on: what each stage's input and output may contain, +and which component owns each decision. Read the +[architecture](asap-aware-mapping-architecture.md) first; use the +[extension guide](extend-asap-aware-mapping.md) when changing one. Field-level +details are in the rustdoc (`cargo doc -p asap-aware-mapping -p asap-types +--no-deps`). + +## 1. IR at each stage + +| After | The DAG holds | Timing | +| --- | --- | --- | +| Stage 0 (frontends) | `NonASAPOp` nodes only; aggregates carry `AggIntent` measures with each query's `AccuracyTarget` as written | none | +| Stage 1 (`compose_logical_candidate`, Pass 2 rules) | `ASAPOp` nodes where a candidate chose an exact accumulator or a summary (`SummaryAgg`, `PaneChain`, `SummaryMerge`, `SummaryEstimate`, `FinalizeExactAccumulator`); everything else stays `NonASAPOp` | none: Stage 1 never decides when anything runs | +| Stage 2 (`optimize_physical_asap`) | the same operators, with a `Materialization` on each summary build | every node timed from the candidate's `MaterializationAssignment` | + +- Structurally equal sub-DAGs are interned before Stage 1 + (`asap_types::ir::cse`), so Stage 1 compares nodes by structure and Pass 2 + matches readers structurally, never by pointer. +- Timing is applied with `apply_materialization_timings`, which validates + every data-state edge. A violation is an `ExecutionDataStateError`: + for example `QueryTimeValueUnderIngestionSummary` (a query-time value + beneath an ingestion-time build), `ConflictingTiming` (one shared node + needs two timings; `split_shared_by_timing` copies it for one side first) + or `IllegalChildDataState`. These are internal errors in a candidate the + stages built, not bad user input. +- A summary build's `SummaryCoverage` (which rows and time range its state + represents) is derived, never declared; a merge's inputs must be disjoint + as its family's `MergeInputRule` requires, or coverage fails + (`InputsOverlap`). + +## 2. Stage 1 candidates + +- **Targets.** `ComputationTarget` is one measure of one `Aggregate`. Its + `choices` list the exact option first (`ExactAccumulator` when the intent + has a mergeable exact state, else `Exact`), then each `Summary(family)` from + `realization::summary_candidates`. Candidates are unranked: Pass 1 never + compares cost. +- **Pruning.** A candidate that cannot meet the target is moved to the + target's pruned list with a `PruneReason` (`RequiresExact`, + `NullableAvgInput`, `UnsupportedByDeployment`); it is never silently + dropped. Pass 1 prunes only what it can prove; accuracy that depends on + data facts is left to Stage 3. +- **Fused computations.** `FusedComputation { absorbs, candidates, pruned }` + replaces the target directly beneath (the whole-expression top-k, the + nested sum). A whole-workload choice keeps one digit per target; a fused + pick with a non-first digit on the absorbed target is skipped, so each + plan is enumerated once. +- **Pass 2.** Each rule adds candidates; none removes a Pass 1 candidate, adds + a summary family, or adds a readout. So a deployment that does not support a + family never sees it again after Pass 1. A Pass 2 candidate records what it + changed in `Pass2Candidate` (shared builds, panes, rollups, top-k reuses, + identical-node sharing) and keeps its Pass 1 candidate's `choice`. +- **Sizing.** A summary shared by several readers is sized with each + parameter's largest value among them. A summary is sized for ε in Stage 1; + a δ that depends on data facts (a distinct-count bound, series counts) is + left symbolic in its guarantee for Stage 3 to bind. + +## 3. Accuracy + +- `AccuracyModel` is the extension point: `local_guarantee` (a family's + guarantee over exact input), `propagate` (composition through an operator) + and `satisfies` (a guarantee against a target). It fails closed: no rule, or + a missing input guarantee, means no guarantee, and the candidate is + rejected rather than assumed accurate. +- A target with no `accuracy_metric` accepts a guarantee in any metric that + meets its ε, δ and scope; Stage 3 reports the metric it accepted + (`AcceptedAccuracy`). +- An approximate input split across several inputs gets its share through + `accuracy::budget::per_input_target`. +- Facts come only from the declared `DataWorkload`; an undeclared fact is + unknown, never zero. + +## 4. Stage 2 + +- Only summary builds (`SummaryAgg`, `PaneChain`) get materialization + options: `NotMaterialized` always; `MatAtIngestion(Memory)` when the build + is eligible to run as data arrives; `MatAtQuery(Memory)` when two or more + queries read it (reused within one evaluation); + `MatAtQueryKeptAcrossEvaluations(Memory)` for a tumbling-pane chain that + builds one new pane per evaluation, when the deployment declares + `query_time_results_retained`. +- One `PhysicalCandidate` per assignment, the all-`NotMaterialized` one first + (`P`, then `P-m`). An assignment that puts a build at ingestion + time over one that is not is not generated. A logical candidate Stage 2 + cannot implement becomes a `PrunedCandidate` with its reason. + +## 5. Stage 3 + +- `select_plan` validates its inputs first: `SelectionError::InvalidSettings`, + `EntryCount` and `InvalidDeploymentCapabilities` are bad input and fail the + whole selection. +- Each candidate is then either invalid or valid, recorded as a `Rejection` + with its reason (`valid: false` failed a check: accuracy, latency, raw data + not retained, or a cluster limit under cost model 3; `valid: true` lost + under the cost model). A candidate that cannot be resized for its failure + probability is rejected alone. +- Usage (`ResourceUsage`) is measured without prices; the `CostModel` in + `PlanSelectionSettings` turns it into a choice. Ties keep enumeration order + under every cost model. +- `Selection.plan` is the selected candidate's roots with the weight + requirements Stage 3 confirmed from the data facts. + +## 6. Ownership + +| Decision | Owner | +| --- | --- | +| What a query means | the frontend (Stage 0) | +| Which computations exist and how each may be computed | Pass 1 | +| What is shared, split into panes or reused | Pass 2 | +| When each build runs and whether it is kept | Stage 2 | +| Which plan is valid and which is chosen | Stage 3 | +| What the executor can build, retain and how it is priced | the deployment, through `PlanningModels` (`DeploymentCapabilities`, `PlanSelectionSettings`, `AccuracyModel`) | +| Running the plan | the executor (`asap-physical-operators`), outside the planner | diff --git a/docs/develop_docs/end-to-end-accuracy-guarantees.md b/docs/develop_docs/end-to-end-accuracy-guarantees.md index 639ab02a0..405f2a80e 100644 --- a/docs/develop_docs/end-to-end-accuracy-guarantees.md +++ b/docs/develop_docs/end-to-end-accuracy-guarantees.md @@ -23,7 +23,7 @@ aggregate intent and AccuracyTarget -> propagate guarantees through the registered composition rule -> check the final guarantee against AccuracyTarget -> keep legal candidate or record AccuracyError - -> rank retained candidates with CostModel (ranking does not prove accuracy) + -> Stage 3 prices the valid candidates under the cost model (cost does not prove accuracy) ``` For a single summary, the child is exact and propagation leaves the local @@ -77,9 +77,9 @@ trait AccuracyModel { approximate inputs of one composition (ε / n for an additive metric, (1 + ε)^(1/n) − 1 for a relative one, δ / n). Every candidate must then be resized, propagated, and checked before being treated as satisfying the target. This is distinct from -candidate visibility: direct DDSketch ratios lacking domain evidence remain in -`CandidateLogicalASAPDAGs` with `guarantee: None` and can appear in `cost_sorted`, but automatic -`global_selection` skips them. Presence and cost are not accuracy certification. +candidate visibility: a Stage 1 candidate whose output has no guarantee is +still enumerated and appears in the stage document, but Stage 3 rejects it +for any approximate target. Presence and cost are not accuracy certification. See the [workflow design](../design_docs/architecture/input-output-workflow.md#planning-evidence-inputs) for this boundary. diff --git a/docs/develop_docs/extend-asap-aware-mapping.md b/docs/develop_docs/extend-asap-aware-mapping.md index 06527ccf5..6de779a82 100644 --- a/docs/develop_docs/extend-asap-aware-mapping.md +++ b/docs/develop_docs/extend-asap-aware-mapping.md @@ -1,1018 +1,118 @@ # Extend ASAP-aware mapping -Use this guide after reading the [mapping architecture](asap-aware-mapping-architecture.md) -and consulting the [mapping contracts](asap-aware-mapping-contracts.md). - -## Add or change planner behavior - -## 1. Adding a new `ReplacementStrategy` - -A new optimization should normally be introduced as a new implementation of `ReplacementStrategy`. - -Do not change the trait just because a new optimization is added. - -Start with this skeleton (illustrative — `MyStrategy` is not a real type in this crate): - -```rust -pub struct MyStrategy; - -impl ReplacementStrategy for MyStrategy { - fn matches(&self, target: &TargetSubDAG<'_>) -> bool { - // Return true only when this transformation is applicable. - todo!() - } - - fn replacements( - &self, - target: &TargetSubDAG<'_>, - ) -> Vec { - if !self.matches(target) { - return Vec::new(); - } - - // Enumerate every semantically valid replacement. - todo!() - } -} -``` - -There are four decisions to make. - ---- - -### Define the target shape - -`matches` should contain the minimum structural and semantic checks needed to determine whether the strategy applies. - -For example, the aggregate path in `ASAPStrategies` requires a -supported shape: - -- the node is an `Aggregate`, -- it has one aggregation intent, -- it does not have `HAVING`. - -The strategy also handles supported binary and temporal-average shapes. Consult -its current `matches` and `propose` implementations instead of assuming that every -target is an aggregate. - -A strategy that depends on cross-query context may additionally inspect `consumer_count`. - -For example: - -```rust -fn matches(&self, target: &TargetSubDAG<'_>) -> bool { - target.consumer_count >= 2 -} -``` - -is enough for the current shared-sub-DAG strategy. - -#### Guideline - -Keep `matches` cheap and unsurprising. - -It should answer *whether the strategy applies here* in the plain English sense — not `explanation.rs`'s formal `ReplacementExplanation` concept (issue #257, a separate reporting layer over this trait's own output — see [mapping contracts](asap-aware-mapping-contracts.md#2-replacement-explanations-explanationrs)). `matches` isn't that machinery and doesn't need to produce anything it consumes; it should also not perform ranking or choose a winner. - ---- - -### Enumerate every valid alternative - -`replacements` should return every semantically valid candidate for a matched target. - -For example, if a quantile can be represented by: - -```text -KLL -DDSketch -``` - -then both should be returned. - -Do not write: - -```rust -if kll_is_cheaper { - vec![kll] -} else { - vec![ddsketch] -} -``` - -inside a strategy. - -That would throw away part of the search space before the cost model can compare complete plans. - -Instead: - -```rust -vec![kll, ddsketch] -``` - -and let costing decide later. - ---- - -### Summary sub-DAG vs. logical rewrite - -Both are returned as: - -```rust -Replacement::SubDAG(node) -``` - -- A fully constructed post-ASAP summary: `node` contains an `ASAPOp`. -- A logical pre-ASAP rewrite: `node` has only `NonASAPOp` nodes and no - guarantee. `is_logical_rewrite(&node)` checks this. - -Set `provenance` to say which one it is (`ReplacementProvenance::SummaryRealization`, -`LogicalRewrite`, ...); selection reads provenance, not the sub-DAG's shape. - -Use `Replacement::ExactComposition` when a candidate depends on a child target -whose implementation must be selected compatibly later. Do not bind it to the -first child alternative during enumeration. - -This distinction matters because a rewrite may enable more transformations -later, while a constructed summary represents a concrete summary realization. - ---- - -### Add a rationale - -Every `ReplacementSubDAG` should explain why the candidate exists. - -Good: - -```rust -ReplacementSubDAG { - replacement: ..., - strategy: "MyStrategy", - provenance: ..., // typed role, e.g. LogicalRewrite - rationale: "derive service-level aggregate by rolling up the \ - mergeable service+region aggregate".into(), -} -``` - -Less useful: - -```rust -rationale: "candidate 2".into() -``` - -The rationale should help a developer understand a planner trace without reading the strategy implementation. - -Do not encode machine-readable state into the string. - ---- - -### Strategy contract - -Every new strategy should follow these rules. - -#### Rule 1: `matches == false` should be safe - -The existing strategies return an empty vector when `replacements` is called on a target they do not match. - -Follow the same convention: - -```rust -if !self.matches(target) { - return Vec::new(); -} -``` - -Do not panic simply because the caller skipped a prior `matches` call. - ---- - -#### Rule 2: enumerate; do not rank - -A strategy owns **legality and enumeration**. - -A cost model owns **preference and costing**. - -This separation is the most important extension rule in this module. - ---- - -#### Rule 3: do not duplicate an existing decision procedure - -If another module already knows how to determine whether something is legal or -how to realize it, wrap that logic. - -Do not create a second implementation of the same semantics inside the strategy. - -The existing `ASAPStrategies` is the model to follow: it reuses -`replacement.rs`'s existing candidate list and summary-construction path. - ---- - -#### Rule 4: preserve semantics - -Every returned replacement must be semantically valid for the target. - -Cost differences do not justify semantic differences. - -If a transformation is only valid under additional summary properties, grouping assumptions, or accuracy constraints, check those conditions before returning the candidate. - ---- - -#### Rule 5: a strategy does not need to discover the whole workload - -`TargetSubDAG` is passed into the strategy. - -The strategy is responsible for deciding what to do with that target, not for walking every workload root to discover targets. - -If your transformation requires context not currently represented in `TargetSubDAG`, that is a design question about target metadata or search/discovery — not a reason to hide a second workload traversal inside `replacements`. - ---- - -### Example: current `ASAPStrategies` - -`ASAPStrategies` is the reference implementation for a strategy that -produces constructed post-ASAP summaries. - -Construction: - -```rust -let strategy = - ASAPStrategies::default_cost_model(); -``` - -or with a custom cost model: - -```rust -let model = MyCostModel; // illustrative -let strategy = ASAPStrategies::new(&model); -``` - -The strategy matches supported aggregate nodes. - -At a high level: - -```mermaid -flowchart LR - A["Input TargetSubDAG
root is a supported Aggregate"] --> B["ASAPStrategies::matches
check whether the target shape can produce summaries"] - B -->|"true"| C["ASAPStrategies::replacements
use CostModel preferences and sizing while preserving
every semantically valid realization"] - B -->|"false"| NONE["Empty candidate list"] - C --> F["Output Vec<ReplacementSubDAG>
each entry contains a constructed summary sub-DAG and rationale;
all candidates retained in preferred order"] -``` - -For an approximate quantile, both KLL and DDSketch remain candidates when -their committed parameters and evidence satisfy the applicable accuracy checks, -even if the cost model prefers one. When only one realization is legal, such as an exact accumulator or pass-through, the strategy returns that single candidate. - ---- - -#### Observable behavior - -Call the public strategy interface and inspect every returned candidate: - -```rust -let strategy = ASAPStrategies::new(&cost_model); -let candidates = strategy.replacements(&target); - -for candidate in candidates { - match candidate.replacement { - Replacement::SubDAG(node) => { - // A constructed summary sub-DAG (`node.is_asap()`), or a kept - // pre-ASAP sub-DAG with an exact guarantee for pass-through. - } - Replacement::ExactComposition(_) => unreachable!( - "ASAPStrategies produces sub-DAG candidates" - ), - } -} -``` - -The public contract is the behavior contributors should preserve: every legal -candidate is returned, ordering follows the supplied `CostModel`, each summary -is fully constructed, and each candidate carries a useful rationale. Nested -aggregate choices remain independent. - ---- - -### Example: current `SharedSubDAGStrategy` - -`SharedSubDAGStrategy` is the reference implementation for a logical rewrite strategy. - -It applies when: - -```rust -target.consumer_count >= 2 -``` - -and returns two alternatives: - -```text -1. Build once and share. -2. Build independently for each consumer. -``` - -The shared candidate reuses the same `Rc`: - -```rust -Replacement::SubDAG(Rc::clone(target.root)) -``` - -The independent candidate creates a structurally equal but separately allocated node: - -```rust -Replacement::SubDAG( - Rc::new((**target.root).clone()) -) -``` - -This strategy does **not** decide whether sharing is cheaper. - -That preference belongs to the cost model. -`CandidateLogicalASAPDAGs::cost_sorted` calls `CostModel::cse_share_decision` when it ranks a -share-versus-recompute candidate pair. The strategy still returns both -alternatives because enumeration and ranking are separate steps: - -- `consumer_count >= 2` means `share_common_sub_dags` has already merged the expression into one shared `Rc`. The shared alternative is therefore an `Rc::clone`; the independent alternative requires a deep clone. -- `cse_share_decision` is used by the ranking path, not by - `SharedSubDAGStrategy`. -- The strategy must return both valid alternatives even if the current cost model strongly prefers one. A future whole-plan search may choose differently from today's local comparison. - -This example is useful when implementing transformations such as: - -- roll-up vs. recompute, -- shared grouping vs. per-group instances, -- semantic rewrite vs. original expression. - -Each strategy can expose the alternatives without choosing between them. - ---- - -### Using a strategy - -The basic calling pattern is: - -```rust -let target = TargetSubDAG::new(&root); -let strategy = - ASAPStrategies::default_cost_model(); - -if strategy.matches(&target) { - let candidates = - strategy.replacements(&target); - - for candidate in candidates { - println!("{}", candidate.rationale); - } -} -``` - -A caller may also safely call `replacements` directly and treat an empty vector as "not applicable": - -```rust -let candidates = - strategy.replacements(&target); - -if candidates.is_empty() { - // No candidate from this strategy. -} -``` - -For strategies that require workload context: - -```rust -let target = - TargetSubDAG::with_consumer_count( - &root, - consumer_count, - ); -``` - -The caller is responsible for providing correct cross-workload metadata. - ---- - -### Testing a new strategy - -Every new strategy should have focused tests for its contract. - -At minimum, test the following. - -#### Applicability - -A matching target should satisfy: - -```rust -assert!(strategy.matches(&target)); -``` - -A non-matching target should satisfy: - -```rust -assert!(!strategy.matches(&target)); -``` - ---- - -#### Safe non-match behavior - -Also call `replacements` on a non-matching target: - -```rust -assert!( - strategy.replacements(&target).is_empty() -); -``` - -This verifies that the strategy does not depend on callers always invoking `matches` first. - ---- - -#### Exhaustive candidate enumeration - -If the target has N valid alternatives: - -```rust -let replacements = - strategy.replacements(&target); - -assert_eq!(replacements.len(), N); -``` - -Check the identities or kinds of all candidates, not just the preferred one. - -The current sketch-family tests explicitly verify that: - -- quantile returns both KLL and DDSketch, -- cardinality returns HLL and KMV (Theta is not offered until `asap_sketchlib` has a Theta sketch). - -This is the most important regression test for a strategy. - ---- - -#### Rationale - -Verify that every candidate has a non-empty rationale: - -```rust -assert!( - replacements - .iter() - .all(|r| !r.rationale.is_empty()) -); -``` - -For a strategy whose explanation includes important context, also test that context. - -For example, the shared-sub-DAG tests verify that the consumer count appears in the rationale. - ---- - -#### Structural semantics - -For logical rewrites, test the structural property that distinguishes the alternatives. - -For example, the current shared-sub-DAG tests verify: - -```rust -Rc::ptr_eq(shared, &q) -``` - -for the shared candidate, and: - -```rust -!Rc::ptr_eq(independent, &q) -``` - -plus structural equality for the independent candidate. - -Do not test only the rationale string; test the actual replacement semantics. - ---- - -#### Custom cost model behavior - -If a strategy accepts a cost model, verify that a custom model changes the intended costing behavior without changing the exhaustive candidate set. - -The current sketch strategy does exactly this: - -```mermaid -flowchart LR - INPUT["Legal candidate set
KLL + DDSketch"] --> MODEL["Custom CostModel
prefers DDSketch for this AggIntent"] - MODEL --> ORDER["rank_candidates output
DDSketch first, KLL second"] - ORDER --> RESULT["Strategy output
both candidates remain; only their order changes"] -``` - -That is the expected separation between enumeration and ranking. - ---- - -## 2. Adding or customizing a `CostModel` - -### Adding a custom `CostModel` - -Use a custom `CostModel` when you want to change preferences or cost assumptions without changing transformation legality. - -A minimal model can override only the hook it cares about. - -For example (illustrative — `PreferDDSketch` is not a real type in this crate): - -```rust -struct PreferDDSketch; - -impl CostModel for PreferDDSketch { - fn rank_candidates( - &self, - _intent: &AggIntent, - candidates: &[SketchAlgorithm], - ) -> Vec { - let mut ranked = candidates.to_vec(); - - if let Some(pos) = - ranked.iter().position( - |k| *k == SketchAlgorithm::DDSketch - ) - { - let dd = ranked.remove(pos); - ranked.insert(0, dd); - } - - ranked - } -} -``` - -Then inject it into code that accepts a `&dyn CostModel`: - -```rust -let model = PreferDDSketch; - -let strategy = - ASAPStrategies::new(&model); - -let replacements = - strategy.replacements(&target); -``` - -Important: changing `rank_candidates` changes the preferred ordering, but `ASAPStrategies` still enumerates every valid sketch candidate. - -A custom cost model should not change which alternatives are semantically legal. - ---- - -### Which `CostModel` hook should I implement? - -Use this as a practical guide. - -#### `rank_candidates` - -Use when you want to change the preference among valid sketch algorithms. - -Example: - -```text -KLL vs. DDSketch -HLL vs. Theta vs. KMV -``` - -Signature: - -```rust -fn rank_candidates( - &self, - intent: &AggIntent, - candidates: &[SketchAlgorithm], -) -> Vec; -``` - -The returned vector should rank candidates from most to least preferred. - -It must return a permutation of the supplied candidates: every input candidate exactly once, with no additions or removals. Planner call sites enforce this contract and panic if a cost model violates it. - ---- - -#### `size_params` - -Use when the sketch algorithm is already known and you want to choose its parameters from an accuracy target. - -Signature: - -```rust -fn size_params( - &self, - kind: SketchAlgorithm, - intent: &AggIntent, - eps: f64, - delta: f64, -) -> SketchParams; -``` - -Typical uses include: - -- choosing KLL capacity, -- choosing HLL precision, -- selecting sketch-specific error parameters. - -Conceptually: - -```mermaid -flowchart LR - ALG["Chosen SketchAlgorithm
for example, KLL or HLL"] --> SIZE["CostModel::size_params
translate a requested accuracy budget into
algorithm-specific storage parameters"] - INTENT["AggIntent
what the query is computing"] --> SIZE - ACC["Accuracy budget
epsilon and delta"] --> SIZE - SIZE --> PARAMS["SketchParams
for example, KLL capacity or HLL precision"] -``` - ---- - -#### `realize_extension` - -Use for extension-defined implementation kinds. - -```rust -fn realize_extension( - &self, - ext_kind: &str, - payload: &serde_json::Value, -) -> Realization; -``` - -This is the hook for turning an extension description into a concrete `Realization`. - -Use it for implementation families that are intentionally outside the built-in enum dispatch. - ---- - -#### `evaluation_extension` - -Use when an extension-defined summary also needs custom query/evaluation behavior. - -```rust -fn evaluation_extension( - &self, - ext_kind: &str, - payload: &serde_json::Value, - col: &ColumnRef, -) -> SketchStatistic; -``` - -This complements `realize_extension`: realization defines what gets maintained; evaluation defines how it is queried (see the [CostModel reference](asap-aware-mapping-contracts.md#costmodel)). - ---- - -#### `cse_recompute_cost` - -Use to estimate the cost of computing a common sub-DAG independently at each consumer. - -```rust -fn cse_recompute_cost( - &self, - candidate: &CseCandidate, -) -> Cost; -``` - ---- - -#### `cse_shared_maintenance_cost` - -Use to estimate the cost of computing and maintaining a shared sub-DAG. - -```rust -fn cse_shared_maintenance_cost( - &self, - candidate: &CseCandidate, -) -> Cost; -``` - ---- - -#### `cse_share_decision` - -Use when the ranking path needs a share-vs.-recompute preference. - -```rust -fn cse_share_decision( - &self, - candidate: &CseCandidate, -) -> ShareDecision; -``` - -The replacement-strategy layer should still expose both valid alternatives where appropriate. This hook is the cost-sensitive decision point for code paths that need to commit to one answer. - ---- - -#### `estimate_cost` - -Use when callers need a comparable numeric cost for each replacement, in addition to relative ordering. - -```rust -fn estimate_cost( - &self, - candidate: &ReplacementSubDAG, - target: &TargetSubDAG<'_>, -) -> f64; -``` - -The default returns `f64::NAN`, making the absence of a numeric model explicit. Override this hook when passing the model to `CandidateLogicalASAPDAGs::cost_sorted` if downstream code displays or otherwise consumes the `costs` values. Prefer to derive the result from the same inputs used by `rank_candidates` and the CSE cost hooks so numeric costs do not disagree with relative ordering. - ---- - -### Testing a new cost model - -A cost-model test should focus on the hook being customized. - -For ranking: - -```rust -let ranked = - model.rank_candidates( - &intent, - &[SketchAlgorithm::Kll, - SketchAlgorithm::DDSketch], - ); - -assert_eq!( - ranked[0], - SketchAlgorithm::DDSketch -); -``` - -Then test integration through a consumer of the cost model. - -For example: - -```rust -let strategy = - ASAPStrategies::new(&model); - -let replacements = - strategy.replacements(&target); -``` - -The important assertion is usually not that other valid candidates disappeared. They should not. - -Instead verify that: - -- the model changes ordering or parameters as intended, -- all legal candidates remain available to the replacement layer. - -For sizing, test representative accuracy targets and assert the resulting `SketchParams`. - -For CSE costing, create a representative `CseCandidate` and test recompute cost, shared-maintenance cost, and the resulting `ShareDecision`. - ---- - -## 3. Adding a new sketch algorithm - -A new sketch algorithm generally touches more than `ReplacementStrategy`. - -Declare built-in sketch applicability through the public candidate registry: - -```rust -summary_candidates(intent) -``` - -`ASAPStrategies` consumes this registry through its public `replacements` method. - -Therefore, when adding a new built-in sketch algorithm, the intended flow is: - -```mermaid -flowchart LR - MAP["1. Declare legality
add the algorithm to summary_candidates
for each AggIntent it can answer"] - MAP --> MODEL["2. Define costing
rank it, derive its SketchParams,
and provide a comparable numeric cost"] - MODEL --> BUILD["3. Define realization behavior
ensure the public strategy output contains a valid summary sub-DAG
with the correct maintained state and evaluation"] - BUILD --> ACC["4. Certify accuracy
derive from committed parameters;
propagate and check the final target"] - ACC --> ENUM["5. Verify integration
ASAPStrategies includes it automatically;
tests confirm enumeration, ordering, sizing, and cost"] -``` - -This keeps one source of truth for sketch applicability. Applicability alone -is not an accuracy certificate: define the estimator-specific metric, bound and -failure probability in `AccuracyModel::local_guarantee`, derive them from the -committed parameters after rounding/clamping, and propagate through supported -composition rules. Supply typed evidence and budget allocation where required. -Missing or insufficient proof must keep the candidate ineligible before cost -ranking; preserve exact fallback and structured rejection information. - -See the [accuracy implementation companion](end-to-end-accuracy-guarantees.md) -for formulas and evidence requirements. For a new algorithm, also update its -parameter, evaluation, schema and serialization definitions in `asap-types`. - -Do not special-case the new sketch inside `ASAPStrategies` unless the strategy itself needs fundamentally new behavior. - -### Verifying a new sketch algorithm - -Cover an admitted guarantee and rejection for insufficient parameters, missing -or malformed evidence, incompatible metrics and unsupported composition. Test -root-target checking before cost ranking, exact fallback, and exported rejection -or guarantee data. A cheaper estimate must never admit an accuracy-illegal plan. - -After wiring the new algorithm into `summary_candidates` and giving the cost model a real `rank_candidates`/`size_params` opinion about it, check two things. First, that `ASAPStrategies::replacements()` for a matching `TargetSubDAG` actually includes a candidate realizing the new algorithm — extend a test shaped like `replacement.rs`'s own test-module coverage-matrix tests (e.g. `agg_intent_to_summary_kind_coverage_matrix`) to cover the new algorithm's `AggIntent`. Second, that `cost_sorted`/`estimate_cost` produce sane, comparable numbers for the new candidate rather than a `NaN` placeholder or an outlier that swamps every other candidate. - ---- - -## 4. Adding both a strategy and a cost model - -Some features require both. - -For example, suppose we add: - -```text -roll up a finer group-by -vs. -compute the coarser group-by independently -``` - -The responsibilities should be divided as follows. - -### Strategy - -The new strategy determines: - -- whether the two group-bys have the required relationship, -- whether the aggregation is mergeable, -- whether roll-up preserves semantics, -- and then returns both valid alternatives. - -Conceptually: - -```rust -vec![ - rollup_candidate, - recompute_candidate, -] -``` - -### Cost model - -The cost model determines: - -- maintenance cost of the finer-grained summary, -- cost of roll-up, -- cost of independent computation, -- expected query frequency, -- and which complete plan is cheaper. - -Do not encode: - -```text -"only return roll-up when roll-up is cheaper" -``` - -inside the strategy. - -That turns a cost decision into a legality decision and prevents later global plan search from seeing both options. - ---- - -## 5. Common mistakes - -### Mistake: choosing the cheapest candidate inside a strategy - -Wrong: - -```rust -fn replacements(...) -> Vec { - vec![choose_cheapest_candidate()] -} -``` - -Right: - -```rust -fn replacements(...) -> Vec { - all_valid_candidates() -} -``` - ---- - -### Mistake: maintaining a second sketch-applicability table - -If `replacement.rs` already defines which sketch algorithms satisfy an `AggIntent`, reuse that source. - -Otherwise implementation enumeration and the replacement strategy can -silently disagree. - ---- - -### Mistake: reimplementing summary construction inside a strategy - -If the candidate should produce a normal summary sub-DAG (`SummaryAgg` / -`SummaryEstimate`), use the existing -summary-construction path. - -A strategy should steer or wrap that path when necessary, not recreate schema derivation, column resolution, evaluation construction, or parameter sizing. - ---- - -### Mistake: treating `rationale` as planner state - -`rationale` is explanatory text. - -If downstream logic needs a fact, represent it in the plan or another typed structure instead of parsing the rationale. - ---- - -### Mistake: hiding workload traversal in a strategy - -A `ReplacementStrategy` operates on the `TargetSubDAG` it is given. - -Workload-wide target discovery, deduplication, and consumer counting are separate concerns. - ---- - -### Mistake: assuming `Rc` structural equality and identity mean the same thing - -For CSE-style decisions, pointer identity can encode actual sharing. - -Two `Rc` values can be structurally equal but deliberately represent independent computation. - -Use the distinction intentionally. - ---- - -## 6. Extension checklist - -When adding a new strategy: - -- [ ] Define the exact target shape. -- [ ] Implement `ReplacementStrategy::matches`. -- [ ] Implement `ReplacementStrategy::replacements`. -- [ ] Return every semantically valid replacement. -- [ ] Return an empty vector for non-matching targets. -- [ ] Return `Replacement::SubDAG` for both constructed post-ASAP output and - logical pre-ASAP alternatives, with the matching `provenance`. -- [ ] Add a useful rationale to every candidate. -- [ ] Reuse existing legality and implementation logic instead of duplicating it. -- [ ] Keep ranking and cost-based pruning out of the strategy. -- [ ] Test positive and negative applicability. -- [ ] Test exhaustive enumeration. -- [ ] Test the actual structural semantics of each replacement. -- [ ] Test behavior with a custom cost model if the strategy uses one. - -When adding a new cost model: - -- [ ] Override only the hooks whose behavior should change. -- [ ] Keep semantic applicability outside the cost model. -- [ ] Use `rank_candidates` for algorithm preference; return every input candidate exactly once. -- [ ] Use `size_params` for accuracy-to-parameter mapping. -- [ ] Use extension hooks for extension-defined implementations/evaluations. -- [ ] Use CSE hooks for recompute-vs.-sharing costs. -- [ ] Override `estimate_cost` if consumers require numeric costs instead of `NaN`. -- [ ] Test the hook directly. -- [ ] Test integration through a consumer such as `ASAPStrategies`. -- [ ] Verify that changing cost preferences does not silently remove valid replacement candidates. - ---- - -## 7. Current extension map - -Use this table to find the right place for a change. - -| I want to... | Primary extension point | -|---|---| -| Add a new logical optimization | new `impl ReplacementStrategy` | -| Add a new replacement for an existing target shape | `ReplacementStrategy::replacements` | -| Change when a strategy applies | `ReplacementStrategy::matches` | -| Add a new built-in sketch candidate | `replacement.rs`'s summary-candidate mapping plus realization and accuracy contracts | -| Prefer one sketch algorithm over another | `CostModel::rank_candidates` | -| Change sketch sizing for an accuracy target | `CostModel::size_params` | -| Add extension-defined implementation behavior | `CostModel::realize_extension` | -| Add extension-defined evaluation behavior | `CostModel::evaluation_extension` | -| Change CSE recomputation cost | `CostModel::cse_recompute_cost` | -| Change shared-maintenance cost | `CostModel::cse_shared_maintenance_cost` | -| Change current share/recompute choice | `CostModel::cse_share_decision` | -| Decide whether an available implementation satisfies a required one | `impl Matcher` | -| Produce a normal (ranked-first) post-ASAP summary for one target | `ASAPStrategies::replacements(...).into_iter().next()` | -| Search a whole workload for supported legal candidates | `search_workload`/`search_workload_with` | -| Enforce per-root result accuracy requirements | `search_workload_with_targets` | -| Coordinate compatible choices across groups | `CandidateLogicalASAPDAGs::global_selection` | -| Assemble the selected logical DAG | `GlobalSelection::assemble_selected_dag` | -| Get every candidate ranked best-first, across a whole workload | `CandidateLogicalASAPDAGs::cost_sorted` | -| Get a real numeric cost per candidate, not just a relative rank | `CostModel::estimate_cost` | -| Enumerate valid sketch algorithms | `summary_candidates` | -| Build a target with no workload context | `TargetSubDAG::new` | -| Build a target with known sharing context | `TargetSubDAG::with_consumer_count` | -| Explain why a replacement exists, where, and why | `explanation::explain_replacements`/`explain_replacements_with` | -| Add a new kind of replacement explanation | new `impl ReplacementStrategy`, wired into `default_strategies`/`default_strategies_with` — not a new explanation-specific trait, see §8 | - ---- - -## 8. Using and extending explanation.rs - -### Using it - -```rust -use asap_aware_mapping::{explain_replacements, ExplanationKind}; - -let explanations = explain_replacements(vec![("dashboard_p99", query)]); -for explanation in &explanations { - match explanation.kind { - ExplanationKind::SketchApproximation => { /* ... */ } - ExplanationKind::CommonSubexpressionReuse => { /* ... */ } - _ => { /* ExplanationKind is #[non_exhaustive] */ } - } -} -``` - -To plug in a deployment-specific strategy or `CostModel`, use `explain_replacements_with` with a strategy set built the same way `default_strategies_with` builds one — see [§2](#2-adding-or-customizing-a-costmodel) and [§4](#4-adding-both-a-strategy-and-a-cost-model). - -### Adding a new kind of replacement explanation - -There is no separate checklist here: follow [§1](#1-adding-a-new-replacementstrategy) to add the new `ReplacementStrategy` and wire it into `default_strategies`/`default_strategies_with`, then add an `ExplanationKind` variant and ensure `explain_replacements` returns that kind for the new public candidate shape. Test the behavior through `explain_replacements` or `explain_replacements_with`; explanation reporting should not introduce a second discovery rule. +Audience: developers adding planner behavior. Read the +[architecture](asap-aware-mapping-architecture.md) and the +[contracts](asap-aware-mapping-contracts.md) first. Each recipe lists the +files to change and the tests to add; keep each change inside the stage that +owns the decision. + +## 1. Add a summary family + +A new sketch that Pass 1 can offer for an intent. + +1. **Type.** Add the variant to `SketchAlgorithm` and its parameters to + `SketchParams` (`crates/types/src/post_asap/sketch.rs`). +2. **Capability.** Add its row to `FamilyCapability::of` + (`crates/types/src/post_asap/family_capability.rs`): which inputs it can + merge (`MergeInputRule`) and which statistics it answers (`StatisticKind`, + with the merged error). Pass 2's sharing, pane and rollup rules read only + this table. +3. **Offer and size.** List it for its intents in + `realization::summary_candidates`, most preferred first, and size it in + `realization::default_size_params`. +4. **Accuracy.** Add its error model under `accuracy/estimators/` and return + it from `DefaultAccuracyModel`'s `local_guarantee`, by inverting the same + sizing formulas. Without a guarantee, every candidate that uses it is + rejected. +5. **Cost.** Give it a state size and per-operation CPU in + `stage3_plan_selection.rs` (and accept its parameters in + `stage3_plan_selection/measured_operation_costs.rs` if deployments measure + it). +6. **Executor.** Wrap the `asap_sketchlib` implementation in + `crates/asap-physical-operators/src/summary_kernels/` and register it in + `factory.rs` and `capability.rs`. Gaps in the library are fixed upstream, + not with a local copy. +7. **Labels.** Name it in `asap_devtools::candidate_labels`. + +Tests: a Pass 1 test in `crates/asap-aware-mapping/tests/per_computation_candidates.rs` +that the intent gets the new candidate (and that a deployment without it +prunes it with `UnsupportedByDeployment`); an estimator test beside the +estimator; an execution test in `crates/integration-tests/tests/` such as +`frequency_execution.rs` or `kll_pane_execution.rs`. + +## 2. Add a Pass 2 rule + +A new way for computations of one Pass 1 candidate to share, split or reuse +work. + +1. In `asap_aware_cse.rs`, add the part it can choose as a `*Candidate` type + with a finder (for example `rollup_candidates(roots)`), and the function + that applies a set of them and returns `*Pass2Candidate`s (for example + `rollup_pass2_candidates`). Read summary families only through + `FamilyCapability`. +2. In `pass/stage1_to_stage3.rs`, call it from Stage 1 enumeration after the + existing rules and record what it applied in a new `Pass2Candidate` field. +3. Rebuild its label from that field in `asap_devtools::candidate_labels`. +4. Add its row to the Pass 2 rule table of the + [design](../design_docs/proposals/planner-layering.md). + +A rule must keep every Pass 1 candidate, must not add a family or a readout, +and must keep the composed DAG's meaning. Stage 3 decides whether it pays off. + +Tests: unit tests beside the rule in `asap_aware_cse.rs` for when it applies +and when it must not; an end-to-end test like +`crates/integration-tests/tests/reuse_rules.rs` that the planned workload +contains the candidate and that Stage 3 prices it. + +## 3. Add a Stage 2 materialization option + +1. Add the `Materialization` variant + (`crates/types/src/post_asap/execution_data_state.rs`) and its timing in + `crates/types/src/ir/timing.rs`, with its data-state edge rules. +2. Offer it in `stage2_physical_asap_optimization.rs` (the + `MaterializationCandidate` options), after the existing options so default + candidate ids stay the same, and gate it on the deployment input that + allows it. +3. Price it in `stage3_plan_selection.rs`: what it keeps in memory or on + disk, and what each evaluation rebuilds. + +Tests: Stage 2 unit tests for when the option is offered; +`crates/integration-tests/tests/materialization_e2e.rs` and the Example 4 +tests in `planner_layering_example4.rs`. + +## 4. Add a Stage 3 check or a deployment input + +- A new validity check goes in `select_plan` and rejects a candidate with a + `Rejection { valid: false, reason }`; it never fails the whole selection + unless the input itself is bad (a new `SelectionError` variant, documented + as bad input). +- A new pricing input goes in `PlanSelectionSettings` (validated in its + `validate`); a new statement about the executor goes in + `DeploymentCapabilities`. Both reach every stage through `PlanningModels`. +- Expose it in `stage_pipeline` and in the + [Stage Viewer guide](../user_guide_docs/stage-viewer.md). + +Tests: a Stage 3 unit test that the check rejects exactly the candidates it +should, and a `crates/devtools/tests/stage_pipeline.rs` test for the flag. + +## 5. Add a query-language construct + +Lower the construct in its frontend (`crates/frontend-promql/src/promql.rs`, +`crates/frontend-sql/src/sql/`, `crates/frontend-metricsql/src/`) into +existing `NonASAPOp` nodes and `AggIntent`s. If it needs a new intent, add +it in `asap-types` and offer its candidates in Pass 1 (recipe 1). A construct +that cannot be represented is rejected with the frontend's +unsupported-feature error, never lowered approximately. + +Tests: frontend tests for the lowering and its rejection cases; the corpus +tests (`analyze_corpora`, `variant_coverage`) show the coverage change. + +## Checklist + +- [ ] The decision is made in the stage that owns it (contracts §6). +- [ ] Every new public type, field and variant has a doc comment. +- [ ] Unit tests beside the code and one end-to-end test through the stages. +- [ ] The design document's examples still plan the same, or the change is + explained: run the `planner_layering_example*` tests and + `stage_pipeline --example planner-layering-1` (and `-2`, `-3a`, `-3b`). +- [ ] The design, developer and user docs that name what changed are updated. diff --git a/docs/develop_docs/library-api.md b/docs/develop_docs/library-api.md index 022fb6dc5..32f254c72 100644 --- a/docs/develop_docs/library-api.md +++ b/docs/develop_docs/library-api.md @@ -1,14 +1,13 @@ # Public library functions -Audience: developers embedding ASAPPlanner or adding strategies/models. This is +Audience: developers embedding ASAPPlanner or supplying deployment models. This is a compact reference for the public workflow APIs, not an exhaustive symbol reference. The [CLI guide](../user_guide_docs/run-a-query.md) covers command-line inspection; the [design overview](../design_docs/architecture/README.md) defines ownership. ASAPPlanner's output is the plan selected by its planning stages (`PlanOutput`, see [Plan a workload through the stages](#plan-a-workload-through-the-stages)). -The legacy search below outputs `CandidateLogicalASAPDAGs`; ranking is a view over its candidates. -Downstream owns physical binding and commitment. Selection/DAG assembly helpers -do not deploy a plan, and a serializable DAG is not evidence of runtime readiness. +Downstream owns physical binding and commitment. The planner does not deploy a +plan, and a serializable DAG is not evidence of runtime readiness. ## Choose a library workflow @@ -16,9 +15,8 @@ do not deploy a plan, and a serializable DAG is not evidence of runtime readines | --- | --- | --- | | Pre-ASAP IR | Frontend `lower_*` | [Lower a query](#lower-a-query-into-pre-asap-ir) | | Selected plan for a workload (Stages 0–3) | `asap_planner::e2e_plan` | [Plan a workload](#plan-a-workload-through-the-stages) | -| All ranked candidates | `search_workload_with_targets` -> `cost_sorted` | [Generate and rank](#generate-and-rank-candidates) | -| Custom optimization set | Construct `Vec>`, then search | [Strategies and models](#choose-strategies-and-models) | -| Selected semantic DAG / export | `global_selection` -> `assemble_selected_dag` -> export | [Selection example](#optional-whole-plan-selection-and-dag-assembly) | +| Every stage's candidates for already lowered roots | `asap_aware_mapping::run_stage1_to_stage3` | [Plan a workload](#plan-a-workload-through-the-stages) | +| Physical DAG for an executor | `PlanOutput::execution_timed_dag`, or timing and export functions | [Export](#export) | Each recipe ends at a different artifact. Use only the stages needed for that artifact, while preserving the checks required by its intended consumer. @@ -170,15 +168,16 @@ asap_aware_mapping::run_stage1_to_stage3( | Input or output | Meaning | | --- | --- | -| `PlanningModels::builtin()` | Built-in accuracy model and no extra evidence. `with_plan_selection(&settings)` sets the deployment's `PlanSelectionSettings`; `with_deployment_capabilities(&capabilities)` sets its `DeploymentCapabilities` (field `deployment_capabilities`) | +| `PlanningModels::new(accuracy, evidence)` | Your `&dyn AccuracyModel` and `&dyn AccuracyEvidenceProvider`, with default plan selection settings and deployment capabilities; `with_accuracy` and `with_evidence` replace one of them | +| `PlanningModels::builtin()` | Built-in accuracy model (`DefaultAccuracyModel`) and no extra evidence (`NoAccuracyEvidence`). `with_plan_selection(&settings)` sets the deployment's `PlanSelectionSettings`; `with_deployment_capabilities(&capabilities)` sets its `DeploymentCapabilities` (field `deployment_capabilities`) | | `DeploymentCapabilities` (`asap_aware_mapping::deployment_capabilities`) | What the executor supports: `supported_summaries` (`None`, the default, supports every summary the planner builds; otherwise `SupportedSummary { family: SketchAlgorithm, readouts: Vec }` entries; Pass 1 prunes others with `PruneReason::UnsupportedByDeployment`), `raw_data_retained` (default `true`; when `false`, a candidate that scans raw data at query time is invalid, and stored data is refused with `SelectionError::InvalidDeploymentCapabilities`) and `query_time_results_retained` (default `false`; when `true`, Stage 2 offers `MatAtQueryKeptAcrossEvaluations` for tumbling panes) | -| `PlanSelectionSettings` (`asap_aware_mapping::stage3_plan_selection`) | `cost_model`, `resource_speed` (`ms_per_cpu_op`, `ms_per_scanned_byte`) `non_repeating_query_horizon_seconds` (default 3,600) and `measured_operation_costs` (default empty: one `MeasuredOperationCost { sketch, operation, cpu_ns_per_operation }` per measured sketch configuration and `SummaryOperation` (`Update`, `Merge`, `Estimate`); a sketch node with an exactly matching entry is priced from it, every other node from the formulas, and each node's `NodeCost::price_source` says which) | +| `PlanSelectionSettings` (`asap_aware_mapping::stage3_plan_selection`) | `cost_model`, `resource_speed` (`ms_per_cpu_op`, `ms_per_scanned_byte`), `non_repeating_query_horizon_seconds` (default 3,600) and `measured_operation_costs` (default empty: one `MeasuredOperationCost { sketch, operation, cpu_ns_per_operation }` per measured sketch configuration and `SummaryOperation` (`Update`, `Merge`, `Estimate`); a sketch node with an exactly matching entry is priced from it, every other node from the formulas, and each node's `NodeCost::price_source` says which) | | `CostModel::UsageOverTime(ResourceUnitCost)` | Cost model 1 (default, `ResourceUnitCost::illustrative()`): unit cost × usage per second; the cheapest valid candidate wins | | `CostModel::ProvisionedCapacity(ResourceUnitCost)` | Cost model 2: unit cost × peak memory, peak CPU cores and disk; the cheapest valid candidate wins | | `CostModel::ClusterCapacity(ClusterLimits)` | Cost model 3: memory, disk and optional CPU cores are hard limits; `ClusterObjective::{LowestLatency, LeastMemory, ParetoFront}` chooses among the candidates that fit | | `PlanOutput.plans` | One `QueryPlan { entry_index, root }` per query, in `QueryWorkload::entries()` order | | `PlanOutput.selection` | Stage 3's `Selection`: `selected`, `costs` (each candidate's `CandidateCost` with its `ResourceUsage` and `cost_per_second`, `None` under cost model 3), `accepted`, `rejected` (with reasons) and `front` | -| `PlanOutput::execution_timed_dag()` | The planned queries as one `PhysicalASAPDAG`, timed with the default `MaterializationAssignment` (see [Export and explain](#export-and-explain)) | +| `PlanOutput::execution_timed_dag()` | The planned queries as one `PhysicalASAPDAG`, timed with the default `MaterializationAssignment` (see [Export](#export)) | Each candidate's `ResourceUsage` holds its usage without prices: `cpu_ops_per_second`, `scanned_bytes_per_second`, `average_memory_bytes`, @@ -274,331 +273,6 @@ The `stage_pipeline` devtool runs the same stages and writes every stage's candidates as one JSON document; see the [Stage Viewer guide](../user_guide_docs/stage-viewer.md). -## Generate and rank candidates - -### Target sub-DAG candidates - -`TargetSubDAGCandidates` collects alternatives for one query subexpression -discovered by search. `CandidateLogicalASAPDAGs` contains these per-target candidate sets and -the workload's query roots. A root is a whole query; an inner expression can -also be a target. - -For example, a supported `quantile(0.99, latency)` subexpression may have multiple -summary alternatives. Those alternatives belong to the same candidate set -because they are choices for the same computation. Another subexpression has its -own candidate set. If two queries reference a shared subexpression, they can -share its selected computation. - -`cost_sorted()` returns a `RankedTargetSubDAGCandidates` for each target: the subexpression, -its candidates in ranked order, and a cost entry aligned with each candidate. -It keeps the alternatives available; it does not select an entire workload plan. - -### API definition - -```text -search_workload_with_targets<'s, Id>( - roots: Vec<(Id, Rc, Option)>, - strategies: &[Box], - accuracy_model: &dyn AccuracyModel, -) -> Result, SearchError> - -CandidateLogicalASAPDAGs::cost_sorted(&self, cost_model: &dyn CostModel) - -> Vec> -``` - -| Argument | Choices / meaning | Required? | -| --- | --- | --- | -| `roots` | One tuple per query: caller ID, canonical IR, and root target | Yes | -| Root target | `Some(AccuracyTarget::…)` applies an explicit end-to-end requirement; `None` adds no explicit root target | Tuple field required; value optional | -| `strategies` | Default factory output or an explicit strategy vector; see option tables below | Yes; even an empty vector does not disable automatic workload strategies | -| `accuracy_model` | `DefaultAccuracyModel` or a custom `AccuracyModel` implementation | Yes | -| Ranking `cost_model` | `DefaultCostModel` or an evidence-backed/custom `CostModel` | Yes | - -`search_workload_with_targets` normally rejects candidates without a guarantee -that satisfies the root target. One exception is a direct DDSketch quantile -ratio: without input-domain evidence, it remains in `CandidateLogicalASAPDAGs` with -`guarantee: None` so the downstream backend can decide whether to select it. -Its presence does **not** mean it satisfies the target. `cost_sorted` still -shows it, but `global_selection` skips it and DAG assembly uses the exact fallback -unless a certified alternative is available. A backend that wants the -uncertified candidate must explicitly inspect it and check its own domain -evidence and execution requirements before selecting or deploying it. - -### Example - - -The following complete Rust example lowers one query, supplies an explicit root -accuracy target, and prints every ranked candidate instead of selecting a winner. -The default cost model is suitable for inspection, not deployment calibration. - -```rust -use asap_frontend_promql::lower_promql_workload; -use asap_types::workload::{ - BatchEntry, DataWorkload, DurationMs, Query, - PlanningWorkload, QueryLanguage, QueryRequirements, QueryWorkload, -}; -use asap_aware_mapping::{ - default_strategies_with, search_workload_with_targets, - DefaultAccuracyModel, DefaultCostModel, -}; -use asap_types::types::AccuracyTarget; - -fn main() -> Result<(), Box> { - let accuracy = AccuracyTarget::max_error(0.01); - let workload = PlanningWorkload { - query_workload: QueryWorkload { - language: QueryLanguage::PromQL, - query_batch: Some(vec![BatchEntry { - query: Query("quantile(0.99, latency)".into()), - requirements: QueryRequirements { - accuracy: accuracy.clone(), - ..Default::default() - }, - predictability: Default::default(), - invocations: 1, - execute_at: None, - }]), - repeating_queries: None, - }, - // `latency` is sampled every second. - data_workload: Some(DataWorkload::sampled_every( - ["latency".to_string()], - DurationMs(1_000), - )), - }; - let root = lower_promql_workload(&workload)?.remove(0); - let cost_model = DefaultCostModel; - let strategies = default_strategies_with(&cost_model); - let space = search_workload_with_targets( - vec![("q1", root, Some(accuracy))], - &strategies, - &DefaultAccuracyModel, - )?; - for group in space.cost_sorted(&cost_model) { - for (candidate, cost) in group.candidates.iter().zip(&group.costs) { - println!("candidate={candidate:?}, reported_cost={cost:?}"); - } - } - Ok(()) -} -``` - -| API (`asap_aware_mapping`, unless qualified) | Inputs | Output and limits | -| --- | --- | --- | -| `search_workload` | `(query_id, Rc)` roots | `CandidateLogicalASAPDAGs` with built-in strategies/model; no explicit per-root target argument | -| `search_workload_with` | Roots, strategy slice | `CandidateLogicalASAPDAGs`; callers choose context-free replacement strategies | -| `search_workload_with_targets` | Roots with optional end-to-end targets, strategies, accuracy model | Candidate space with supplied root-target checks; `None` does not supply a root-level requirement; uncertified direct DDSketch ratios remain available for backend selection | -| `CandidateLogicalASAPDAGs::cost_sorted` | Cost model | `Vec`; retains alternatives and pairs `candidates[i]` with `costs[i]` | -| `CandidateLogicalASAPDAGs::cost_sorted_with_recurrence` | Cost model, recurrence profiles, optional horizon | Ranked per-target candidate sets or `RecurrenceError`; uses recurrence for applicable share/recompute comparisons | -| `ASAPStrategies::replacements` through `ReplacementStrategy` | One `TargetSubDAG` | Alternatives at that target; not whole-workload search | - -`cost_sorted` is a ranking view, not a request to discard all but the first -candidate. Display costs follow model hooks and may be unavailable/non-finite; -they are not necessarily a globally sortable physical-cost scalar. Unavailable -cost alternatives may remain for explanation. Inspect eligibility and evidence -before physical selection; do not treat their presence as deployment permission. - -### Enumerate candidate DAGs per root - -```text -CandidateLogicalASAPDAGs::enumerate_candidate_dags_for_root(&self, id: &Id, expansion_limit: usize) - -> Result, RealizationError> -``` - -Returns every distinct finalized DAG for one root, unranked; other roots' -choices are not multiplied in. Exceeding `expansion_limit` is an error, never a -partial inventory. - -For PromQL roots that carry a target, `search_workload_with_targets` also asks -each strategy's `ReplacementStrategy::propose_for_root`. `ASAPStrategies` -answers an instant-vector TopK with current-series heap realizations over rows -carrying the complete series identity (`$promql_series_identity`). They are -finalized, deduplicated, and marked `ReplacementProvenance::RootPhysicalRealization`. -Callers do not apply `with_series_identity` themselves. Compile each with -`promql_rows::compile_current_series_evaluation`; other queries keep their previous -inventory. `global_selection` never commits these candidates; the backend -compiles and prices them. CandidateLogicalASAPDAGs lists no placement variants: node timing -comes from a `MaterializationAssignment` (all query time for the legacy search; -Stage 2 materialization chooses one per physical candidate). - -## Choose strategies and models - -### Strategy options - -The `strategies` argument takes Rust objects implementing `ReplacementStrategy`, -not string names or a closed enum. These built-in context-free choices can be -combined in one vector; each proposes candidates where its applicability checks -pass. An omitted strategy contributes no proposals of its own. - -| Value to put inside `Box::new(...)` | Meaning | In default factories? | -| --- | --- | --- | -| `ASAPStrategies::new(&model)` | Enumerates supported exact/sketch implementations and parameter choices for aggregate targets | Yes | -| `HydraGroupingStrategy::new(&model)` | Considers a shared multi-subpopulation structure for supported grouped sketch families, subject to accuracy evidence | Yes | -| `SharedSubDAGStrategy` | Proposes sharing versus independent recomputation at reused sub-DAGs | Yes | -| `SemanticEquivalentRewriteStrategy` | Proposes supported equivalent aggregate rewrites, including decomposing average into sum/count | Yes | -| `ExactCompositionStrategy::new(&model)` | Proposes supported exact operations around summary evaluations or in maintenance | Yes | -| Your `ReplacementStrategy` implementation | Adds domain-specific legal replacement proposals | No | - -`AvgToSumOverCountStrategy` is an alias for `SemanticEquivalentRewriteStrategy` -at this revision; it is not a separate narrow rewrite to enable alongside it. - -The following are derived automatically from the workload by `search_workload*`: - -| Automatic behavior | Meaning | Can the strategy vector disable it? | -| --- | --- | --- | -| Canonical sharing/CSE | Interns structurally equal input subexpressions | No | -| `RollupStrategy` | Proposes compatible reuse across grouping granularities | No | -| `AccuracyReconciliationStrategy` | Proposes compatible sharing across different accuracy requirements | No | -| `TopKLimitReuseStrategy` | Proposes reuse among compatible top-k limits | No | - -The current API does not expose a universal enable/disable flag for every pass. -For inspecting only one strategy at one target, use -`ReplacementStrategy::replacements(&TargetSubDAG)`; this does not perform the -whole-workload search. Selecting a strategy does not force its candidate to win. - -### Factory choices - -```text -default_strategies() -> Vec> -default_strategies_with<'a>(cost_model: &'a dyn CostModel) - -> Vec> -replacement::default_strategies_with_evidence<'a>( - cost_model: &'a dyn CostModel, evidence: &'a dyn AccuracyEvidenceProvider, -) -> Vec> -``` - -| Factory | Use when | Models used | -| --- | --- | --- | -| `default_strategies()` | Exploring with built-in defaults | Built-in cost/accuracy defaults | -| `default_strategies_with(&model)` | Supplying deployment-specific costing/sizing | Supplied cost model; default accuracy | -| `default_strategies_with_evidence(&model, &evidence)` | Supplying planning-time accuracy evidence as well | Supplied cost and evidence; default accuracy | -| Explicit vector | Controlling which context-free strategies are supplied | Models passed into each constructor | - -### Example: supply two strategies and run search - -```rust -use asap_frontend_promql::lower_promql_workload; -use asap_types::workload::{ - BatchEntry, DataWorkload, DurationMs, Query, - PlanningWorkload, QueryLanguage, QueryRequirements, QueryWorkload, -}; -use asap_aware_mapping::{ - search_workload_with_targets, DefaultAccuracyModel, DefaultCostModel, - ReplacementStrategy, ASAPStrategies, SharedSubDAGStrategy, -}; -use asap_types::types::AccuracyTarget; - -fn main() -> Result<(), Box> { - let accuracy = AccuracyTarget::max_error(0.01); - let workload = PlanningWorkload { - query_workload: QueryWorkload { - language: QueryLanguage::PromQL, - query_batch: Some(vec![BatchEntry { - query: Query("quantile(0.99, latency)".into()), - requirements: QueryRequirements { - accuracy: accuracy.clone(), - ..Default::default() - }, - predictability: Default::default(), - invocations: 1, - execute_at: None, - }]), - repeating_queries: None, - }, - // `latency` is sampled every second. - data_workload: Some(DataWorkload::sampled_every( - ["latency".to_string()], - DurationMs(1_000), - )), - }; - let root = lower_promql_workload(&workload)?.remove(0); - let model = DefaultCostModel; - let strategies: Vec> = vec![ - Box::new(ASAPStrategies::new(&model)), - Box::new(SharedSubDAGStrategy), - ]; - let space = search_workload_with_targets( - vec![("q1", root, Some(accuracy))], &strategies, &DefaultAccuracyModel, - )?; - println!("{:#?}", space.cost_sorted(&model)); - Ok(()) -} -``` - -This omits Hydra and semantic/exact-composition strategies from the supplied -vector. Automatic workload strategies still run. Omitting an optimization does -not waive semantic or accuracy requirements. - -### Model and evidence options - -Traits permit custom implementations; the following are concrete built-in options. -Module-qualified paths below are relative to `asap_aware_mapping`. - -| Parameter | Available value / constructor | Meaning | -| --- | --- | --- | -| `&dyn CostModel` | `DefaultCostModel` | Built-in ordering/sizing and structural estimates; no measured deployment guarantee | -| `&dyn AccuracyModel` | `DefaultAccuracyModel` | Built-in guarantee rules and satisfaction checks | -| `&dyn AccuracyEvidenceProvider` | `NoAccuracyEvidence` | No extra planning-time statistics; evidence-dependent claims remain unavailable | -| Any provider trait above | Your implementation | Supplies alternative models/evidence under the same contracts | - -### Measured sketch costs - -Measured CPU time per sketch operation now reaches plan selection, not the -legacy search: put it in `PlanSelectionSettings::measured_operation_costs` -(see [Plan a workload through the stages](#plan-a-workload-through-the-stages)). -The legacy offline sketch-benchmark and physical-plan cost models are removed. - -### Example: configure all sketch-strategy providers - -```rust -use asap_aware_mapping::{ - DefaultAccuracyModel, DefaultCostModel, - NoAccuracyEvidence, ReplacementStrategy, ASAPStrategies, -}; - -fn main() { - let cost = DefaultCostModel; - let accuracy = DefaultAccuracyModel; - let evidence = NoAccuracyEvidence; - let strategies: Vec> = vec![Box::new( - ASAPStrategies::new_with_planning_inputs_and_evidence( - &cost, &accuracy, &evidence, - ), - )]; - // Use &strategies and &accuracy in search_workload_with_targets. - println!("{} explicitly configured strategy", strategies.len()); -} -``` - -Constructor definition: - -```text -ASAPStrategies::new_with_planning_inputs_and_evidence( - cost_model: &dyn CostModel, - accuracy_model: &dyn AccuracyModel, - evidence: &dyn AccuracyEvidenceProvider, -) -> ASAPStrategies -``` - -All provider arguments are required for this constructor. They must outlive the -strategy vector. `ASAPStrategies::new(&cost_model)` is the shorter -constructor using the default accuracy model and no extra evidence. - -| Extension point | What it controls | What it cannot establish alone | -| --- | --- | --- | -| `ReplacementStrategy` | Proposed semantic alternatives | Permission to violate query semantics or downstream support | -| `CostModel` | Candidate ordering/sizing hooks and recurrence cost hooks | Correctness, measured costs without evidence, or installed runtime support | -| `AccuracyModel` | Derivation, propagation and satisfaction of guarantees | A meaningful guarantee without its required assumptions/evidence | -| `AccuracyEvidenceProvider` | Planning-time statistics used by supported strategies | Authority to change query requirements | - -Models may be consumed during generation as well as ranking. Construct strategies -with the intended model/evidence; replacing only the final sorting model does not -regenerate parameter choices. For evidence-aware defaults, use -`asap_aware_mapping::replacement::default_strategies_with_evidence`. -For custom accuracy/evidence on sketches, -`ASAPStrategies::new_with_planning_inputs_and_evidence` exposes these providers. -Keep each provider's evidence scope and freshness valid for the query population. - ## Workload inputs and defaults `PlanningWorkload` holds `QueryWorkload` and optional `DataWorkload` as peer @@ -626,128 +300,23 @@ stream norms finite and nonnegative, and distinct bounds positive. | Type/input | Current behavior | Caller responsibility | | --- | --- | --- | -| `QueryRequirements::default()` | `AccuracyTarget::exact()`, no latency bound (`max_latency_ms: None`) | Pass approximation explicitly and thread per-root requirements into search | +| `QueryRequirements::default()` | `AccuracyTarget::exact()`, no latency bound (`max_latency_ms: None`) | Pass approximation explicitly on each workload entry | | `DataWorkload::default()` | Unknown arrival, no source facts | Supply facts needed for the requested comparisons | | `SourceFacts::default()` | Every fact unknown | Unknown facts are not zero; drop facts that no longer hold before planning | -| `DefaultCostModel` | Built-in ordering/sizing and structural cost hooks | Supply deployment evidence for calibrated comparisons | +| `PlanSelectionSettings::default()` | Cost model 1 with the illustrative `ResourceUnitCost`, no measured operation costs | Supply the deployment's unit costs, speed and measurements for calibrated comparisons | +| `DeploymentCapabilities::default()` | Every summary supported, raw data retained, query-time results not kept | Declare what the executor supports | `Default` is a Rust constructor contract, not a general serde omission rule. Several workload fields require explicit serialized values. A struct field being optional also does not guarantee every planning operation can succeed without it. -## Optional whole-plan selection and DAG assembly - -### What does global selection mean? - -`global_selection()` coordinates choices **across target sub-DAG candidate sets -in the workload**. Here, “global” describes that cross-target scope. It does not -mean a proven globally optimal solution over every possible physical plan, nor -selection across every machine in a deployment. - -Consider this conceptual dependency DAG: - -```text -Q1 --+ - +--> A --> B -Q2 --+ - -A's candidate set: alternatives for computing A -B's candidate set: alternatives for computing B -``` - -Both queries need A, and computing A needs B. Choosing to compute A once and -share it, versus recomputing it for each consumer, changes how many evaluations -of B are needed. That can change which choice for B is preferable. - -`cost_sorted()` ranks each target's alternatives using that target's recorded -consumer count. `global_selection()` accounts for ancestor sharing decisions -when deriving effective usage counts, and keeps coupled parent/child composition -choices consistent. The result records coordinated choices; `assemble_selected_dag()` then -constructs the selected semantic DAG while preserving shared nodes. - -| Operation | Question answered | Result | -| --- | --- | --- | -| `cost_sorted()` | How are the alternatives ranked for each subexpression? | Ranked alternatives per target | -| `global_selection()` | Which compatible choices should be used together, accounting for sharing and dependencies? | A coordinated selection across targets under the supplied model | - -Plain `global_selection()` does not decide materialization or establish -physical deployment feasibility. Stage 2 materialization owns -materialization; downstream still owns physical commitment. - -| Method on `CandidateLogicalASAPDAGs` / `GlobalSelection` | Behavior | -| --- | --- | -| `CandidateLogicalASAPDAGs::global_selection(&model)` | Compatible structural selection across targets; no recurrence or materialization planning implied | -| `CandidateLogicalASAPDAGs::global_selection_with_recurrence(...)` | Compatible selection using supplied recurrence profiles/horizon; no materialization commitments implied | -| `GlobalSelection::assemble_selected_dag(&target)` | `Result>, RealizationError>`; constructs untimed semantic IR, not stored summary data | - -Use a target associated with the searched space; DAG assembly can return `None` -when that target is absent. A downstream integration can use these convenience -APIs when its supplied model/evidence supports the intended comparison. Neither -plain structural selection nor taking each target's first candidate substitutes -for checking complete physical alternatives and deployment constraints. - -### API definition and example - -```text -CandidateLogicalASAPDAGs::global_selection(&self, cost_model: &dyn CostModel) -> GlobalSelection<'_> -GlobalSelection::assemble_selected_dag(&self, target: &Rc) - -> Result>, RealizationError> -``` - -For structural inspection only, this complete example selects a semantic root -and prints it. It performs no materialization or deployment -planning. - -```rust -use asap_frontend_promql::lower_promql_workload; -use asap_types::workload::{ - BatchEntry, DataWorkload, DurationMs, Query, - PlanningWorkload, QueryLanguage, QueryRequirements, QueryWorkload, -}; -use asap_aware_mapping::{search_workload, DefaultCostModel}; -use asap_types::types::AccuracyTarget; - -fn main() -> Result<(), Box> { - let workload = PlanningWorkload { - query_workload: QueryWorkload { - language: QueryLanguage::PromQL, - query_batch: Some(vec![BatchEntry { - query: Query("sum(latency)".into()), - requirements: QueryRequirements { - accuracy: AccuracyTarget::exact(), - ..Default::default() - }, - predictability: Default::default(), - invocations: 1, - execute_at: None, - }]), - repeating_queries: None, - }, - // `latency` is sampled every second. - data_workload: Some(DataWorkload::sampled_every( - ["latency".to_string()], - DurationMs(1_000), - )), - }; - let root = lower_promql_workload(&workload)?.remove(0); - let space = search_workload(vec![("q1", root)])?; - let selection = space.global_selection(&DefaultCostModel); - // Search may canonicalize roots; use the root returned by CandidateLogicalASAPDAGs. - if let Some(selected) = selection.assemble_selected_dag(&space.roots[0].1)? { - println!("{selected:#?}"); - } - Ok(()) -} -``` - -## Export and explain +## Export | Function/type | Purpose | | --- | --- | | `asap_types::ir::apply_materialization_timings(&root, &assignment, &mut TimingMemo::new())` | Write execution timing into every node from a `MaterializationAssignment` (default: all query time) and validate the data-state edges; `PlanOutput::execution_timed_dag()` applies the default to a planned workload | | `asap_types::ir::physical_export::compile_physical_asap_dag(&timed_root)` | Export a timed DAG as a `PhysicalASAPDAG` (`PHYSICAL_ASAP_DAG_WIRE_VERSION`); rejects an untimed node; `compile_physical_asap_workload(&roots)` exports a batch with one root per query | | `PhysicalASAPDAGDocument::new(dag)` and `.validate()` | Versioned semantic envelope and explicit validation; constructing it alone does not validate | -| `explain_replacements` / `explain_replacements_with` | Findings from default/custom-strategy search; not a complete physical feasibility report | Choose the export matching your intended handoff: an untimed semantic DAG is not interchangeable with a versioned execution contract. Preserve @@ -762,7 +331,6 @@ cargo doc -p asap-aware-mapping -p asap-types --no-deps - [Frontend PromQL](../../crates/frontend-promql/src/lib.rs), [SQL](../../crates/frontend-sql/src/lib.rs), [MetricsQL](../../crates/frontend-metricsql/src/lib.rs) - [Planner facade](../../crates/planner/src/lib.rs), [planning stages](../../crates/asap-aware-mapping/src/pass/stage1_to_stage3.rs), [plan selection and cost models 1–3](../../crates/asap-aware-mapping/src/stage3_plan_selection.rs) -- [Search, ranking and selection](../../crates/asap-aware-mapping/src/replacement.rs) -- [Cost models](../../crates/asap-aware-mapping/src/cost_model.rs) +- [Deployment capabilities](../../crates/asap-aware-mapping/src/deployment_capabilities.rs) - [Workload types](../../crates/types/src/workload.rs) - [Planner-runtime contract](../design_docs/architecture/planner-runtime-contract.md) diff --git a/docs/develop_docs/metrics-observability-corpora.md b/docs/develop_docs/metrics-observability-corpora.md index 129ea0af1..f9a928529 100644 --- a/docs/develop_docs/metrics-observability-corpora.md +++ b/docs/develop_docs/metrics-observability-corpora.md @@ -42,30 +42,25 @@ o11y-bench, and awesome-prometheus-alerts. They are not duplicated here. formats and writes deterministic fixtures; it does not parse or judge PromQL. - `metrics_observability.rs` is the executable corpus test. It runs the - PromQL parser/lowerer and the sketch-only post-ASAP replacement pass, and - prints the current measurements. + PromQL parser/lowerer and Stage 1 (logical ASAP-aware optimization) on each + query, and prints the current measurements. - `summarize_misses.py` is an optional report formatter. It consumes the test's opt-in `POST_ASAP_MISS` lines and groups misses by coarse root shape. The test prints totals, parse errors, lowering errors, pre-ASAP successes, post-ASAP candidates, unchanged queries, and post-ASAP errors. `Pre-ASAP` means that parsing and lowering produced an `OperatorNode` DAG. `Post-ASAP candidate` -means the isolated `ASAPStrategies` produced a candidate that contains -an ASAP operator (`contains_asap()`). `Unchanged` is a successful pre-ASAP query -for which that strategy returned only the kept pre-ASAP sub-DAG (`retain_exact`). - -## Strategies - -The corpus measurement deliberately uses only -`ASAPStrategies::default_cost_model().replacements(...)` on each -query root. It does not measure workload-wide search or the other default -strategies. - -The default workload search currently registers `ASAPStrategies`, -`HydraGroupingStrategy`, `SharedSubDAGStrategy`, and -`AvgToSumOverCountStrategy`. Workload context can additionally contribute -`RollupStrategy` and `AccuracyReconciliationStrategy`. This baseline is -therefore a sketch-only comparison point. +means that some Stage 1 candidate of the query, run alone through +`run_stage1_to_stage3` with the built-in models, contains an ASAP operator +(`contains_asap()`). `Unchanged` is a successful pre-ASAP query whose Stage 1 +candidates are all exact. + +## What is measured + +Each query is planned alone, with an approximate target (ε = 0.01) on +lowering and no data facts, so the measurement shows which queries Pass 1 +and Pass 2 can give a summary candidate. It does not measure sharing across +queries or which plan Stage 3 selects. ## Alerts diff --git a/docs/develop_docs/offline-sketch-evidence.md b/docs/develop_docs/offline-sketch-evidence.md index 9e4bc02f1..d3dd03cdb 100644 --- a/docs/develop_docs/offline-sketch-evidence.md +++ b/docs/develop_docs/offline-sketch-evidence.md @@ -84,12 +84,12 @@ include generator configuration and seed, or trace checksum and sampling rules. Validity intervals are supplied by the producer or deployment policy; they are an explicit applicability assumption, not a measured property. -`EmpiricalCostModel` implements the planner's existing `CostModel` boundary. +`EmpiricalCostModel` implemented the planner's former `CostModel` boundary. It derives the planner's default parameter configurations for the requested accuracy, and orders algorithms by mean measured update CPU only when all candidates have applicable measurements. Otherwise it preserves discovery order. It returns every candidate and keeps default sizing. Its final candidate scores -retain `DefaultCostModel`'s dimensionless structural meaning; do not label those +retained `DefaultCostModel`'s dimensionless structural meaning; do not label those scores as CPU or measured savings. Deployment cost models can own the provider and call `lookup` with their own diff --git a/docs/develop_docs/planner-vocabulary-migration.md b/docs/develop_docs/planner-vocabulary-migration.md index 80ef6de72..c29982a95 100644 --- a/docs/develop_docs/planner-vocabulary-migration.md +++ b/docs/develop_docs/planner-vocabulary-migration.md @@ -1,5 +1,10 @@ # Planner vocabulary migration (#427) +> **Status: historical.** Most names in this table belonged to the legacy +> replacement search, which has been removed. The planner now plans through +> the stages described in [the architecture](asap-aware-mapping-architecture.md). +> The table is kept as a record of the rename. + This is a Rust source API rename. Update imports and call sites using the table below. Planning, ranking, schema resolution, cost arithmetic, and window coverage rules are unchanged. Old Rust names are removed rather than retained as a second diff --git a/docs/develop_docs/replacement-explanations.md b/docs/develop_docs/replacement-explanations.md index 4cd97300a..d8b22082a 100644 --- a/docs/develop_docs/replacement-explanations.md +++ b/docs/develop_docs/replacement-explanations.md @@ -1,5 +1,12 @@ # Explaining a Replacement +> **Status: historical.** `explanation.rs` and `explain_replacements` were +> removed with the legacy replacement search. What the planner considered, +> and why each candidate was pruned or rejected, is now in the stages' +> output: Pass 1's pruned candidates with their `PruneReason`, Stage 2's +> pruned candidates, and Stage 3's `Rejection` reasons, all written by +> `stage_pipeline` (see the [Stage Viewer guide](../user_guide_docs/stage-viewer.md)). + ASAP-aware mapping should make its discovered opportunities visible to users and other components. For each relevant region of the plan, the planner should be able to explain: diff --git a/docs/develop_docs/target-candidate-api-migration.md b/docs/develop_docs/target-candidate-api-migration.md index e9c9e5436..849a10a9d 100644 --- a/docs/develop_docs/target-candidate-api-migration.md +++ b/docs/develop_docs/target-candidate-api-migration.md @@ -1,5 +1,12 @@ # Target candidate and DAG assembly API migration (#456) +> **Status: historical.** The candidate and DAG-assembly API this migration +> renamed (the legacy search's target candidates, `CandidateLogicalASAPDAGs`, +> `GlobalSelection`) has been removed. Stage 1 now produces +> `LogicalCandidate`s, Stage 2 `PhysicalCandidate`s, and Stage 3's +> `select_plan` the `Selection`; see [the architecture](asap-aware-mapping-architecture.md). +> The table is kept as a record of the rename. + This source-only rename follows the input/output/workflow design in #445. Candidate generation, ordering, accuracy checks, selection, and runtime behavior are unchanged. #453 separately defines the integration API surface. From 327677137a33f4e39ee41716c3ed68ceefaedab5 Mon Sep 17 00:00:00 2001 From: zzylol <50204836+zzylol@users.noreply.github.com> Date: Sat, 17 Oct 2026 20:25:28 +0000 Subject: [PATCH 09/10] docs(design): the stage pipeline replaces the legacy search in the design docs input-output-workflow.md, the architecture overview, the planner pipeline, the planner-runtime contract and the physical-planning doc describe Stages 0-3 and PlanOutput instead of CandidateLogicalASAPDAGs ranking and global_selection. Docs about the removed search carry a historical note. --- docs/design_docs/architecture/README.md | 108 +++---- .../architecture/asap-aware-mapping.md | 6 + .../architecture/asap-aware-plan-search.md | 8 +- .../evidence-dependent-candidates.md | 6 + .../architecture/input-output-workflow.md | 264 +++++------------- .../architecture/planner-runtime-contract.md | 10 +- ...d_interface_with_pluggable_optimization.md | 7 + docs/design_docs/concepts/accuracy-models.md | 14 +- docs/design_docs/concepts/planner-pipeline.md | 40 +-- docs/design_docs/concepts/post-asap-ir.md | 2 +- .../decisions/concat-unique-keys.md | 3 +- docs/design_docs/decisions/cse-cost-model.md | 6 + .../physical-planning-and-deployment.md | 10 +- .../analytical-resource-cost.md | 4 +- .../ddsketch-quantile-ratios.md | 5 + .../maintained-populations.md | 4 +- .../asap-aware-mapping/summary-properties.md | 5 +- .../proposals/asap-primitive-schema.md | 2 +- .../proposals/asapquery-rule-coverage.md | 6 + docs/develop_docs/README.md | 2 +- docs/develop_docs/library-api.md | 2 +- 21 files changed, 211 insertions(+), 303 deletions(-) diff --git a/docs/design_docs/architecture/README.md b/docs/design_docs/architecture/README.md index d0cc61026..86346e156 100644 --- a/docs/design_docs/architecture/README.md +++ b/docs/design_docs/architecture/README.md @@ -1,50 +1,41 @@ # ASAPPlanner design overview -ASAPPlanner is a reusable planning library. It converts queries and workload -requirements into a deployment-independent space of logical Post-ASAP candidates. -It does not commit, deploy, or execute a physical plan; downstream systems such -as ASAPQuery-backend bind the candidates to physical alternatives, make the -deployment-level decision, and run the selected contract. +ASAPPlanner is a reusable planning library. It converts queries, workload +requirements and the deployment's inputs into one selected physical plan +that may use Approximate Streaming Analytics Primitives (sketches and exact +summaries). It does not deploy or execute the plan; downstream systems such +as ASAPQuery-backend run it. For the integration workflow, start with [ASAPPlanner input, output, and -workflows](input-output-workflow.md). It defines inputs, `CandidateLogicalASAPDAGs`, selection -and assembly workflows, and future replanning support. +workflows](input-output-workflow.md). The stages are defined in the +[planning stages design](../proposals/planner-layering.md). ## Planner component flow ```mermaid flowchart TD - W["PlanningWorkload: query demand + optional data facts"] - F["Frontend dependencies: SQL catalog or PromQL time"] - E["Strategy, accuracy model, and applicable evidence"] - PRE["Frontend lowering → canonical Pre-ASAP OperatorNode roots"] - SEARCH["Whole-workload candidate search: sharing, legality, accuracy"] - SPACE["CandidateLogicalASAPDAGs: compact logical candidate DAG space"] - RANK["Optional cost_sorted: ranked inspection view"] - SELECT["Optional global_selection + assemble_selected_dag"] - DAG["Selected logical Post-ASAP DAG"] - BACKEND["Downstream: bind physical alternatives, decide deployment, compile and execute"] + W["PlanningWorkload: query workload + optional data workload"] + F["Frontend dependencies: SQL catalog or PromQL histogram catalog"] + D["Deployment inputs: accuracy model, plan selection settings (cost model), deployment capabilities"] + PRE["Stage 0: query-language frontends → logical DAG (Pre-ASAP OperatorNode roots)"] + S1["Stage 1: logical ASAP-aware optimization (Pass 1, Pass 2) → logical ASAP candidates"] + S2["Stage 2: physical ASAP-aware optimization (materialization) → physical ASAP candidates"] + S3["Stage 3: plan selection → selected PhysicalASAPDAG, costs and rejection reasons"] + BACKEND["Downstream: compile, deploy and execute"] W --> PRE F --> PRE - PRE --> SEARCH - E --> SEARCH - SEARCH --> SPACE - SPACE --> RANK --> BACKEND - SPACE --> SELECT --> DAG --> BACKEND + PRE --> S1 --> S2 --> S3 --> BACKEND + D --> S1 + D --> S2 + D --> S3 ``` -`CandidateLogicalASAPDAGs` is the output of logical candidate search. Each target's candidate set holds -alternatives and rejection reasons, but no materialization decision. -Choose between two branches: inspect candidates (optionally ranked), or select -and assemble logical DAGs. Stage 2 materialization (#509) will decide per -sub-DAG whether to materialize and whether at ingestion or query time; until -then every summary runs at query time. No branch by itself deploys or executes -a physical plan. -Known-invalid evidence rejects a logical candidate. Missing accuracy evidence -leaves a constructible candidate visible in `CandidateLogicalASAPDAGs` but uncertified; default -selection does not commit it without the required guarantee. Cost evidence can -rank eligible candidates, but it cannot establish a missing guarantee or turn -an unsupported physical alternative into a deployable plan. +Stages 1 and 2 enumerate candidates without comparing them. Stage 3 rejects +the candidates that miss a query's accuracy target or latency bound, or that +the deployment cannot run, and selects one of the rest under the deployment's +cost model. Missing accuracy evidence never certifies a candidate: a +candidate whose output has no guarantee is rejected for an approximate +target. ## Module map @@ -52,49 +43,32 @@ an unsupported physical alternative into a deployable plan. |---|---|---| | Shared IR | `asap-types` | The unified operator IR (`ir`: one `OperatorNode` before and after ASAP optimization), schemas, workloads, guarantees, and exported plan data | | Front-end common | `frontend-common` | Name-based `UnresolvedOp` tree shared by the front ends, and `resolve_root` into the operator IR | -| Query frontends | `frontend-sql`, `frontend-promql`, `frontend-metricsql` | Parse source languages and produce canonical Pre-ASAP queries | -| ASAP-aware mapping | `asap-aware-mapping` | Candidate generation, CSE, legality, accuracy propagation, costing, and ranking | -| Developer inspection | `devtools` | Expose planner DAGs, alternatives, decisions, and explanations for inspection | -| End-to-end validation | `integration-tests` | Verify behavior across frontends, mapping, and output IR | +| Query frontends | `frontend-sql`, `frontend-promql`, `frontend-metricsql` | Stage 0: parse source languages and produce canonical Pre-ASAP queries | +| ASAP-aware mapping | `asap-aware-mapping` | Stages 1–3: candidate generation, ASAP-aware CSE, materialization, accuracy checks, costing and selection | +| Planner facade | `planner` | `e2e_plan`: Stage 0 followed by Stages 1–3 | +| Executor | `asap-physical-operators` | Runs a `PhysicalASAPDAG` over `asap_sketchlib` summaries | +| Developer inspection | `devtools` | `stage_pipeline` and the Stage Viewer documents, corpus coverage tools | +| End-to-end validation | `integration-tests` | Verify behavior across frontends, stages and execution | ## Inputs and outputs -Planner inputs are intentionally broader than a query string. Selection may -depend on query recurrence, data arrival and distribution, accuracy and latency -requirements, the planning horizon, available materialized state, downstream -capabilities, and complete cost evidence. Missing or stale evidence must remain -explicit rather than being treated as zero. +Planner inputs are intentionally broader than a query string. Selection +depends on query recurrence, data arrival and per-source facts, accuracy and +latency requirements, and the deployment's cost model and capabilities. +Missing facts stay unknown rather than being treated as zero. -The primary output is `CandidateLogicalASAPDAGs`; `cost_sorted` derives an optional ranked -view with index-aligned costs. Downstream may inspect compatible choices -across targets rather than assuming the first candidate is a feasible -physical workload plan. Candidates carry logical summary algorithms, -parameters, and guarantees, but no materialization decision. Rejection reasons -are retained in the candidate space. - -ASAPQuery-backend and other downstream applications translate the candidates -into physical alternatives. They own concrete implementations, storage layout, -placement, sharding, deployment-level cost and compatibility, final commitment, -serving, and operational feedback. Their physical planning can reorder -candidates because it has evidence that the reusable Planner does not, but it -must not silently change Planner-owned semantics. - -`CandidateLogicalASAPDAGs::global_selection` optionally coordinates structural choices across -targets; `GlobalSelection::assemble_selected_dag` constructs a selected semantic DAG. -Those APIs do not establish physical feasibility or a -materialization decision. See the [library guide](../../develop_docs/library-api.md#optional-whole-plan-selection-and-dag-assembly) -for the distinction. Downstream may consume candidates directly and retains -responsibility for physical commitment. +The output is the selected plan (`PlanOutput`: one plan root per query and +Stage 3's `Selection` with every candidate's cost or rejection reason). +Downstream owns concrete storage layout, placement, sharding, serving and +operational feedback, and must not silently change Planner-owned semantics. ## Further reading +- [Planning stages design](../proposals/planner-layering.md) - [Parsing and canonicalization](parse-and-canonicalize.md) - [Pre-ASAP IR](../concepts/pre-asap-ir.md) - [Post-ASAP IR](../concepts/post-asap-ir.md) -- [ASAP-aware mapping](asap-aware-mapping.md) - [Accuracy guarantees](../proposals/asap-aware-mapping/end-to-end-accuracy-guarantees.md) - [Physical-plan integration](physical-plan-integration.md) -- [Analytical resource cost](../proposals/asap-aware-mapping/analytical-resource-cost.md) -- [Searching over plans](asap-aware-plan-search.md) -- [Explainability](../../develop_docs/replacement-explanations.md) - [ASAPPlanner planner-runtime contract](planner-runtime-contract.md) +- Historical: [ASAP-aware mapping](asap-aware-mapping.md), [Searching over plans](asap-aware-plan-search.md), [Analytical resource cost](../proposals/asap-aware-mapping/analytical-resource-cost.md) diff --git a/docs/design_docs/architecture/asap-aware-mapping.md b/docs/design_docs/architecture/asap-aware-mapping.md index c04fa530a..6ee996b77 100644 --- a/docs/design_docs/architecture/asap-aware-mapping.md +++ b/docs/design_docs/architecture/asap-aware-mapping.md @@ -1,5 +1,11 @@ # ASAP-Aware Mapping +> **Status: historical.** This document describes the legacy replacement +> search (strategies, `CandidateLogicalASAPDAGs` as a Rust type, cost-model +> ranking and `global_selection`), which has been removed. The planner now +> plans through the stages of the [planning stages design](../proposals/planner-layering.md). The +> document is kept as a record of the design. + ## Overview ASAP-aware mapping decides **whether and how a query intent can be answered using summaries instead of scanning raw data**. diff --git a/docs/design_docs/architecture/asap-aware-plan-search.md b/docs/design_docs/architecture/asap-aware-plan-search.md index e612740b6..b00ea1d23 100644 --- a/docs/design_docs/architecture/asap-aware-plan-search.md +++ b/docs/design_docs/architecture/asap-aware-plan-search.md @@ -1,5 +1,11 @@ # Candidate Plan Search +> **Status: historical.** This document describes the legacy replacement +> search (strategies, `CandidateLogicalASAPDAGs` as a Rust type, cost-model +> ranking and `global_selection`), which has been removed. The planner now +> plans through the stages of the [planning stages design](../proposals/planner-layering.md). The +> document is kept as a record of the design. + ASAP-aware mapping should consider all alternatives holistically rather than optimize prematurely. Suppose a plan contains several independent-looking decision points: @@ -91,6 +97,6 @@ order. The [code architecture](../../develop_docs/asap-aware-mapping-architecture.md) describes current discovery and registry behavior; the -[library guide](../../develop_docs/library-api.md#optional-whole-plan-selection-and-dag-assembly) +[library guide](../../develop_docs/library-api.md) shows selection and its evidence boundaries. Broader optimization dimensions are tracked in the [proposal](../proposals/asap-aware-mapping/optimizations.md). diff --git a/docs/design_docs/architecture/evidence-dependent-candidates.md b/docs/design_docs/architecture/evidence-dependent-candidates.md index b01cc8d21..c815bf04f 100644 --- a/docs/design_docs/architecture/evidence-dependent-candidates.md +++ b/docs/design_docs/architecture/evidence-dependent-candidates.md @@ -1,5 +1,11 @@ # Evidence-dependent candidates +> **Status: historical.** This document describes the legacy replacement +> search (strategies, `CandidateLogicalASAPDAGs` as a Rust type, cost-model +> ranking and `global_selection`), which has been removed. The planner now +> plans through the stages of the [planning stages design](../proposals/planner-layering.md). The +> document is kept as a record of the design. + Audience: ASAPPlanner library integrators, especially ASAPQuery-backend. `CandidateLogicalASAPDAGs` is a space of constructible logical alternatives, not a list of diff --git a/docs/design_docs/architecture/input-output-workflow.md b/docs/design_docs/architecture/input-output-workflow.md index a079b3c2e..13a76adb3 100644 --- a/docs/design_docs/architecture/input-output-workflow.md +++ b/docs/design_docs/architecture/input-output-workflow.md @@ -5,10 +5,11 @@ This document is for library integrators such as ASAPQuery-backend, not users submitting queries through a backend. -ASAPPlanner is a **logical planning library**. Its input is a planning workload -plus the models, evidence, and deployment capabilities needed by the requested -planning workflow. Its canonical output is a `CandidateLogicalASAPDAGs` containing the legal -Post-ASAP alternatives for the workload. +ASAPPlanner is a **planning library**. Its input is a planning workload plus +the deployment's inputs: its accuracy model, plan selection settings (the +cost model) and capabilities. Its output is one selected physical plan for +the workload, with every alternative it considered and why each was not +selected. ### Input fields at a glance @@ -17,40 +18,31 @@ Post-ASAP alternatives for the workload. | `PlanningWorkload.query_workload` | Query language and one-time/repeating query workloads | Yes | | `PlanningWorkload.data_workload` | How the data arrives, and facts about each source | No implicit default. Set `None` when unavailable; a PromQL query with a bare instant selector needs a nonzero `sample_interval` on the metric it reads. | | Frontend-specific dependencies (outside `PlanningWorkload`) | `SqlCatalog` for SQL; when needed, `HistogramCatalog` for PromQL | `SqlCatalog` is required for SQL lowering | -| Planning models | Candidate cost/ranking and accuracy composition/checking | Used by the relevant APIs; built-in `DefaultCostModel` and `DefaultAccuracyModel` are available | -| External evidence and capabilities | Domain facts, measured costs, workload statistics, and runtime support | Supply when available and when the chosen optimization depends on them; absence is not proof | +| `PlanningModels` | Accuracy model and evidence, `PlanSelectionSettings` (cost model 1, 2 or 3, resource speed, measured operation costs), `DeploymentCapabilities` | `PlanningModels::builtin()` uses `DefaultAccuracyModel`, no evidence, cost model 1 with illustrative unit costs, and a deployment that supports everything | -Frontend lowering and candidate search are stages within this workflow, not -additional end-to-end inputs. See [Inputs](#inputs) for the nested workload -fields and [frontend dependencies](#frontend-specific-dependencies). +Frontend lowering (Stage 0) and Stages 1–3 run inside the workflow; they are +not additional inputs. See [Inputs](#inputs) for the nested workload fields +and [frontend dependencies](#frontend-specific-dependencies). ### Output at a glance | Output | Fields or contents | Meaning | |---|---|---| -| `CandidateLogicalASAPDAGs` | The legal candidate Post-ASAP DAGs for the workload, represented compactly as canonical roots, one candidate set per target sub-DAG, and cross-target composition information | The ASAPPlanner output | +| `PlanOutput` | `plans`: one `QueryPlan { entry_index, root }` per query; `selection`: Stage 3's `Selection` (`selected`, `plan`, `costs`, `accepted`, `rejected`, `front`), set by the default pass `Stage1ToStage3Pass`; standalone scalar queries in `scalar_roots` and `scalar_plans` | The selected plan; `execution_timed_dag()` exports it as one `PhysicalASAPDAG` | +| `Stage1ToStage3Run` (from `run_stage1_to_stage3`) | Every stage's candidates: Pass 1's per-computation candidates, the logical candidates, the physical candidates, Stage 2's pruned candidates, and the selection | For callers that inspect or display the alternatives | -[Ranking](#ranked-view) and [selection and -DAG assembly](#selection-and-dag-assembly) APIs operate on this `CandidateLogicalASAPDAGs`. -These are alternative uses of the candidate space, not mandatory sequential -stages. Its candidates do not choose ingestion-time versus query-time -placement: a `MaterializationAssignment` sets each node's execution timing. -Stage 2 materialization (#509) will decide per sub-DAG whether to materialize -and whether at ingestion or query time; until then every summary runs at query -time. - -The candidate DAGs are logical planning artifacts. ASAPPlanner does **not** -produce a deployed executable plan; downstream systems bind physical operators, -choose placement and storage, deploy state, and execute queries. +The selected plan is a planning artifact. ASAPPlanner does **not** deploy +it; downstream systems compile it, choose placement and storage, deploy state +and execute queries. ```text -PlanningWorkload + frontend dependencies + planning models/evidence +PlanningWorkload + frontend dependencies + PlanningModels | v - ASAPPlanner + ASAPPlanner: Stage 0 → Stage 1 → Stage 2 → Stage 3 | v - CandidateLogicalASAPDAGs: candidate Post-ASAP DAGs + PlanOutput: selected plan, costs and rejection reasons ``` --- @@ -58,51 +50,34 @@ PlanningWorkload + frontend dependencies + planning models/evidence ## Running example: a recurring PromQL query Suppose a dashboard evaluates `count_over_time(up[5m])` once a minute, and -`up` receives a sample every 15 seconds. This diagram traces the concrete -inputs and the two possible uses of the same candidate space: +`up` receives a sample every 15 seconds: ```mermaid flowchart TD - Q["query_workload: PromQL; repeating query count_over_time(up[5m]); every 60 s; exact; real-time; predictable"] + Q["query_workload: PromQL; repeating query count_over_time(up[5m]); every 60 s; exact; predictable"] D["data_workload: ingesting; up sampled every 15 s"] - F["PromQL lowering"] - R["One canonical OperatorNode root"] - S["Candidate search"] - P["CandidateLogicalASAPDAGs: logical choices for this root"] - I["cost_sorted: inspect choices"] - G["global_selection + assemble_selected_dag(root)"] - L["One selected Post-ASAP DAG; the exact pre-ASAP sub-DAG if no optimization is selected"] - B["Backend: bind physical operators, deploy, and execute"] + M["PlanningModels: accuracy model, cost model, deployment capabilities"] + F["Stage 0: PromQL lowering → one OperatorNode root"] + S1["Stage 1: exact accumulator or plain exact; panes"] + S2["Stage 2: each summary build at query time or maintained at ingestion time"] + S3["Stage 3: check and price every physical candidate; select one"] + B["Backend: compile, deploy and execute the selected plan"] Q --> F D --> F - F --> R --> S --> P - P --> I - P --> G --> L --> B + F --> S1 --> S2 --> S3 --> B + M --> S1 + M --> S2 + M --> S3 ``` -“Predictable” says the query is known in advance; it is independent of its -one-minute recurrence. The `CandidateLogicalASAPDAGs` may contain an exact count-summary -realization, but it is not a deployed query. The caller can inspect candidates -or obtain a logical DAG; deciding whether maintaining a summary is cheaper than -recomputing raw results belongs to Stage 2 materialization (#509). +"Predictable" says the query is known in advance; it is independent of its +one-minute recurrence. Whether maintaining the count as data arrives is +cheaper than recomputing it at every evaluation is Stage 3's comparison of +Stage 2's candidates under the deployment's cost model. For contrast, a one-time SQL query needs a catalog but need not supply data -arrival evidence merely to inspect logical alternatives: - -```mermaid -flowchart LR - Q["query_batch: SELECT COUNT(*) FROM metrics; invocations 1; AdHoc"] - C["SqlCatalog: resolves metrics and its columns"] - F["SQL lowering"] - R["One OperatorNode root"] - P["Candidate search → CandidateLogicalASAPDAGs"] - Q --> F - C --> F - F --> R --> P -``` - -In this SQL example, `data_workload` can be `None` if the chosen lowering and -search rules do not consume it. +arrival facts: `data_workload` can be `None`, and the data is then treated as +of unknown arrival with no source facts. --- @@ -264,11 +239,11 @@ latter cannot be fabricated by one. | Input | Where it enters / default | Why it matters | |---|---|---| -| Accuracy model | Target-aware search takes an `AccuracyModel`; `DefaultAccuracyModel` is available. Default strategies also use it for candidate construction. | Composes candidate guarantees and checks them against requested accuracy. The model does not itself provide missing data-domain facts. | -| Cost model | Candidate strategies and `cost_sorted`/`global_selection` use a `CostModel`; `DefaultCostModel` is available. | Ranks or selects candidates. The built-in model is not a measured deployment cost for every physical implementation. | -| Accuracy/domain evidence | `AccuracyEvidenceProvider`; default strategies use `NoAccuracyEvidence` when no provider is supplied. | Input ranges, nonempty populations, Top-K intervals, and similar facts can certify or rule out particular approximations. Missing facts remain unknown. | -| Measured cost evidence | Supplied through a deployment-specific cost model or physical-evidence provider when cost-based physical comparison is needed. | CPU, memory, and I/O estimates must be comparable before claiming a summary beats raw recomputation. | -| Runtime capabilities | Checked by deployment-specific providers. | Prevents choosing a maintenance/window operation the intended executor cannot implement. | +| Accuracy model | `PlanningModels` (`AccuracyModel`; `DefaultAccuracyModel` by default) | Composes candidate guarantees and checks them against requested accuracy. The model does not itself provide missing data-domain facts. | +| Accuracy/domain evidence | `PlanningModels` (`AccuracyEvidenceProvider`; `NoAccuracyEvidence` by default) | Input ranges, nonempty populations and similar facts can certify or rule out particular approximations. Missing facts remain unknown. | +| Cost model | `PlanSelectionSettings.cost_model` (cost model 1 by default) | Turns each valid candidate's resource usage into a choice. The illustrative unit costs are not a measured deployment cost. | +| Measured cost evidence | `PlanSelectionSettings.measured_operation_costs` | Measured CPU time per sketch update, merge and estimate replaces the formulas for the sketch nodes it matches. | +| Runtime capabilities | `DeploymentCapabilities` (`supported_summaries`, `raw_data_retained`, `query_time_results_retained`) | Prevents choosing a summary, raw-data scan or kept state the intended executor cannot provide. | For example, the query `quantile_over_time(0.9, data[5m]) / quantile_over_time(0.5, data[5m])` does not tell Planner whether the windows @@ -279,144 +254,53 @@ backend observation must cover the relevant data, not just an earlier window. Evidence is input; a derived guarantee or rejection reason is output. Missing facts do not establish either validity or invalidity: Planner cannot claim a -guarantee that depends on them. Candidate retention and selection depend on -the applicable strategy, accuracy target, and helper; missing evidence is not -a blanket reason to discard unrelated candidates. For the direct DDSketch -ratio above, search retains a candidate without a proven root guarantee when -domain evidence is missing; automatic `global_selection` does not choose it. -See the [candidate-search reference](../../develop_docs/library-api.md#generate-and-rank-candidates) -for this backend-selection path. +guarantee that depends on them, so Stage 3 rejects, with its reason, a +candidate whose output has no guarantee for an approximate target. --- ## Output -### `CandidateLogicalASAPDAGs` - -`CandidateLogicalASAPDAGs` is Planner's canonical output. It contains: - -* canonical workload roots; -* one `TargetSubDAGCandidates` entry for each discovered target sub-DAG; -* legal replacement candidates; -* rejected candidates and reasons; and -* information needed to select compatible candidates across targets. - -A `CandidateLogicalASAPDAGs` represents a **space of logical DAG choices**, not a single plan. -It is exposed to integrators because the backend may choose among candidates -using implementation support, measured costs, and available resources that -candidate search does not have. A summary that is cheap on one backend may be -expensive or unsupported on another. Returning only one plan during search -would discard those choices too early. - -Callers with suitable models can instead use the [selection workflows](#workflows) -below. A future higher-level API could hide `CandidateLogicalASAPDAGs` behind those decisions; -the current interface lets an integrator own them. DAG assembly connects choices -after selection and does not replace this candidate interface. - -Here, a **root** is the top-level `Rc` for a workload query. A -**target** is any discovered sub-DAG that may be replaced, including roots. -For `count(up) + 1`, the addition is a root and `count(up)` can be an inner -target. `TargetSubDAGCandidates` holds the alternatives for one such target. - -It represents that space compactly instead of eagerly copying every complete -DAG. `CandidateLogicalASAPDAGs` stores the workload's canonical roots once, creates one -`TargetSubDAGCandidates` entry for each distinct target sub-DAG, and stores -that target's replacement alternatives once inside the entry. Candidate -children refer back to canonical -targets, so common subexpressions and shared alternatives are not duplicated -across roots. - -For example, if one target has three alternatives and its child has two, -eager enumeration could create six complete DAGs. `CandidateLogicalASAPDAGs` stores the three -parent alternatives, the two child alternatives, and their relationship. -Whole-plan selection chooses compatible alternatives across those targets; -`assemble_selected_dag(root)` then recursively substitutes the selected alternatives to -construct a complete Post-ASAP DAG. Sharing each target's candidate set avoids the -Cartesian-product expansion of complete DAGs and preserves shared nodes. +### `PlanOutput` -## Workflows +`PlanOutput` holds the selected plan and Stage 3's record of the selection: -All paths start by lowering the workload and searching for candidates: +* `plans`: one plan root per query, in `QueryWorkload::entries()` order; +* `selection.selected`: the id of the selected physical candidate, and + `selection.plan` its roots; +* `selection.costs`: each valid candidate's resource usage and, under cost + models 1 and 2, its cost per second; +* `selection.accepted`: per valid candidate and query, the accuracy metric + Stage 3 accepted; +* `selection.rejected`: every other candidate with its reason, whether it + failed a check or lost under the cost model; +* `selection.front`: the Pareto front under cost model 3's Pareto objective. -```text -PlanningWorkload + frontend dependencies + planning models/evidence - -> frontend lowering: one OperatorNode root per normalized query entry - -> search_workload_with_targets - -> CandidateLogicalASAPDAGs -``` +Planner selects because it now has the inputs a backend used to own: the +deployment declares its cost model, measurements and capabilities, and every +alternative stays visible in `selection` and in `Stage1ToStage3Run`. -The integration associates each lowered root with a caller-owned `Id` and its -accuracy target for search; `Id` correlates results, not query semantics. -See the [public library reference](../../develop_docs/library-api.md) for arguments. +## Workflows -Then choose the operation matching the caller's responsibility: +```text +PlanningWorkload + frontend dependencies + PlanningModels + -> Stage 0: frontend lowering, one root per normalized query entry + -> Stage 1: Pass 1 per-computation candidates, Pass 2 sharing, panes, reuse + -> Stage 2: one physical candidate per materialization assignment + -> Stage 3: check, measure and select + -> PlanOutput +``` | Purpose | Operation | Result | |---|---|---| -| Inspect candidates or let the backend choose | [Ranked view](#ranked-view), if ranking is useful | Per-target candidate lists and costs | -| Ask Planner to choose logical computations | [Selection and DAG assembly](#selection-and-dag-assembly) | One selected Post-ASAP DAG root per query | - -### Ranked view - -`CandidateLogicalASAPDAGs::cost_sorted` returns one `RankedTargetSubDAGCandidates` for each -`TargetSubDAGCandidates` entry. Conceptually, it is the same target's -alternatives in cost-model preference order where the model defines one -(otherwise discovery order), with one displayed cost per alternative. It is -a **view of one decision point**, not a complete DAG or a selected plan. +| Plan a workload end to end | `asap_planner::e2e_plan` | `PlanOutput` | +| Plan already-lowered roots and inspect every stage | `asap_aware_mapping::run_stage1_to_stage3` | `Stage1ToStage3Run`, whose `stage3_plan_selection` is the `Selection` | +| Inspect a workload's stages as a document | `stage_pipeline` (devtools) | The stage document the [Stage Viewer](../../user_guide_docs/stage-viewer.md) shows | -The return type is `Vec>`; each element has this shape: - -```rust -struct RankedTargetSubDAGCandidates<'a> { - target: &'a Rc, - consumer_count: usize, - candidates: Vec<&'a ReplacementSubDAG>, - costs: Vec, // costs[i] describes candidates[i] -} -``` - -`consumer_count` counts references to this target in the workload. `costs` -may contain `NaN` when the model has no numeric estimate; the ordering is not -itself a deployment-cost certificate. This view is useful for debugging, -explanation, or downstream optimization. Candidate presence does not imply -physical deployability. - -### Selection and DAG assembly - -The input is `CandidateLogicalASAPDAGs` and a cost model. Call -`CandidateLogicalASAPDAGs::global_selection(&cost_model)` once for the workload, then -`GlobalSelection::assemble_selected_dag(root)` for each wanted query root. -These are two public APIs, not one combined call: N roots require one selection -and N assembly calls. Each successful assembly returns one DAG root; the caller -collects them for the workload, with shared nodes where applicable. - -Selection chooses compatible alternatives across the workload. For example, -if two queries can share a summary, their choices must agree on the shared -computation. `assemble_selected_dag(root)` then connects the chosen alternatives -into each query's DAG in memory, preserving shared nodes. Candidate search has -already built candidate sub-DAGs; assembly connects the selected choices into -the result for one query root. - -| Input → decisions → output (click a step for details) | -|:---:| -| **Input:** [CandidateLogicalASAPDAGs](asap-aware-plan-search.md) + cost model | -| ↓ | -| **Select:** [global_selection](../../develop_docs/library-api.md#what-does-global-selection-mean) chooses compatible alternatives | -| ↓ | -| **Assemble:** [assemble_selected_dag(root)](../../develop_docs/library-api.md#api-definition-and-example) connects those choices for each query root | -| ↓ | -| **Output:** one selected logical [Post-ASAP DAG](../concepts/post-asap-ir.md) per query root | - -Each output DAG specifies the chosen operators, parameters, and accuracy -guarantees. Its root is an `Rc` (the same IR as the input, -with some nodes now ASAP operators) and carries no execution timing yet; the -[API reference](../../develop_docs/library-api.md#api-definition-and-example) -describes the function signatures and return handling. - -This path selects how to compute the query, not whether summary state is -materialized. - ---- +The integration associates each lowered root with a caller-owned `Id` and its +workload entry (accuracy target, latency bound, recurrence); `Id` correlates +results, not query semantics. See the +[public library reference](../../develop_docs/library-api.md) for arguments. ## Replanning (future support) @@ -435,5 +319,5 @@ responsibilities. * [Pre-ASAP IR](../concepts/pre-asap-ir.md) * [Post-ASAP IR](../concepts/post-asap-ir.md) * [Planner/runtime responsibilities](planner-runtime-contract.md) -* [Plan search internals](asap-aware-plan-search.md) +* [Planning stages design](../proposals/planner-layering.md) * [Public library reference](../../develop_docs/library-api.md) diff --git a/docs/design_docs/architecture/planner-runtime-contract.md b/docs/design_docs/architecture/planner-runtime-contract.md index 4ac46c4be..51bd9bd98 100644 --- a/docs/design_docs/architecture/planner-runtime-contract.md +++ b/docs/design_docs/architecture/planner-runtime-contract.md @@ -2,11 +2,11 @@ ## Purpose -ASAPPlanner produces `CandidateLogicalASAPDAGs`, a compact logical candidate space. Integrators -may select candidates downstream or ask Planner's helpers to select and assemble -DAGs. Stage 2 materialization (#509) will decide per sub-DAG whether to -materialize and whether at ingestion or query time; until then every summary -runs at query time. Physical deployment and execution remain downstream. The +ASAPPlanner produces one selected physical plan for a workload: Stage 1 +enumerates the logical candidates, Stage 2 decides per summary build whether +and when it is materialized, and Stage 3 checks every physical candidate and +selects one under the deployment's cost model. Physical deployment and +execution remain downstream. The [input/output/workflow design](input-output-workflow.md) defines this boundary. ## Three decision layers diff --git a/docs/design_docs/architecture/updated_interface_with_pluggable_optimization.md b/docs/design_docs/architecture/updated_interface_with_pluggable_optimization.md index 2c31be92e..a77a602d4 100644 --- a/docs/design_docs/architecture/updated_interface_with_pluggable_optimization.md +++ b/docs/design_docs/architecture/updated_interface_with_pluggable_optimization.md @@ -1,5 +1,12 @@ # One entry point, and a pluggable optimization pass +> **Status:** the end-to-end function (`asap_planner::e2e_plan`) and the +> pluggable pass (`OptimizationPass`, with `Stage1ToStage3Pass` as the default) +> are implemented. Passages that name the legacy search (`MajorPass`, +> `search_workload`, `global_selection`) describe code that has been removed; +> the default pass runs the stages of the +> [planning stages design](../proposals/planner-layering.md). + ## 1. What is new 1. An **end-to-end function API** to lib users, giving them a one-function-call abstraction from a prepared input to the selected Post-ASAP DAG. diff --git a/docs/design_docs/concepts/accuracy-models.md b/docs/design_docs/concepts/accuracy-models.md index 8d7595462..4bf309b1f 100644 --- a/docs/design_docs/concepts/accuracy-models.md +++ b/docs/design_docs/concepts/accuracy-models.md @@ -131,14 +131,12 @@ another implementation with the same algorithm name. In particular, a named empirical calibration is different from an arbitrary benchmark's maximum observed error; both its confidence and applicability must remain explicit. -The current interfaces still expose general parameter proposal through -`CostModel::size_params`. Default sizing is dispatched to the estimator modules through the existing -public candidate-construction entry point. Accuracy validation is independent of those proposals. The new -source-contract path centralizes HLL sizing and guarantee derivation in -Planner's accuracy module, overriding the generic proposal when the applicable -contract is supplied. It does not yet move every algorithm's sizing interface -out of CostModel. The design boundary is that parameter proposals never grant -accuracy authority to the cost model. +Default sizing is `realization::default_size_params`, which dispatches to the +estimator modules; a summary shared by several readers is sized with each +parameter's largest value among them. Accuracy validation is independent of +sizing: Stage 3 checks the guarantee derived from the committed parameters. +The design boundary is that sizing never grants accuracy authority to the +cost model. ## Composing guarantees through a DAG diff --git a/docs/design_docs/concepts/planner-pipeline.md b/docs/design_docs/concepts/planner-pipeline.md index c07d17775..9294ce0dc 100644 --- a/docs/design_docs/concepts/planner-pipeline.md +++ b/docs/design_docs/concepts/planner-pipeline.md @@ -1,26 +1,32 @@ # Planner pipeline -ASAPPlanner accepts a planning workload and produces `CandidateLogicalASAPDAGs`, a compact -representation of candidate Post-ASAP DAGs. Ranking and selection are operations -over that output, not mandatory stages of candidate search. +ASAPPlanner accepts a planning workload and the deployment's inputs, and +produces one selected physical plan for the whole workload. The +[planning stages design](../proposals/planner-layering.md) defines each +stage. - SQL / PromQL / MetricsQL queries - | parse and canonicalize + SQL / PromQL / MetricsQL queries + data workload + | Stage 0: query-language frontends v - Pre-ASAP IR: exact, language-independent query intent - | enumerate legal summary-aware alternatives + Logical DAG (Pre-ASAP IR): exact, language-independent query intent + | Stage 1: logical ASAP-aware optimization + | Pass 1: exact or a summary per computation + | Pass 2: share, split into panes, or reuse across computations v - CandidateLogicalASAPDAGs: candidate Post-ASAP DAGs - | - +--> inspect candidates, optionally using cost_sorted - +--> select and assemble logical DAGs + Logical ASAP candidates (Post-ASAP IR, untimed) + | Stage 2: physical ASAP-aware optimization + | materialization: not materialized, at ingestion time, + | or at query time (shared, or kept across evaluations) + v + Physical ASAP candidates (timed) + | Stage 3: plan selection + | accuracy, latency and deployment checks; cost model 1, 2 or 3 + v + Selected PhysicalASAPDAG, with every candidate's cost or rejection reason -Stage 2 materialization (#509) will decide per sub-DAG whether to materialize -and whether at ingestion or query time; until then every summary runs at query -time. All physical binding, deployment, and execution remain downstream -responsibilities. +Every stage keeps all its candidates; only Stage 3 compares them. The +deployment supplies its cost model, accuracy model and capabilities, and runs +the selected plan; physical deployment and execution stay downstream. The [input, output, and workflows](../architecture/input-output-workflow.md) document defines the public boundary and helper call order. - -The [Pre-ASAP IR](pre-asap-ir.md) captures what a query means without any summary implementation. The [Post-ASAP IR](post-asap-ir.md) represents the same intent using possible ASAP primitives. See the [architecture overview](../architecture/README.md) for the full component flow and boundary details. diff --git a/docs/design_docs/concepts/post-asap-ir.md b/docs/design_docs/concepts/post-asap-ir.md index 936c544ea..20abf5a8f 100644 --- a/docs/design_docs/concepts/post-asap-ir.md +++ b/docs/design_docs/concepts/post-asap-ir.md @@ -62,7 +62,7 @@ Reserved (migrated but unimplemented; schema derivation rejects them with The earlier draft listed `SummaryCreate` and `SummaryInsert`. These are not separate variants. `SummaryAgg` describes the state-producing -computation and its update input. Stage 2 materialization (#509) will decide +computation and its update input. Stage 2 (physical ASAP-aware optimization) decides whether and when that state is maintained. Physical binding and runtime execution implement the actual build and update operations. Not every summary family supports incremental maintenance. diff --git a/docs/design_docs/decisions/concat-unique-keys.md b/docs/design_docs/decisions/concat-unique-keys.md index 08b811b94..dad5ae692 100644 --- a/docs/design_docs/decisions/concat-unique-keys.md +++ b/docs/design_docs/decisions/concat-unique-keys.md @@ -1,6 +1,7 @@ # `Concat` and `unique_keys`: the discriminator override (issue #228) -> Status: accepted decision for the implementation described here. +> Status: accepted decision for the implementation described here. The +> legacy `replacement.rs` and `explanation.rs` it mentions have been removed. > Historical: `crates/types/src/pre_asap/query_expr.rs` was removed in #543; > `Concat` is now `NonASAPOp::Concat` (`crates/types/src/ir/non_asap.rs`). diff --git a/docs/design_docs/decisions/cse-cost-model.md b/docs/design_docs/decisions/cse-cost-model.md index 10bb5739a..2952e39ce 100644 --- a/docs/design_docs/decisions/cse-cost-model.md +++ b/docs/design_docs/decisions/cse-cost-model.md @@ -1,5 +1,11 @@ # Common sub-DAG sharing: rule-based vs. cost-based framework (issue #237) +> **Status: historical.** This document describes the legacy replacement +> search (strategies, `CandidateLogicalASAPDAGs` as a Rust type, cost-model +> ranking and `global_selection`), which has been removed. The planner now +> plans through the stages of the [planning stages design](../proposals/planner-layering.md). The +> document is kept as a record of the design. + > Status: accepted decision for the implementation described here. ## Context diff --git a/docs/design_docs/physical-planning-and-deployment.md b/docs/design_docs/physical-planning-and-deployment.md index e839b796e..dedec5971 100644 --- a/docs/design_docs/physical-planning-and-deployment.md +++ b/docs/design_docs/physical-planning-and-deployment.md @@ -28,10 +28,12 @@ implementation library. Deployment systems such as ASAPQuery and asap-fusion own deployment compilation and operation. Materialization is a planning contract associated with the logical DAG, not a separate computation IR. -> **Status:** Stage 2 materialization (#509) will decide per sub-DAG whether to -> materialize and whether at ingestion or query time. It is not implemented yet; -> until then the planner times every summary at query time. Sections 2 and 3 -> describe the intended contract. +> **Status:** Stage 2 (physical ASAP-aware optimization) decides, per summary +> build, whether it is not materialized, materialized at ingestion time, or +> materialized at query time (shared within one evaluation, or kept across +> evaluations), and Stage 3 selects among the results. Sections 2 and 3 +> describe the intended contract, which is wider than what Stage 2 offers +> today. The Logical Post-ASAP DAG is preceded by the Pre-ASAP DAG, the language-independent query semantics before summary selection. Both are diff --git a/docs/design_docs/proposals/asap-aware-mapping/analytical-resource-cost.md b/docs/design_docs/proposals/asap-aware-mapping/analytical-resource-cost.md index 0564acb38..15f1b54b2 100644 --- a/docs/design_docs/proposals/asap-aware-mapping/analytical-resource-cost.md +++ b/docs/design_docs/proposals/asap-aware-mapping/analytical-resource-cost.md @@ -2,7 +2,9 @@ > Status: implemented model with explicit support limits. The > [analytical estimator](../../../../crates/asap-aware-mapping/src/analytical_cost.rs) -> and physical-plan adapter implement supported evidenced comparisons. +> is what Stage 3 measures resource usage with. The physical-plan adapter and +> the legacy `CostModel` described below have been removed; Stage 3's cost +> models 1–3 replace them. > Unsupported operators, arrival modes and missing evidence remain unavailable; > proposed extensions are not implied by the implemented formulas. diff --git a/docs/design_docs/proposals/asap-aware-mapping/ddsketch-quantile-ratios.md b/docs/design_docs/proposals/asap-aware-mapping/ddsketch-quantile-ratios.md index 8bb452783..f35fa93d1 100644 --- a/docs/design_docs/proposals/asap-aware-mapping/ddsketch-quantile-ratios.md +++ b/docs/design_docs/proposals/asap-aware-mapping/ddsketch-quantile-ratios.md @@ -1,5 +1,10 @@ # DDSketch ratio certification (planner integration) +> **Status: historical.** The DDSketch ratio candidate and its +> `quantile_input_domain` evidence belonged to the legacy replacement search, +> which has been removed; Pass 1 does not offer a ratio candidate. The +> document is kept as a record of the bound and its preconditions. + Planner uses `asap_sketchlib` commit `da3635a80f8f854d47b772d49d5a9e5fb6927d8e` from [sketchlib PR #141](https://github.com/ProjectASAP/asap_sketchlib/pull/141) for mapping bounds and numerical integration tests. This pin does not update Collector or backend deployments. For valid relative-value operand bounds a and b, with b < 1 and a nonzero true denominator, the ratio bound is `(a+b)/(1-b)`. No independence assumption is needed. Sizing both DDSketch operands to `epsilon/(2+epsilon)` meets the ratio target, and compatible readouts can share one producer. diff --git a/docs/design_docs/proposals/asap-aware-mapping/maintained-populations.md b/docs/design_docs/proposals/asap-aware-mapping/maintained-populations.md index d12f833ff..78edc6f8d 100644 --- a/docs/design_docs/proposals/asap-aware-mapping/maintained-populations.md +++ b/docs/design_docs/proposals/asap-aware-mapping/maintained-populations.md @@ -50,8 +50,8 @@ column and grouping determine whether consumers refer to the same population. ## Rule: share one population across compatible readouts -**Realization:** `MaintainedPopulationStrategy`, an opt-in `ReplacementStrategy` -in [maintained_population.rs](../../../../crates/asap-aware-mapping/src/maintained_population.rs). +**Realization:** `MaintainedPopulationStrategy::candidate` builds the sub-DAG; +Pass 1 does not offer it yet. Code: [maintained_population.rs](../../../../crates/asap-aware-mapping/src/maintained_population.rs). **Target sub-DAGs:** diff --git a/docs/design_docs/proposals/asap-aware-mapping/summary-properties.md b/docs/design_docs/proposals/asap-aware-mapping/summary-properties.md index 888f103ea..08e27de74 100644 --- a/docs/design_docs/proposals/asap-aware-mapping/summary-properties.md +++ b/docs/design_docs/proposals/asap-aware-mapping/summary-properties.md @@ -110,7 +110,7 @@ candidate generation -> guarantee propagation (AccuracyModel::propagate) -> AccuracyTarget satisfaction (AccuracyModel::satisfies) -> legal candidates only (illegal ones -> TargetSubDAGCandidates::rejected) - -> cost ranking / global selection (CostModel) + -> plan selection (Stage 3 cost model) ``` Every finalized post-ASAP value carries a machine-readable @@ -144,5 +144,4 @@ registered same-metric additive, relative, and Lipschitz rules and exact sum/max/min over approximate inputs. It uses union-bound probabilities without assuming independence, preserves unknown statistics as unknown, and rejects unsupported composition instead of treating the child as exact. -An `AccuracyBudgetAllocator` can propose resized layers; the `CostModel` -ranks only candidates that satisfy the accuracy requirement. +Plan selection prices only candidates that satisfy the accuracy requirement. diff --git a/docs/design_docs/proposals/asap-primitive-schema.md b/docs/design_docs/proposals/asap-primitive-schema.md index c83ce0066..0031c0aa3 100644 --- a/docs/design_docs/proposals/asap-primitive-schema.md +++ b/docs/design_docs/proposals/asap-primitive-schema.md @@ -93,7 +93,7 @@ Based on our requirement, each field should contain the following information. | `FrequencyL2`, `FrequencyEntropy` | `Sketch`: `UnivMon` | `SummaryEstimate(FrequencyL2 \| FrequencyEntropy)` | | `Sum`, `Count`, `Min`, `Max`, `Rate`, `IRate`, `Increase` (exact) | `ExactAggregate(ExactKind, …)` | `FinalizeExactAccumulator` | - The sketch candidates are `summary_candidates(intent)` in `crates/asap-aware-mapping/src/replacement.rs`; exact accumulators come from `exact_realization` there (`Count`) and from `function_rules.rs` (the others); the evaluations are `SketchStatistic` ([§6.5](#65-update-input-and-evaluations-post_asapsketchrs-post_asapmaintained_populationrs)). + The sketch candidates are `summary_candidates(intent)` in `crates/asap-aware-mapping/src/realization.rs`; exact accumulators come from Pass 1's `ExactAccumulator` candidates in `per_computation_candidates.rs`; the evaluations are `SketchStatistic` ([§6.5](#65-update-input-and-evaluations-post_asapsketchrs-post_asapmaintained_populationrs)). - **Time window aggregation intents**: whether states built over smaller windows can answer a larger one. This depends on how the family combines states: - **Merge** (`SummaryMerge`, §5.6): states over disjoint panes combine into the state of their union, e.g. two 1-minute KLL states answer a 2-minute quantile. Requires a mergeable family. diff --git a/docs/design_docs/proposals/asapquery-rule-coverage.md b/docs/design_docs/proposals/asapquery-rule-coverage.md index 9ba999292..beac2f5df 100644 --- a/docs/design_docs/proposals/asapquery-rule-coverage.md +++ b/docs/design_docs/proposals/asapquery-rule-coverage.md @@ -1,5 +1,11 @@ # ASAPQuery rule coverage +> **Status: historical.** This document describes the legacy replacement +> search (strategies, `CandidateLogicalASAPDAGs` as a Rust type, cost-model +> ranking and `global_selection`), which has been removed. The planner now +> plans through the stages of the [planning stages design](planner-layering.md). The +> document is kept as a record of the design. + > Status: proposal and coverage analysis. Check the implementation and tests before relying on it as a supported-behavior catalog. This document compares ASAPPlanner's rule system with the Rust planner in diff --git a/docs/develop_docs/README.md b/docs/develop_docs/README.md index a49707790..65c63f563 100644 --- a/docs/develop_docs/README.md +++ b/docs/develop_docs/README.md @@ -10,7 +10,6 @@ formats, evidence, and verification workflows. - [Pre-ASAP IR reference](pre-asap-ir.md) - [Post-ASAP IR](../design_docs/concepts/post-asap-ir.md) - [Accuracy implementation companion](end-to-end-accuracy-guarantees.md) -- [DDSketch ratio certification](../design_docs/proposals/asap-aware-mapping/ddsketch-quantile-ratios.md) - [Offline sketch evidence](offline-sketch-evidence.md) - [Metrics-observability corpora](metrics-observability-corpora.md) - [Physical handoff cost references](physical-handoff-costs.md), [storage operations](storage-operation-costs.md) @@ -21,3 +20,4 @@ Historical records of removed APIs: - [Planner vocabulary migration](planner-vocabulary-migration.md) - [Target candidate API migration](target-candidate-api-migration.md) - [Replacement explanations](replacement-explanations.md) +- [DDSketch ratio certification](../design_docs/proposals/asap-aware-mapping/ddsketch-quantile-ratios.md) diff --git a/docs/develop_docs/library-api.md b/docs/develop_docs/library-api.md index 32f254c72..6e7f6495f 100644 --- a/docs/develop_docs/library-api.md +++ b/docs/develop_docs/library-api.md @@ -176,7 +176,7 @@ asap_aware_mapping::run_stage1_to_stage3( | `CostModel::ProvisionedCapacity(ResourceUnitCost)` | Cost model 2: unit cost × peak memory, peak CPU cores and disk; the cheapest valid candidate wins | | `CostModel::ClusterCapacity(ClusterLimits)` | Cost model 3: memory, disk and optional CPU cores are hard limits; `ClusterObjective::{LowestLatency, LeastMemory, ParetoFront}` chooses among the candidates that fit | | `PlanOutput.plans` | One `QueryPlan { entry_index, root }` per query, in `QueryWorkload::entries()` order | -| `PlanOutput.selection` | Stage 3's `Selection`: `selected`, `costs` (each candidate's `CandidateCost` with its `ResourceUsage` and `cost_per_second`, `None` under cost model 3), `accepted`, `rejected` (with reasons) and `front` | +| `PlanOutput.selection` | `Some` with Stage 3's `Selection` under the default pass: `selected`, `costs` (each candidate's `CandidateCost` with its `ResourceUsage` and `cost_per_second`, `None` under cost model 3), `accepted`, `rejected` (with reasons) and `front` | | `PlanOutput::execution_timed_dag()` | The planned queries as one `PhysicalASAPDAG`, timed with the default `MaterializationAssignment` (see [Export](#export)) | Each candidate's `ResourceUsage` holds its usage without prices: From f50431fb4a43d93b3f77e451c746da059e14d852 Mon Sep 17 00:00:00 2001 From: zzylol <50204836+zzylol@users.noreply.github.com> Date: Sat, 17 Oct 2026 20:39:15 +0000 Subject: [PATCH 10/10] docs: keep the design doc's own wording; current pins and tenses planner-layering.md and the acceptance docs keep the original overview diagram wording and the code font on lookback and as_of. library-api.md pins a current revision and lists asap-planner and tokio. The analytical resource cost proposal and the planner-runtime contract use the past tense for removed code, and the Example 4 acceptance doc no longer claims B2 >= B1, B3 under every model. --- .../architecture/planner-runtime-contract.md | 5 +++-- .../analytical-resource-cost.md | 10 ++++----- .../planner-layering-example1-acceptance.md | 2 +- .../planner-layering-example2-acceptance.md | 2 +- .../planner-layering-example3-acceptance.md | 8 +++---- .../planner-layering-example4-acceptance.md | 12 ++++++---- .../design_docs/proposals/planner-layering.md | 22 +++++++++---------- docs/develop_docs/library-api.md | 13 +++++++---- 8 files changed, 42 insertions(+), 32 deletions(-) diff --git a/docs/design_docs/architecture/planner-runtime-contract.md b/docs/design_docs/architecture/planner-runtime-contract.md index 51bd9bd98..c9079b376 100644 --- a/docs/design_docs/architecture/planner-runtime-contract.md +++ b/docs/design_docs/architecture/planner-runtime-contract.md @@ -69,7 +69,8 @@ activation, and rollback are not an end-to-end Planner protocol. scan selection, input/output edges, operation counts, update and bootstrap fanout, retained state, CPU, memory, I/O, and accuracy facts. 4. ASAPPlanner keeps constructible candidates with missing evidence visible - in `CandidateLogicalASAPDAGs` but does not certify unknown accuracy. + among its Stage 1 candidates, and Stage 3 rejects them with the reason; it + does not certify unknown accuracy. Materialization compares supported alternatives over the same workload horizon. Missing or incomparable costs do not establish that maintaining a summary beats raw recomputation; structural scores and @@ -126,7 +127,7 @@ in one cost formula. execution. - A selected realization framework is a contract, not executor code. - Physical capabilities and evidence constrain deployment choices, not every - logical candidate's presence in `CandidateLogicalASAPDAGs`. Known unsupported capabilities + logical candidate's presence among the Stage 1 candidates. Known unsupported capabilities and unknown algorithms cannot become deployable alternatives. - Complete physical alternatives need identity and comparable evidence for cost-based deployment decisions. Stale evidence cannot certify or cost a diff --git a/docs/design_docs/proposals/asap-aware-mapping/analytical-resource-cost.md b/docs/design_docs/proposals/asap-aware-mapping/analytical-resource-cost.md index 15f1b54b2..e53862bda 100644 --- a/docs/design_docs/proposals/asap-aware-mapping/analytical-resource-cost.md +++ b/docs/design_docs/proposals/asap-aware-mapping/analytical-resource-cost.md @@ -19,8 +19,8 @@ plans. These are separate concerns: evidence to estimate CPU work, peak memory, and source/disk I/O. The physical-resource estimator itself is independent of the arrival mode. -One planner adapter currently lowers work into it: `PhysicalPlanCostModel` -compares complete at-rest plans. Continuously-ingesting, `Mixed` and `Unknown` +One planner adapter lowered work into it before it was removed: +`PhysicalPlanCostModel` compared complete at-rest plans. Continuously-ingesting, `Mixed` and `Unknown` comparisons are unavailable; costing summary maintenance over arriving data belongs to Stage 2 materialization (#509). Evidence from one arrival mode must not be reused for another. @@ -797,8 +797,8 @@ windows, nested aggregates, Top-K, and shared sub-DAGs therefore use the same estimation path. The generic query lowerer recursively maps the supported raw query operators into that representation. -`PhysicalPlanCostModel` is the final-selection adapter. For every -candidate it obtains one canonical `ComparisonScope`, recursively lowers the +`PhysicalPlanCostModel` was the final-selection adapter (now removed). For every +candidate it obtained one canonical `ComparisonScope`, recursively lowers the actual raw target, and then lowers a logical rewrite or requests the fully bound physical DAG for a `SummaryExpr` candidate. The deployment implements `PlannerPhysicalPlanProvider`: query-node evidence is consumed atomically by @@ -911,7 +911,7 @@ The intended end-to-end selection pipeline is: 6. applies calibration and ranks candidates by ascending cost. The query lowerer and physical estimator cover the supported raw-query shapes -listed above. `PhysicalPlanCostModel` executes this pipeline for every +listed above. `PhysicalPlanCostModel`, now removed, executed this pipeline for every candidate supplied to `CandidateLogicalASAPDAGs::global_selection`. Logical rewrites are lowered recursively. Summary candidates participate only after the deployment has bound their complete `SummaryExpr` DAG; there is no optimistic generic diff --git a/docs/design_docs/proposals/planner-layering-example1-acceptance.md b/docs/design_docs/proposals/planner-layering-example1-acceptance.md index 50153e5d9..bf9b97a9b 100644 --- a/docs/design_docs/proposals/planner-layering-example1-acceptance.md +++ b/docs/design_docs/proposals/planner-layering-example1-acceptance.md @@ -31,7 +31,7 @@ Shared data workload: `arrival` = `Ingesting { rows_per_second }`, about 66,667 samples/s; `sources["http_requests_total"]` has `series_count` = 1,000,000, `sample_interval` = 15 s and `metric_type` = counter. -| Query | Repeats | lookback | as_of | Accuracy | Latency | +| Query | Repeats | `lookback` | `as_of` | Accuracy | Latency | |---|---|---|---|---|---| | Q1 `sum by (job) (rate(http_requests_total[1m]))` | 10 s | 1 m | evaluation time | exact | none | | Q2 `topk by (job) (10, sum_over_time(http_requests_total[1m]))` | 10 s | 1 m | evaluation time | ε = 0.01, δ = 0.001 | ≤ 100 ms | diff --git a/docs/design_docs/proposals/planner-layering-example2-acceptance.md b/docs/design_docs/proposals/planner-layering-example2-acceptance.md index f69473929..ff2594c51 100644 --- a/docs/design_docs/proposals/planner-layering-example2-acceptance.md +++ b/docs/design_docs/proposals/planner-layering-example2-acceptance.md @@ -14,7 +14,7 @@ needs a missing feature is `#[ignore]`d with the PR that delivers it. ## Workload -Three SQL dashboard panels over `flows`, every 10 s, lookback 1 m, as_of +Three SQL dashboard panels over `flows`, every 10 s, `lookback` 1 m, `as_of` evaluation time, no latency requirement. | Query | Computation | SQL (abridged) | Accuracy | diff --git a/docs/design_docs/proposals/planner-layering-example3-acceptance.md b/docs/design_docs/proposals/planner-layering-example3-acceptance.md index 41ae5c459..b0b30f810 100644 --- a/docs/design_docs/proposals/planner-layering-example3-acceptance.md +++ b/docs/design_docs/proposals/planner-layering-example3-acceptance.md @@ -25,7 +25,7 @@ Shared data workload, except Pattern A's `arrival` is `Mixed`. **Pattern A.** One `query_batch`, `invocations: 1` at T, `ad_hoc`, ε = 0.005, δ = 0.01. Tests use T = 2026-01-01T00:00:00Z and a 365-day year. -| Query | lookback | as_of | +| Query | `lookback` | `as_of` | |---|---|---| | q1 `quantile_over_time(0.99, latency_ms[5y])` | 5 y | T | | q2 `quantile_over_time(0.99, latency_ms[1y])` | 1 y | T | @@ -34,7 +34,7 @@ Shared data workload, except Pattern A's `arrival` is `Mixed`. | q5 `quantile_over_time(0.99, latency_ms[3y] offset 2y)` | 3 y | T − 2 y | **Pattern B.** `quantile_over_time(0.99, latency_ms[5m])`, every 1 min, -lookback 5 m, as_of evaluation time, ε = 0.01, δ = 0.01, ≤ 200 ms. +`lookback` 5 m, `as_of` evaluation time, ε = 0.01, δ = 0.01, ≤ 200 ms. ## Expected candidates per stage @@ -106,8 +106,8 @@ and the `SummaryEstimate`. them. 5. **EH accuracy.** A boundary inside an EH bucket is approximate. Whether the accuracy model charges that against ε = 0.005 is not stated; not tested. -6. **as_of and `offset`.** The table gives as_of = T − 1 y and the query - text has `offset 1y`. We read as_of as the window's end (the result of +6. **`as_of` and `offset`.** The table gives `as_of` = T − 1 y and the query + text has `offset 1y`. We read `as_of` as the window's end (the result of the offset at T), not as a second shift. The frontend lowers the offset from the text. 7. **Plan selection statistics.** A scan reads the window above it, through diff --git a/docs/design_docs/proposals/planner-layering-example4-acceptance.md b/docs/design_docs/proposals/planner-layering-example4-acceptance.md index 8d1b135e3..5ecacc468 100644 --- a/docs/design_docs/proposals/planner-layering-example4-acceptance.md +++ b/docs/design_docs/proposals/planner-layering-example4-acceptance.md @@ -80,7 +80,7 @@ it. |---|---|---| | Every node upstream of an ingestion-time node also runs at ingestion time (all variants) | `stage2_physical_asap_optimization_ingestion_time_upstream_is_ingestion_time` | passes: B1 and the monthly variant's maintained panes | | With data at rest, no node runs at ingestion time | `stage2_physical_asap_optimization_at_rest_runs_nothing_at_ingestion_time` | passes | -| A materialized output covers its consumers: retention ≥ max over its readers of lookback + offset | `stage2_physical_asap_optimization_materialized_output_covers_its_consumers` | passes: B1 keeps 6 min ≥ 5 min | +| A materialized output covers its consumers: retention ≥ max over its readers of `lookback` + offset | `stage2_physical_asap_optimization_materialized_output_covers_its_consumers` | passes: B1 keeps 6 min ≥ 5 min | ### Pattern A @@ -109,8 +109,12 @@ it. ### Selection direction, relative to the cost model -* A3 > A1 and B2 ≥ B1, B3 hold under any model that charges each build: A3 - and B2 do strictly more of the same work. +* A3 > A1 holds under any model that charges each build: A3 does strictly + more of the same work. B2 also rebuilds more per evaluation than B1 and B3, + but it keeps nothing between evaluations, so B2 ≥ B1, B3 holds only under a + model that charges rebuilds more than kept state. Cost model 1 does not: + B2 costs 2.183 per second, B3 8,884.5 and B1 13,326.5 (see the table + above). * A1 ≤ A2 as given, and B1 ≤ B3, depend on the cost model (cost model 1, 2 or 3). The doc's reasons are that A2 maintains years of history for one batch, and that B3 adds the @@ -140,7 +144,7 @@ it. repeating entry whose window ends at each run, known at T. 4. **Retention unit.** "Kept 5 min" is read as the event-time span the stored state covers. For A1, which lives only for the batch, that is 5 y of - history. The test requires retention ≥ each reader's lookback + offset. + history. The test requires retention ≥ each reader's `lookback` + offset. 5. **Backfill.** A2 backfills old data once. Whether that one-time cost is charged to the plan, and how it is amortized, is not stated; not tested. 6. **Pruning A2 at rest.** "A2 is not generated" is read as physical diff --git a/docs/design_docs/proposals/planner-layering.md b/docs/design_docs/proposals/planner-layering.md index 11c2cb643..6d94576a7 100644 --- a/docs/design_docs/proposals/planner-layering.md +++ b/docs/design_docs/proposals/planner-layering.md @@ -162,7 +162,7 @@ flowchart TB subgraph INPUTS["Inputs"] direction LR QW["Query workload
(PromQL / SQL / MetricsQL,
query recurrence,
accuracy requirements,
latency requirements)"]:::input - DW["+ Data workload
(streaming vs. data at rest,
per-source series counts,
sample intervals)"]:::input + DW["+ Data workload
(streaming vs. data at rest,
data distribution,
cardinality)"]:::input DI["+ Deployment inputs
(cost model,
accuracy model,
deployment capabilities)"]:::input end @@ -920,12 +920,12 @@ To see examples in DAG Viewer, following the instructions [here](TODO: write an Each example's workload is shown as tables. Field names in code font are the fields of -[`workload.rs`](https://github.com/ProjectASAP/ASAPPlanner/blob/main/crates/types/src/workload.rs). -The lookback and as_of columns are not fields: the query text states them (its range and -offset). Each query reads the event-time window [as_of − lookback, as_of]. -as_of is the window's end; lookback is its length. An -as_of of "evaluation time" means no offset: the window ends whenever the -query runs, so it moves forward with each evaluation. A fixed as_of, such as +[`workload.rs`](https://github.com/ProjectASAP/ASAPPlanner/blob/main/crates/types/src/workload.rs), +except `lookback` and `as_of`, which the query text states (its range and +offset). Each query reads the event-time window [`as_of` − `lookback`, `as_of`]. +`as_of` is the window's end; `lookback` is its length. An +`as_of` of "evaluation time" means no offset: the window ends whenever the +query runs, so it moves forward with each evaluation. A fixed `as_of`, such as T − 1 y, pins the window to a historical interval. An approximate accuracy target (`AccuracyTarget::max_error(ε).failure_probability(δ)`) means the answer's error is at most ε with probability at least 1 − δ. @@ -959,7 +959,7 @@ With 1,000,000 series each sampled every 15 s, the ingestion rate is **Query workload.** Two PromQL dashboard panels over the last minute. The first needs an exact total; the second tolerates error. -| Query | Repeats | lookback | as_of | Accuracy requirement | Latency requirement | +| Query | Repeats | `lookback` | `as_of` | Accuracy requirement | Latency requirement | |---|---|---|---|---|---| | Q1: `sum by (job) (rate(http_requests_total[1m]))` | every 10 s | 1 m | evaluation time | exact | none | | Q2: `topk by (job) (10, sum_over_time(http_requests_total[1m]))` | every 10 s | 1 m | evaluation time | ε = 0.01, δ = 0.001 | ≤ 100 ms | @@ -1220,7 +1220,7 @@ The SQL frontend recognizes Q2 as `Entropy` and Q3 as `L2` when Q3 casts its counts to DOUBLE (`CAST(c AS DOUBLE) * CAST(c AS DOUBLE)`): the Int64 `c * c` above can overflow, so only the Float64 product is recognized. -| Query | Computation | Repeats | lookback | as_of | Accuracy requirement | +| Query | Computation | Repeats | `lookback` | `as_of` | Accuracy requirement | |---|---|---|---|---|---| | Q1 | `Distinct(src_ip)` | every 10 s | 1 m | evaluation time | ε = 0.02, δ = 0.01 | | Q2 | `Entropy(src_ip)` | every 10 s | 1 m | evaluation time | ε = 0.05, δ = 0.01 | @@ -1338,7 +1338,7 @@ all executed together at time T. | `predictability` | `ad_hoc` | | Accuracy requirement | ε = 0.005, δ = 0.01 | -| Query | lookback | as_of | +| Query | `lookback` | `as_of` | |---|---|---| | `quantile_over_time(0.99, latency_ms[5y])` | 5 y | T | | `quantile_over_time(0.99, latency_ms[1y])` | 1 y | T | @@ -1401,7 +1401,7 @@ flowchart LR **Pattern B: one repeating query with overlapping windows.** A real-time p99 panel over the last 5 min, refreshed every minute. -| Query | Repeats | lookback | as_of | Accuracy requirement | Latency requirement | +| Query | Repeats | `lookback` | `as_of` | Accuracy requirement | Latency requirement | |---|---|---|---|---|---| | `quantile_over_time(0.99, latency_ms[5m])` | every 1 min | 5 m | evaluation time | ε = 0.01, δ = 0.01 | ≤ 200 ms | diff --git a/docs/develop_docs/library-api.md b/docs/develop_docs/library-api.md index 6e7f6495f..9a8cd9033 100644 --- a/docs/develop_docs/library-api.md +++ b/docs/develop_docs/library-api.md @@ -26,13 +26,18 @@ artifact, while preserving the checks required by its intended consumer. Inside this workspace, depend on the frontend you need, `asap-aware-mapping`, and `asap-types`, or on `asap-planner`, which wraps every frontend and the planning stages. External users can use Git dependencies pinned to a compatible -revision; use the same revision across these crates. For the example below: +revision; use the same revision across these crates, and pick a revision at +least as new as the APIs you call (this guide describes `main` at `775e21ee`). +For the examples below: ```toml [dependencies] -asap-frontend-promql = { git = "https://github.com/ProjectASAP/ASAPPlanner", rev = "e7fdb2492c42c9f5b34760706a5162aa586d3025" } -asap-aware-mapping = { git = "https://github.com/ProjectASAP/ASAPPlanner", rev = "e7fdb2492c42c9f5b34760706a5162aa586d3025" } -asap-types = { git = "https://github.com/ProjectASAP/ASAPPlanner", rev = "e7fdb2492c42c9f5b34760706a5162aa586d3025" } +asap-planner = { git = "https://github.com/ProjectASAP/ASAPPlanner", rev = "775e21ee38879631fe9752c105d50bea890d3d4a" } +asap-frontend-promql = { git = "https://github.com/ProjectASAP/ASAPPlanner", rev = "775e21ee38879631fe9752c105d50bea890d3d4a" } +asap-aware-mapping = { git = "https://github.com/ProjectASAP/ASAPPlanner", rev = "775e21ee38879631fe9752c105d50bea890d3d4a" } +asap-types = { git = "https://github.com/ProjectASAP/ASAPPlanner", rev = "775e21ee38879631fe9752c105d50bea890d3d4a" } +# For the async e2e_plan example: +tokio = { version = "1", features = ["rt", "macros", "rt-multi-thread"] } ``` ## Lower a query into Pre-ASAP IR