refactor!: plan with ASAPPlanner main's stage pipeline and drop the old execution stack

[zzylol]

Stack 1/5 for executing the plan ASAPPlanner selects (#509's physical ASAP DAG). Next: #819.

Why

The backend selected its own plans, with its own ERP, SDS, precompute and query engines, and each part pinned a different Planner revision, so none of it ran the plan the Planner selects. The Planner now selects plans end to end on its main; the backend should only supply deployment inputs and execute the result.

What

• Control plane: plans a PromQL PlanningWorkload with the Planner's Stages 0–3 under the backend's deployment inputs and writes plan.json (InstalledPlan): the selected PhysicalASAPDAG plus the query each root answers.
• Data plane: loads the plan, accepts Prometheus Remote Write, and forwards every query to Prometheus. Later PRs in the stack answer planned queries from the plan.
• Planner pin: ASAPPlanner main 58999df2 for all five crates the backend uses: asap-types (as planner-types), asap-frontend-promql, asap-logical-asap-optimization, asap-plan-selection, asap-executor. scripts/sync-asapplanner-main.sh and its test now update five pins.
• Examples: examples/planner-layering-example1.json, test(promql): enforce range step parity in production E2E #509 Example 1 in the current workload format (DataWorkload { arrival, sources }, AccuracyTarget, QueryRequirements { accuracy, max_latency_ms }), generated from the Planner's own example definitions.
• Deleted: everything else in the old backend (−176k lines), including crates/, docs/, demos/, tools/ and the Python package. The table in Deleted capabilities lists each piece and its replacement; the ones not replaced yet are tracked in Capabilities removed with the old backend that are not replaced yet (#818) #823.

Supersedes #788–#812, #791 and #779.

Deleted capabilities

Every piece of the old backend this PR deletes, and what replaces it. "Deferred" rows have no replacement yet and are tracked in #823.

Deleted	What it did	Replaced by / tracked in
control_plane/src/planner_selection.rs, workload_cost.rs, physical/cost_model.rs, physical/tco.rs	Backend-side plan selection and workload cost	ASAPPlanner Stage 3 (select_plan, cost models 1–3)
control_plane/src/physical/erp.rs, tools/empirical-bench	ERP and measured-cost evidence	ASAPPlanner measured_operation_costs, imported from sketch-bench
control_plane/src/accuracy.rs, epsilon_alloc.rs	Accuracy checks and error-budget allocation	ASAPPlanner Stage 1 accuracy rules and Stage 3 AccuracyModel
control_plane/src/physical/materialization_candidates.rs, pane_reuse.rs, maintained_population.rs	Materialization choice, pane reuse, maintained populations	ASAPPlanner Stage 2 materialization and Pass 2 sharing
control_plane/src/physical/sketch_catalog.rs, summary_catalog.rs, runtime_capability.rs, control_plane/sketch_capabilities.yml, sketch_params_default.yml	Which sketches exist and their default parameters	ASAPPlanner sizing; the backend's DeploymentCapabilities
control_plane/src/query_parser, physical/compiler.rs, lower.rs, matcher.rs, deployment_expr.rs, realization.rs, executable_binding.rs, query_plan.rs, query_plan/query_time.rs	Parsing, lowering and compiling plans	ASAPPlanner frontends and executor; the control plane installs plan.json
control_plane/src/physical/wire.rs, emit/backend_wire.rs, backend_client.rs, publication.rs, plan_dot.rs, status.rs, types.rs, workload.rs, control_plane/proto	Plan transport between control and data plane	plan.json (InstalledPlan), loaded by the data plane
control_plane/src/clickhouse.rs, query_plan/clickhouse_exact.rs	ClickHouse planning	Deferred, #823
control_plane/src/opamp, emit/monitor.rs, physical/online.rs, runtime_samples.rs, metrics_exposer.rs	OpAMP control, monitoring, online re-planning	Deferred, #823 (re-planning is out of scope; runtime feedback is ASAPPlanner #250)
crates/asap_types	A copy of the Planner's types	ASAPPlanner asap-types, pinned
crates/asap_otel_proto, data_plane/src/drivers/ingest/otel.rs	OTel ingestion with a sketch-metric proto	Deferred, #823
data_plane/src/drivers/ingest/prometheus_remote_write.rs, series_resolver.rs	Remote Write ingestion and series routing	Remote Write in this PR; samples selected by the plan's Scan (#820)
data_plane/src/drivers/query/adapters/prometheus_http.rs, query/fallback/prometheus.rs, query/servers	Prometheus query API and fallback	Prometheus HTTP API and forwarding (#821)
data_plane/src/drivers/query/adapters/victoriametrics_http.rs, query/fallback/victoriametrics.rs	VictoriaMetrics adapter	Deferred, #823
data_plane/src/precompute_engine (window manager, ingest handler, maintenance runtime, output sink, raw DAG, group keys, metrics)	Building stored summaries	The precompute engine (#820)
data_plane/src/precompute_engine/revisions.rs, frame_lineage.rs	Late-sample revisions	Deferred, #823 (late samples are dropped and counted)
data_plane/src/precompute_engine/partitioning.rs, series_router.rs, subdag_scheduler.rs, worker.rs	Multi-worker precompute	Deferred, #823
data_plane/src/precompute_engine/erp_observer.rs	ERP observations at runtime	ASAPPlanner measured costs (offline, sketch-bench)
data_plane/src/query_engines/asap_query_engine	Answering PromQL from summaries	The query engine (#821) running the plan's roots
data_plane/src/query_engines/routing (capability matching, freshness probe, storage routing)	Choosing per query between summaries and raw data	ASAPPlanner (the plan says what each query reads)
data_plane/src/query_engines/asap_clickhouse_query_engine	SQL acceleration over ClickHouse	Deferred, #823
data_plane/src/storage_engines/sketch_db/sds.rs, index, lifecycle, current_series.rs, data, query	Stored summaries, index and lifecycle	SDS and SummaryStore (#819)
data_plane/src/storage_engines/sketch_db/persistence	S3 / Gorilla object-store persistence	Deferred, #823
data_plane/src/storage_engines/sketch_db/backfill	Backfill from ClickHouse or Prometheus history	Deferred, #823
data_plane/src/storage_engines/sketch_db/accuracy.rs, metrics.rs	Runtime accuracy and storage metrics	ASAPPlanner guarantees; experiment metrics come with the benefit harness
data_plane/src/update_sampling	Update sampling	Deferred, #823 (not planned)
data_plane/src/bin, benches, examples, tests (old)	Old binaries, benchmarks, examples and tests	New data_plane binary and tests (#819–#822)
control_plane/Dockerfile, data_plane/Dockerfile, data_plane/*.yml.j2, demos/asapquery	Container images, compose files, demo	Deferred, #823 (with the benefit harness)
tools/autosketch-comparison, tools/erp-figure1	AutoSketch and ERP experiments	sketch-bench (where the AutoSketch evaluation lives)
tools/o11y-execution, tools/shared-workload, tools/run-offline-planner-replay.py, tools/test_shared_panes.py	Old acceptance and workload replay tooling	The correctness e2e test (#822) and the planned benefit harness
tools/clickhouse-benefit, tools/o11y-sql-main-eval	SQL evaluation	Deferred, #823
tools/cluster-data-exporter	Google/Alibaba trace exporter	Deferred, #823
docs/ (getting started, guides, design, developer, evaluation, user guide)	Docs of the old architecture	The design doc for this stack; new docs follow with the stack
pyproject.toml and the Python package	Python tooling	Deferred, #823 (not planned)

How

plan_workload lowers the workload with asap_frontend_promql::lower_promql_query_workload, adds each series' identity to PromQL roots (as the Planner's Stage1ToStage3Pass does), and runs asap_plan_selection::run_stage1_to_stage3 with PlanningModels::builtin() plus the backend's DeploymentCapabilities and PlanSelectionSettings. The backend never chooses: it takes Stage 3's Selection::plan (the selected candidate's timed roots), expands each PaneChain into its panes with expand_pane_chains (the executor's compiler rejects unexpanded chains), and exports it with compile_physical_asap_workload.

run_stage1_to_stage3 is used instead of the asap-planner facade's e2e_plan because e2e_plan returns PlanOutput, whose DAG export resets every node to query time, and because the facade pulls in the SQL frontend (DataFusion), which a PromQL-only backend does not need.

The backend declares query_time_results_retained: false: the data plane does not implement MatAtQueryKeptAcrossEvaluations, so the Planner never offers it.

Key code interfaces added / modified

// control_plane/src/lib.rs
pub struct InstalledPlan {
    /// PromQL text of each root of `dag`, in root order.
    pub queries: Vec<String>,
    /// The selected candidate's DAG, with every `PaneChain` expanded into
    /// the panes the executor runs.
    pub dag: PhysicalASAPDAGDocument,
}
impl InstalledPlan {
    /// The DAG validates and has one root per query.
    pub fn validate(&self) -> Result<(), PlanError>;
    /// Reads and validates a plan written by `save`.
    pub fn load(path: &Path) -> Result<Self, PlanError>;
    /// Writes the plan as JSON.
    pub fn save(&self, path: &Path) -> Result<(), PlanError>;
}

/// What the backend gives the Planner about its deployment.
pub struct DeploymentInputs {
    /// What the data plane can execute and keep (Stages 1–3).
    pub deployment_capabilities: DeploymentCapabilities,
    /// How Stage 3 prices candidates: cost model, resource speed, and any
    /// measured operation costs.
    pub plan_selection: PlanSelectionSettings,
}
/// `Default`: `backend_capabilities()` with `PlanSelectionSettings::default()`
/// (cost model 1, illustrative unit costs).
impl Default for DeploymentInputs;

/// Stages 0–3; returns the plan Stage 3 selected.
pub fn plan_workload(workload: &PlanningWorkload, inputs: &DeploymentInputs)
    -> Result<InstalledPlan, PlanError>;

/// Every summary the Planner can build (`supported_summaries: None`), raw
/// samples kept in Prometheus, no query-time result kept across evaluations.
pub fn backend_capabilities() -> DeploymentCapabilities;

pub enum PlanError {
    /// The workload is not PromQL.
    UnsupportedLanguage(QueryLanguage),
    /// `PlanningWorkload::validate` failed.
    Workload(String),
    /// Query `i` lowers to a standalone scalar, which the data plane cannot answer.
    ScalarRoot(usize),
    /// The PromQL frontend failed.
    Lowering(String),
    /// Stage 1 or Stage 3 failed (including no valid candidate).
    Planning(String),
    /// Pane-chain expansion, export or document validation failed.
    InvalidDag(String),
    /// The DAG's root count differs from the query count.
    RootCount { roots: usize, queries: usize },
    /// Reading or writing `plan.json` failed.
    File { path: String, message: String },
}

control_plane CLI: --workload, --deployment-capabilities (Planner DeploymentCapabilities JSON), --measured-operation-costs (Planner MeasuredOperationCost list JSON), --out.

Before this PR

The backend selected its own plan with its own cost model and engines, on Planner revisions that differed per crate.

After this PR

control_plane --workload examples/planner-layering-example1.json --out plan.json
wrote plan.json (2 queries, 13 nodes)

The plan is the one the Planner selected (for Example 1, P1: both queries exact at query time). The data plane installs it and forwards queries with x-asap-execution: forwarded.

Usage

# Workload: #509 Example 1 (two dashboard panels over http_requests_total).
WORKLOAD=examples/planner-layering-example1.json
# Where the control plane writes the selected plan.
PLAN=plan.json
# Plan with the default deployment inputs.
cargo run -p control_plane -- --workload "$WORKLOAD" --out "$PLAN"
# -> wrote plan.json (2 queries, 13 nodes)

# Prometheus that answers forwarded queries.
PROMETHEUS=http://localhost:9090
# Serve Remote Write (/api/v1/write) and the query API on 8088.
cargo run -p data_plane -- --plan "$PLAN" --prometheus-url "$PROMETHEUS" --listen 127.0.0.1:8088 &
# Any query is forwarded unchanged; the response says so.
curl -s -D - -o /dev/null 'http://127.0.0.1:8088/api/v1/query?query=up' | grep x-asap
# -> x-asap-execution: forwarded

Verification

• cargo test -p control_plane -p data_plane (control plane 3 tests: plans Example 1 and round-trips plan.json, rejects SQL, rejects a plan without a query per root; data plane: Remote Write decoding and forwarding).
• cargo fmt --all -- --check; cargo clippy --workspace --all-targets -- -D warnings; python3 -m unittest discover -s scripts/ci -p 'test_*.py'.
• Usage run on this PR's tree: the plan above, and x-asap-execution: forwarded with the upstream's status passed through, checked against a stand-in HTTP server.