Skip to content

feat(types): three error metrics, a shared AccuracySpec and metric-aware satisfies (#690) - #708

Merged
zzylol merged 4 commits into
mainfrom
feat/accuracy-spec-690
Oct 15, 2026
Merged

zzylol merged 4 commits into
mainfrom
feat/accuracy-spec-690

Conversation

@zzylol

@zzylol zzylol commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Closes #690. Supersedes #627: its separate answers check is folded into the metric-aware satisfies.

Why

AccuracyTarget (Exact | Epsilon | EpsilonDelta) had no error metric, and satisfies compared only the number. Each summary states ε in its own form: KLL as a rank error, DDSketch as a relative error, Count-Min as a fraction of the stream's L1 norm, Count Sketch of its L2 norm, HLL as a relative distinct-count error. All of these were compared against the same ε. A bound in one metric could pass a target that meant another. On the guarantee side, ErrorMetric had seven variants, and most were the same error under a different normalization.

What

  • Three error metrics. ErrorMetric is now Absolute { scale }, Relative and Rank, and stays #[non_exhaustive].

    • scale is None (value units), StreamL1 or StreamL2.
    • Count-Min guarantees Absolute@StreamL1 and Count Sketch Absolute@StreamL2.
    • HLL, KMV and the UnivMon distinct count become Relative.
    • TopKMembership is removed.
  • One outer shape. AccuracySpec<E, P> { metric, error, failure_probability, scope } is shared by both sides:

    • AccuracyTarget = AccuracySpec<f64, Option<f64>>. δ None means no δ requirement; sizing then uses the default δ. Constructors are exact(), epsilon(ε) and epsilon_delta(ε, δ), plus with_metric and with_scope.
    • AccuracyGuarantee { spec: AccuracySpec<BoundExpr, ProbabilityExpr>, provenance }. provenance is the explanation trail, and no planning decision reads it.
    • AccuracyScope is PerValue | WholeResult. Guarantees are PerValue, and targets default to WholeResult.
  • Unresolved input vs concrete target. Workloads and the query-language frontends take a RequestedAccuracy, whose metric may be omitted.

    • It still reads the old "Exact", {"Epsilon": ε} and {"EpsilonDelta": …} JSON, as well as {"metric", "error", "failure_probability", "scope"}.

    • Plan selection resolves it per query root (RequestedAccuracy::for_root), and the metric applies only at the root:

      Root Default metric
      count, top-k Absolute@StreamL1
      distinct count, L2 frequency moment Relative
      quantile, histogram_quantile none; the request must name one (SelectionError::AccuracyRequest)
      anything else (sums, arithmetic, rates, extrema, entropy) Absolute
  • Metric-aware satisfies.

    • Same metric: compare ε, and δ when the target states one.
    • An L2-scaled bound also meets an L1-scaled target with the same ε, since ‖f‖₂ ≤ ‖f‖₁ over the root's one stream (stated in metric_meets and NormScale).
    • An exact guarantee meets every target.
    • Any other pair does not satisfy. Absolute ↔ Relative would need operand-domain evidence, and Rank → value needs the distribution; neither conversion is in this PR.
    • Plan selection's rejection reason now names the guarantee's metric.
  • Sizing stays native (§4). A summary is checked against its intent's target restated in the summary's own metric (accuracy::native_target). Only the root target's metric is checked, by Stage 3.

  • stage_pipeline gains --metric <absolute|absolute_stream_l1|absolute_stream_l2|relative|rank>.

  • Viewer. The dag-viewer editor has a metric dropdown next to ε (default, Absolute, Relative, Rank). /api/plan passes a non-empty metric to stage_pipeline --metric and rejects an unknown metric or a metric without ε. The query header shows the metric a target names.

How

  • crates/types/src/types.rs: ErrorMetric, NormScale, AccuracyScope, AccuracySpec, RequestedAccuracy, the root-metric derivation and AccuracyRequestError. post_asap::guarantee re-exports them, and AccuracyGuarantee::new builds a per-value guarantee.
  • pass/stage1_to_stage3.rs: resolve_targets turns each query's RequestedAccuracy into the root's AccuracyTarget before select_plan.
  • accuracy/mod.rs: satisfies with metric_meets, and native_target. Pass 1's per-summary checks (replacement.rs, grouping.rs) use native_target.
  • Everything else is the mechanical constructor and field rewrite (.bound → .spec.error, AccuracyTarget::Epsilon(e) → epsilon(e), …).
  • Example and test workloads with quantile targets now name their metric: Rank on KLL paths (Examples 3 and 4, the e2e and summary-sharing tests), and Relative where the test checks a relative error (DDSketch paths, the tumbling-pane p99). Example 2's Q3 (an L2 norm) uses the new Relative default, the metric UnivMon certifies its L2 in.
  • tools/dag-viewer/{index.html,editor.js,app.js,server.py}: the metric dropdown, its pass-through and its display.

Key code interfaces added / modified

crates/types/src/types.rs

/// The stream norm a scaled absolute error is a multiple of: `‖f‖₁` (Count-Min)
/// or `‖f‖₂` (Count Sketch) of the stream the summaries read. New type.
pub enum NormScale {
    /// `‖f‖₁`, the stream's total weight (Count-Min's scale).
    StreamL1,   // new
    /// `‖f‖₂` of the stream's frequencies (Count Sketch's scale).
    StreamL2,   // new
}

/// Which error quantity an accuracy statement bounds; non-exhaustive, so
/// matches must not assume the list is closed. The type stays, moved to `types`;
/// seven variants become three.
#[non_exhaustive]
#[serde(into = "MetricName", from = "MetricName")]              // wire: absolute, absolute_stream_l1, absolute_stream_l2, relative, rank
pub enum ErrorMetric {                                          // moved here from post_asap::guarantee
    /// `|estimate − truth| ≤ ε`, in value units (`scale: None`) or multiples of a stream norm.
    Absolute { scale: Option<NormScale> },                      // was AbsoluteValue, Frequency (L1), L2Frequency
    /// `|estimate − truth| ≤ ε · |truth|`, e.g. a distinct count's or DDSketch's error.
    Relative,                                                   // was RelativeValue, Cardinality
    /// The returned value's rank is within `ε · n` of the requested rank (a quantile's KLL ε).
    Rank,   // unchanged
    /// The returned key set equals the true top-k set.
    TopKMembership,   // removed
}
impl ErrorMetric {
    /// Absolute error in the value's own units.
    pub const ABSOLUTE: Self = Self::Absolute { scale: None };
    /// Absolute error in multiples of the stream's L1 norm (Count-Min).
    pub const ABSOLUTE_STREAM_L1: Self = Self::Absolute { scale: Some(NormScale::StreamL1) };
    /// Absolute error in multiples of the stream's L2 norm (Count Sketch).
    pub const ABSOLUTE_STREAM_L2: Self = Self::Absolute { scale: Some(NormScale::StreamL2) };
}

/// Which output values one (ε, δ) statement covers simultaneously: each value
/// separately (sketch and composed guarantees), or the whole result (a target's default). New type.
pub enum AccuracyScope {
    /// Each output value is within ε with probability ≥ 1 − δ, separately; what sketches
    /// and composition produce.
    PerValue,   // new
    /// Every value of one evaluation's result is within ε at once, with probability ≥ 1 − δ;
    /// a target's default.
    WholeResult,   // new
}

/// Pr[err_metric(estimate, truth) > error] <= failure_probability over the values `scope` names.
/// New type, shared by targets and guarantees.
pub struct AccuracySpec<E, P> {
    /// The error quantity bounded.
    pub metric: ErrorMetric,   // new
    /// ε, in `metric`'s units.
    pub error: E,   // new
    /// δ.
    pub failure_probability: P,   // new
    /// Which output values the statement covers.
    pub scope: AccuracyScope,   // new
}

/// A concrete requirement; `failure_probability: None` imposes no δ (sizing uses the
/// default δ). Exact is `error = 0`, `failure_probability = Some(0)`.
pub type AccuracyTarget = AccuracySpec<f64, Option<f64>>;      // was enum AccuracyTarget { Epsilon(f64), EpsilonDelta { epsilon, delta }, Exact }
impl AccuracySpec<f64, Option<f64>> {
    /// No approximation permitted.
    pub const fn exact() -> Self;
    /// Error at most `epsilon` in absolute value units, no δ requirement.
    pub const fn epsilon(epsilon: f64) -> Self;
    /// Error at most `epsilon` in absolute value units with probability at least `1 − delta`.
    pub const fn epsilon_delta(epsilon: f64, delta: f64) -> Self;
    /// This target stated in `metric`.
    pub fn with_metric(self, metric: ErrorMetric) -> Self;
    /// This target over `scope`.
    pub fn with_scope(self, scope: AccuracyScope) -> Self;
    /// Whether this target permits no approximation.
    pub fn is_exact(&self) -> bool;
}

/// Unresolved user input. Reads the old "Exact", {"Epsilon": ε}, {"EpsilonDelta": {..}} JSON
/// and {"metric", "error", "failure_probability", "scope"} (only "error" required).
#[serde(from = "RequestedAccuracyWire")]
/// New type.
pub struct RequestedAccuracy {
    /// The requested metric; `None` (default) takes the root's default metric.
    pub metric: Option<ErrorMetric>,   // new
    /// ε.
    pub error: f64,   // new
    /// δ; `None` (default) imposes no δ requirement.
    pub failure_probability: Option<f64>,   // new
    /// Which output values the request covers; defaults to `WholeResult`.
    pub scope: AccuracyScope,   // new
}
impl RequestedAccuracy {
    /// No approximation permitted.
    pub const fn exact() -> Self;
    /// Error at most `epsilon` in the query's default metric, no δ requirement.
    pub const fn epsilon(epsilon: f64) -> Self;
    /// Error at most `epsilon` in the query's default metric with probability at least `1 − delta`.
    pub const fn epsilon_delta(epsilon: f64, delta: f64) -> Self;
    /// This request stated in `metric`.
    pub fn with_metric(self, metric: ErrorMetric) -> Self;
    /// Whether this request permits no approximation.
    pub fn is_exact(&self) -> bool;
    /// This request's ε, δ and scope stated in `metric`; used to size a summary for an
    /// aggregate intent. An exact request stays exact.
    pub fn target_in(&self, metric: ErrorMetric) -> AccuracyTarget;   // in an aggregate intent's own metric
    /// The target a quantile intent is sized against: the requested metric, or `Rank`.
    pub fn for_quantile(&self) -> AccuracyTarget;
    /// The target plan selection checks at `root`: the requested metric or the
    /// root's default. Errors for a quantile root whose request names no metric.
    pub fn for_root(&self, root: &QueryRoot) -> Result<AccuracyTarget, AccuracyRequestError>;  // resolves the root's default metric
}

/// A requested accuracy that cannot be resolved for a query root. New type.
pub enum AccuracyRequestError {
    /// A quantile query (`intent` names it) whose request names no metric; bad input,
    /// since rank or value error is the user's choice and the planner never guesses it.
    MissingQuantileMetric { intent: &'static str },   // new
}

crates/types/src/post_asap/guarantee.rs, workload.rs

/// Re-exports the shared accuracy types, so existing `post_asap::guarantee` paths keep working.
pub use crate::types::{AccuracyScope, AccuracySpec, ErrorMetric, NormScale};  // ErrorMetric was defined here

/// The accuracy statement attached to a finalized post-ASAP value; symbolic, for plan selection to bind.
/// The type stays; its three statement fields move into `spec`.
pub struct AccuracyGuarantee {
    /// Metric, symbolic ε and δ, and scope.
    pub spec: AccuracySpec<BoundExpr, ProbabilityExpr>,   // new
    /// The error quantity bounded.
    pub metric: ErrorMetric,   // removed: now spec.metric
    /// The symbolic ε.
    pub bound: BoundExpr,   // removed: now spec.error
    /// The symbolic δ.
    pub failure_probability: ProbabilityExpr,   // removed: now spec.failure_probability
    /// Why the bound is what it is; no planning decision reads it.
    pub provenance: Vec<GuaranteeSource>,   // unchanged
}
impl AccuracyGuarantee {
    /// A per-value guarantee: error at most `error` in `metric`, failing with
    /// probability at most `failure_probability`.
    pub fn new(                                          // new: a PerValue guarantee
        metric: ErrorMetric,
        error: BoundExpr,
        failure_probability: ProbabilityExpr,
        provenance: Vec<GuaranteeSource>,
    ) -> Self;
}

/// Whether the caller requested accuracy explicitly or inherited the exact default.
/// The type stays; `Explicit` now carries the unresolved request.
pub enum AccuracyRequirement {
    /// A caller-stated request; its metric may be omitted.
    Explicit(RequestedAccuracy),   // was Explicit(AccuracyTarget)
    /// No request given: exact.
    ImplicitExact,   // unchanged
}
impl AccuracyRequirement {
    /// The request, with `ImplicitExact` as `RequestedAccuracy::exact()`.
    pub fn requested(&self) -> RequestedAccuracy;        // was target(&self) -> AccuracyTarget
}

crates/asap-aware-mapping/src/accuracy/mod.rs, stage3_plan_selection.rs, pass/stage1_to_stage3.rs

impl AccuracyModel for DefaultAccuracyModel {
    /// Whether `guarantee` meets `target`: a bound in a comparable metric within ε,
    /// and within δ when stated; unknown required dimensions fail.
    fn satisfies(&self, guarantee: &AccuracyGuarantee, target: &AccuracyTarget) -> bool;  // now metric-aware via metric_meets
}
/// Scope is not compared yet.
/// Same metric, or an L2-scaled bound for an L1-scaled target; any other pair fails.
/// (An exact guarantee meets every target; `satisfies` checks that before calling this.)
fn metric_meets(guarantee: ErrorMetric, target: ErrorMetric) -> bool;                   // new
/// Used to check one summary against the target it was sized for.
/// `target` restated in the summary's own metric, for Pass 1 sizing.
pub fn native_target(target: &AccuracyTarget, guarantee: &AccuracyGuarantee) -> AccuracyTarget;  // new

/// Why plan selection could not finish. The type stays; it gains one variant.
pub enum SelectionError {
    /// The number of roots, targets and recurrences differ (internal bug in the caller).
    TargetCount { candidate: String, roots: usize, targets: usize, recurrences: usize },   // unchanged
    /// The cost model could not price one node of a candidate (internal bug).
    Cost { candidate: String, node: PhysicalASAPNodeId, error: AnalyticalCostError },   // unchanged
    /// No candidate meets every target (unsatisfiable request); every rejection with its reason.
    NoValidCandidate(Vec<Rejection>),   // unchanged
    /// A query's requested accuracy cannot be resolved for its root (bad input,
    /// e.g. a quantile with no metric).
    AccuracyRequest { query: usize, error: asap_types::types::AccuracyRequestError },  // new
}

/// Each query's request resolved for its frontend root; the first failure is a
/// `SelectionError::AccuracyRequest`.
fn resolve_targets<Id>(                                  // new: resolves each query's RequestedAccuracy before select_plan
    roots: &[(Id, QueryRoot)],
    targets: &[Option<RequestedAccuracy>],
) -> Result<Vec<Option<AccuracyTarget>>, SelectionError>;

crates/frontend-metricsql/src/lib.rs, frontend-sql/src/lib.rs, frontend-promql/src/promql.rs

/// Lowers one MetricsQL query to the IR with `accuracy` as its requested accuracy.
pub fn lower_metricsql(query: &str, accuracy: RequestedAccuracy, ingestion_interval: Duration)
    -> Result<Rc<OperatorNode>, MetricsqlError>;                     // accuracy was AccuracyTarget
/// Same, returning a `QueryRoot` (a relation or a scalar).
pub fn lower_metricsql_query(query: &str, accuracy: RequestedAccuracy, ingestion_interval: Duration)
    -> Result<asap_types::ir::QueryRoot, MetricsqlError>;            // accuracy was AccuracyTarget
/// Lowers one SQL query over `catalog` with `accuracy` as its requested accuracy.
pub async fn lower_sql(query: &str, catalog: &SqlCatalog, accuracy: RequestedAccuracy)
    -> Result<Rc<OperatorNode>, SqlError>;                           // accuracy was AccuracyTarget
/// Same, in an explicit SQL dialect.
pub async fn lower_sql_dialect(query: &str, catalog: &SqlCatalog, dialect: SqlDialect, accuracy: RequestedAccuracy)
    -> Result<Rc<OperatorNode>, SqlError>;                           // accuracy was AccuracyTarget
impl PromqlLowerer {
    /// Lowers one PromQL query given the source's ingestion interval.
    pub(crate) fn lower_query_with_ingestion_interval(query: &str, accuracy: &RequestedAccuracy, interval: Duration)
        -> Result<asap_types::ir::QueryRoot>;                        // accuracy was &AccuracyTarget
}
// Each intent now gets accuracy.target_in(its metric) or accuracy.for_quantile().

crates/integration-tests/src/lib.rs, crates/devtools/src/lib.rs, planning_stages.rs, bin/stage_pipeline.rs, crates/asap-physical-operators/tests/common/mod.rs

/// Every planning stage's output for the single query `root`.
pub fn run_stage1_to_stage3(root: Rc<OperatorNode>, accuracy: RequestedAccuracy)
    -> asap_aware_mapping::Stage1ToStage3Run<usize>;                // accuracy was AccuracyTarget
/// The logical DAG the planner selects for `root` at `accuracy` (test helper in asap-physical-operators/tests/common).
pub fn selected_dag(root: Rc<OperatorNode>, accuracy: RequestedAccuracy) -> Rc<OperatorNode>;  // was AccuracyTarget
/// Lowers one developer-supplied PromQL query with an explicit source cadence.
pub fn lower_promql_with_data_ingestion_interval(query: &str, accuracy: asap_types::types::RequestedAccuracy, interval_ms: u64)
    -> Result<std::rc::Rc<asap_types::ir::OperatorNode>, PromqlError>;  // was AccuracyTarget
/// Runs the planning stages over roots from any mix of query languages:
/// query `i` is `roots[i]` with request `targets[i]`.
pub fn run_planning_stages_over(
    roots: Vec<QueryRoot>,
    targets: &[RequestedAccuracy],                                   // was &[AccuracyTarget]
    recurrence: &[QueryRecurrence],
    data: &DataWorkload,
    max_combinations: Option<usize>,
) -> Result<PlanningStagesRun, PlanningStagesError>;
// --metric states the target's error metric; omitted, each root's default metric applies.
// stage_pipeline: new flag --metric <absolute|absolute_stream_l1|absolute_stream_l2|relative|rank>;
// it needs --epsilon. The per-query JSON accuracy gains an optional "metric" field.

tools/dag-viewer/server.py, editor.js, index.html

// POST /api/plan request body: new optional "metric" ("" = the query's default metric)
// epsilon: the requested ε (0 < ε < 1), forwarded as --epsilon.
{ "epsilon": 0.01, "delta": 0.001, "metric": "rank" }   // "absolute" | "relative" | "rank"; needs epsilon; forwarded as --metric

Usage, end to end (numbers from stage_pipeline at this PR's merge commit ac3c852a):

// 1. What the user writes: p99 latency over 5 min within ε = 0.01, δ = 0.001, as a rank error.
//    `requested` is the unresolved request on the workload entry (metric optional); stage_pipeline
//    builds the same from `--promql "quantile_over_time(0.99, latency_ms[5m])" --epsilon 0.01
//    --delta 0.001 --metric rank`. The frontend and Stage 3 both read it.
let requested = RequestedAccuracy::epsilon_delta(0.01, 0.001).with_metric(ErrorMetric::Rank);
// `q1` is the workload entry carrying it in `requirements.accuracy`.
let q1 = BatchEntry {
    query: Query("quantile_over_time(0.99, latency_ms[5m])".into()),
    requirements: QueryRequirements {
        accuracy: AccuracyRequirement::Explicit(requested),
        response_latency: LatencyRequirement::Unspecified,   // no latency bound
    },
    predictability: Predictability::Unknown,   // not used by this check
    invocations: 1,                            // a one-time query, run once
    execute_at: None,                          // at planning time
    time_selection: TimeSelection {            // real time, over the last 5 min
        scope: QueryTimeScope::RealTime,
        lookback: Some(DurationMs(300_000)),
        as_of: None,
    },
};

// 2. Stage 1 (stage1_logical_asap_optimization) offers L1 exact, L2 KLL and L3 DDSketch. Each is
//    sized in its own metric (accuracy::native_target): KLL { k: 345, failure_probability: 0.001 }
//    with a Rank guarantee of 0.009995 at δ 0.001; DDSketch { alpha: 0.01 } with a Relative
//    guarantee of 0.01 at δ 0.

// 3. Stage 3 (stage3_plan_selection): resolve_targets calls requested.for_root(&root) per query.
//    `target` is the concrete AccuracyTarget select_plan checks every candidate against:
//      AccuracySpec { metric: Rank, error: 0.01, failure_probability: Some(0.001), scope: WholeResult }
//    satisfies() compares only guarantees in the same metric, so DDSketch's Relative bound misses it.

// 4. What the user gets back (stage3_plan_selection in the document):
//    selected: "P2" (KLL, 0.000517 cost/s)
//    P1 exact: valid, costlier (0.0006 cost/s)
//    P3 DDSketch: rejected, "q1: the output guarantees a Relative bound Some(0.01), failure
//      probability Some(0.0), which misses target AccuracySpec { metric: Rank, error: 0.01, ... }"
//    With `--metric relative` the roles swap: P3 DDSketch is selected and P2 KLL is rejected
//    ("guarantees a Rank bound Some(0.009995...)").
//    Without `--metric`: no plan; error "q1: the accuracy target of a quantile query must name its
//    metric (\"rank\", \"relative\" or \"absolute\")" (AccuracyRequestError::MissingQuantileMetric).

The same request in the workload JSON: "requirements": {"accuracy": {"metric": "rank", "error": 0.01, "failure_probability": 0.001}} (scope defaults to whole result; the old {"EpsilonDelta": {"epsilon": 0.01, "delta": 0.001}} still parses, with no metric).

Before this PR

satisfies(g, t) checked only g.bound ≤ ε (and δ). A DDSketch relative bound of 0.01 met a rank target of 0.01. A Count-Min L1 frequency bound met a distinct-count target whenever its number was small enough. A quantile target needed no metric.

After this PR

$ stage_pipeline --promql "quantile_over_time(0.99, m[1m])" --epsilon 0.01 --out q.json
stage3_plan_selection.error: q1: the accuracy target of a quantile query must name its metric ("rank", "relative" or "absolute")

$ stage_pipeline --promql "quantile_over_time(0.99, m[1m])" --epsilon 0.01 --metric rank --out q.json
selected: P2
P3 invalid: q1: the output guarantees a Relative bound Some(0.01), failure probability Some(0.0), which misses target AccuracySpec { metric: Rank, error: 0.01, failure_probability: None, scope: WholeResult }, …
P1 valid: costlier

(The DDSketch candidate P3 is rejected for a rank target; KLL P2 is selected.) The committed planner-layering-example1 viewer fixture is regenerated for the new JSON shape, and its selection is unchanged (P1; P4–P9 rejected for q2).

Verification

  • New tests:
    • accuracy::tests::satisfies_requires_the_targets_metric (every metric pair).
    • a_target_without_delta_ignores_the_failure_probability.
    • per_value_guarantees_meet_whole_result_targets_directly_until_669b, which pins the current scope behavior.
    • stage3_plan_selection::tests::a_bound_in_another_metric_misses_the_target.
    • types::tests::{requested_accuracy_reads_old_and_spec_forms, metric_wire_names, root_metric_defaults_follow_the_query_intent, a_quantile_root_must_name_its_metric, exact_targets}. root_metric_defaults_follow_the_query_intent covers L2 → Relative and entropy → Absolute.
    • stage_pipeline::a_quantile_target_names_its_metric.
    • Viewer: test_server.test_metric_names_the_targets_error_metric, the new metric validation cases, test_server.PlanEndpointTests.test_a_quantile_query_plans_with_the_metric_it_names (plans a quantile query through /api/plan with and without a metric), and test_viewer.test_the_editor_names_a_target_metric.
  • Failing first:
    • The two metric tests (satisfies_requires_the_targets_metric, a_bound_in_another_metric_misses_the_target) fail with the metric check disabled.
    • The L2 default assertion failed before the default changed (Absolute { scale: None } vs Relative).
    • The viewer metric tests failed before the dropdown and server change.
  • Run locally on head 6e4768e (rebased on origin/main 2357ece), and after the rebase onto cebffbd: fmt, clippy and the touched crates (asap-types, asap-aware-mapping, asap-physical-operators, asap-integration-tests: 1234 passed):
    • cargo fmt --all -- --check passes.
    • cargo clippy --workspace --all-targets --all-features --locked -- -D warnings is clean.
    • cargo nextest run over the whole workspace, in two runs: 535 + 1188 passed, 0 failed.
    • python3 -m unittest test_viewer test_server in tools/dag-viewer: 48 passed.

Limitations and follow-ups

  • Scope is not compared yet: a PerValue guarantee is checked against a WholeResult target directly. The G·δ union bound and budget allocation are #669b.
  • Absolute ↔ Relative conversion from OperandDomain evidence is not implemented in satisfies. Such pairs are unmet unless a composition rule already produced the target's metric.
  • An aggregate intent still carries an AccuracyTarget with a metric (used only for sizing, in native terms). A quantile intent without a named metric is sized as Rank.
  • Doc updates (accuracy-models.md) are tracked in Update the docs after the G1–G11 consolidation groups land #677.

@zzylol
zzylol force-pushed the feat/accuracy-spec-690 branch from 4ed75f4 to 6e4768e Compare October 15, 2026 14:21
…targets (#690)

ErrorMetric shrinks to Absolute, Relative and Rank. Absolute carries an
optional norm scale (StreamL1 for Count-Min, StreamL2 for Count Sketch);
Cardinality becomes Relative and TopKMembership is removed.

AccuracySpec<E, P> {metric, error, failure_probability, scope} is the
outer shape of both a target and a guarantee. AccuracyTarget is
AccuracySpec<f64, Option<f64>> (no δ requirement when None), with
exact()/epsilon()/epsilon_delta() constructors. AccuracyGuarantee holds a
symbolic AccuracySpec<BoundExpr, ProbabilityExpr> plus its provenance.
AccuracyScope is PerValue or WholeResult; guarantees are PerValue and
targets default to WholeResult.

Workloads and the query-language frontends take a RequestedAccuracy whose
metric may be omitted. It still reads the old "Exact"/Epsilon/EpsilonDelta
JSON. Plan selection resolves it per query root: counts and top-k default
to an L1-scaled absolute error, distinct counts to a relative error, and
any other root to an absolute value error. A quantile or
histogram_quantile root without a metric is an input error. Example and
test workloads with quantile targets now name Rank. A summary is still
checked against its intent's target in its own native terms.

stage_pipeline gains --metric. The committed planner-layering-example1
viewer fixture is regenerated for the new JSON shape; its selection is
unchanged.
DefaultAccuracyModel::satisfies now compares a guarantee with a target
only when both are stated in the same metric. An L2-scaled bound also
meets an L1-scaled target of the same stream, since ‖f‖₂ ≤ ‖f‖₁. A bound
in any other metric does not satisfy, however small, and an exact
guarantee meets every target. This takes over the separate answers check
proposed in #627. A target without a δ checks only the error. A per-value
guarantee is still compared with a whole-result target directly until
the union bound in #669b; a test pins this behavior.

Plan selection's rejection reason now names the guarantee's metric.

Example 2's Q3 (an L2 norm, whose root defaults to an absolute error) now
names a relative target, the metric UnivMon certifies its L2 in.
)

An L2 norm root without a named metric now resolves to a Relative target,
like a distinct count; entropy stays Absolute. UnivMon certifies its L2 in
a relative error, so Example 2's Q3 no longer names its metric and still
selects the UnivMon L2 candidates.
The editor gains a metric dropdown next to ε (default, Absolute, Relative,
Rank). /api/plan passes a non-empty metric to `stage_pipeline --metric`
and rejects an unknown metric or a metric without ε. The query header
shows the metric a target names. A quantile query planned with a metric
selects a plan; without one, the page shows the missing-metric error.
@zzylol
zzylol force-pushed the feat/accuracy-spec-690 branch from 6e4768e to 50256ca Compare October 15, 2026 14:35
@zzylol
zzylol merged commit ac3c852 into main Oct 15, 2026
2 of 4 checks passed
zzylol added a commit that referenced this pull request Oct 15, 2026
…t() (#710)

fix(tools): the MetricsQL consumer check uses RequestedAccuracy::exact()

#708 replaced AccuracyTarget::Exact with constructors and made the frontends
take a RequestedAccuracy, so the generated consumer crate no longer compiled
and main's test job failed.
@zzylol zzylol added the accuracy Accuracy requirements, guarantees, and propagation label Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

accuracy Accuracy requirements, guarantees, and propagation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Accuracy targets and guarantees: three error metrics (absolute, relative, rank), a shared AccuracySpec, and metric-aware satisfies

1 participant