Repository navigation
Counter window ignores the @ anchor (and its offset): rate(m[5m] @ 300) reads the window ending at the evaluation time #748
Description
Problem, by example
rate(m[5m] @ 300) must read the window (0 s, 300 s] whatever the evaluation time (PromQL's @ pins the evaluation time of the selector). The executor reads the window ending at the evaluation time instead, as if the @ were not there. offset combined with @ is dropped too.
Two series a and b, a counter sample every 15 s from 7 s to 292 s (the samples(7_000) fixture in crates/integration-tests/tests/tumbling_pane_merge.rs), the selected exact plan, evaluated at 360 s:
| Query, evaluated at 360 s | Today | Expected (the same counter without @, evaluated at the anchor) |
|---|---|---|
rate(m[5m] @ 300) |
[0.065, 0.065] (= rate(m[5m]) at 360 s) |
[0.0667, 0.0667] (= rate(m[5m]) at 300 s) |
increase(m[5m] @ 300) |
[19.5, 19.5] |
[20.0, 20.0] |
rate(m[5m] @ 300 offset 1m) |
[0.065, 0.065] |
rate(m[5m]) at 240 s |
increase(m[5m] @ 300 offset 1m) |
[19.5, 19.5] |
increase(m[5m]) at 240 s |
The test anchored_counter_window_ends_at_the_anchor in crates/integration-tests/tests/tumbling_pane_merge.rs shows it (committed #[ignore]d with a link to this issue; it fails when run):
let at_300 = whole_build_values_at("rate(m[5m])", 300_000, &samples);
let anchored = whole_build_values_at("rate(m[5m] @ 300)", 360_000, &samples);
assert_eq!(anchored, at_300); // fails: [0.065, 0.065] != [0.0667, 0.0667]Offsets without @ are right: offset_counter_window_ends_at_the_offset and a_counter_over_panes_with_an_outer_offset_matches_single_build pass.
Why it happens
counter_window in crates/executor/src/physical_planner/mod.rs turns the TimeShifts below a counter's TimeRange into an (offset, lookback) pair relative to the evaluation time, and skips every shift with an anchor:
if shift.at.is_none() {
offset = offset.checked_add(shift.offset_ms)?;
}Operator::evaluation_range then ends the window at evaluation_time_ms - offset_ms. ExactEvaluation has no way to say "end at an absolute time", so an anchored shift is dropped along with its offset. The raw rows the deployment supplies are the whole input, so the counter evaluation's window is the only thing that picks the samples, and it picks the wrong ones.
Proposal
- Give the counter window an anchor:
ExactEvaluationcarries the window end as eitherRelativeToEvaluation { offset_ms }(today's case) orAnchored { at, offset_ms }, withatanAtModifier(Timestamp(ms),Start,End). counter_windowfolds the shifts from the range down: an anchored shift fixes the end atat - offset(offsets above the anchor still add), and an unanchored one adds its offset as today.evaluation_rangeresolvesStart/Endfrom the run scope (the query's range start and end), asseries_windowalready does for range bounds.- Alternative: refuse a counter with an
@anchor at compile time (Error::Invalid) until the above lands, so the query fails instead of returning a wrong number.
Scope and acceptance
- In scope: exact
rate/increaseover one build and over merged panes, with@ <ts>,@ start(),@ end(), each with and withoutoffset. - Out of scope: sketch-backed counters (there are none today).
- Done when
anchored_counter_window_ends_at_the_anchorpasses without#[ignore], plus a pane variant likea_counter_over_panes_with_an_outer_offset_matches_single_buildwith@ 300.
Status and related work
Found while checking a review note on the counter-window change (#713). Today's numbers come from running the test on main at 20d2923.
Activity
fix: review nits after #745: cap message from its constant, Hydra con…