Repository navigation
fix(asap-tools): score SQL query results in calculate_fidelity - #896
Merged
milindsrivastava1997 merged 1 commit intoOct 15, 2026
Merged
milindsrivastava1997 merged 1 commit into
milindsrivastava1997 merged 1 commit into
Conversation
SQL results arrive as ClickHouse TabSeparated text in
QueryResult.raw_text_result with `result` left None, so
get_all_timeseries found no series and SQL queries were never scored.
Parse the TSV into the same {frozenset(label -> value): float} form as
`result`. TSV has no header, so label names come from the query's
outermost SELECT list (bare columns are labels, the single other item is
the value), read in SELECT order as ClickHouse returns them. Parsing
happens in get_all_timeseries via QueryResult.keyed_result(), so both
freshly-run and JSONL-loaded results are covered.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011gKTCAEz3sJ1Hzde16xxU1
This was referenced Oct 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #885.
Problem: SQL results are stored as TabSeparated text in
raw_text_result, andget_all_timeseriesonly readresult, socalculate_fidelitynever scored a SQL query.Fix:
parse_tabseparated_result(query, raw_text)turns TSV rows into the same{frozenset(label → value): float}shaperesultuses.QueryResult.keyed_result()returnsresult, or the parsed TSV;get_all_timeseriesuses it. Parsing happens at scoring time, so loaded JSONL results are covered too.rqeroute (feat(asap-tools): rqe query protocol and frontend switch for e2e runs #890).Known limitation: ASAPQuery's ClickHouse adapter writes label columns in alphabetical order (
KeyByLabelNames::newsorts them), while ClickHouse uses SELECT order. If a query's GROUP BY columns aren't alphabetical in its SELECT list, an ASAPQuery TSV estimate's series won't match the exact series and get skipped; in rare cases values could pair with the wrong labels. Answers through therqeroute are unaffected, since they carry label names. A follow-up could make the adapter emit SELECT order.Also not covered:
raw_text_result;recommended_sketch_configs.pyandkll_rank_error.pystill read.result;Tests:
asap-tools/experiments/tests/test_calculate_fidelity.py(13).🤖 Generated with Claude Code