Skip to content

Ground CIP custom SQL in the Calcite dialect - #870

Open
clavery wants to merge 2 commits into
mainfrom
feature/cip-calcite-grounding
Open

clavery wants to merge 2 commits into
mainfrom
feature/cip-calcite-grounding

Conversation

@clavery

@clavery clavery commented Nov 18, 2026

Copy link
Copy Markdown
Collaborator

Summary

When agents moved past the curated CIP reports and wrote their own SQL, the queries often failed on the first try. The agents guessed column names (orders, units, site_name) and used Calcite reserved words as bare aliases (AS month, AS date). These cases come from real agent sessions using the CIP tools.

  • MCP CIP skill (b2c-mcp-cip): a new workflow step says to describe each table with cip_discover action: "table" before writing custom SQL and to use only the column names it returns. It also lists the real names that agents guessed wrong (num_orders/num_units, site_display_name, npromotion_id/display_name). A new SQL dialect section covers reserved-word aliases, date literals, month grouping with EXTRACT, the lack of DATEADD, and safe division and rounding.
  • CLI CIP skill (STARTER_QUERIES.md): adds a one-line warning about reserved words.
  • Eval: a new tier0 case, cip/custom-sql-grounding, has regex graders that fail when the trace contains SUM(orders)/SUM(units) or a bare reserved-word alias.

The column names come from the curated report SQL in operations/cip/reports.ts.

Testing

The new eval case has not been run yet (pnpm run mcp:eval --case custom-sql-grounding).

clavery and others added 2 commits November 17, 2026 20:51
The b2c-mcp-cip skill now says to describe tables before custom SQL and
names the real columns agents guess wrong (num_orders, site_display_name),
and adds a SQL dialect section: reserved words as aliases, DATE literals,
EXTRACT for months, no DATEADD. Adds an eval case with regex graders for
reserved aliases and guessed columns.

Co-Authored-By: Claude <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant