Skip to content

Opt-in whole-response cache for GraphQL queries - #2649

Merged
shangyian merged 14 commits into
DataJunction:mainfrom
shangyian:gql-response-cache
Oct 24, 2026
Merged

shangyian merged 14 commits into
DataJunction:mainfrom
shangyian:gql-response-cache

Conversation

@shangyian

@shangyian shangyian commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Some GraphQL queries are expensive and identical across callers. This PR lets a deployment serve those responses from the shared cache.

What it does

  • graphql_cacheable_operations lists operation names to cache. It is empty by default, which turns the feature off. Only named queries are eligible.
  • The key is a hash of the operation name, the query and the variables. The query is parsed with graphql-core, so formatting doesn't change the key.
  • Responses with errors aren't cached.

How it's invalidated

  • Two version keys live in the cache: one for all data and one for availability. A cached response is stored under the current version, so changing the version invalidates the response cache.
  • Availability has its own version because it is written thousands of times a day. It also only applies to queries that mention availability.
  • A session listener replaces the version after the outermost commit of any write, including bulk statements.
  • Cacheable queries read from the writer, so a lagging read replica can't cache old rows.
  • Response caching is skipped, with a warning, when the cache is in-process.

Test Plan

  • PR has an associated issue: #
  • make check passes
  • make test shows 100% unit test coverage

Deployment Plan

An operation named in graphql_cacheable_operations has its whole response
stored in the shared cache for graphql_response_cache_ttl seconds, so a
repeated identical request skips the database and the field engine. Off by
default; responses with errors are not stored.
The response is stored under the latest non-availability history id, read
before the query runs, so any later node, link or tag change makes it
unreachable. The TTL becomes a backstop rather than the staleness bound.
@netlify

netlify Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for thriving-cassata-78ae72 canceled.

Name Link
🔨 Latest commit d63a01e
🔍 Latest deploy log https://app.netlify.com/projects/thriving-cassata-78ae72/deploys/6adcf34b6980f1adfb37dc52

@shangyian
shangyian marked this pull request as ready for review October 24, 2026 07:58
@greptile-apps

greptile-apps Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The changes since the previous review appear safe to merge.

Summary

Adds opt-in caching for whole GraphQL responses.

  • Opted-in GraphQL queries reuse responses until their data changes.

No new actionable issues were found in the changes since the previous review. The earlier threads were unnumbered, so they have no numbered classifications.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[GraphQL POST] --> B{Allowed query and shared cache?}
  B -- No --> C[Run query normally]
  B -- Yes --> D[Choose writer and read data versions]
  D --> E{Cached response exists?}
  E -- Yes --> F[Return cached response]
  E -- No --> G[Run query against writer]
  G --> H{Successful response without errors?}
  H -- Yes --> I[Compress and cache response]
  H -- No --> J[Return without caching]
  K[Database write commits] --> L[Replace affected data versions]
Loading

Reviews (6) · Last reviewed commit: "Give the fake requests in session tests ..." · Reviewed by Greptile

Comment thread datajunction-server/datajunction_server/api/graphql/main.py Outdated
Comment thread datajunction-server/datajunction_server/api/graphql/main.py Outdated
Comment thread datajunction-server/datajunction_server/api/graphql/main.py Outdated
Comment thread datajunction-server/datajunction_server/api/graphql/main.py Outdated
…collections

Pick the operation with graphql-core (operationName, or the only operation),
accept queries only, and key on the printed document so formatting does not
matter. The data version now also covers availability for a document that
selects it and collections for one that lists them.
…response

The history version is now the latest id plus the count of recent rows, from
one statement, so a row that commits after one with a higher id changes it.
A response is skipped if "errors" appears anywhere in its body, not just in
the last 4096 bytes.
Parse the HTTP body with the router's own parse_http_body, so
_cacheable_request receives a typed GraphQLRequestData instead of an
arbitrary JSON value, and a batch (a list) is excluded by type.
Pick the operation with graphql-core's get_operation_ast, look for availability
and listCollections in the query text instead of walking the document, and move
the collections signature into its own function.
Comment thread datajunction-server/datajunction_server/api/graphql/main.py Outdated
Replace the history and collections versions with a version in the shared
cache that a committing session replaces. The history id version never
changed on data with old, very large ids.

Import HTTPException from cross_web and upgrade strawberry-graphql.
Comment thread datajunction-server/datajunction_server/utils.py Outdated
Comment thread datajunction-server/datajunction_server/api/graphql/main.py
@shangyian shangyian changed the title Gql response cache Opt-in whole-response cache for GraphQL queries Oct 24, 2026
@shangyian
shangyian merged commit ccd0f8d into DataJunction:main Oct 24, 2026
28 checks passed
@shangyian
shangyian deleted the gql-response-cache branch October 24, 2026 19:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant