Skip to content

EQL: = between an eql_v3 column and an eql_v4 query term returns no rows silently — add cross-version blocker operators #1160

Open

Description

@coderdan

Background

EQL (Encrypt Query Language) is the SQL we install into a customer's Postgres so it can store and search encrypted values. Every encrypted column has an EQL type, such as public.eql_v3_text_eq. Under the hood each type is a Postgres domain over jsonb: a named jsonb with checks on its shape. EQL defines operators (=, <, …) for each type, and blockers (operators that always raise "not supported") for comparisons that must not run.

ADR-0002 (#1139) installs the same SQL under two names. eql_v3 holds terms written by cipherstash-client, the engine behind the TypeScript SDK. eql_v4 holds terms written by Stack Encrypt, the Rust engine the Go SDK runs. The two engines never produce matching search terms for the same value, so a query from one against a column written by the other can never find anything.

Problem

Comparing an eql_v3 column with an eql_v4 query term does not fail. It returns no rows, and nothing warns the caller.

EQL defines no operator for the pair (eql_v3_text_eq, eql_v4.query_text_eq). Postgres then treats both sides as their base type, jsonb, and uses the built-in jsonb = jsonb. The two payloads are never byte-equal, so the result is always empty.

Verified on Postgres 17, with the v3 and v4 bundles from #1141 installed into one database:

Query Result
v3col = '…'::eql_v4.query_text_eq plans as (c)::jsonb = (…)::jsonb, returns 0 rows
v4col = '…'::eql_v3.query_text_eq returns 0 rows
t3 JOIN t4 ON t3.c = t4.c returns 0 rows
eql_v3.eq_term(v3col) = eql_v4.eq_term(v4term) compares the two terms as text, so a match is possible
inserting a "v":4 payload into an eql_v3 column error (the domain's CHECK tests the version)
eql_v3.eq(v3col, v4term) (the function form) error: function is not unique
t3.c OPERATOR(public.=) t4.c error: operator is not unique

So writes and the function form fail loudly, but the ordinary WHERE col = $1 form does not. That is exactly the mistake a Go service querying a column the TypeScript SDK wrote would make. The corrected ADR-0002 (#1139) records this limit and points here.

The same fallback already happens inside v3 across type families: eql_v3_text_eq = eql_v3.query_integer_eq also plans as jsonb = jsonb. That is a wider gap and is out of scope here. This issue covers only the v3/v4 pairs that ADR-0002 created.

Proposal

  1. When an EQL bundle installs and the other version's schema is already present, create blocker operators for every v3/v4 pair of the same type family and capability, in both directions. Examples: eql_v3_text_eq with eql_v4.query_text_eq, eql_v4_text_eq with eql_v3_text_eq. Each one raises a clear error naming both versions. The second bundle installed creates them, whichever order the two are installed in.
  2. Generate them from the catalog with eql-codegen, as the existing blockers are. Like those, they must be LANGUAGE plpgsql and must not be STRICT. Today v3 has 1,142 operators with an EQL type on both sides, so mirroring them across versions means about 2,300 operators. That number is an estimate from the installed catalog, not a design.
  3. Put the blocker functions in the second-installed version's _internal schema. Dropping either version's schemas (ADR-0001's reinstall) then removes the operators that depend on them with CASCADE, and reinstalling recreates them. Check that reinstalling either version on its own leaves the pair fully blocked.
  4. Turn the "returns 0 rows" rows in the table above into SQLx tests that expect an error.

A cheap interim step, if this waits: eql_v4.lints() (and eql_v3.lints()) could report columns whose indexes or views compare across versions. That would catch stored definitions, though not ad-hoc queries.

Relationship to other work

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    SDKenhancementNew feature or requestneeds-triageMaintainer needs to evaluate the issue

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions