Skip to content

stashgen refuses TextOrdOre and TextSearchOre — EQL stores block ORE terms and the engine derives CLLW ORE #1132

Open

Description

@coderdan

Background

EQL (Encrypt Query Language) is the SQL library we install into a customer's PostgreSQL database. It stores encrypted values and lets the database search them without decrypting. Each EQL type (for example TextEq, IntegerOrd, TextMatch) fixes what a column stores: the ciphertext, plus the search terms that type needs. A search term is a small encrypted value the database can compare, such as an HMAC (a keyed hash, used for equality) or an ORE term (order-revealing encryption, used for < and >).

Stack Encrypt (packages/stack-encrypt) is our Rust encryption engine. Since #1095, a plan field can name an EQL type as its target: the engine then writes the exact value that EQL type stores. The Go SDK uses this through its WASI guest (a WebAssembly build of the engine) and the stashgen code generator.

A target type is producible when the engine can write it. The engine runs the type's own Rust plan, which is the EncryptFrom derive on the type's generated struct. eql-codegen adds that derive only to the types listed in ENCRYPTION_DOMAINS (packages/eql/crates/eql-codegen/src/bindings.rs:316), and today that list is only ("text", "eq"). So of the 51 EQL types, only TextEq is producible. The generated table packages/eql/crates/eql-bindings/src/v3/targets.rs records a reason for each of the other 50, and stashgen refuses them with that reason. #1062 scoped this on purpose to TextEq only. This issue tracks one group of the remaining types.

Only EQL v4 types derive Stack Encrypt's traits. ADR-0002 (#1139) installs the EQL SQL twice: eql_v3 for payloads written by cipherstash-client (the engine behind the TypeScript SDK), and eql_v4 for payloads written by Stack Encrypt. Stack Encrypt's output is not byte-for-byte compatible with cipherstash-client's, so deriving EncryptFrom or DecryptInto on an eql_v3 struct would claim a compatibility that doesn't exist. Every type in this issue becomes producible as its eql_v4_* struct, and no eql_v3_* struct derives either trait. Today's v3 TextEq and TextEqQuery derives move to v4 in #1142. File paths above name the current v3 modules; the v4 ones are generated in #1142.

Problem

Two text types cannot be written because the engine and EQL use different ORE schemes:

EQL type Terms EQL stores
TextOrdOre HMAC + ORE
TextSearchOre HMAC + ORE + bloom filter

EQL's ore term is block ORE: the SQL compares it with the ore_block functions (for example packages/eql/src/v3/common.sql and the *_ord_ore_functions.sql files). The engine derives CLLW ORE terms, a different scheme. The two are not interchangeable: a CLLW term cannot be compared with block-ORE functions. The generated table refuses both types with: "EQL stores a block ORE term and the engine derives a CLLW ORE term; the two are different algorithms". The engine's test target resolver says the same (packages/stack-encrypt/src/dynamic/target.rs:454).

The same mismatch blocks every *OrdOre type in the numeric and date families (for example IntegerOrdOre, DateOrdOre), once their plaintext encoding exists.

Impact: a developer who needs range queries on text with ORE (for example because their column already uses TextOrdOre) cannot write it from Go or a Rust data plan. Today they can only use the OPE types, once those are producible.

Proposal

Decide which side moves. The options:

  1. The engine derives block-ORE terms. Add a block-ORE index to Stack Encrypt whose terms EQL's ore_block functions can compare, then add TextOrdOre and TextSearchOre to ENCRYPTION_DOMAINS for the v4 structs.
  2. EQL accepts CLLW ORE. Add CLLW ORE comparison to EQL (new SQL functions, or a new ORE domain), keeping block ORE for existing data.
  3. Do not produce ORE types from the engine. Mark them permanently unproducible and point users at the OPE types (TextOrdOpe, TextSearch), which do not have this problem.

Whatever is chosen, prove it end to end in PostgreSQL with a fixture that pins the term bytes, as TextEq was.

Relationship to other work

Activity

  1. added
    enhancementNew feature or request
    rustPull requests that update Rust code
    on Oct 7, 2026
  2. self-assigned this
    on Oct 7, 2026
  3. coderdan commented on Oct 8, 2026

    @coderdan
    ContributorAuthor

    Decided in ADR-0002 (#1139): option 1. The engine derives block-ORE terms,
    through a new vitaminc crate, vitaminc-ore. Stack Encrypt's terms are
    produced only into EQL v4 domains (eql_v4_*), which are separate Postgres
    types from the eql_v3_* domains cipherstash-client writes, so the engine
    doesn't need to match cipherstash-client's block-ORE bytes. Block ORE uses
    each kind's natural width, without cipherstash-client's 8-byte padding.

    Text under block ORE waits for ore.rs's chained, variable-length block ORE
    (cipherstash/ore.rs#95). Every other kind gets block ORE in vitaminc 0.6.0.
    Work: cipherstash/vitaminc#374, then #1140 and #1142 here.

  4. coderdan commented on Oct 17, 2026

    @coderdan
    ContributorAuthor

    Related: #1234 covers the engine work for ordering text (Lt, Gt and Between over ADR-0002's folded order key, on the typed and data-plan paths, and the Go ordering methods). The EQL text order types build on it.

  5. coderdan commented on Oct 17, 2026

    @coderdan
    ContributorAuthor

    A design question to settle before block ORE is built here, which came up while reviewing #1237's range domains.

    ADR-0009 decision 7 says a Between query's RangeQuery carries "the ORE left ciphertexts, once block ORE lands". EQL's comparator doesn't fit that. compare_ore_block_256_term(a, b) reads a's left blocks and b's right half (packages/eql/src/v3/sem/ore_block_256/functions.sql, around lines 183–203). Both eql_v4.between and a btree index scan pass the stored value as a, so each query bound has to supply the right half, which means a full ciphertext.

    If Stack Encrypt sends left-only query terms as the ADR says, between and every single-bound ORE comparison would compare wrongly. #1237's tests use full block-ORE terms (from cipherstash-client), so nothing is wrong today.

    Either the ADR wording or the engine's query term needs reconciling before the block ORE work starts.

    https://claude.ai/code/session_01BZhkvKLpNtKuzj3HCfMXoL

  6. coderdan commented on Oct 18, 2026

    @coderdan
    ContributorAuthor

    128-bit block ORE (u128/i128 on ore.rs's chained scheme, cipherstash/ore.rs#113) builds on this; tracked in #1251.

  7. coderdan commented on Oct 18, 2026

    @coderdan
    ContributorAuthor

    Scope check against next plus the open stack (#1235 → #1242, #1243, #1244). Nothing here is producible yet, and none of tonight's PRs makes it so.

    What TextOrdOre, TextSearchOre and every number *OrdOre type need:

    1. Block-ORE terms in the engine. feat(stack-encrypt): order text over ADR-0002's folded order key #1235 and feat(stack-encrypt)!: derive every order term over its ADR-0002 order key, and order floats #1242 move every order term onto cllw-ore's vitaminc schemes, but the ORE term is still CLLW ORE (OreCllw8VariableV1). EQL's ob field holds block ORE and is compared by ore_block_256. So the engine needs a block-ORE index through vitaminc-ore (ADR-0002, option 1).
    2. The query-term shape, settled first. As noted above, compare_ore_block_256_term(a, b) reads b's right half, and both eql_v4.between and the btree pass the stored value as a. So a query bound needs a full ciphertext, not the left-only term ADR-0009 describes. This should be decided before the index is built.
    3. Text block ORE waits on Tracking: review and release chained block ORE (ore v2) so text can use block ORE ore.rs#95 (variable-length, chained). Numbers can go first.
    4. The EQL side is then the same small change as feat(eql-bindings): produce the OPE-ordered EQL v4 types #1266 for OPE: add the ORE domains to ENCRYPTION_DOMAINS, write the term into ob, and test against PostgreSQL with a fixture that pins the bytes.

    #1266 makes the OPE twins producible (IntegerOrd/IntegerOrdOpe, …, TextOrd/TextOrdOpe). Until block ORE lands, those are the ordered types to point users at.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

SDKenhancementNew feature or requestrustPull requests that update Rust code

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions