Repository navigation
Stack Encrypt cannot order text: answer Lt, Gt and Between on text over ADR-0002's folded order key #1234
Description
Background
Stack Encrypt is CipherStash's Rust encryption engine; the Go SDK (through its WASI guest and the stashgen generator) and data plans use it too. Next to each ciphertext it can store order terms, which let the database answer <, >, BETWEEN and ORDER BY without decrypting. It has two order schemes: ORE (order-revealing encryption, compared with a special function) and OPE (order-preserving encryption, compared as plain bytes).
A query names an operator, such as lt or between (ADR-0009). The engine picks the index that answers it.
Problem
Text cannot be ordered today:
- No ordering operator is answered on text. The typed path's
op::Orderedcovers only the integer types (packages/stack-encrypt/src/op/mod.rs:381), and the data-plan path answers the orderings only for integer kinds (ordered(kind),packages/stack-encrypt/src/dynamic/query.rs:193). A Go fieldName stringwithindex=oreorindex=opegets noLt/Betweenmethod. stashgen now prints a notice that the stored terms have no query (feat(golang)!: generate one query method per operator, with Verify and TargetLimit on a matches query #1216). - The order term over text is wrong anyway. ORE and OPE terms for text are derived from the raw UTF-8 bytes (
ore_of/ope_of,packages/stack-encrypt/src/dynamic/term.rs:442-472). Code-point order putséafterzandZoebeforealice. Two Unicode forms ofcaféalso order differently, although equality treats them as one value since feat(stack-encrypt)!: derive terms through vitaminc-normalize and walk dynamic values as Plaintext #1214.
ADR-0002 (accepted) defines the order key for text: "text fold v1", meaning normalise, fold case, strip accents, normalise again, pinned to Unicode 16. Values that fold the same tie. ADR-0009 now states that Stack Encrypt orders text this way. vitaminc 0.6 ships the fold and the order encoding in vitaminc-ore (vitaminc_ore::text, OrderTarget).
Impact: names, titles and any text a user sorts or pages through (ORDER BY name, "the page after Nguyen") cannot be queried in order. It is a core search feature.
Proposal
Unblocked for CLLW ORE and OPE (2026-10-18). The ORE crates own how each type is encoded for ordering. cllw-ore 0.5.2-rc.1, a pre-release, implements vitaminc-ore 0.6's OrderScheme behind its vitaminc feature and encodes text with the fold internally. Stack Encrypt passes the String through and names its target (Text). It does not fold text or re-derive order keys itself. Block ORE, which EQL's ob term stores, still waits on an ore-rs release (#1132).
With the crate available:
- Take
cllw-ore0.5.2-rc.1 with thevitamincfeature and pass text straight to its scheme, on the typed and the data-plan paths, so both produce the same bytes. This is a breaking change to text order terms; none are deployed (ADR-0002 says so). - Answer
Lt,Lte,Gt,GteandBetweenon text: extendop::OrderedtoString, and the data path'sordered(kind)to text.Between's inverted-range refusal compares the bounds in the crate's order, not by a key Stack Encrypt computes. - The Go generator then writes the ordering methods on a text field with
oreorope, and the stashgen notice for an order index on text goes. Update the Go README and the stashgen tests. - Fixtures that pin the text order term bytes, and a test that folded equivalents tie (
Café,cafe,CAFÉ) whilealice < Zoe. - Follow-up, not in this change: the EQL types
TextOrd/TextOrdOpe(stashgen refuses TextMatch, TextSearch, TextOrd and TextOrdOpe — the engine derives their terms, but no EQL type is built from them #1131), andTextOrdOre/TextSearchOre, which need block ORE (stashgen refuses TextOrdOre and TextSearchOre — EQL stores block ORE terms and the engine derives CLLW ORE #1132, stack-encrypt: deletedynamic::ValueandScalar, derive terms through vitaminc, add block ORE, dropBoolorder terms #1140).
#1235 implements this. It was reworked on 2026-10-18 to pass values straight through cllw-ore's vitaminc schemes; its first draft folded text in Stack Encrypt in front of cllw-ore 0.5.
Relationship to other work
- stack-encrypt: ordering operators Lt, Lte, Gt, Gte and Between, and remove QueryUsing::index #1186 added the ordering operators for integers.
- stashgen refuses TextMatch, TextSearch, TextOrd and TextOrdOpe — the engine derives their terms, but no EQL type is built from them #1131 and stashgen refuses TextOrdOre and TextSearchOre — EQL stores block ORE terms and the engine derives CLLW ORE #1132 make the text order EQL types producible; this issue is the engine work they build on.
- Tracking: implement ADR-0009, operator-typed query terms for Stack Encrypt #1178 tracks ADR-0009; Tracking: implement ADR-0002, new value kinds and EQL v4 for Stack Encrypt #1145 tracks ADR-0002.
Activity
feat(stack-encrypt)!: order text by its folded order key, and answer …
feat(stack-encrypt)!: order text by its folded order key, and answer …
feat(stack-encrypt)!: order text, encrypting every order term through…