Skip to content

refactor(eql): write the SQL source with schema and version placeholders - #1147

Merged
coderdan merged 5 commits into
nextfrom
refactor/eql-schema-placeholders
Oct 16, 2026
Merged

coderdan merged 5 commits into
nextfrom
refactor/eql-schema-placeholders

Conversation

@coderdan

@coderdan coderdan commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Summary

EQL (Encrypt Query Language) is the SQL we install into a customer's Postgres so it can store and search encrypted values. ADR-0002 (#1139) decides that one SQL source should be installed under two names: eql_v3 for payloads written by cipherstash-client (the TypeScript SDK's engine) and eql_v4 for payloads written by Stack Encrypt. Postgres then refuses to compare a query of one with a column of the other, instead of silently returning no rows.

This PR is the first step: the SQL source stops naming its schemas and payload version literally, so it can be written out as either version. Nothing a customer installs changes. The v3 installer and uninstaller this branch builds are byte-identical to the ones main builds.

Changes

  • SQL source: every file under packages/eql/src/v3, plus the version template, the search_path pin script and the uninstaller, writes {{prefix}} where it wrote eql_v3 (about 24k sites), and {{eql_version}} in the 90 domain CHECKs that compared VALUE->>'v' to '3'.
  • eql-codegen: the generated SQL emits the same placeholders. The Rust bindings keep the literal eql_v3_ prefix, because they describe v3 payloads.
  • eql-codegen render <3|4> <out> <file>...: new. It substitutes the placeholders, and refuses an unknown or unterminated placeholder and any literal eql_v3/eql_v4. A literal name would render the same for both versions, so the v4 bundle would quietly reach into v3.
  • Build: tasks/build.sh renders the ordered surface into build/render/eql_v3/ (gitignored), runs the symbol-order and installer-completeness gates on the rendered files, and assembles release/ from them.
  • Docs tooling: docs:validate:documented-sql runs psql over the rendered tree, and the Doxygen filter renders eql_v3 names, so the generated reference is unchanged.
  • .gitattributes: marks src/v3/scalars/ore_fallback.sql as generated, as the per-type generated files already are.
  • Docs: packages/eql/AGENTS.md gains a "Schema and version placeholders" section. The scalar-type reference shows extension SQL with placeholders.

Verification

  • release/cipherstash-encrypt.sql, release/cipherstash-encrypt-uninstall.sql and src/deps-ordered-v3.txt: compared with cmp against a build of the base commit. All three are byte-identical.
  • The placeholder rewrite cross-checks the generator: I rewrote the committed generated files mechanically, then regenerated them from the changed templates, and the generator's output matched the rewrite exactly.
  • Passed: test:crates (eql-codegen 150 unit tests, including 8 new render tests), codegen:parity, types:check, typescript:check, test:schemas:parity, test:self_contained_v3, test:symbol_order_v3, test:symbol_order_selftest, test:installer_complete, test:build_ordering_helpers, test:doc-anchors, test:matrix:inventory, test:matrix:catalog-coverage, test:public_identifiers, release:prepare-bindings-assets.test, and root pnpm run test:scripts (71 files, 1296 tests).
  • Not run locally: the SQLx suites, docs:validate:documented-sql, test:surface:snapshot, splinter and the clean-install smoke test. All need Postgres, and Docker wasn't running. test:docs_v3_grep also wasn't run, because it needs bash 4 and this Mac has bash 3. I edited its one source grep and checked that grep by hand. The installer is byte-identical, so the database suites see the same SQL; CI runs them.

Related

Refs #1141. Stacked on #1139 (the ADR). The next two PRs in the stack:

  • The build also writes the eql_v4 bundle, with a gate that no eql_v3 name survives in it, and a test that an eql_v4 query term can't be compared with an eql_v3 column.
  • The SQLx suites run against both installs.

Review notes

Two commits, reviewed differently:

  1. refactor(eql): write the SQL source… (46 files, about 1.2k lines): the codegen constants, crates/eql-codegen/src/render.rs, tasks/build.sh, the docs tooling, and the hand-written SQL. Read this one. The hand-written SQL is the same mechanical substitution, so start with render.rs and build.sh.
  2. chore(eql): regenerate the scalar SQL surface… (246 files, about 23k lines): only the eql-codegen output, with no hand edits. You don't need to read it: mise run codegen:parity reproduces it, and CI checks that.

GitHub collapses the generated files in the Files tab (linguist-generated in packages/eql/.gitattributes). This PR adds the one generated file the existing globs missed, src/v3/scalars/ore_fallback.sql. The +/− counters still count generated files, so the PR header still shows about 24k lines.

https://claude.ai/code/session_01CA4pFfaN4DPNSTLnJVNxsu

@changeset-bot

changeset-bot Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 4a04c66

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@coderabbitai

coderabbitai Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 08507e8d-ee7e-4df8-9e4a-e31e9863c684

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@coderdan
coderdan force-pushed the refactor/eql-schema-placeholders branch from 4edd586 to 80d3e99 Compare October 9, 2026 23:15
@coderdan

coderdan commented Oct 9, 2026

Copy link
Copy Markdown
Contributor Author

Force-pushed: split the single commit into the source change and a separate regenerate commit (generated SQL only), and marked src/v3/scalars/ore_fallback.sql as linguist-generated. The final tree is unchanged apart from that 3-line .gitattributes edit.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Several rendered v4 SQL comments still hard-code v3 schema or payload-version descriptions.

3 open findings
What changed in this PR

This PR prepares EQL’s SQL source for rendering as either EQL v3 or v4 while preserving the current v3 release artifacts.

Changes:

  • Introduces strict schema/version placeholder rendering in eql-codegen.
  • Builds and validates installers from rendered SQL.
  • Converts handwritten and generated SQL to placeholders and updates related tooling.
File Description
packages/​eql/​.gitattributes Marks generated fallback SQL.
packages/​eql/​.gitignore Ignores rendered SQL.
packages/​eql/​AGENTS.md Documents placeholders.
packages/​eql/​crates/​eql-codegen/​src/​bindings.rs Preserves literal v3 binding names.
packages/​eql/​crates/​eql-codegen/​src/​consts.rs Defines SQL placeholders.
packages/​eql/​crates/​eql-codegen/​src/​context.rs Supplies placeholder context.
packages/​eql/​crates/​eql-codegen/​src/​generate.rs Tests rendered generator output.
packages/​eql/​crates/​eql-codegen/​src/​lib.rs Exports renderer.
packages/​eql/​crates/​eql-codegen/​src/​main.rs Adds render command.
packages/​eql/​crates/​eql-codegen/​src/​render.rs Implements strict rendering.
packages/​eql/​crates/​eql-codegen/​templates/​ore_fallback.sql.j2 Parameterizes fallback docs.
packages/​eql/​crates/​eql-codegen/​templates/​query_types.sql.j2 Parameterizes query versions.
packages/​eql/​crates/​eql-codegen/​templates/​types.sql.j2 Parameterizes domain versions.
packages/​eql/​crates/​eql-codegen/​tests/​cli.rs Updates CLI expectations.
packages/​eql/​docs/​reference/​adding-a-scalar-encrypted-domain-type.md Documents extension placeholders.
packages/​eql/​mise.toml Uses rendered validation inputs.
packages/​eql/​src/​v3/​aggregates.sql Replaces literal schema names.
packages/​eql/​src/​v3/​common.sql Replaces literal schema names.
packages/​eql/​src/​v3/​crypto.sql Replaces literal schema names.
packages/​eql/​src/​v3/​json/​aggregates.sql Replaces literal schema names.
packages/​eql/​src/​v3/​json/​blockers.sql Replaces literal schema names.
packages/​eql/​src/​v3/​json/​functions.sql Replaces literal schema names.
packages/​eql/​src/​v3/​json/​operators.sql Replaces literal schema names.
packages/​eql/​src/​v3/​json/​types.sql Parameterizes JSON domains.
packages/​eql/​src/​v3/​lint/​lints.sql Parameterizes lint schemas.
packages/​eql/​src/​v3/​scalars/​bigint/​bigint_eq_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​bigint/​bigint_eq_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​bigint/​bigint_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​bigint/​bigint_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​bigint/​bigint_ord_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​bigint/​bigint_ord_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​bigint/​bigint_ord_ope_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​bigint/​bigint_ord_ope_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​bigint/​bigint_ord_ope_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​bigint/​bigint_ord_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​bigint/​bigint_ord_ore_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​bigint/​bigint_ord_ore_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​bigint/​bigint_ord_ore_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​bigint/​bigint_types.sql Parameterizes domains.
packages/​eql/​src/​v3/​scalars/​bigint/​json_entry_bigint_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​bigint/​json_entry_bigint_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​bigint/​query_bigint_eq_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​bigint/​query_bigint_eq_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​bigint/​query_bigint_ord_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​bigint/​query_bigint_ord_ope_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​bigint/​query_bigint_ord_ope_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​bigint/​query_bigint_ord_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​bigint/​query_bigint_ord_ore_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​bigint/​query_bigint_ord_ore_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​bigint/​query_bigint_types.sql Parameterizes domains.
packages/​eql/​src/​v3/​scalars/​boolean/​boolean_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​boolean/​boolean_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​boolean/​boolean_types.sql Parameterizes domains.
packages/​eql/​src/​v3/​scalars/​date/​date_eq_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​date/​date_eq_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​date/​date_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​date/​date_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​date/​date_ord_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​date/​date_ord_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​date/​date_ord_ope_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​date/​date_ord_ope_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​date/​date_ord_ope_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​date/​date_ord_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​date/​date_ord_ore_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​date/​date_ord_ore_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​date/​date_ord_ore_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​date/​date_types.sql Parameterizes domains.
packages/​eql/​src/​v3/​scalars/​date/​json_entry_date_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​date/​json_entry_date_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​date/​query_date_eq_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​date/​query_date_eq_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​date/​query_date_ord_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​date/​query_date_ord_ope_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​date/​query_date_ord_ope_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​date/​query_date_ord_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​date/​query_date_ord_ore_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​date/​query_date_ord_ore_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​date/​query_date_types.sql Parameterizes domains.
packages/​eql/​src/​v3/​scalars/​double/​double_eq_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​double/​double_eq_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​double/​double_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​double/​double_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​double/​double_ord_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​double/​double_ord_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​double/​double_ord_ope_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​double/​double_ord_ope_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​double/​double_ord_ope_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​double/​double_ord_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​double/​double_ord_ore_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​double/​double_ord_ore_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​double/​double_ord_ore_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​double/​double_types.sql Parameterizes domains.
packages/​eql/​src/​v3/​scalars/​double/​json_entry_double_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​double/​json_entry_double_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​double/​query_double_eq_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​double/​query_double_eq_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​double/​query_double_ord_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​double/​query_double_ord_ope_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​double/​query_double_ord_ope_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​double/​query_double_ord_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​double/​query_double_ord_ore_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​double/​query_double_ord_ore_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​double/​query_double_types.sql Parameterizes domains.
packages/​eql/​src/​v3/​scalars/​functions.sql Parameterizes shared helpers.
packages/​eql/​src/​v3/​scalars/​integer/​integer_eq_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​integer/​integer_eq_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​integer/​integer_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​integer/​integer_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​integer/​integer_ord_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​integer/​integer_ord_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​integer/​integer_ord_ope_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​integer/​integer_ord_ope_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​integer/​integer_ord_ope_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​integer/​integer_ord_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​integer/​integer_ord_ore_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​integer/​integer_ord_ore_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​integer/​integer_ord_ore_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​integer/​integer_types.sql Parameterizes domains.
packages/​eql/​src/​v3/​scalars/​integer/​json_entry_integer_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​integer/​json_entry_integer_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​integer/​query_integer_eq_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​integer/​query_integer_eq_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​integer/​query_integer_ord_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​integer/​query_integer_ord_ope_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​integer/​query_integer_ord_ope_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​integer/​query_integer_ord_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​integer/​query_integer_ord_ore_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​integer/​query_integer_ord_ore_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​integer/​query_integer_types.sql Parameterizes domains.
packages/​eql/​src/​v3/​scalars/​json/​json_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​json/​json_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​json/​json_types.sql Parameterizes domains.
packages/​eql/​src/​v3/​scalars/​numeric/​json_entry_numeric_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​numeric/​json_entry_numeric_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​numeric/​numeric_eq_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​numeric/​numeric_eq_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​numeric/​numeric_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​numeric/​numeric_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​numeric/​numeric_ord_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​numeric/​numeric_ord_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​numeric/​numeric_ord_ope_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​numeric/​numeric_ord_ope_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​numeric/​numeric_ord_ope_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​numeric/​numeric_ord_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​numeric/​numeric_ord_ore_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​numeric/​numeric_ord_ore_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​numeric/​numeric_ord_ore_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​numeric/​numeric_types.sql Parameterizes domains.
packages/​eql/​src/​v3/​scalars/​numeric/​query_numeric_eq_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​numeric/​query_numeric_eq_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​numeric/​query_numeric_ord_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​numeric/​query_numeric_ord_ope_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​numeric/​query_numeric_ord_ope_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​numeric/​query_numeric_ord_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​numeric/​query_numeric_ord_ore_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​numeric/​query_numeric_ord_ore_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​numeric/​query_numeric_types.sql Parameterizes domains.
packages/​eql/​src/​v3/​scalars/​ore_fallback.sql Parameterizes generated fallback.
packages/​eql/​src/​v3/​scalars/​real/​json_entry_real_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​real/​json_entry_real_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​real/​query_real_eq_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​real/​query_real_eq_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​real/​query_real_ord_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​real/​query_real_ord_ope_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​real/​query_real_ord_ope_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​real/​query_real_ord_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​real/​query_real_ord_ore_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​real/​query_real_ord_ore_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​real/​query_real_types.sql Parameterizes domains.
packages/​eql/​src/​v3/​scalars/​real/​real_eq_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​real/​real_eq_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​real/​real_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​real/​real_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​real/​real_ord_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​real/​real_ord_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​real/​real_ord_ope_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​real/​real_ord_ope_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​real/​real_ord_ope_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​real/​real_ord_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​real/​real_ord_ore_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​real/​real_ord_ore_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​real/​real_ord_ore_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​real/​real_types.sql Parameterizes domains.
packages/​eql/​src/​v3/​scalars/​smallint/​json_entry_smallint_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​smallint/​json_entry_smallint_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​smallint/​query_smallint_eq_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​smallint/​query_smallint_eq_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​smallint/​query_smallint_ord_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​smallint/​query_smallint_ord_ope_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​smallint/​query_smallint_ord_ope_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​smallint/​query_smallint_ord_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​smallint/​query_smallint_ord_ore_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​smallint/​query_smallint_ord_ore_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​smallint/​query_smallint_types.sql Parameterizes domains.
packages/​eql/​src/​v3/​scalars/​smallint/​smallint_eq_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​smallint/​smallint_eq_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​smallint/​smallint_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​smallint/​smallint_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​smallint/​smallint_ord_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​smallint/​smallint_ord_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​smallint/​smallint_ord_ope_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​smallint/​smallint_ord_ope_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​smallint/​smallint_ord_ope_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​smallint/​smallint_ord_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​smallint/​smallint_ord_ore_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​smallint/​smallint_ord_ore_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​smallint/​smallint_ord_ore_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​smallint/​smallint_types.sql Parameterizes domains.
packages/​eql/​src/​v3/​scalars/​text/​json_entry_text_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​json_entry_text_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​query_text_eq_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​query_text_eq_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​query_text_match_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​query_text_match_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​query_text_ord_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​query_text_ord_ope_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​query_text_ord_ope_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​query_text_ord_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​query_text_ord_ore_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​query_text_ord_ore_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​query_text_search_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​query_text_search_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​query_text_search_ore_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​query_text_search_ore_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​query_text_types.sql Parameterizes domains.
packages/​eql/​src/​v3/​scalars/​text/​text_eq_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​text_eq_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​text_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​text_match_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​text_match_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​text_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​text_ord_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​text_ord_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​text_ord_ope_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​text_ord_ope_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​text_ord_ope_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​text_ord_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​text_ord_ore_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​text_ord_ore_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​text_ord_ore_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​text_search_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​text_search_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​text_search_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​text_search_ore_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​text_search_ore_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​text_search_ore_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​text/​text_types.sql Parameterizes domains.
packages/​eql/​src/​v3/​scalars/​timestamp/​json_entry_timestamp_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​timestamp/​json_entry_timestamp_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​timestamp/​query_timestamp_eq_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​timestamp/​query_timestamp_eq_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​timestamp/​query_timestamp_ord_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​timestamp/​query_timestamp_ord_ope_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​timestamp/​query_timestamp_ord_ope_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​timestamp/​query_timestamp_ord_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​timestamp/​query_timestamp_ord_ore_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​timestamp/​query_timestamp_ord_ore_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​timestamp/​query_timestamp_types.sql Parameterizes domains.
packages/​eql/​src/​v3/​scalars/​timestamp/​timestamp_eq_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​timestamp/​timestamp_eq_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​timestamp/​timestamp_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​timestamp/​timestamp_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​timestamp/​timestamp_ord_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​timestamp/​timestamp_ord_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​timestamp/​timestamp_ord_ope_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​timestamp/​timestamp_ord_ope_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​timestamp/​timestamp_ord_ope_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​timestamp/​timestamp_ord_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​timestamp/​timestamp_ord_ore_aggregates.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​timestamp/​timestamp_ord_ore_functions.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​timestamp/​timestamp_ord_ore_operators.sql Parameterizes generated SQL.
packages/​eql/​src/​v3/​scalars/​timestamp/​timestamp_types.sql Parameterizes domains.
packages/​eql/​src/​v3/​schema.sql Parameterizes schema creation.
packages/​eql/​src/​v3/​sem/​bloom_filter/​functions.sql Parameterizes SEM functions.
packages/​eql/​src/​v3/​sem/​bloom_filter/​types.sql Parameterizes SEM types.
packages/​eql/​src/​v3/​sem/​hmac_256/​functions.sql Parameterizes SEM functions.
packages/​eql/​src/​v3/​sem/​hmac_256/​types.sql Parameterizes SEM types.
packages/​eql/​src/​v3/​sem/​ope_cllw/​functions.sql Parameterizes SEM functions.
packages/​eql/​src/​v3/​sem/​ope_cllw/​types.sql Parameterizes SEM types.
packages/​eql/​src/​v3/​sem/​ore_block_256/​functions.sql Parameterizes SEM functions.
packages/​eql/​src/​v3/​sem/​ore_block_256/​operator_class.sql Parameterizes operator class.
packages/​eql/​src/​v3/​sem/​ore_block_256/​operators.sql Parameterizes SEM operators.
packages/​eql/​src/​v3/​sem/​ore_block_256/​types.sql Parameterizes SEM types.
packages/​eql/​src/​v3/​version.template Parameterizes version function.
packages/​eql/​tasks/​build.sh Renders before validation and assembly.
packages/​eql/​tasks/​docs/​doxygen-filter.sh Renders documentation input.
packages/​eql/​tasks/​docs/​validate/​documented-sql.sh Validates rendered SQL.
packages/​eql/​tasks/​pin_search_path_v3.sql Parameterizes schema references.
packages/​eql/​tasks/​test/​docs_v3_grep.sh Updates placeholder grep.
packages/​eql/​tasks/​test/​verify_installer_complete.sh Validates rendered installer inputs.
packages/​eql/​tasks/​test/​verify_symbol_order_v3.sh Checks rendered ordering.
packages/​eql/​tasks/​uninstall-v3.sql Parameterizes uninstaller schemas.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread packages/eql/src/v3/json/types.sql
Comment thread packages/eql/src/v3/schema.sql
Comment thread packages/eql/tasks/pin_search_path_v3.sql
@coderdan
coderdan force-pushed the refactor/eql-schema-placeholders branch from 80d3e99 to 7573c58 Compare October 12, 2026 08:41
@coderdan
coderdan added this pull request to stack #1168 October 12, 2026 10:47
@coderdan
coderdan force-pushed the refactor/eql-schema-placeholders branch from 7573c58 to 122dc42 Compare October 12, 2026 10:48

@coderdan coderdan left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of this PR's own two commits (diff against docs/adr-value-encodings-eql-v4). I checked the "nothing a customer installs changes" claim and it holds. The rewrite is mechanical, and the one real defect is in the installer-completeness gate.

What I verified

  • The substitution is mechanical. I rendered every *.sql / *.template file under src/ and tasks/ at the head commit with {{prefix}}→eql_v3 and {{eql_version}}→3, then compared each with cmp against the base commit. All 271 files are byte-identical. No literal eql_v3/eql_v4 is left in the rendered inputs, and no {{…}} other than the two known placeholders appears.
  • The installer and uninstaller are byte-identical. I built the base and head with mise run build, and again with mise run --force build --version 9.8.7 (the release path, stamp COMMENT ON SCHEMA eql_v3 IS '9.8.7'). release/cipherstash-encrypt.sql, release/cipherstash-encrypt-uninstall.sql and src/deps-ordered-v3.txt are identical in both builds.
  • The generated files reproduce. The build regenerates src/v3/scalars in place, and the tree stayed clean afterwards.
  • The Doxygen output is unchanged. doxygen Doxyfile on base and head produces identical docs/api/xml (297 files), so the filter's on-the-fly render is faithful.
  • Payload-version checks are all converted. Every VALUE->>'v' / val ->> 'v' comparison now uses {{eql_version}}, and no hard-coded '3' version check is left in executable SQL. The three prose-only leftovers are already covered by Copilot's comments, so I haven't repeated them.

Findings

  1. Major: packages/eql/tasks/test/verify_installer_complete.sh:79 — the non-vacuity gate (Gate 1) can no longer fail.
    Gate 1 now runs find "$ROOT/src/v3" over the rendered tree. But build.sh (assemble, around line 94–98) does rm -rf "$root" and then renders exactly the files listed in src/deps-ordered-v3.txt. So "files on disk" and "files in the order" are now the same list by construction. The gate's own comment says it must "Compare against an INDEPENDENT find(1) rather than trusting the list's own length", and that independence is gone.
    Reproduced: I dropped src/v3/lint/lints.sql from the order, then rendered and assembled exactly as assemble does.

    • Head: 267 files, order matches disk … installer completeness gate OK, exit 0.
    • Base, same truncation: ERROR: 1 file(s) on disk but absent from … src/v3/lint/lints.sql, installer completeness gate FAILED, exit 1.

    A fully empty order still fails at head, but only by accident: find errors on the missing $ROOT/src/v3 directory and pipefail aborts the script. The intended "refusing to validate an empty surface" message never prints.
    The order-layer invariant is still pinned by install_order_contains_every_v3_sql_file in the codegen crate, so the risk is limited to the layer this gate guards: something between eql-codegen order and the concat that drops a file. That is now unguarded, and it will matter more once assemble runs for v4.
    Suggested fix: keep the disk side on the source tree and map it into the rendered namespace. For example, find src/v3 -type f -name '*.sql' ! -name '*_test.sql' | sed "s|^|$ROOT/|" | LC_ALL=C sort > "$work/disk". Then rerun the truncation above to confirm the gate fails.

  2. Minor: packages/eql/crates/eql-codegen/src/render.rs:173,182 — render overwrites its input when given an absolute path.
    out.join(rel) returns rel unchanged when rel is absolute, so the rendered file is written over the placeholder source and nothing goes to <out-dir>. Reproduced with the built binary: eql-codegen render 4 out /abs/path/a.sql exits 0, rewrites a.sql to SELECT eql_v4.f();, and creates no out/. A .. component escapes <out-dir> the same way.
    build.sh passes only relative paths today, so this is latent. But it is a new CLI that the next PR drives for v4, and the failure mode is a destroyed source tree rather than an error.
    Suggested fix: in render_files, reject any rel that is absolute or has a Component::ParentDir (or strip_prefix it against root), and add a test for it.

  3. Minor: packages/eql/tasks/docs/doxygen-filter.sh:107 — the filter now exits 0 on unreadable input.
    awk … < <(sed … "$1") hides sed's failure: neither set -e nor pipefail covers a process substitution. Reproduced: doxygen-filter.sh src/v3/does-not-exist.sql exits 0 with empty output at head, and exits 2 at base (awk: can't open file).
    tasks/test/docs_v3_grep.sh:84-87 depends on that exit status: "A separate assignment also means set -e aborts if the filter itself fails". After this change, a filter that cannot read its input yields empty output, and the leak check passes.
    Suggested fix: use a real pipe, sed -e … "$1" | awk '…'. The script already sets pipefail, and awk reads to EOF, so the SIGPIPE issue described in docs_v3_grep.sh does not apply.

  4. Minor: packages/eql/.gitignore:266 — unanchored build/ also ignores the tracked tasks/build/ directory.
    git check-ignore -v --no-index packages/eql/tasks/build/new_helper.sh reports packages/eql/.gitignore:266:build/. ordering.sh stays tracked, but a new helper added beside it is silently left out by git add -A. The root .gitignore anchors its own pattern as /build.
    Suggested fix: /build/.

  5. Nit: packages/eql/crates/eql-codegen/src/bindings.rs:85-98 — the new doc comment is attached to the wrong function.
    v3_domain_name's /// lines were added directly after struct_doc_lines's doc block, with no item between them. Rustdoc therefore merges both blocks onto v3_domain_name, and struct_doc_lines is left undocumented.
    Suggested fix: move v3_domain_name and its doc above struct_doc_lines's doc comment.

Not counted above, and outside this PR's diff: tasks/build.sh's #MISE sources lists crates/eql-codegen/src/**/*.rs but not crates/eql-codegen/templates/**, which context.rs pulls in with include_str!. A template-only edit like the two in this PR is therefore a cache hit for mise run build until a .rs file changes.

No changeset is needed as far as I can tell: the published SQL, the bindings and the docs output are byte-identical, and eql-codegen is not published.

@coderdan
coderdan force-pushed the refactor/eql-schema-placeholders branch from 122dc42 to af8c6f8 Compare October 12, 2026 11:58
@coderdan

Copy link
Copy Markdown
Contributor Author

Rebased onto the updated #1139, then pushed af8c6f8, which answers the review above:

  1. Gate 1 can fail again. The disk side lists the source tree (src/v3) and maps the paths into the rendered namespace. I reran your truncation (dropping src/v3/lint/lints.sql from the order): 1 file(s) on disk but absent … src/v3/lint/lints.sql, installer completeness gate FAILED, exit 1.
  2. render with an absolute or .. path. It is refused with NotRelative before anything is read. A test checks that the source is untouched and no output dir is created.
  3. Doxygen filter exit status. It uses a real pipe under pipefail. doxygen-filter.sh src/v3/does-not-exist.sql now exits 1.
  4. .gitignore. The pattern is now /build/. git check-ignore no longer matches tasks/build/new_helper.sh.
  5. Doc comment. v3_domain_name is moved above struct_doc_lines's doc block.

I also took the side note: crates/eql-codegen/templates/** is now in build.sh's #MISE sources. Copilot's three comment fixes use the placeholders, so the v3 render is unchanged. release/cipherstash-encrypt.sql and the uninstaller are byte-identical before and after this commit (mise run --force build, then cmp). cargo test -p eql-codegen and clippy pass. I couldn't run test:docs_v3_grep locally because macOS bash 3.2 lacks mapfile, so CI covers it.

@coderdan
coderdan marked this pull request as ready for review October 12, 2026 13:18
@coderdan
coderdan requested a review from a team as a code owner October 12, 2026 13:18
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 12, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-12T13:25:21.329760Z af8c6f8 Draft marked ready
🔒 Security Review ✅ Completed 2026-10-12T13:29:02.346693Z af8c6f8 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@cipherstash-agent cipherstash-agent left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review attestation — af8c6f8

Source: automated code review (/code-review medium) followed by a cross-check of the findings against the code. This is an attestation that the review was run.

Critical / blocking: none. Rendering the new src/ and tasks/ SQL as eql_v3 gives text identical to the pre-PR files, so the source change is lossless. I read the changed SQL and the Rust and shell tooling by hand. I did not run the build or the test suites.

Non-blocking findings (inline comments, except the first):

  • eql-codegen/src/lib.rs:43 — owned_schemas() / list-schemas return unrendered {{prefix}} names, while the doc says "the schemas the eql_v3 surface owns". Render them or document the placeholder contract.
  • eql-codegen/src/main.rs:124 — render accepts an empty file list (>= 4) and exits 0 having written nothing. Use >= 5.
  • tasks/test/verify_installer_complete.sh:30 — ROOT defaults independently of ORDERED.
  • tasks/docs/doxygen-filter.sh:50 — the lenient sed duplicates render_str without its strict checks. I did not re-verify this one.

Nits:

  • tasks/pin_search_path_v3.sql:4 — "appended verbatim" is stale.
  • eql-codegen/src/render.rs:301 — temp dirs are not cleaned up when an assertion fails.

Generated by Claude Code

Comment thread packages/eql/crates/eql-codegen/src/main.rs
Comment thread packages/eql/tasks/test/verify_installer_complete.sh Outdated
Comment thread packages/eql/tasks/pin_search_path_v3.sql Outdated
Comment thread packages/eql/crates/eql-codegen/src/render.rs
Comment thread packages/eql/tasks/docs/doxygen-filter.sh Outdated

@tobyhede tobyhede left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review (medium): 7 findings, inline where the line is in the diff.

One not anchored inline: packages/eql/tasks/schemas-parity.sh:33 now compares placeholder text with placeholder text (consts::SCHEMA and schema.sql both say {{prefix}}), so it no longer catches a list that diverges on the real rendered names, and list-schemas prints an unrendered token. Render the output or assert the rendered form.


Generated by Claude Code

Comment thread packages/eql/tasks/docs/doxygen-filter.sh Outdated
Comment thread packages/eql/tasks/build.sh
Comment thread packages/eql/tasks/test/docs_v3_grep.sh Outdated
Comment thread packages/eql/tasks/pin_search_path_v3.sql Outdated
Comment thread packages/eql/crates/eql-codegen/src/main.rs
Comment thread packages/eql/crates/eql-codegen/src/render.rs Outdated

Copy link
Copy Markdown
Contributor

Review attestation (/code-review medium)

  • Files reviewed: the diff of the PR at af8c6f8, mainly eql-codegen (render.rs, main.rs) and the EQL tasks/ scripts; generated SQL was not read line by line. No build or tests were run.
  • Models: Claude Sonnet 5.5 (review ran in a forked skill agent).

Generated by Claude Code

@coderdan
coderdan force-pushed the refactor/eql-schema-placeholders branch from af8c6f8 to f4c245c Compare October 12, 2026 13:36
@coderdan
coderdan removed this pull request from stack #1168 October 12, 2026 13:40
@coderdan
coderdan added this pull request to stack #1174 October 12, 2026 13:40
@coderdan
coderdan force-pushed the docs/adr-value-encodings-eql-v4 branch from 19fabd5 to 19b1042 Compare October 13, 2026 01:03
@coderdan
coderdan force-pushed the refactor/eql-schema-placeholders branch 2 times, most recently from 49bfcaa to 54e3f73 Compare October 13, 2026 02:23

@freshtonic freshtonic left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of this PR's own five commits (c8e5f5c7f..eb1050cae). I found no blocking defect.

What I verified

  • I built the base and the head in separate worktrees with mise run --force build --version 3.0.99. Both builds pass. The two worktrees stay clean after the build, so the generated scalars reproduce.
  • release/cipherstash-encrypt-uninstall.sql and src/deps-ordered-v3.txt are byte-identical.
  • release/cipherstash-encrypt.sql is not byte-identical at the head. The one difference is the header comment of tasks/pin_search_path_v3.sql (installer lines 59502-59506: 3 old lines, 5 new lines). Commit 54e3f73 made this change. The thread reply says so, but the PR description still says the installer is byte-identical.
  • I compared all 270 files in build/render/eql_v3/ with the base source by cmp. 269 are identical. The one that is different is tasks/pin_search_path_v3.sql.
  • No {{ is left in the rendered tree or in the two release files. The source has only the two known placeholders (30941 × {{prefix}}, 99 × {{eql_version}}). No literal eql_v3/eql_v4 is left in the source, in any letter case.
  • The stamp is COMMENT ON SCHEMA eql_v3 IS '3.0.99';. The sed stamp grep in tasks/release/prepare-bindings-assets.sh:45 still matches it.
  • Every Doxygen input (src/**/*.sql except *_test.sql) has a rendered copy. Thus tasks/docs/doxygen-filter.sh:62-68 refuses no file at the moment.
  • crates/eql-codegen/templates/** and the render inputs are in tasks/build.sh:4's #MISE sources. The new files are under packages/eql/crates/** and packages/eql/tasks/**, so the test-eql.yml push filter selects them.

Effect on the release gates

The bundle digest changes, but no gate fires because of this PR:

  • The committed bundles and manifests (packages/eql/packages/eql/sql/, crates/eql-bindings/sql/) do not change. eql-sql-asset-freshness compares the tree with itself, so it stays green.
  • scripts/sync-lockstep-versions.mjs regenerates the assets only when eqlLockstepSkew finds a skew. A release that does not bump EQL thus does not pick up the new comment.
  • FROZEN_PUBLISHERS is empty, so the frozen-bytes check is not active.

The next EQL version bump ships the new comment under a new digest. That is correct. It is a comment-only change in the installer, so a changeset is not necessary.

Suggested follow-up PR

  1. PR description. Change "byte-identical" in "Changes" and "Verification" to "identical except for the pin_search_path_v3.sql header comment". A reviewer who uses cmp must not think the build is broken.
  2. packages/eql/src/v3/json/functions.sql:42. The COMMENT ON FUNCTION {{prefix}}.meta_data(jsonb) text says "used by v3 jsonb wrappers". This is not a -- comment. It is an object comment that goes into the database, so the v4 bundle will install "v3" text. Write v{{eql_version}}. The v3 render stays byte-identical. The LIKE 'eql-inline-critical%' match in tasks/pin_search_path_v3.sql:101 reads only the prefix, so it is not affected.
  3. packages/eql/mise.toml:63-69. mise run clean removes release/ but not build/render/. docs:validate:documented-sql and the Doxygen filter read that tree. At the moment build always runs before them, so the risk is low. But clean must also remove /build/, to agree with the new .gitignore entry.

ADR-0002 separates the two producers of EQL payloads by Postgres type:
cipherstash-client's terms stay in eql_v3, Stack Encrypt's move to eql_v4,
and both come from one SQL source. This is the first step of #1141: the
source stops naming its schemas and payload version literally.

The hand-written SQL under src/v3, the version template, the search_path pin
script and the uninstaller now write {{prefix}} where they wrote eql_v3, and
{{eql_version}} where domain CHECKs compared VALUE->>'v' to '3'. eql-codegen
emits the same placeholders: its schema, typname-prefix and version constants
are placeholder strings, and its unit tests render output as v3 before
asserting. The Rust bindings keep the literal eql_v3_ prefix because they
describe v3 payloads.

A new `eql-codegen render <3|4> <out> <file>...` substitutes them. It
refuses an unknown or unterminated placeholder and any literal eql_v3 or
eql_v4, since a literal name would render the same for both versions and
let the v4 bundle reach into v3. build.sh renders the ordered surface into
build/render/eql_v3/ and runs the symbol-order and installer-completeness
gates on the rendered files. The docs tooling reads rendered SQL too.

The committed generated surface is regenerated in the next commit, so
codegen:parity fails at this one alone.

Refs #1141
Output of `cargo run -p eql-codegen` after the previous commit; no hand
edits. `mise run codegen:parity` reproduces it.

Only v3 is rendered so far, and the installer and uninstaller built from
this commit are byte-identical to the ones built before the placeholders.

Refs #1141
Gate 1 of verify_installer_complete.sh compared the order against the
rendered tree, which build.sh writes from that same order, so a dropped
file could no longer fail it. It now lists the source tree and maps the
paths into the rendered namespace (a truncated order fails again).

The Doxygen filter read through a process substitution, whose exit status
is lost, so an unreadable input passed docs_v3_grep.sh with empty output;
it is a plain pipe under pipefail now. eql-codegen render refuses an
absolute or .. path instead of writing over the source. build/ is
anchored so tasks/build/ is not ignored, the codegen templates are build
cache sources, a moved doc comment is back on its function, and the
schema, JSON and pin-script comments name the rendered version (v3 output
is byte-identical).
- `eql-codegen render` refuses an empty file list instead of exiting 0
  having written nothing.
- The literal-name guard no longer matches a longer version such as
  `eql_v30`, and its error says to write the placeholder, comments
  included.
- `list-schemas [3|4]` prints rendered names, and the schema parity gate
  renders schema.sql with the build's renderer before comparing.
- The Doxygen filter refuses any `{{` left after its substitution.
- verify_installer_complete.sh takes its rendered root from the order
  file's directory by default.
- The render tests clean up their scratch directories on failure.
- pin_search_path_v3.sql's header says it is rendered, not appended
  verbatim. This is the only change to the v3 installer, and it is a
  comment.
…endering it

The filter substituted the placeholders with its own sed, a second
renderer without eql-codegen's checks. It now reads a src/ or tasks/
file's rendered copy from build/render/eql_v3/, which every caller
already builds, and refuses a file with no rendered copy. Other files,
such as the filter's own test fixtures, are read as given. The Doxygen
XML is byte-identical (297 files).

docs_v3_grep.sh counts the hidden ste_vec_contains aliases in the
rendered file too, so it no longer depends on the placeholder's
spelling, and now depends on build.

Claude-Session: https://claude.ai/code/session_01VYKAWbd361xDLmqBdLoUcB
@coderdan
coderdan force-pushed the refactor/eql-schema-placeholders branch from eb1050c to 4a04c66 Compare October 16, 2026 10:35
@coderdan
coderdan removed this pull request from stack #1174 October 16, 2026 11:02
Base automatically changed from docs/adr-value-encodings-eql-v4 to next October 16, 2026 11:06
@coderdan
coderdan merged commit 595bb65 into next Oct 16, 2026
41 checks passed
@coderdan
coderdan deleted the refactor/eql-schema-placeholders branch October 16, 2026 11:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants