Document JSON path access in restriction and projection - #291
Open
dimitri-yatsenko wants to merge 1 commit into
Open
dimitri-yatsenko wants to merge 1 commit into
dimitri-yatsenko wants to merge 1 commit into
Conversation
`attribute.path` works in both restriction and projection, and DataJoint
translates it per backend — json_value() on MySQL, jsonb_extract_path_text()
on PostgreSQL. None of that was documented. reference/operators.md does not
mention JSON at all, and the type-system spec said only "Supports JSON path
queries where available", whose "where available" implies a portability caveat
that does not exist.
Adds a JSON path access section to the type-system spec covering the syntax for
both surfaces, nested and array paths, the path:type annotation, and ordering
comparisons via a typed projection.
Also states the two defects a reader will otherwise discover by being given
wrong answers:
- a restriction value that is a Python int or bool is not coerced to match the
extracted text, so {"specs.calibrated": True} returns no rows on MySQL and
raises on PostgreSQL (#1564);
- :int works on PostgreSQL and raises on MySQL, where RETURNING does not accept
it; unsigned and signed are portable (#1563).
Verified every example on MySQL 8.0 and postgres:15.
dimitri-yatsenko
requested review from
MilagrosMarin,
lum-agilyti and
ttngu207
October 1, 2026 15:32
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
attribute.pathworks in both restriction and projection, and DataJoint translates it per backend. None of it was documented.reference/operators.md— zero mentions of JSON.reference/specs/type-system.md— one line: "Supports JSON path queries where available." Eleven words, no syntax, and "where available" implies a portability caveat that does not exist._prov.What this adds
A JSON path access section in the type-system spec:
Equipment & {"specs.vendor": "Acme"}andEquipment.proj(vendor="specs.vendor")— and that both emitjson_value()on MySQL andjsonb_extract_path_text()on PostgreSQL.specs.probe.serial,specs.channels[0].path:typeannotation that fixes it.proj(ch="specs.channels:unsigned") & "ch > 32".Two defects stated rather than left to be discovered
A reader who follows the obvious path gets wrong answers, so the section says so:
& {"specs.calibrated": True}returns no rows on MySQL and raises on PostgreSQL — the extraction yields text and the Python value is not coerced (#1564). The MySQL case is the dangerous one: an empty result reads as "none are calibrated.":intis not portable — works on PostgreSQL, raises on MySQL, whoseRETURNINGclause does not accept it.unsignedandsignedwork on both (#1563).Still to come
tutorials/advanced/json-type.ipynbteaches "Filtering on JSON Content — fetch then filter in Python" and lists "Filter in Python" as an inherent property of JSON in its Design Guidelines. That is wrong — it teaches a full table scan into Python for something the database does — and it is almost certainly a workaround for #1564 mistaken for a property of the type.Rewriting it is deliberately held until #1564 merges, so the tutorial is not rewritten twice. Tracked in #292.
Verification
Every example run against MySQL 8.0 and postgres:15.
mkdocs build --strictclean;check_links.pypasses across 142 pages.