feat(writer): cascade delta like Rust's DeltaScheme, edition-gated (issue #410) - #439
Merged
Merged
Conversation
…ssue #410) Parity port 4/4 for #410. fastlanes.delta overrides encodeCascade: bases and deltas (Rust ids 0/1, also our wire order) become open children with Rust's exclusion (Delta on both), and arrays under one FastLanes chunk are skipped (Rust's MIN_DELTA_LEN). Like Rust, Delta is registered as a cascade candidate but only offered when an enabled edition contains it (unstable2025.05.0+); default writes never emit it. On jittered timestamps with the unstable edition the cascade now picks it. Fix found on the way: the edition exclusion set only covered the encoders the writer itself lists, so a candidate kept in an encoder's private list (Sparse's bool patch-index cascade offers delta) escaped it. Cascading delta then won there and a default-edition write failed the backstop guard. editionExcluded now covers every well-known id outside the enabled editions. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
dfa1
force-pushed
the
feat/410-delta-cascade
branch
from
October 3, 2026 07:30
1fcc907 to
9882bc6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #440.
Closes #410: Rust-parity port 4/4, after RunEnd (#435), ZigZag (#436) and RLE (#437, plus float RLE in #438).
Delta
DeltaEncodingEncoder.encodeCascade:basesanddeltas(Rust ids 0/1, also our wire order) become open children, with Rust'sDeltaSchemeexclusion (Delta on both, so data is never delta-encoded twice).MIN_DELTA_LEN = 1024).DeltaScheme, but notes "no edition includesfastlanes.deltayet, so the session's enabled editions decide". It is offered as a cascade candidate only when an enabled edition contains it (unstable2025.05.0+, the same gate as OnPair). Default writes never emit it.CODES_EXCLUDEDalready bars Sequence there. Noted for a follow-up if we want it literal.Deltashelper used byencodeandencodeCascade; the terminal output is unchanged.Bug fix found on the way (edition guard)
editionExcludedonly covered the encoders the writer itself lists.SparseEncodingEncoderkeeps a private candidate list for compressing a validity bitmap's patch indices, and it offersfastlanes.delta. Plain delta always lost there, so this never surfaced. Once delta cascades it wins, and a default-edition write then failed the backstop guard (fastlanes.delta: outside the configured edition(s) [core2026.08.0]). This reproduced in 3 FileSizeComparison tests.editionExcludednow covers every well-known id outside the enabled editions. That is safe by construction: anything it removes would have failed the backstop anyway.Tests
WriterEditionGuardTest: with default editions, cascading never emits delta; the unstable edition picks it on jittered timestamps and reads back exactly; and the private-candidate-list regression (fails without the fix, verified with a clean build).DeltaEncodingEncoderTest.Cascade: children equal the terminal buffers, exclusions, and the under-one-chunk skip.javaWriter_jniReader_delta_cascading_unstableEdition: vortex-jni reads the cascaded delta file on a full scan, a mid-chunk row range and a zone-pruned filter.Docs:
docs/compatibility.mddelta row, plus the CHANGELOG.🤖 Generated with Claude Code