Repository navigation
[Python] Commit snapshot properties through Native - #10505
Merged
JingsongLi merged 1 commit intoOct 11, 2026
Merged
Conversation
leaves12138
approved these changes
Oct 11, 2026
leaves12138
left a comment
Contributor
There was a problem hiding this comment.
Reviewed at e43cb4f. The Native routing preserves the existing preparation/publication exception boundary, passes snapshot properties per call, and keeps callback/non-REST fallback behavior. The tests cover append, PK, Data Evolution, stream checkpoint isolation, empty commits, static/dynamic overwrite, and publication response loss. No blocking correctness issue found. This remains dependent on Rust #1104 reaching main, and the latest Native CI was still running at review time.
JingsongLi
marked this pull request as ready for review
October 11, 2026 11:49
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PyPaimon already supports caller
snapshot_propertiesin batch, stream and overwrite commits. The presence of an explicit property map previously forced the Python publication path, even for{}and REST tables which otherwise qualified for Native.Properties describe the current commit, are not inherited by subsequent checkpoints, and empty maps are absent from snapshot metadata. These rules follow Java
ManifestCommittable.propertiesandFileStoreCommitImplsnapshot construction. No new Python capability or API is introduced.Dependency
Depends on apache/paimon-rust#1104 for Rust core snapshot-property publication and the optional binding argument. This PR is draft until that change reaches Rust main. Native CI continues to build upstream
apache/paimon-rust@main.Verification
Built the companion Rust extension and enabled Native plan/read/write/commit for:
table_commit_test.pyandtable/simple_table_test.py: 41 passed.New cases assert that property maps cannot leak between stream checkpoints, empty-map metadata is omitted, static empty overwrite publishes properties while dynamic empty overwrite creates no snapshot, and publication response loss preserves both metadata and referenced data files.