Repository navigation
[core] Unify catalog locks for snapshot commits - #10489
Closed
JingsongLi wants to merge 7 commits into
Closed
JingsongLi wants to merge 7 commits into
JingsongLi wants to merge 7 commits into
Conversation
JingsongLi
marked this pull request as draft
October 10, 2026 10:30
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Configured catalog locks should protect reading the current head, validating changes, preparing manifests and publishing each snapshot. This change uses one table-bound
LockfromCatalog.createLockfor every attempt, including the first, with separate scopes for APPEND and COMPACT publications. Post-commit callbacks run after release. It removes the retry-only option and theCommitAttempt/CatalogCommitLockinterfaces while preserving the publicCatalogLockFactorySPI and directsnapshotCommit()publication-lock behavior.The REST protocol and minimal public contract are proposed separately in #10491. This branch currently contains that foundation as well as the writer integration; it can be rebased after the foundation lands.
Hive locks now heartbeat and verify ownership before publication. JDBC leases renew and release by an internal owner, reject expired renewal, and keep renewal deadlines monotonic. Existing JDBC lock tables receive a nullable
lock_ownercolumn on catalog initialization, requiring ALTER permission. All JDBC writers should upgrade before relying on owner-checked release after expiry. Filesystem locking does not add a storage fencing token.REST automatic conflict rollback carries the table UUID and commit user so a server can atomically validate the expected head and lease, roll back and clear the lease. Snapshot/file conflict checks remain required; leases do not make stale file changes valid.
Validation: