Repository navigation
Conversation
…onformity child deduplication - Clean up orphaned locks and stale checksum indices on expired uploads in cleanupExpiredUploads across Disk, S3, and Azure backends - Remove redundant cleanupLock invocation from terminateUpload, aligning with UploadLock try-with-resources lifecycle - Remove redundant uploadLockingService != null guards and restore default locking service fallback in cleanupExpiredUploads - Enhance Utils.createScheduledDaemonExecutor to support multi-core pools sized by available processors - Add active child preservation tests in Tus and RUFH conformity suites - Add unit and multi-backend integration tests covering expired upload lock cleanup
… lock release, redundant call elimination, and NFS directory hardening
…lock blobs and stop signals
|
coverage: 94.308% (-0.08%) from 94.39% — feature/cloud-locking-performance-hardening into master |
…nage locks via UploadLock and cleanupLock
…and RUFH handlers
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR hardens distributed cloud locking (AWS S3 & Azure Blob Storage), resolves performance bottlenecks, unifies upload lock lifecycles, and ensures complete cleanup for expired uploads and checksum deduplication indexes.
Key Changes
S3 Upload Performance & Latency Reduction (~3x Speedup)
If-None-Match: *) during S3 lock acquisition, eliminating the initial GET request pre-check when no lock is present.statObjectround-trips when appending parts or calculating offsets.Unified Lock Lifecycle & Expired Upload Cleanup
Runtime.getRuntime().availableProcessors().cleanupExpiredUploads()across all storage backends (Disk, S3, Azure Blob) to explicitly calluploadLockingService.cleanupLock(id)for expired uploads and prune stale checksum index entries.cleanupLock(id)fromterminateUpload()across all storage backends, ensuring it aligns with theUploadLocktry-with-resources lifecycle where the request-scoped lock handle releases and cleans itself.cleanupExpiredUploads(UploadLockingService)and removed all redundant null checks.Active Child Deduplication & Conformity Testing
scripts/tus_conformity_test.py) and IETF RUFH (scripts/rufh_conformity_test.py) conformity test suites to guarantee parent tombstoning preserves child readability.ITAzureBlobStorageService,ITS3StorageService,DiskStorageServiceTest) covering expired upload lock cleanup and checksum index lifecycle.AGENTS.mdwith Section 25 documenting theUploadLockcleanup and non-null invariant.CHANGELOG.mdwith the release notes for version 2.0.0.Verification
mvn -P codestyle com.spotify.fmt:fmt-maven-plugin:format.