Repository navigation
objectstorage: size small snapshot upload buffers to actual data - #2945
Merged
Haven Xia (HavenXia) merged 1 commit intoOct 20, 2026
Merged
Conversation
On GCS, streamed snapshot files smaller than uploadCompositeMin (64 MiB) allocated and reserved 64 MiB for PutObject's lookahead buffer, 64 MiB for putSingle's Writer.ChunkSize, and 64 MiB for parZstd's 8 MiB chunk buffers. Each small file reserved 192 MiB against the 512 MiB default upload budget, serializing a 7-file actor snapshot into two-file waves. Wrap bounded upload streams in boundedReader so PutObject and putSingle size their single-request buffers to the next 256 KiB multiple above the stream bound. Scale parZstd's frame chunk size down to a power of two in [1 MiB, 8 MiB] for sparse files with less than 8 MiB of populated data, and update planStream and streamBufferMemory to reserve the corresponding memory.
Chenyi Wang (chw120)
force-pushed
the
fix-small-upload-memory
branch
from
October 20, 2026 04:09
4be58fa to
e73dcca
Compare
Aditya Shantanu (aditya-shantanu)
approved these changes
Oct 20, 2026
Haven Xia (HavenXia)
approved these changes
Oct 20, 2026
2 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #2848 (and #2847). While #2848 fixed upload memory accounting for multi-part ranged sparse files, small single-request files (
< 64 MiB) still allocated and reserved worst-case64 MiBbuffers at three layers (64 MiBGCS lookahead +64 MiBGCS chunk buffer +64 MiBzstd worker =192 MiBper file, and128 MiBformanifest.json).Under the default
--upload-memory-budget=512Mi, a small gVisor snapshot (~7 files +manifest.json, ~17 MB total) still required> 1.2 GiBof budget reservations, forcing concurrentSuspendActoruploads to queue behind the semaphore (2–3 files at a time).Changes
64 MiB, size both thePutObjectlookahead buffer and the single-requestWriter.ChunkSizeto the actual object size (rounded up to256 KiBmultiples,512 KiBminimum total instead of128 MiB).< 8 MiBof populated data, scale the per-frame compression chunk and zstd window down to a power of two in[1 MiB, 8 MiB], reducing per-worker memory from64 MiBdown to as low as8 MiB.Uploaderplanning to reserve these proportional buffer sizes (8.5 MiBminimum for small sparse files instead of192 MiB), allowing all files in a snapshot burst to upload concurrently within the default512Mibudget. Large files (>= 64 MiB) and ranged sparse uploads (parts >= 2) are unchanged.Test Plan
1. GKE Cluster Benchmark (
glutton_baseline_10_users, 10 gVisor workers, default512Mibudget)grpc_SuspendActor)main(90de075f, with #2848)main2. Microbenchmarks (
pkg/objectstorage/upload_bench_test.go)512 MiBbudget, 50 ms RTT)main(#2848)BenchmarkConcurrentSnapshotUploads/gcs_shared_512MiB(80 small files)BenchmarkLargeSparseUpload/4_actors_gvisor_7files_each(4× [256 MiB ranged + 6× 64 KiB])BenchmarkLargeSparseUpload/1_actor_256MiB_ranged(large file only)Part of #2553