Repository navigation
Measure what the block codec costs a call - #193
Merged
Merged
Conversation
Every row in the allocation report stores its value raw: the fixtures fill
with incompressible bytes, on purpose, because a compressible value shrinks to
a forty-four byte block and stops exercising the tier. The consequence is that
the compressor has been running on every one of those writes and having its
output discarded, and the decompressor has never run at all.
Two rows for the case where compression is kept.
allocations bytes
get, promoting a compressed block 14.00 153,389.0
get, promoting a raw block 12.00 24,310.0
put, compressing a block, Rust 25.86 49,498.2
put, compressing a block, the other
store backend 10.83 48,342.4
A read of a compressed block allocates **153,389 bytes to return 1,024** --
six times the same read when the block is stored raw. The figure is identical
on both feature arms to the byte, so none of it is the store: it is
`zstd::stream::decode_all` building a decompression context and its window on
every call.
The write is the same cost in the other direction. A write that keeps its
compressed form pays 49,498 bytes; a write whose value does not compress pays
58,224 for a payload of the same size, because the compressor runs there too
and the result is thrown away. The difference between those two is not the
compressor, it is the block that gets stored.
Both rows assert the regime they are named for. The read checks that nine in
ten of its reads reached a lower tier, and that nine in ten of those were
served from a block with the compressed codec byte set -- so if the fixture's
value ever stopped compressing, the row would fail rather than quietly become
a second copy of the raw one.
The bounds are where the codec is today, not an endorsement of it. A reused
context would move both figures, and these come down with it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every row in the allocation report stores its value raw. The fixtures fill
with incompressible bytes on purpose — a compressible value shrinks to a
forty-four byte block and stops exercising the tier, a trap that has cost this
work two measurements already.
The consequence is that the compressor has been running on every one of those
writes and having its output discarded, and the decompressor has never run at
all.
get, promoting a compressed blockget, promoting a raw blockput, compressing a block (pure-Rust store)put, compressing a block (the other backend)A read of a compressed block allocates 153,389 bytes to return 1,024 —
six times the same read when the block is stored raw. The figure is identical
on both feature arms to the byte, so none of it is the store: it is
zstd::stream::decode_allbuilding a decompression context and its window onevery call.
The write is the same thing in the other direction. A write that keeps its
compressed form pays 49,498 bytes; a write whose value does not compress
pays 58,224 for a payload of the same size, because the compressor runs there
too and the result is thrown away. The difference between those two rows is
not the compressor — it is the block that gets stored.
Both rows assert the regime they are named for
The read checks that nine in ten of its reads reached a lower tier, and that
nine in ten of those were served from a block with the compressed codec byte
set. If the fixture's value ever stopped compressing, the row fails rather
than quietly becoming a second copy of the raw one.
The bounds are where the codec is today, not an endorsement of it. A reused
context would move both figures, and these come down with it.
Gates on both feature arms:
cargo fmt --check,clippy --all-targets -D warnings, both suites, the vocabulary check, and both reports with--require-passed.🤖 Generated with Claude Code