Repository navigation
The deleted flag stops holding a word of its own - #222
Merged
matrixarkai merged 1 commit intoOct 25, 2026
Merged
Conversation
`MetadataRecord` was 56 bytes: two sixteen-byte handles, two `u64` stamps, and a
`bool` that alignment gave a whole eight-byte word to. The version is a per-key
mutation counter that cannot reach 2^63, so the flag rides its top bit.
MetadataRecord 56 -> 48
the metadata service's share 112 -> 104
the metadata line 298 -> 290, fixed term 164,240 -> 164,232
THE WIRE IS UNCHANGED, WHICH IS WHAT MAKES IT A REPRESENTATION CHANGE
`append_metadata_record` and the snapshot encoder both write the version and the
flag as separate `u64`s, and they still do -- through `version()` and `deleted()`.
No snapshot, no proof and no checksum sees a different byte, so there is no stamp
to bump.
THE BOUNDARY IS RULED OUT, NOT MADE UNLIKELY
A version at 2^63 would flip the flag. `set_version` masks to `MAX_VERSION` and
`version()` masks on the way out, so a value at the bound is CAPPED rather than
corrupting the flag, and a `debug_assert` names it. Reaching 2^63 versions of one
key is not a rate any store sustains, but "unlikely" is not the same as "cannot",
and a shared word has to rule the failure out.
THE COMPILER ENUMERATED THE SITES, BECAUSE A REGEX WOULD HAVE BEEN WRONG
`version` and `deleted` are field names on `ObjectMetadata`, `MetadataChange` and
`MetadataMutation` as well. A sweep over `.version` would have rewritten all of
them. Making the two fields private turns the compiler into the enumerator: every
site it reports is a `MetadataRecord` site and nothing else is. 84 of them were
patched from the error locations -- file, line and column -- bottom-up so earlier
edits could not shift later columns.
AND SEVEN STRUCT LITERALS, WHERE THE ORDER IS THE FORMAT
Two of them read their fields straight off a snapshot cursor:
version: cursor.read_u64()?,
update_time_ms: cursor.read_u64()?,
deleted: cursor.read_bool()?,
A struct literal evaluates its fields in source order and a call evaluates its
arguments left to right, so `MetadataRecord::new` takes them in the order the
literal listed them and reads the same bytes in the same order. Reordering those
arguments would decode a different record and compile perfectly, which is why the
constructor's signature is the literal's order and the comment says so.
A FOURTH STALE MIRROR, AND THE REASON NOTHING CAUGHT IT
`measure_what_a_one_word_text_handle_would_save` mirrors this record to price a
thinner handle, and its mirror still spelled the version and the flag as separate
fields -- so it measured 56 against a real 48. Its own control assertion compares
the mirror to `size_of::<MetadataRecord>()` and would have failed on the first
run. It is `#[ignore]`d, so there was no first run.
That is the fourth stale mirror in this effort. They are all the same shape: a
by-construction measurement is only as good as its control, and an `#[ignore]`d
control is run by nothing.
WHAT IS LEFT, AND WHAT THE COMPILER SAYS ABOUT EACH
104 today
97 the kind out of the stored key -- every object record's key begins
`object/`, seven characters of kind tag stored once per record
96 an eight-byte value slot
SAFE RUST CANNOT HOLD TEXT-OR-DIGEST IN EIGHT BYTES, and that is measured rather
than argued: `Box<Box<str>>` is a thin pointer and `NonZeroU64` is one word, but
their niches are the SAME VALUE, so an enum over them is sixteen bytes either
way. Only a hand-tagged word reaches eight, and that needs `unsafe` to read a
pointer back out. The route that does not is splitting the collection by kind,
which also makes the `object/` prefix implicit.
The allocation header on the key's characters is NOT a candidate and the tree
refuses it: `an_object_key_costs_two_bytes_a_character_however_many_places_name_it`
measures resident copies as the SLOPE of metadata against key length, and the
bounded change log names those same characters -- so owning them costs a byte per
character per holder, which for a 1,024-byte key is a megabyte.
THE FIGURES, EACH RE-MEASURED
1 KiB row 3.33x -> 3.32x 4 KiB and 64 KiB rows unchanged
striped 1.98x -> 1.97x replicated 3.47x -> 3.46x
metadata an object at 100 / 1,000 / 4,000 / 8,000 498 / 454 / 331 / 310
delete trade holding 1,771,625, emptied 2,240,638, 26.4% larger
the share table objects 178 (61.4%), records 104 (35.9%), keys 8 (2.8%)
MEASURED
Suite: 412 passing on the default feature set and 433 with every
feature, with the same 21 failures in both, name for name the baseline in
tools/known_failures.txt. Green on cargo build --all-targets for both feature sets with
warnings denied, and on cargo doc for both.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
MetadataRecordwas 56 bytes: two sixteen-byte handles, twou64stamps, and aboolthatalignment gave a whole eight-byte word to. The version is a per-key mutation counter that cannot
reach 2^63, so the flag rides its top bit.
MetadataRecordThe wire is unchanged, which is what makes it a representation change
append_metadata_recordand the snapshot encoder both write the version and the flag as separateu64s, and they still do — throughversion()anddeleted(). No snapshot, no proof and nochecksum sees a different byte, so there is no stamp to bump.
The boundary is ruled out, not made unlikely
A version at 2^63 would flip the flag.
set_versionmasks toMAX_VERSIONandversion()maskson the way out, so a value at the bound is capped rather than corrupting the flag, and a
debug_assertnames it. Reaching 2^63 versions of one key is not a rate any store sustains — but"unlikely" is not "cannot", and a shared word has to rule the failure out.
The compiler enumerated the sites, because a regex would have been wrong
versionanddeletedare field names onObjectMetadata,MetadataChangeandMetadataMutationas well, so a sweep over.versionwould have rewritten all of them. Makingthe two fields private turns the compiler into the enumerator: every site it reports is a
MetadataRecordsite and nothing else is. 84 of them were patched from the error locations —file, line and column — bottom-up, so earlier edits could not shift later columns.
And seven struct literals, where the order is the format
Two of them read their fields straight off a snapshot cursor:
A struct literal evaluates its fields in source order and a call evaluates its arguments left to
right, so
MetadataRecord::newtakes them in the order the literal listed them and reads the samebytes in the same order. Reordering those arguments would decode a different record and compile
perfectly, which is why the signature is the literal's order and the comment says so.
A fourth stale mirror, and the reason nothing caught it
measure_what_a_one_word_text_handle_would_savemirrors this record to price a thinner handle, andits mirror still spelled the version and the flag as separate fields — so it measured 56 against a
real 48. Its own control assertion compares the mirror to
size_of::<MetadataRecord>()and wouldhave failed on the first run. It is
#[ignore]d, so there was no first run.That is the fourth stale mirror in this effort, all the same shape: a by-construction measurement
is only as good as its control, and an
#[ignore]d control is run by nothing.What is left, and what the compiler says about each
Safe Rust cannot hold text-or-digest in eight bytes, and that is measured rather than argued:
Box<Box<str>>is a thin pointer andNonZeroU64is one word, but their niches are the samevalue, so an enum over them is sixteen bytes either way. Only a hand-tagged word reaches eight,
and that needs
unsafeto read a pointer back out. The route that does not is splitting thecollection by kind — which also makes the
object/prefix implicit.The allocation header on the key's characters is not a candidate, and the tree refuses it:
an_object_key_costs_two_bytes_a_character_however_many_places_name_itmeasures resident copies asthe slope of metadata against key length, and the bounded change log names those same characters
— so owning them costs a byte per character per holder, which for a 1,024-byte key is a megabyte.
The figures, each re-measured
Measured
cargo build --all-targets --lockedand--all-features,RUSTFLAGS=-D warnings— greencargo doc --no-deps --lockedand--all-features,RUSTDOCFLAGS=-D warnings— green🤖 Generated with Claude Code