Repository navigation
The objects map stops keying itself by a copy of what it holds - #221
Merged
matrixarkai merged 1 commit intoOct 24, 2026
Merged
Conversation
`objects` was a map from an `ObjectId` to an `ObjectMetadata` that already held that `ObjectId`: two shared-text handles, 32 bytes, once per object. The metadata knows its own identity, so the collection is ordered BY that identity rather than beside it -- the same change `meta.records` took, on the storage node's side. objects.mapkey_inline 32 -> 0 a storage node's own share 218 -> 186 the metadata line 330 -> 298, fixed term unchanged at 164,240 AND THE BYTES ARE A SLOPE, NOT A FIXED TERM, BECAUSE NOTHING COPIES CHARACTERS This removes a HANDLE, not a copy of the name. The map key and the metadata's identity already shared one allocation, so no character moves and `an_object_key_costs_two_bytes_a_character_however_many_places_name_it` is unaffected -- which is the difference between this and owning the key's characters, a change that guard refused for the records collection on the same day: there the bounded change feed named the same characters, so an owned copy cost 1,024 times the key length. WHAT IT COST TO WRITE: 218 COMPILE ERRORS, AND THREE THINGS WORTH KEEPING Most of them were field reads on values that came out of the collection, so four helpers -- `object`, `objects_iter`, `objects_values`, `objects_range` -- put the map's own shapes back and took 218 to 13. `len` and `is_empty` did not change at all. 1. A STRUCTURAL SWEEP NEEDS ITS EXCLUSIONS DERIVED, NOT REMEMBERED. The accesses are mostly multi-line -- `.objects` on one line, `.get(` on the next -- so line anchors miss them and a regex is the only practical tool. Mine rewrote THREE fields that are not this collection: `ObjectListPage`, `ObjectTagPage` and `ObjectMutationRollback` each have an `objects` of their own. The compiler caught them because a `Vec` has no `objects_iter`; the dangerous version is a sweep whose over-reach still compiles. 2. A HELPER BORROWS ALL OF `self`, WHERE A FIELD BORROWS ONE FIELD. The snapshot install loop reads every object and inserts into `self.content_types` in the same body. `self.objects.iter()` let the borrow checker split those; a method call does not, so that loop reads the field directly and says why. 3. THERE IS NO `get_mut`, AND THERE SHOULD NOT BE. Changing an object in place could change where it belongs in a collection ordered by its elements. The two callers -- one in the install path, one in a test -- take it out, change it, and file it back. The install path uses `replace` rather than `file_object`, because that funnel re-interns the content type and would undo the assignment the loop exists to make. THE REGRESSION I INTRODUCED, AND WHAT CAUGHT IT `file_object` began with `let object_id = self.filed_object_key(object_id);`. That helper does TWO things: it adopts the identity the collection already holds, AND it interns the bucket name from the store's own set, which every object in a bucket names through one allocation. I reimplemented the first inline and lost the second, so every object began owning its own copy of the bucket name. -32 for the map key, +22 for the bucket name, measured as -10. The total still moved the right way and the whole suite passed. What showed it was the published per-term figure: `objects.mapkey_heap 42 -> 64`. A decomposition that only has to SUM would not have caught it, because it still summed. Read a helper's body before reimplementing it -- `filed_object_key` names one of the two things it does. `objects.mapkey_inline` now reports ZERO rather than being dropped, for the same reason the records side kept its counters: it is what the document names, and a change that reintroduced a second copy of the identity would move it. A STALE CONSTANT IN A MEASUREMENT I WROTE AN HOUR AGO `measure_the_metadata_per_object_at_the_published_counts` printed what the line predicts at each published count -- from a slope and fixed term COPIED INTO the test. The slope moved twice today, so those rows were stale while still looking deliberate. It measures what has to be measured and says that the rest follow from the published line, whose own guard checks that line against the tree. THE FIGURES, EACH RE-MEASURED 4 KiB row 3.09x -> 3.08x 1 KiB row 3.36x -> 3.33x striped 2.01x -> 1.98x replicated 3.50x -> 3.47x metadata an object at 100 / 1,000 / 4,000 / 8,000 506 / 462 / 339 / 318 delete trade holding 1,775,633, emptied 2,244,646, 26.1% -> 26.4% larger the share table objects 178 (59.7%), records 112 (37.6%), keys 8 (2.7%) MEASURED Suite: 412 passing on the default feature set and 433 with every feature, with the same 21 failures in both, name for name the baseline in tools/known_failures.txt. Green on cargo build --all-targets for both feature sets with warnings denied, and on cargo doc for both. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
objectswas a map from anObjectIdto anObjectMetadatathat already held thatObjectId: two shared-text handles, 32 bytes, once per object. The metadata knows its ownidentity, so the collection is ordered by that identity rather than beside it — the same
change
meta.recordstook, on the storage node's side.objects.mapkey_inlineThe bytes are a slope, not a fixed term, because nothing copies characters
This removes a handle, not a copy of the name. The map key and the metadata's identity
already shared one allocation, so no character moves and
an_object_key_costs_two_bytes_a_character_however_many_places_name_itis unaffected.That is exactly the difference between this and owning the key's characters — a change the
same guard refused for the records collection earlier today, because the bounded change feed
names those characters too, so an owned copy costs 1,024 × the key length.
What it cost to write: 218 compile errors, and three things worth keeping
Most were field reads on values that came out of the collection, so four helpers —
object,objects_iter,objects_values,objects_range— put the map's own shapes back and took 218 to13.
lenandis_emptydid not change at all.1. A structural sweep needs its exclusions derived, not remembered. The accesses are mostly
multi-line (
.objectson one line,.get(on the next), so line anchors miss them and a regexis the only practical tool. Mine rewrote three fields that are not this collection:
ObjectListPage,ObjectTagPageandObjectMutationRollbackeach have anobjectsof theirown. The compiler caught them because a
Vechas noobjects_iter; the dangerous version is asweep whose over-reach still compiles.
2. A helper borrows all of
self, where a field borrows one field. The snapshot install loopreads every object and inserts into
self.content_typesin the same body.self.objects.iter()let the borrow checker split those; a method call does not, so that loop reads the field directly
and says why.
3. There is no
get_mut, and there should not be. Changing an object in place could changewhere it belongs in a collection ordered by its elements. The two callers — one in the install
path, one in a test — take it out, change it, and file it back. The install path uses
replacerather than
file_object, because that funnel re-interns the content type and would undo theassignment the loop exists to make.
The regression I introduced, and what caught it
file_objectbegan withlet object_id = self.filed_object_key(object_id);. That helper doestwo things: it adopts the identity the collection already holds, and it interns the bucket
name from the store's own set, which every object in a bucket names through one allocation. I
reimplemented the first inline and lost the second, so every object began owning its own copy of
the bucket name.
The total still moved the right way and the whole suite passed. What showed it was the
published per-term figure:
objects.mapkey_heap 42 → 64. A decomposition that only has to sumwould not have caught it, because it still summed. Read a helper's body before reimplementing it
—
filed_object_keynames one of the two things it does.objects.mapkey_inlinenow reports zero rather than being dropped, for the same reason therecords side kept its counters: it is what the document names, and a change that reintroduced a
second copy of the identity would move it.
A stale constant in a measurement I wrote an hour ago
measure_the_metadata_per_object_at_the_published_countsprinted what the line predicts at eachpublished count — from a slope and fixed term copied into the test. The slope moved twice
today, so those rows were stale while still looking deliberate. It measures what has to be
measured and says that the rest follow from the published line, whose own guard checks that line
against the tree.
The figures, each re-measured
Measured
cargo build --all-targets --lockedand--all-features,RUSTFLAGS=-D warnings— greencargo doc --no-deps --lockedand--all-features,RUSTDOCFLAGS=-D warnings— green🤖 Generated with Claude Code