Repository navigation
Omit redundant fixed-array lengths in Cereal, YAS and Boost.Serialization - #755
Merged
liuzicheng1987 merged 1 commit intoOct 25, 2026
Conversation
Collaborator
|
@Altren thanks! |
liuzicheng1987
merged commit Oct 25, 2026
ea0c104
into
munich-data-engineering:main
176 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixed-size arrays currently carry a redundant length prefix in the Cereal, YAS and Boost.Serialization adapters, even though the extent is already part of the C++ type. Serialize std::array and C arrays without that prefix, including nested and empty fixed-size arrays. This saves one length field per fixed array (8 bytes in the 64-bit binary configurations).
Add an optional read_fixed_array adapter API and matching writer operations. Fixed and dynamic archive reads share the element-reading loop; fixed reads take their extent from the type, while dynamic reads consume their existing length prefix. Cereal and YAS input array handles remain pointer-only. Map keys, variant tags and element parsing through processors are preserved, and formats without the new API retain their existing array representation.
Compatibility: this changes the fixed-array wire format in these three adapters. Existing archives need to be read with the previous version and rewritten. Readers and writers must agree on the array type and extent; without the prefix, an extent mismatch cannot be reliably detected.
This is extremely useful for data storages like
std::vector<Eigen::Vector2d>and custom reflectors, that represent such vectors asstd::array<double, N>. Reducing stored size by 33% and improving both read and write performance by 30% for Cereal and Boost and 10% for YAS