[mrxz]

The RAD format uses deflate compression through miniz_oxide. Seeing #433 achieve a large decompression performance uplift by using zstd, made me wonder whether this is due to the different algorithm or implementation. It seems that miniz_oxide is one of the slower implementations in the Rust ecosystem.

This PR switches to using flate2 which supports various backends, including zlib-rs. Performing some basic tests show a huge improvement in compression speed (~50% shorter), slightly smaller outputs (~2-5%) and an improvement in overall decoding time (~20%). While the result isn't byte identical, both zlib-rs and miniz_oxide are compatible with each others output.

One annoying downside of Cargo is the way features are additive throughout the dependency tree. Even without specifying the zlib-rs feature, flate2 would already switch to it as the zip dependency for sogs depends on flate2 and sets the zlib-rs feature as well. As such, I've made the exact feature for zip explicit as deflate was a shorthand for ["deflate-flate2-zlib-rs", "deflate-zopfli"]. The latter is only for compression with compression levels > 10, which spark-rs/-lib doesn't actually need.

[oscarlorentzon]

Looks good.

Could we use zlib-rs for spz as well, so that one library handles compression and decompression throughout and miniz_oxide can be dropped from spark-lib's dependencies?

[mrxz]

Could we use zlib-rs for spz as well, so that one library handles compression and decompression throughout and miniz_oxide can be dropped from spark-lib's dependencies?

Yes, wanted to do that as a follow-up to this PR. Would also allow us to get rid of the manual gzip header parsing, as that's built-in to flate2 as well.

[dmarcos]

Thanks!