Bzip2 crate switches from C to 100% rust

Summary

Today we published bzip2 version 0.6.0, which uses our rust implementation of the bzip2 algorithm, libbz2-rs-sys, by default. The bzip2 crate is now faster and easier to cross-compile. The libbz2-rs-sys crate can also be built as a C dynamic library, if you have a C project that would benefit from these improvements. Why though? Why bother working on this algorithm from the 90s that sees very little use today? The thing is that many protocols and libraries still need to support bzip2 to be compliant with their specification, so many project still, deep down in their dependency tree, depend on bzip2. We've used our experience from zlib-rs to modernize the bzip2 implementation. We've previously written about the implementation details of libbz2-rs-sys in "Translating bzip2 with c2rust", now let's look at the benefits of this work. Improved performance Our rust implementation generally outperforms the C implementation, though there are a couple of cases where we only match C performance. We are not aware of any cases where we are substantially slower. For compression, we are a fair amount faster. For bzip2, the level indicates how much working memory is used. It doesn't influence performance by much, and for sample3.ref level 1 already allocates more memory than the file is large, so higher levels are irrelevant. namec (cpu cycles)rust (cpu cycles)Δ sample3.ref (level 1)38.51M ± 77.03K33.53M ± 90.52K-14.87% silesia-small.tar (level 1) 3.43G ± 2.06M 3.00G ± 6.31M-14.30% silesia-small.tar (level 9) 3.47G ± 4.86M 3.17G ± 4.43M- 9.66% For decompression there is a bit more of a spread, but we again see significant speedups across the board. namec (cpu cycles)rust (cpu cycles)Δ sample3.bz2 2.53M ± 30.08K 2.42M ± 8.95K- 4.48% sample1.bz2 9.63M ± 40.44K 8.86M ± 10.64K- 8.63% sample2.bz220.47M ± 55.28K19.02M ± 36.13K- 7.67% dancing-color.ps.bz287.46M ± 481.02K83.16M ± 548.86K- 5.17% re2-exhaustive.txt.bz2 1.89G ± 12.29M 1.76G ± 12.64M- 7.65% zip64support.tar.bz2 2.32G ± 12.09M ...