Repository navigation
Reduce Flate decoding cost: reserve output capacity, use zlib-rs - #655
Merged
Merged
Conversation
decode_flate grew its output from an empty Vec, so large streams were reallocated and copied as they decoded. Reserve four times the encoded length, capped at 16 MiB so a small stream that expands enormously cannot reserve memory before it produces it, and give back a reservation a low-ratio stream left mostly unused. Add a flate_decode benchmark over content-, image- and font-like streams. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Switch flate2 from miniz_oxide to its pure-Rust zlib-rs backend, built from source like the rest of the dependency tree. Under the shipped release profile it decodes the flate_decode benchmark streams 1.7x to 4.6x faster and opens the example documents 6-12% faster. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Nov 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #621.
Changes
decode_flatenow reserves 4× the encoded length up front instead of growing from an emptyVec. The reservation is capped at 16 MiB, so a small stream that expands enormously can't use it to grab memory. When a low-ratio stream leaves more than half of the reservation unused, the output is shrunk, because decoded data is kept asBytes.default-features = false, features = ["zlib-rs"]. zlib-rs is pure Rust, built from source, with no prebuilt binaries or C build. pdf-filter is flate2's only runtime user (the parser and object-collection crates use it only in tests), andminiz_oxideis no longer in pdf-filter's dependency tree.crates/pdf-filter/benches/flate_decode.rsdecodes synthetic content (2 MiB of operator text), image (3 MiB RGB gradient) and font-like (512 KiB, low ratio) streams throughpdf_filter::filter::decode.Benchmarks
Shipped release profile (opt-level "z", fat LTO), 5 interleaved rounds,
--measurement-time 5, mean of criterion's point estimates.flate_decode:A rebuilt copy of the original measured 0.983 / 0.170 / 3.570 ms, so the reservation alone (with miniz_oxide) is within the noise. With zlib-rs, removing the reservation measured 0.228 / 0.048 / 2.119 ms, so the reservation helps image-sized streams by about 2%. ×2 and ×8 multipliers were not better than ×4.
document_open:The release
web-canvaswasm grows by 20 KB (23,767,525 → 23,787,586 bytes, +0.08%).Verification
cargo testwith 859 passing, andcargo clippy --all-targets -- -D warnings. Both were run on the workspace excludingpdf-graphics-skiaandpdf-conformance, whose Skia build script downloads prebuilt binaries, which the local sandbox blocks. CI covers those crates.cargo fmt --check,cargo build --workspace(same exclusions), andcargo xtask web-canvas --profile release.🤖 Generated with Claude Code