[vangel76]

Fixes #1138

Cause

MAX_DECODE_PIXELS (2^28) bounded one decoded image only. composite() kept the ZIP output (want * rowbytes * h), one Vec<f32> per channel and the result Vec<[f32; 4]> alive together, layer_pixels() the channel planes, alpha, mask and output. A 16384 x 16384 header with a small ZIP stream of zeros therefore asked for roughly 10-14 GiB (an uncatchable abort), and on wasm32 vec![[0f32; 4]; n] panics with a capacity overflow for n >= 2^27. The smart-object bake size came from the file: a quad corner of 1e6 gave needs_bake and a 16384 x 16384 (4 GiB) bake.

Change

• crates/psd/src/lib.rs: a Budget (4 GiB; 1 GiB under target_pointer_width = "32") that every buffer of one decode is charged against. composite() and layer_pixels() first check their whole expected peak (result at 16 bytes a pixel, CMYK black plate, the channel or ZIP buffer, the mask) and return Error::Unsupported("... over the N MiB memory budget") before allocating anything. Allocation goes through try_reserve_exact, not vec![..; n]. The ZIP limit is the size the channels need.
• Channels are written straight into the result (no Vec<f32> per channel, no second result; audit item O4). The CMYK black plate is the only extra float plane.
• crates/psd/src/placed.rs: SmartObject::bake_scale(w, h, k, doc) lowers k so a bake is at most 4x the document across either way and at most MAX_BAKE_PIXELS (2^24); render_placed returns None for an output over the cap and reserves with try_reserve_exact. crates/engine/src/psd_import.rs uses bake_scale. Interpretation of "4x the document": 4x per side, not 4x area.
• Over-budget decodes reach the user through the existing MediaError::Decode mapping in media/layered.rs, an ordinary decode error.

Tests (crates/psd/src/memory_tests.rs, placed.rs)

• Equality against the previous decoder, kept in the test module as a reference: composite and every layer (canvas and own bounds, partly off-canvas layer, masks incl. inverted/default colour) for 8/16-bit x RGB/Gray/CMYK x RLE/raw; composite for 32-bit float raw and for ZIP with and without prediction (8/16-bit) per mode. All bit-identical.
• A 16384 x 16384 header with a small ZIP stream is refused with the real default budget, for compression 0, 1, 2 and 3, without allocating.
• The budget counts the sum: limits between the result size and result + channel size fail; exact peak passes; layer path likewise.
• An absurd quad corner (1e4, 1e6, 1e12) stays within 4x the document and 2^24 pixels; a normal quad keeps its previous scale and size; render_placed refuses a 16384 x 16384 output.

The budget and bake tests use new APIs (composite_with_budget, bake_scale) and do not compile on main; I did not run the huge-header test against the old code (it would allocate GiBs).

Commands run

• cargo test -p effectcraft-psd (22 passed), cargo test -p effectcraft-engine --lib (546 passed), cargo test -p effectcraft-media, cargo test -p effectcraft-script, cargo test -p effectcraft-host --test vector_import
• cargo clippy --workspace --all-targets -- -D warnings, cargo fmt --all --check, cargo xtask layers
• cargo check --target wasm32-unknown-unknown -p effectcraft-psd -p effectcraft-media -p effectcraft-engine

Not run: cargo xtask ci as a whole, the full workspace test suite (including ui-egui), real Photoshop files from the corpus.

Deliberately left out

• No consumer-side size pre-check (engine commands/file.rs, footage_reload.rs, media/layered.rs): those files are touched by feat(import): import an EffectCraft project into the open project (file.importProject) #1104/Refresh layered composition sources on import #1087/#1070: a fresh import or opened project reads overwritten footage again #1086/feat(aep): import After Effects projects through an external py-aep converter #860/perf: faster effects, expressions, text, caches, movie reading, project open, export and UI playback #856; the library now refuses on its own.
• The bake's per-triangle fill loop can still take long for a degenerate quad (time, not memory).
• A 16384 x 16384 composite no longer decodes (the result alone is the whole 4 GiB budget), so the largest decodable image is somewhat under 2^28 pixels. That is the cost of a sum budget; the budget constant is one line.
• The embedded non-PSD image path in media/layered.rs (load_oriented) is unchanged.

Expect small conflicts in crates/psd/src/lib.rs with the parallel PSD PRs (#1137, #1134, #1131, #1129); I will merge upstream/main after they land.

🤖 Generated with Claude Code

[echelon]

Merged. Thank you for the contribution, @vangel76! It landed on main together with a batch of other community PRs, after a full workspace build and test run. If you are not on our Discord yet, please join us: https://discord.gg/artcraft