Skip to content

render: opt-in reuse-only input cache (--exp-input-cache) - #1421

Draft
MatejKozlovsky wants to merge 2 commits into
storytold:mainfrom
MatejKozlovsky:input-cache
Draft

MatejKozlovsky wants to merge 2 commits into
storytold:mainfrom
MatejKozlovsky:input-cache

Conversation

@MatejKozlovsky

@MatejKozlovsky MatejKozlovsky commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

What

An opt-in mode for the layer cache (--exp-input-cache, or EFFECTCRAFT_EXP_INPUT_CACHE=1), off by default:

  • Store at once while under budget. While the cache has room, a mode-on run stores results
    exactly like the default cache, so projects that fit their budget behave as they do on main.
  • Reuse-only admission under pressure. Once a new result would go over the budget, it's stored
    only when it's rendered a second time, so one-off results never push reused ones out.
  • Cropped storage, full-extent hits. Under pressure, a stored layer is cropped to its non-zero
    pixels (plus a 2 px margin), but only when the crop is at most a quarter of the frame. Marginal
    crops cost a scan plus a full-extent copy on each hit for little memory. Hits are served at the
    full extent and origin, with the original buffer's offset restored bit for bit (not recomputed
    from the crop's), so output is byte-identical to an uncached render.
  • One full-extent copy for hot entries. A hot cropped entry keeps one full-extent copy (one
    copy and one GPU upload, not one per frame), but only within the budget: other entries' copies
    are dropped, least recently used first, until the new copy fits; if it still doesn't, the hit
    goes without one.
  • Cycle-aware eviction. Entries reused every frame are kept over a cycle bigger than the
    budget, instead of being LRU-evicted just before they're needed.

Based directly on main. It doesn't depend on #898 or any other open PR. The switch is a small
static in render::cache that the CLI sets (the CLI also reads EFFECTCRAFT_EXP_INPUT_CACHE; the
render crate reads no environment), so there's no new settings plumbing. EFFECTCRAFT_CACHE_STATS=1
also reports how many inserts were stored cropped.

Results

GPU render (--gpu), RTX 3090, 30 frames, H.264, benchmark resolution (4K samples at half), layer
cache budget 1024 MB (the default), 3 runs each, interleaved:

Sample main --exp-input-cache
b (text) 50.3 / 50.5 / 50.4 s 15.4 / 15.6 / 15.4 s (3.3× faster)
d (blur/glow) 34.6 / 34.3 / 34.3 s 5.7 / 5.8 / 5.8 s (6.0× faster)
f (Singularity) 8.0 / 8.0 / 7.9 s 8.2 / 8.0 / 8.3 s (about +2%, close to main's run-to-run spread)
c (effects) 4.0 / 4.0 / 4.0 s 4.0 / 4.0 / 3.9 s (same)

The gain is on projects whose reused layers don't all fit the budget (b, d). An earlier version
was slower on c and f. Profiling showed two causes:

  • On c, which fits the budget, reuse-only admission rendered every layer twice, and every hit made a
    full-extent copy of a cropped entry. Results are now stored at once while under budget.
  • On f, which just reaches the budget, cropping layers that were only slightly smaller than the
    frame cost the scan plus per-hit copies. The crop threshold went from 75% to 25% of the frame.
    With crop thresholds of 25% and 50%, b and d stay at 15 s and 5.5 s. With cropping off, they
    drop back to about 17.8 s and 7.6 s.

Output

Byte-identical to main (per-frame MD5, 15 PNG frames each) on samples b, c, d and f and on three
small tailored projects (a single blur/glow layer, a single 10-effect layer, and a 4K full-stack
layer):

  • CPU-only renders: all identical.
  • GPU renders: all identical. The 30-frame H.264 benchmark outputs above are identical too.

Earlier run-to-run differences seen on b/f came from #898's adaptive CPU workers. They render a
timing-dependent set of frames on the CPU, and CPU and GPU differ by up to 2 levels. That has
nothing to do with this change, and this branch (on main) has no such scheduler.

Testing

  • cargo fmt --all
  • cargo clippy -p effectcraft-render -p effectcraft-cli --all-targets -- -D warnings: clean.
  • cargo test -p effectcraft-render: 221 passed, 0 failed, 1 ignored.
  • cropped_entries_come_back_bit_for_bit_with_fractional_offsets: cropped entries under pressure
    come back with the same pixels and bit-identical offsets (fractional, -0.0, large). It fails with
    the old offset - x0 + x0 restore.
  • reuse_only_cache_under_pressure_crops_and_matches_uncached: a render under a tight budget with
    small masked patches at fractional positions; asserts some entries were stored cropped and every
    frame is byte-identical to an uncached render.
  • New/updated tests: reuse_only_admits_at_once_within_the_budget_and_on_reuse_over_it, reuse_only_cache_matches_uncached_with_expressions_and_time_effects.

…layer cache budget

Layer results are admitted on their second render, stored cropped to their pixels (64-px grid) and served at their full extent and origin, so output is byte-identical to an uncached render. A hot cropped entry keeps one full-extent copy (one copy and one GPU upload, not one per frame) only within the budget; eviction keeps entries reused every frame over a cycle bigger than the budget. Off by default; --exp-input-cache or EFFECTCRAFT_EXP_INPUT_CACHE=1 turns it on.
… expansions until the new copy fits, and the env switch is read by the CLI
@MatejKozlovsky

Copy link
Copy Markdown
Contributor Author

Parking this as a draft for now. It's the one change with a measured 3–6× on the text and blur/glow samples, but those are the same samples #1441 and #1523 target, and I want to re-measure it on top of them before asking for review. Known fixes before un-drafting: the Drop stats print (and its env read) moves out of the render crate, and the crop scan compares bits so −0.0 pixels survive (today only the offset is restored exactly). If the gain on top of #1441/#1523 is small, I'll close it.

@MatejKozlovsky
MatejKozlovsky marked this pull request as draft October 17, 2026 19:55

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant