Skip to content

GPU: resident effect stacks, in-frame texture reuse, upload-cache budget, stable footprint keys (#687) - #688

Merged
h-a-n-a-s-h-i merged 5 commits into
storytold:mainfrom
MatejKozlovsky:gpu-resident-effects
Oct 13, 2026
Merged

h-a-n-a-s-h-i merged 5 commits into
storytold:mainfrom
MatejKozlovsky:gpu-resident-effects

Conversation

@MatejKozlovsky

Copy link
Copy Markdown
Contributor

Fixes #687.

The overall series vs this PR. This is PR 2 of a series that reworks GPU export. The full series,
measured on a test PC (Ryzen 7 5800X, RTX 3090 24 GB, Windows 10) with the same 10 s 4K comp: 0.7.0
stock --gpu 575.1 s vs the full series 33.5 s (~17.2×). This PR alone delivers much less; its
numbers are below. The batch-parallel export that keeps frames off the GPU on main is addressed by a
later PR (the scheduler); here exports are measured with RAYON_NUM_THREADS=1 so frames stay on the GPU.

Changes (4 commits, one change each)

  1. Render: an adjustment layer's footprint key ignores its effects (the footprint is the source
    through the masks), so a time-dependent effect such as Noise no longer re-keys it every frame.
  2. GPU: an Enc reuses the working textures it released at once. Its later commands run after
    everything it recorded before (one queue, recording order), so it may overwrite them; other encoders
    only get them after the Enc is gone, through the existing pool.
  3. GPU: GpuContext::set_upload_budget / upload_budget (default unchanged, 1536 MB), and one
    warning when the upload cache is full and starts evicting layer textures.
  4. GPU: walk::draw_resident_fx: a plain layer's effect stack runs resident on the GPU (GpuFx, in
    effect space, as 0.7.0 defines it, then rebased to layer space) and is placed directly; CPU-only effects
    in the stack read back, run and upload as before. A layer whose processed buffer is cached and unchanged
    since the last frame (Renderer::layer_is_static) keeps the usual path, so it uploads once instead of
    re-running its effects every frame. Layers with layer styles, Preserve Transparency or a track matte
    keep the usual path.

Numbers (measured, test PC, main 2ec87e9 vs main + this PR)

4K comp with effect stacks, 3 s / 90 frames, --gpu, RAYON_NUM_THREADS=1:

wall fps GPU util avg/max VRAM peak
main, CPU (reference) 173.3 s 0.52 – –
main --gpu 81.1 s 1.11 29% / 100% 17,185 MiB
main + this PR --gpu 49.3 s 1.83 (1.65×) 18% / 95% 9,376 MiB

Both GPU exports differ from the CPU export (MD5), so the frames really came from the GPU. They are
bit-identical to each other (PSNR ∞) and both 50.9 dB average PSNR vs CPU (Y 49.7 dB). No GPU-declined or
upload-cache-full lines in either log.

effectcraft-cli bench --gpu, same comp, median of 5 (ms/frame):

main this PR
4K GPU cold 1181.0 491.2
4K GPU warm 208.1 128.4
4K GPU view 172.4 64.1
1080p GPU cold 325.6 177.9
1080p GPU warm 51.3 48.9
1080p GPU view 35.4 22.1

GPU vs CPU: 0.000% of pixels over 1/255 on both. Upload traffic on this comp is 0.0 MB/frame warm on both
(nothing in it re-keys), so the footprint-key and upload-budget changes don't show here; their tests
measure them. An earlier version of this PR re-ran static effect stacks every frame and was slower at
1080p warm (83.7 ms vs 51.8 ms); commit 4's layer_is_static check fixed that (48.9 ms above).

Tests

  • effect_stacks_render_resident: GPU vs CPU parity, 8 and 32 bpc, full and half resolution: GPU effects
    with a CPU-only effect in the middle, opacity, a blend mode, rotation, and a shape layer (effect space
    from its bounds' corner).
  • static_effect_layers_draw_from_the_cache: a static layer with effects uploads nothing after it is
    cached, frames repeat exactly, matches the CPU.
  • texture_reuse_does_not_change_output: a busy frame is bit-identical with in-Enc reuse on and off,
    and matches the CPU. enc_reuses_its_own_released_textures: the Enc gets its own texture back at once;
    another Enc does not.
  • adjustment_footprint_uploads_once_under_time_dependent_effects: the footprint key is equal at two
    times (no GPU needed); on the GPU, frames 3 and 4 upload 0 bytes (before the fix: 64,000 bytes, the
    footprint, every frame); output still changes every frame and matches the CPU.
  • upload_budget_bounds_the_layer_textures_kept: within the budget static layers upload once; under a
    1 MB budget they upload again; the frame is the same.
  • All existing GPU parity suites now go through the resident path and pass.

CI

Rust 1.95, on top of main 2ec87e9:

  • cargo fmt --check: clean. xtask layers, assets (39 assets), wasm (32 crates): ok.
  • Workspace tests: all pass except 2 that also fail on unmodified main (same pixels): effectcraft-gpu
    tests_fx_pixel2::stylize (cchextile, 2 of 770 px) and
    tests_fx_stylize::posterize_thresholds_strobe_vignette (ccthresholdrgb, 13 of 5917 px), on a software
    Vulkan adapter.
  • clippy -D warnings: clean for this PR's code. Upstream errors on unmodified main, not touched here:
    collapsible_match in crates/psd/src/lib.rs:660, crates/text/src/path_text.rs:36,
    crates/effects/src/audio_fx.rs:249, crates/effects/src/generate3.rs:507, and nonminimal_bool in
    crates/ui-egui/src/panels/text_panels.rs:856.
  • CPU export (no --gpu) of the 4K comp is byte-identical between main and this PR.

Limitations

  • Exports on main still render a batch of frames in parallel, which keeps many frames off the GPU; that
    is the next PR (the scheduler). The export numbers above use RAYON_NUM_THREADS=1 for that reason.
  • Layer styles, Preserve Transparency and track mattes keep the existing path for effect layers.
  • The upload budget is an API; picking it from the device's memory is left to the embedder (a later PR).

… time-dependent effect (Noise) no longer re-keys it and the GPU re-uploads it every frame (storytold#687)
…f allocating new ones for every intermediate of a frame (storytold#687)
…en full, instead of evicting layer textures silently (storytold#687)
…rectly, instead of a readback and re-upload around every GPU effect run (storytold#687)
@MatejKozlovsky

Copy link
Copy Markdown
Contributor Author

Merged current main into this branch to resolve the conflict; build and tests pass locally. #688 and #898 are both clean now and independent. If it helps, #688 then #898 is a sensible merge order.

MatejKozlovsky added a commit to MatejKozlovsky/effectcraft that referenced this pull request Oct 13, 2026
…o PRs merge without a duplicate field

Both this PR and storytold#688 add `GpuContext::upload_budget`. Here the field and its
initialiser sat in different places than in storytold#688, so git merged both copies
and the result failed with E0124/E0062; `set_pool_budget` and the `Pool`
budget field also overlapped storytold#688's hunks and conflicted.

The upload-budget field, its initialiser, `set_upload_budget`/`upload_budget`,
the `UPLOAD_BUDGET` doc and the check in `upload_buf` now match storytold#688 line for
line, including storytold#688's one-time "upload cache full" warning, so either merge
order applies them once. `set_pool_budget` moves next to `check` and the
`Pool::budget` field above `bytes`, away from storytold#688's changes. Budgets and
eviction behave as before.
@h-a-n-a-s-h-i
h-a-n-a-s-h-i merged commit 9aabb5d into storytold:main Oct 13, 2026
@echelon

echelon commented Oct 13, 2026

Copy link
Copy Markdown
Member

Merged. Thank you for the contribution, @MatejKozlovsky! It landed on main together with a batch of other community PRs, after a full workspace build and test run. If you are not on our Discord yet, please join us: https://discord.gg/artcraft

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants