Skip to content

GPU: stop 4K effect frames from running out of video memory (and log GPU declines in the CLI) - #1415

Closed
MatejKozlovsky wants to merge 3 commits into
storytold:mainfrom
MatejKozlovsky:gpu-oom-submit
Closed

MatejKozlovsky wants to merge 3 commits into
storytold:mainfrom
MatejKozlovsky:gpu-oom-submit

Conversation

@MatejKozlovsky

@MatejKozlovsky MatejKozlovsky commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Problem

With GPU acceleration on, the 4K benchmark samples c (effects stack) and d (blur/glow) rendered at
CPU speed, byte-identical to a CPU render, and printed nothing.

Frame 0 ran out of video memory part-way through the layer stack (on an RTX 3090, 24 GB). The
"Out of Memory" error retires the GPU context, so every later frame and every effect falls back to
the CPU for the rest of the export.

Why memory runs out: all layers in these samples are static, so each layer's effect run uses its
own Enc (upload → kernels → readback), while the frame's own encoder stays open for the whole
frame. A texture released while any encoder is open is parked until that encoder is submitted
(it may still be read by recorded commands). The frame's encoder only submits every 256
dispatches, and a frame of placements doesn't reach that, so every effect run's 4K f32
intermediates stay parked. After about 20 blurred/glowing 4K layers, the device is out of memory.

The CLI installed no logger, so the "GPU declined the frame" warning was never shown.

Fix

  1. gpu: submit once parked and new textures pass 1 GiB (crates/gpu/src/context.rs)
    • The pool tracks the bytes of parked textures.
    • Each Enc tracks the bytes of textures it newly allocated since its last submit.
    • Before recording more work, an encoder submits when either 256 dispatches are pending (as
      before) or its new bytes plus the parked bytes reach 1 GiB, so parked textures go back to the pool.
    • Parked bytes count only for the oldest open encoder (usually the frame's). A parked texture is
      freed only when every encoder open at its release has been submitted, so submitting a newer one
      (an effect chain or Advanced 3D pass inside the frame) frees nothing. Newer encoders count only
      their own allocations and don't submit on every step.
    • The parked bytes and the oldest open encoder are published as atomics under the pool lock, so a
      dispatch reads them without taking the lock.
    • Counting only the encoder's own allocations was not enough. The parked textures are what
      build up.
  2. cli: log warnings to stderr (RUST_LOG overrides)
    • effectcraft-cli installs the engine's logger at startup with default filter warn.
    • engine::logging gains install_with(default); install() keeps the desktop default.
    • A GPU that declines frames or runs out of memory now says so.

Results

4K (full resolution), --gpu, H.264 output, single runs.

RTX 3090 (24 GB), 10 frames:

Sample Before After
c (effects) 166.0 s (CPU fallback) 65.0 s
d (blur/glow) 115.2 s (CPU fallback) 76.4 s

RTX 3090, sample b (text), 15 frames, two runs each:

Run 1 Run 2
Before 214.7 s (CPU fallback) 213.8 s (CPU fallback)
After 179.0 s 185.9 s

GTX 1070 (8 GB), 3 frames:

Sample Before After
c (effects) 110.6 s (CPU fallback) 32.3 s
d (blur/glow) 78.5 s (CPU fallback) 44.8 s
  • The GPU really renders now. With the fix, process CPU use drops from about 55–65% to about
    5% on the RTX 3090, and from 321–639 CPU-seconds to 15–28 on the GTX 1070.
  • The 1 GiB threshold works on 8 GB as well. wgpu doesn't report device memory, so the
    threshold isn't scaled by adapter memory; it is fixed and documented at PENDING_BYTES. It is small next to typical VRAM (8 parked 4K f32 textures) and only adds a
    submit when that much is parked.
  • Output differs from main where main fell back to the CPU, because those frames now render
    on the GPU. Each configuration is byte-identical from run to run.
  • Samples b and f at the benchmark resolution (1080p-class) are unchanged. Neither runs out of
    memory there, before or after. Output is byte-identical between main and this branch, and the
    fix is slightly faster (b 26.4 → 25.3 s, f 4.77 → 4.47 s).

After the oldest-encoder change (4K, --gpu, same samples), the output is byte-identical to
the previous revision and no out-of-memory or decline warnings appear:

Previous revision This revision
RTX 3090, c, 10 frames 66.9 / 62.3 s 71.5 / 63.5 s
RTX 3090, d, 10 frames 72.1 s 72.2 s
RTX 3090, b, 15 frames 179.0 / 186.6 s 183.0 / 186.7 s
GTX 1070, c, 3 frames 32.3 s 31.2 s
GTX 1070, d, 3 frames 37.8 / 36.3 s 45.9 / 37.4 s

These differences are within run-to-run noise (the order of the runs was swapped for the repeats).

Overlap

Validation

  • cargo clippy -p effectcraft-gpu --all-targets -- -D warnings: clean.
  • cargo test -p effectcraft-gpu: 212 passed, 1 ignored.
  • cargo clippy -p effectcraft-cli -p effectcraft-engine --lib --bins -- -D warnings: clean.
  • cargo fmt --all.
  • CLI logging: silent by default on a clean render; RUST_LOG=info shows info records.

@bflatastic

Copy link
Copy Markdown
Member

Thanks @MatejKozlovsky! This and #1523 (with #1550 on top) attack the same 4K VRAM exhaustion in context.rs, and they conflict. Which approach do you want to keep? Your call, as the author of both. Also, the CLI logging::install_with("warn") change is independent and could land on its own as a small PR, and released_bytes += / pending_bytes + parked should use saturating_add. This one also needs a rebase onto main (CLI main.rs, after #1357).

@MatejKozlovsky

Copy link
Copy Markdown
Contributor Author

@bflatastic thanks! Makes sense, I'll go with #1523 + #1550 and close this one with a pointer there. The install_with("warn") CLI bit is split out into its own small PR (#1651), rebased on top of #1357, and I'll move the saturating_add bits into #1523 where they apply.

@MatejKozlovsky

Copy link
Copy Markdown
Contributor Author

Closing this in favour of #1523 (with #1550 on top), which tackle the same 4K VRAM exhaustion in context.rs and conflict with this one (see your comment above). The CLI logging change from here now lives in #1651.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants