Repository navigation
GPU: stop 4K effect frames from running out of video memory (and log GPU declines in the CLI) - #1415
Closed
MatejKozlovsky wants to merge 3 commits into
Closed
MatejKozlovsky wants to merge 3 commits into
MatejKozlovsky wants to merge 3 commits into
Conversation
…mes don't run out of memory
…e whose submit frees them), read lock-free
This was referenced Oct 14, 2026
Member
|
Thanks @MatejKozlovsky! This and #1523 (with #1550 on top) attack the same 4K VRAM exhaustion in |
Contributor
Author
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
With GPU acceleration on, the 4K benchmark samples c (effects stack) and d (blur/glow) rendered at
CPU speed, byte-identical to a CPU render, and printed nothing.
Frame 0 ran out of video memory part-way through the layer stack (on an RTX 3090, 24 GB). The
"Out of Memory" error retires the GPU context, so every later frame and every effect falls back to
the CPU for the rest of the export.
Why memory runs out: all layers in these samples are static, so each layer's effect run uses its
own
Enc(upload → kernels → readback), while the frame's own encoder stays open for the wholeframe. A texture released while any encoder is open is parked until that encoder is submitted
(it may still be read by recorded commands). The frame's encoder only submits every 256
dispatches, and a frame of placements doesn't reach that, so every effect run's 4K f32
intermediates stay parked. After about 20 blurred/glowing 4K layers, the device is out of memory.
The CLI installed no logger, so the "GPU declined the frame" warning was never shown.
Fix
gpu: submit once parked and new textures pass 1 GiB(crates/gpu/src/context.rs)Enctracks the bytes of textures it newly allocated since its last submit.before) or its new bytes plus the parked bytes reach 1 GiB, so parked textures go back to the pool.
freed only when every encoder open at its release has been submitted, so submitting a newer one
(an effect chain or Advanced 3D pass inside the frame) frees nothing. Newer encoders count only
their own allocations and don't submit on every step.
dispatch reads them without taking the lock.
build up.
cli: log warnings to stderr (RUST_LOG overrides)effectcraft-cliinstalls the engine's logger at startup with default filterwarn.engine::logginggainsinstall_with(default);install()keeps the desktop default.Results
4K (full resolution),
--gpu, H.264 output, single runs.RTX 3090 (24 GB), 10 frames:
RTX 3090, sample b (text), 15 frames, two runs each:
GTX 1070 (8 GB), 3 frames:
5% on the RTX 3090, and from 321–639 CPU-seconds to 15–28 on the GTX 1070.
threshold isn't scaled by adapter memory; it is fixed and documented at
PENDING_BYTES. It is small next to typical VRAM (8 parked 4K f32 textures) and only adds asubmit when that much is parked.
on the GPU. Each configuration is byte-identical from run to run.
memory there, before or after. Output is byte-identical between main and this branch, and the
fix is slightly faster (b 26.4 → 25.3 s, f 4.77 → 4.47 s).
After the oldest-encoder change (4K,
--gpu, same samples), the output is byte-identical tothe previous revision and no out-of-memory or decline warnings appear:
These differences are within run-to-run noise (the order of the runs was swapped for the repeats).
Overlap
the context) and pool/VRAM budget knobs. That is complementary: Export: stream frames through a shared CPU/GPU work queue, convert to YUV 4:2:0 on the GPU, and tune to the hardware #898 recovers after running out,
this change avoids running out in the first place. Both touch
crates/gpu/src/context.rs(
Pool,Enc), so a small textual conflict is possible; the logic is independent.Export: stream frames through a shared CPU/GPU work queue, convert to YUV 4:2:0 on the GPU, and tune to the hardware #898 also adds CLI options in
main.rs, which may conflict textually with the logger line.main.rs.bound the memory used inside one frame, which is what ran out here.
Validation
cargo clippy -p effectcraft-gpu --all-targets -- -D warnings: clean.cargo test -p effectcraft-gpu: 212 passed, 1 ignored.cargo clippy -p effectcraft-cli -p effectcraft-engine --lib --bins -- -D warnings: clean.cargo fmt --all.RUST_LOG=infoshows info records.