[MatejKozlovsky]

Problem

On heavy 4K comps the GPU runs out of memory partway through frame 0. The compositor keeps drawing the rest of that frame's layers on a device that has already failed. Each later layer fails as well, and frame 0 only goes back to the CPU after the full comp has been walked. In the trace, frame 0 of c_effects is declined 20.4 s into the export, 10 s after the GPU retired.

Change

walk::draw_comp now checks check_health() before each layer (draw_precomp already does this). Once the device has failed, the frame returns None right away and goes to the CPU renderer as before. That is 3 lines in crates/gpu/src/walk.rs. There is no behaviour change while the GPU is healthy.

Independent of #1511 (the fallback-parallel PR); the two stack cleanly. No overlap with #1415, #1441 or #898.

Results

Vulkan, RTX 3090, 6 frames at 4K, 16 threads, H.264. Output compared as decoded framemd5.

Comp	CPU only	GPU, main	GPU + #1511	GPU, this PR	GPU, this PR + #1511
c_effects	76.5 s	113.1 s	100.7 s	100.4 s	88.4 s
d_blurglow	47.9 s	75.2 s	61.8 s	68.7 s	56.4 s

• Frame 0 decline moves from 20.4 s to 10.4 s on a cold start, the first run of a new binary while the driver shader cache is cold. On a warm start the OOM is reported 3.6 s in, and the frame is declined 0.17 s after that.
• Warm re-run of c_effects: 93.3 s with this PR, 81.4 s with this PR + Export: render in parallel once the GPU stops for good #1511.
• Decoded frames are identical to the CPU render in every run.
• Healthy GPU comp (1080p demo, 30 frames, 3 runs each): 1.97–2.01 s on main vs 1.85–2.00 s with this PR. That is within noise, and the output is identical.

Not covered

The OOM is already caught as soon as it happens: on_uncaptured_error retires the context synchronously, about 20 ms after the failing submit. It is not waiting on the readback timeout, so error scopes would not detect it any sooner. The remaining gap to CPU only (about 5 s with #1511) is the GPU work done before the OOM plus frame 0 rendering alone on the CPU. Skipping the GPU when a frame's estimated footprint does not fit would be the follow-up.

Testing

cargo fmt --check and clippy --no-deps -D warnings are clean. cargo test -p effectcraft-gpu gives 210 passed and 2 failed (tests_fx_pixel2::stylize, tests_fx_stylize::posterize_thresholds_strobe_vignette). Both also fail on unmodified main on the same software adapter, so they are not caused by this change.

[bflatastic]

Merged, thank you @MatejKozlovsky! It landed on main with a batch of community PRs, after a full workspace build and test run on Windows: fmt, clippy -D warnings, all 3,463 tests, layers, assets and wasm.