Repository navigation
Description
macOS unified-memory OOM during YuE2 ABC/music generation (rc=137 after 88GB compressor balloon)
Hardware: Mac Studio M1 Ultra 192GB unified memory
OS: macOS 15.7 (Sequoia)
ComfyUI: v0.42.0 (commit 1a82e6d, master pulled 2026-10-22)
Regression range: Known-good d52a7a5 (Oct-20) � Broken 1a82e6d (Oct-22)
Symptom
Nightly YuE2 song generation (ABC�Music pipeline) crashes with rc=137 (SIGKILL from jetsam) after ballooning to 88GB compressed RAM during the music sampling stage at ABC_CAP=4096. ComfyUI process vanishes mid-render; prompt disappears from /queue and /history.
Memory pressure trace (1Hz sampling):
23:31:32 memfree=97% compressedMB=50 � fresh restart, models loading
23:40:40 memfree=21% compressedMB=89302 � attempt 1 peaks, jetsam imminent
23:41:11 memfree=16% compressedMB=96643 � stuck 2min before crash
23:42:02 memfree=53% compressedMB=2392 � restart, prompt resubmitted
23:53:14 memfree=21% compressedMB=88406 � attempt 2 same pattern
23:54:30 memfree=58% compressedMB=224 � crash, service down
Error log:
[23:42:25] â� ï¸� prompt b67cacf1 vanished (comfy bounce mid-render) â�� resubmitting (attempt 2)
[23:55:15] � NIGHTLY SONG FAILED: prompt 49b886a1 vanished from /queue AND /history � ComfyUI restarted mid-render (rc=137)
Workflow (simplified)
{
"1": {"class_type": "YuE2TextEncode", "inputs": {"clip_name": "qwen3vl_8b_int8_convrot.safetensors", ...}},
"9": {"class_type": "LoraLoader", "inputs": {"model": ["1", 0], "clip": ["1", 1], "lora_name": "ar_lora_inst_v3abc_comfyui.safetensors", ...}},
"2": {"class_type": "YuE2GenerateABC", "inputs": {"clip": ["9", 1], "max_abc_tokens": 4096, ...}},
"3": {"class_type": "YuE2GenerateMusic", "inputs": {"clip": ["9", 1], "abc": ["2", 0], "steps": 6000, ...}},
"10": {"class_type": "VAEDecode", "inputs": {"samples": ["3", 0], "vae": ["1", 2]}},
"11": {"class_type": "SaveAudio", "inputs": {"audio": ["10", 0], ...}}
}The script explicitly calls POST /free with {"unload_models": true, "free_memory": true} before each prompt submit, but the ABC�Music handoff still balloons the compressor to 88GB+ within 10 minutes.
What worked before (d52a7a5, Oct-20)
Tested 8192-token ABC challenge on Oct-21: hit the same 88GB peak but survived (attempt 2 succeeded after attempt 1 hit the peak at ~7k tokens). Full 240s FLAC delivered.
What broke (d52a7a5..1a82e6d, 10 commits)
The diff between the two SHAs shows no changes to comfy/text_encoders/llama.py, comfy/memory_efficiency.py, or comfy/model_prefetch.py. The commits that landed:
1a82e6d Fix Gemma4 31B global attention: k_eq_v with 4 KV heads (#17220)
d2c1a79 Add model integration instructions to AGENTS.md (#17279)
46aad33 Update workflow templates to v0.11.94 (#17268)
5841c18 [Partner Nodes] feat(Mirelo): add Mirelo Video to SFX...
faa7e49 [Partner Nodes] feat(VEED): add Fabric 1.0, Lipsync 2.0...
a50102b [Partner Nodes] feat(Krea): add ultra thinking mode...
d80736d [Partner Nodes] fix(HeyGen): update voice price badge
7a28332 Load the optional local agent through comfy-agent's v1 host (#17241)
f5c611e Add Pad Image node with mirror padding and pad to multiple (#17224)
37ff76b Set minimum Pillow version in requirements (#17249)
Most suspicious: #17220 (Gemma4 attention fix) and #17241 (local agent host integration) � neither obviously related to YuE2, but one of them reintroduced the memory leak.
Hypothesis
This looks like the same class of bug as #17175 / PR #17180 (stale prefetch queue cleanup after MTP interruption), but manifesting differently on macOS unified memory:
- On CUDA: the bug was
TypeError: cannot unpack non-iterable MTPHead objectincleanup_prefetch_queues(), which failed subsequent nodes with an exception but didn't crash the process. - On macOS MPS: the cleanup either doesn't run at all (different code path?) or fails silently, leaving pinned models + staging buffers + KV cache alive. The compressor balloons to 88GB and jetsam SIGKILLs the process.
Or #17220's Gemma4 fix touched shared attention.k_eq_v logic that YuE2/Qwen3-VL also uses, introducing a subtle reference-cycle that prevents model unload on MPS.
Request
- Is this the same root cause as PR Fix MTPHead unpack crash in prefetch queue cleanup (CORE-644) #17180 (
cleanup_prefetch_queuesfailing to reset after MTP)? If so, does the fix apply to the YuE2 text-encoder�music-model handoff path on macOS? - Can we get a targeted
--disable-smart-memoryor--lowvramworkaround documented for YuE2 on macOS unified memory systems? - Will the Fix MTPHead unpack crash in prefetch queue cleanup (CORE-644) #17180 fix (if it's merged) resolve this, or does YuE2 need a separate cleanup hook in
comfy/text_encoders/llama.pyline ~122?
Current workaround
Pinned ComfyUI to d52a7a5 via .PINNED file to disable auto-update. Nightly songs resume successfully (though still balloon to 88GB and sometimes fail once before succeeding on reroll).
Happy to test candidate fixes or provide additional traces.
Activity
astro-15-alive commentedon Oct 22, 2026
linked upstream work on same failure class: #17175 (stale prefetch-queue state after partial dynamic-VRAM prefetch failure) and PR #17180 (fixes MTPHead unpack crash inside
cleanup_prefetch_queues). our macOS evidence suggests the same root cause surfaces as jetsam SIGKILL (rc=137) rather than Python exception, because unified memory lets compressor balloon to ~88GB before anything raises.regression window on our side:
d52a7a5(Oct-20, survived 8192-token ABC run at ~88GB peak) ->1a82e6d(Oct-22, dies at ABC_CAP=4096).git diff d52a7a5..1a82e6d -- comfy/text_encoders/llama.py comfy/memory_efficiency.pyis empty, so suspect the trigger is in shared loading/attention code touched by #17220 (Gemma4 k_eq_v 4 KV heads) or #17241 (local agent host). happy to bisect and re-test candidates against pinned nightly workflow.astro-15-alive commentedon Nov 10, 2026
Updating with a controlled rollback test on an M3 Ultra 256GB (2026-11-10).
Correction to our Oct-22 note:
d52a7a5(Oct-20) did not clear rc=137 on our box. Engine at d52a7a5+1 (1045d48, v0.42.0):Key finding — not a plain jetsam balloon. At least one kill occurred with near-zero memory pressure (10MB compressed, 69% free, RSS 710MB steady). launchd reports
last exit code = 137;~/logs/comfyui-ram.logshows no pressure ramp before that kill. Kills land in decode after ~4k tokens (80%+ of a 4-min segment), not prefill.Env already mitigated (none prevented kills):
APPLESILICON_FP8_MPS_WATERMARK=offMTLFLASHATTN_SDPA=off,MTLFLASHATTN_SHIM=off(kills were consistently >4096 tokens; completions ~2800)MPS_PREALLOC=0--reserve-vram20–30GB music lane; full model-cache flush before renderSo the >4k-token decode cliff reproduces at d52a7a5 too — predates the attention refactor and d52a7a5's window for us. Pin
1045d48held (auto-update blocked), rollback reverted post-test since it changed nothing.@thoward6446-betteryour's paged KV idea is the only direction consistent with our data — the cliff is AR-length-triggered, and neither allocator watermarks, SDPA backend, nor version pin touches it.