Skip to content

macOS unified-memory OOM during YuE2 ABC/music generation (rc=137 after 88GB compressor balloon) #17305

Open

Description

@astro-15-alive

macOS unified-memory OOM during YuE2 ABC/music generation (rc=137 after 88GB compressor balloon)

Hardware: Mac Studio M1 Ultra 192GB unified memory
OS: macOS 15.7 (Sequoia)
ComfyUI: v0.42.0 (commit 1a82e6d, master pulled 2026-10-22)
Regression range: Known-good d52a7a5 (Oct-20) � Broken 1a82e6d (Oct-22)

Symptom

Nightly YuE2 song generation (ABC�Music pipeline) crashes with rc=137 (SIGKILL from jetsam) after ballooning to 88GB compressed RAM during the music sampling stage at ABC_CAP=4096. ComfyUI process vanishes mid-render; prompt disappears from /queue and /history.

Memory pressure trace (1Hz sampling):

23:31:32 memfree=97% compressedMB=50     � fresh restart, models loading
23:40:40 memfree=21% compressedMB=89302  � attempt 1 peaks, jetsam imminent  
23:41:11 memfree=16% compressedMB=96643  � stuck 2min before crash
23:42:02 memfree=53% compressedMB=2392   � restart, prompt resubmitted
23:53:14 memfree=21% compressedMB=88406  � attempt 2 same pattern
23:54:30 memfree=58% compressedMB=224    � crash, service down

Error log:

[23:42:25] �� prompt b67cacf1 vanished (comfy bounce mid-render) � resubmitting (attempt 2)
[23:55:15] � NIGHTLY SONG FAILED: prompt 49b886a1 vanished from /queue AND /history � ComfyUI restarted mid-render (rc=137)

Workflow (simplified)

{
  "1": {"class_type": "YuE2TextEncode", "inputs": {"clip_name": "qwen3vl_8b_int8_convrot.safetensors", ...}},
  "9": {"class_type": "LoraLoader", "inputs": {"model": ["1", 0], "clip": ["1", 1], "lora_name": "ar_lora_inst_v3abc_comfyui.safetensors", ...}},
  "2": {"class_type": "YuE2GenerateABC", "inputs": {"clip": ["9", 1], "max_abc_tokens": 4096, ...}},
  "3": {"class_type": "YuE2GenerateMusic", "inputs": {"clip": ["9", 1], "abc": ["2", 0], "steps": 6000, ...}},
  "10": {"class_type": "VAEDecode", "inputs": {"samples": ["3", 0], "vae": ["1", 2]}},
  "11": {"class_type": "SaveAudio", "inputs": {"audio": ["10", 0], ...}}
}

The script explicitly calls POST /free with {"unload_models": true, "free_memory": true} before each prompt submit, but the ABC�Music handoff still balloons the compressor to 88GB+ within 10 minutes.

What worked before (d52a7a5, Oct-20)

Tested 8192-token ABC challenge on Oct-21: hit the same 88GB peak but survived (attempt 2 succeeded after attempt 1 hit the peak at ~7k tokens). Full 240s FLAC delivered.

What broke (d52a7a5..1a82e6d, 10 commits)

The diff between the two SHAs shows no changes to comfy/text_encoders/llama.py, comfy/memory_efficiency.py, or comfy/model_prefetch.py. The commits that landed:

1a82e6d Fix Gemma4 31B global attention: k_eq_v with 4 KV heads (#17220)
d2c1a79 Add model integration instructions to AGENTS.md (#17279)
46aad33 Update workflow templates to v0.11.94 (#17268)
5841c18 [Partner Nodes] feat(Mirelo): add Mirelo Video to SFX...
faa7e49 [Partner Nodes] feat(VEED): add Fabric 1.0, Lipsync 2.0...
a50102b [Partner Nodes] feat(Krea): add ultra thinking mode...
d80736d [Partner Nodes] fix(HeyGen): update voice price badge
7a28332 Load the optional local agent through comfy-agent's v1 host (#17241)
f5c611e Add Pad Image node with mirror padding and pad to multiple (#17224)
37ff76b Set minimum Pillow version in requirements (#17249)

Most suspicious: #17220 (Gemma4 attention fix) and #17241 (local agent host integration) � neither obviously related to YuE2, but one of them reintroduced the memory leak.

Hypothesis

This looks like the same class of bug as #17175 / PR #17180 (stale prefetch queue cleanup after MTP interruption), but manifesting differently on macOS unified memory:

  • On CUDA: the bug was TypeError: cannot unpack non-iterable MTPHead object in cleanup_prefetch_queues(), which failed subsequent nodes with an exception but didn't crash the process.
  • On macOS MPS: the cleanup either doesn't run at all (different code path?) or fails silently, leaving pinned models + staging buffers + KV cache alive. The compressor balloons to 88GB and jetsam SIGKILLs the process.

Or #17220's Gemma4 fix touched shared attention.k_eq_v logic that YuE2/Qwen3-VL also uses, introducing a subtle reference-cycle that prevents model unload on MPS.

Request

  1. Is this the same root cause as PR Fix MTPHead unpack crash in prefetch queue cleanup (CORE-644) #17180 (cleanup_prefetch_queues failing to reset after MTP)? If so, does the fix apply to the YuE2 text-encoderâ��music-model handoff path on macOS?
  2. Can we get a targeted --disable-smart-memory or --lowvram workaround documented for YuE2 on macOS unified memory systems?
  3. Will the Fix MTPHead unpack crash in prefetch queue cleanup (CORE-644) #17180 fix (if it's merged) resolve this, or does YuE2 need a separate cleanup hook in comfy/text_encoders/llama.py line ~122?

Current workaround

Pinned ComfyUI to d52a7a5 via .PINNED file to disable auto-update. Nightly songs resume successfully (though still balloon to 88GB and sometimes fail once before succeeding on reroll).

Happy to test candidate fixes or provide additional traces.

Activity

  1. astro-15-alive commented on Oct 22, 2026

    @astro-15-alive
    Author

    linked upstream work on same failure class: #17175 (stale prefetch-queue state after partial dynamic-VRAM prefetch failure) and PR #17180 (fixes MTPHead unpack crash inside cleanup_prefetch_queues). our macOS evidence suggests the same root cause surfaces as jetsam SIGKILL (rc=137) rather than Python exception, because unified memory lets compressor balloon to ~88GB before anything raises.

    regression window on our side: d52a7a5 (Oct-20, survived 8192-token ABC run at ~88GB peak) -> 1a82e6d (Oct-22, dies at ABC_CAP=4096). git diff d52a7a5..1a82e6d -- comfy/text_encoders/llama.py comfy/memory_efficiency.py is empty, so suspect the trigger is in shared loading/attention code touched by #17220 (Gemma4 k_eq_v 4 KV heads) or #17241 (local agent host). happy to bisect and re-test candidates against pinned nightly workflow.

  2. astro-15-alive commented on Nov 10, 2026

    @astro-15-alive
    Author

    Updating with a controlled rollback test on an M3 Ultra 256GB (2026-11-10).

    Correction to our Oct-22 note: d52a7a5 (Oct-20) did not clear rc=137 on our box. Engine at d52a7a5+1 (1045d48, v0.42.0):

    • d52a7a5+1 (head): one full song tonight, dying on 4/6 segment attempts — seg1 ×2 bounces (completed try3 at 588s), seg2 clean, seg3 ×2 bounces (died at 900s timeout, ~80% sampling).
    • d52a7a5 (pin): seg1 completed on try 2 (634s); seg2 then died on 3/3 attempts at 80–83% music sampling. Compressor balloon to 149GB / memfree 36% on one death, but a later kill at 83% had compressed=10MB, memfree=69%.

    Key finding — not a plain jetsam balloon. At least one kill occurred with near-zero memory pressure (10MB compressed, 69% free, RSS 710MB steady). launchd reports last exit code = 137; ~/logs/comfyui-ram.log shows no pressure ramp before that kill. Kills land in decode after ~4k tokens (80%+ of a 4-min segment), not prefill.

    Env already mitigated (none prevented kills):

    • APPLESILICON_FP8_MPS_WATERMARK=off
    • MTLFLASHATTN_SDPA=off, MTLFLASHATTN_SHIM=off (kills were consistently >4096 tokens; completions ~2800)
    • MPS_PREALLOC=0
    • --reserve-vram 20–30GB music lane; full model-cache flush before render
    • oMLX/27B neighbors evicted (memfree 91–98% at start)

    So the >4k-token decode cliff reproduces at d52a7a5 too — predates the attention refactor and d52a7a5's window for us. Pin 1045d48 held (auto-update blocked), rollback reverted post-test since it changed nothing.

    @thoward6446-betteryour's paged KV idea is the only direction consistent with our data — the cliff is AR-length-triggered, and neither allocator watermarks, SDPA backend, nor version pin touches it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions