[RixinLiu]

Problem

When a kvcached pool cannot allocate a prefill batch, kvcached frees the batch's Mamba slots and puts its requests back in the waiting queue (#547). By then SGLang 0.5.20 has already started the batch's HiCache loads from the host (ready_to_load_host_cache() runs before prepare_for_extend), and the forward that would have waited for them never runs. So:

• a freed Mamba slot can be given to another request while a load is still writing into it;
• a later request can match the nodes being loaded and read them before their data arrives.

THIS IS A PROBLEM BY READING CODE. NO REAL REPRODUCTION.

Change

For native HiCache, unwind synchronizes the batch's layer_done_counter.events[consumer].finish_event before releasing admission locks or Mamba slots. This covers its MHA/Mamba transfers and retains their forward-stream dependency, without synchronizing unrelated device streams. Batches without loads do not wait. External linker mode is outside this change's scope.

Validation

• At a74ee23: 78 targeted tests passed, including consumer -1/0/2, event selection, and wait-before-release ordering. The load cases fail on main.
• CPU suite: 1591 passed, 15 skipped. The extra skip caused by running as root passed separately as non-root (1 passed).
• GitHub CI: 9/9 jobs passed; Python 3.10-3.13 each reported 1592 passed, 14 skipped on the same source tree.
• Ruff, isort and MyPy passed. Private Codex review exited 0 with no actionable findings for native SGLang 0.5.20.
• No real GPU capacity-miss/HiCache reproduction or performance measurements.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

[rishabhsinha17]

LGTM, the hazard is real at 0.5.20 and the wait sits where it has to.

start_loading runs inside ready_to_load_host_cache before prepare_for_extend (scheduler.py:4012-4018) and submit_host_to_device enqueues every layer's copy on host_to_device_stream (l2_transfer.py:74-118), so at the unwind the copies are already queued and a device sync covers them. the mamba load-back writes into req.kv.mamba_pool_idx (unified_cache/components/mamba.py:674-679, 752-755), the slot the unwind frees, and a requeued batch gets hicache_consumer_index -1 from an empty load queue, so nothing else orders its kernels behind the copies.

the sync comes before dec_lock_ref and the mamba free, after the tp all-reduce with no collective until the next check_hicache_events, and is a no-op at -1. a cpu probe recording the sync, the dec_lock_ref and the free in order fails on main like the new test, 34 passed in the file, cpu manifest 1509 passed with the failure set identical to clean main. lint and ci-exact mypy are clean, gpu coverage stays on your side.

[rishabhsinha17]

optional, layer_done_counter.events[batch.hicache_consumer_index].finish_event.synchronize() on the cache controller would wait for this load alone instead of the whole device (with overlap scheduling the device sync also waits for the forward in flight), a miss is rare so i am fine either way. one limit for the record, the unified cache external linker sets the same index but loads on its own thread (mooncake_direct_linker.py:177), which no device sync orders, and kvcached does not run that path.

[shipiyouniao]

LGTM for native HiCache. The loads are already queued before allocation can fail, so this wait closes the gap before unwind releases the Mamba slots and admission locks.

[shipiyouniao]

Updated the unwind to wait for the batch's HiCache finish_event, with regressions for event selection and wait-before-release ordering. Targeted/full CPU tests, static checks and CI passed; private Codex review found no actionable issues. I'll merge using the existing approvals.