[RFC]: FlashLoop integration — token-sparse prefill, sparse decode attention, and KV residual quantization #156

Status: RFC draft for discussion. Date: 2026-10-27.

Feedback period: at least one week after publication. Keep this issue open afterward for ownership and implementation tracking.

1. Motivation, goal and scope

Integrate FlashLoop as three independently selectable execution features:

1. Token-sparse prompt updates across recurrent loops.
2. Sparse decode attention with a cached reference contribution.
3. Packed KV storage using a quantized base and cross-loop residuals.

The existing stage, scheduler, executor and loop execution boundaries remain. Scheduler continues to choose operations, token ranges and recurrence depths. FlashLoop changes the work performed inside those operations and the physical representation of their KV. A broad runtime refactor is not a prerequisite.

The integration targets the existing legal combinations of PD, mixed scheduling, WCPB, asynchronous execution, CUDA Graphs, block speculation and token pipelines. It does not make currently unsupported combinations legal: in particular, mixed scheduling still excludes speculative work. Compatibility is a delivery target, not a claim that these combinations already work with FlashLoop.

The first model target is four-loop Ouro with BF16 compute and LAST_EXITED history. Other models require their own sparsity schedules and numerical qualification. SHARED history is outside the initial scope. All features default to disabled, preserving the current dense execution path.

FlashLoop is approximate. Quality preservation must be measured; it does not imply identical logits, greedy sequences or sampling distributions to dense Ouro. Speculative acceptance must be evaluated against a clearly defined target using the same FlashLoop configuration and cache representation rules. Strict equivalence to serial execution is optional; an execution-dependent quantization policy must be identified as approximate and evaluated separately.

2. Baseline and existing execution boundaries

The code mapping below uses runtime revision 8fbf4cd and the accompanying local pipeline integration changes reviewed on 2026-10-27. Pipeline-specific contracts describe that working tree, not a claim that every change is in the committed revision. Implementation PRs must pin their actual base and patch revision.

The relevant existing contracts are documented in unified execution, runtime configuration, WCPB and self-speculation.

Caller process
  LLM / StageClient
       | IPC
StageWorker process
  StageEngine
    Scheduler + KVCacheManager + optional PD connector
    Executor
      ordinary ModelRunner or speculative round controller
        shared ModelRuntime
          model + attention + device KV operations

StageWorker owns process/device initialization and model loading. ModelRunner prepares batches, manages device state and buffers, orders streams/events and collects results. The speculative controllers additionally own draft, verify, accept/reject and token-pipeline progression.

ModelRuntime is the thin shared model invocation layer in executor.py. It consumes ForwardBatch, executes PRELUDE, RECURRENT or CODA, and returns ForwardResult. It also invokes the selected graph execution path. It does not own admission, request queues or speculative acceptance. Ordinary and speculative runners reference the same runtime, model and cache resources.

Consecutive loops may remain in one submission. FlashLoop must work both when execution returns to Scheduler after each loop and when a submission executes several loops locally. It must not add a CPU synchronization at every loop.

3. Algorithm contracts

Loop numbers below are one-based; runtime depth indices are zero-based. Every KV version refers to the same request, token position and physical Transformer layer. Attention statistics are also specific to a query and attention head.

3.1 Token-sparse prefill

Execute loops 1 and 2 densely. Before loop r, score eligible prompt tokens using the preceding completed loop states:

score[i] = norm(H[r-1, i] - H[r-2, i]) / max(norm(H[r-1, i]), epsilon)
active[r] = top_k(score over active[r-1], budget[r])

Scores use FP32 arithmetic. Retention budgets refer to the original selection window length, not the preceding active count. Active sets are nested. Break equal-score ties by absolute token position so packing and batch order do not change the result. Preserve the old active hidden rows before overwriting them; two tensor aliases cannot provide a meaningful cross-loop change score.

For selected rows, execute every physical layer's normalization, QKV projection, attention, output projection and MLP, followed by the end-of-loop normalization. Inactive rows retain their previous final hidden state and each layer's previous KV. They do not receive another normalization. Each active query still attends to every legal causal position, including inactive tokens' inherited KV.

An illustrative eight-token window with retention budgets 4 and 2 could execute:

             t1 t2 t3 t4 t5 t6 t7 t8
loop 1        x  x  x  x  x  x  x  x
loop 2        x  x  x  x  x  x  x  x
loop 3        .  x  .  x  .  x  .  x
loop 4        .  .  .  x  .  .  .  x

At loop 3, active queries use the logical KV sequence [KV2, KV3, KV2, KV3, KV2, KV3, KV2, KV3]. The query for t4 sees t1 through t4. Compaction must preserve original positions for both RoPE and causal masking. Passing the compacted rows to a contiguous packed-prefill kernel without that mapping is incorrect. Existing per-query paged attention can serve as a reference; an optimized path must explicitly support nonconsecutive query positions.

For the integrated serving path, use fixed prompt selection windows. Each window has a stable identity and absolute boundaries independent of the temporary batch. Select by score within each window's budget, with no reserved slot for a window's final token or the final prompt token. Unselected positions retain their previous hidden state and KV. Window-local selection is an explicit variation from global prompt selection.

For window length N, candidate Ouro retention ratios are [1, 1, 0.25, 0.10]; these are calibration starting points, not validated defaults for this engine. All-off execution and 100% retention must bypass unnecessary selection work.

3.2 Sparse decode attention

Use loop 2 as the reference for loops 3 and 4, matching the reviewed standalone engine. Select keys independently for each physical layer, query position and attention head. Selection includes the current position if its probability ranks within the budget; it always excludes future positions and padding.

At the reference loop, compute full causal attention and retain:

S       selected absolute key positions
m       sum of reference probabilities on S
O_ref   full reference attention output
C_ref   reference weighted value contribution on S

At a later loop r:

p_local = softmax(Q[r] @ K[r, S].T / sqrt(head_dim))
C_new   = m * (p_local @ V[r, S])
O[r]    = O_ref - C_ref + C_new

Current-token Q/K/V are computed normally at each loop. Historical K/V are read at the requested depth, reconstructed if compressed. This feature reduces attention reads and QK/PV work; it does not eliminate the current token's KV write, projections or MLP.

For a scalar example, suppose the reference probabilities are [0.1, 0.3, 0.6] and values are [10, 20, 30]. Full output is 25. Selecting the last two keys gives m=0.9, C_ref=24, and an unselected contribution of 1. If the later selected softmax is [0.75, 0.25] and selected values are [20, 40], then:

O_new = 25 - 24 + 0.9 * (0.75 * 20 + 0.25 * 40) = 23.5

The approximation preserves the reference contribution outside S and its mass m; it does not compute the current full-softmax denominator. With all keys retained, m is 1 and the formula reduces to current dense attention, up to numerical error.

Reference state lasts only for that input token and its dependent loops. A new token establishes its own state. Loop 4 continues to use loop 2 as its reference; recursively updating selection from loop 3 is a separate algorithm variant. If execution skips the reference loop, sparse correction is unavailable and must not consume stale state. Fixed loop bounds below the sparse start remain dense.

The dense source kernel must supply selection statistics. Existing attention outputs alone cannot reconstruct key probabilities. Measure the cost of collecting statistics, top-k and cache writes; avoid permanently materializing a full attention matrix. A reference implementation may materialize probabilities.

3.3 Quantized base and cross-loop KV residuals

For X equal to K or V, store:

X_hat[1] = dequantize(quantize(X[1]))
delta[r] = X[r] - X_hat[r-1]
X_hat[r] = X_hat[r-1] + dequantize(quantize(delta[r]))

Subtract the reconstructed predecessor, not its uncompressed source value. For example, if X[1]=10.03 reconstructs as 10.00 and X[2]=10.08, the residual is 0.08. Storing the raw difference 0.05 would preserve the first round's error even if the residual were encoded exactly.

Use affine INT4 groups with stored minimum and scale. For a nonconstant group, scale=(max-min)/15, code is the rounded/clipped (x-min)/scale, and decoding is min+scale*code. Define constant groups explicitly to avoid division by zero. The reference configuration uses group size 64, FP16 quantization parameters, post-RoPE K grouped across token rows per channel, and V grouped across channels per token. BF16 remains the compute dtype. A packed storage dtype must not be mistaken for the model activation dtype.

Only active prefill tokens produce late-loop residual payload. Maintain explicit logical-position-to-compact-row mappings. An absent residual means inheritance; it is not equivalent to an INT4 code of zero, which may decode to a nonzero minimum. A logical loop-4 read can resolve to:

t1: base + delta2
t2: base + delta2 + delta3
t4: base + delta2 + delta3 + delta4

Keep logical LAST_EXITED history separate from physical storage encoding. Early-exited decode tokens inherit their final depth at deeper loops. Encoding that inheritance as absent residuals must establish the same logical validity as the existing finalization operation, with completion dependencies intact.

Optimized readers consume packed base/residual tiles directly during QK/PV, rather than constructing a full persistent BF16 shadow cache. Prefill scratch, conversion buffers and the recent BF16 tail remain part of peak memory accounting. Compacting residual payload is required for token sparsity to reduce physical capacity; merely writing zeros into fully reserved depth planes does not do so.

The reference rolling tail retains at least 64 recent tokens and flushes complete 64-token groups; it can therefore contain 64 through 127 tokens after a step. Its standalone flush timing cannot be copied unchanged into speculative or wavefront execution. Section 6 defines the additional representation constraints.

4. Local implementation changes

Extend current contracts with small named views rather than adding a new engine or a second scheduler. Proposed execution metadata carries absolute positions, depths, row modes, token-state slots and optional active-row mappings. The runner owns the tensors; the runtime and kernels borrow them for the submission.

Mixed rows can share projections/MLP where their shapes permit, while attention dispatches dense-source, sparse-correction and sparse-prefill segments. All segments restore results to their original logical rows. A backend may use separate kernels for these segments without separating their request lifecycle.

Extend backend capabilities for the operations actually required: source statistics, nonconsecutive causal queries, packed KV reads, device-owned positions and graph-safe sparse execution. Existing FA4 support does not imply support for these new operations. Explicit incompatible backend choices fail clearly; auto chooses a compatible implementation and reports the choice. Do not silently turn off a requested algorithm component.

5. State ownership and lifetime

For R live token slots, L layers, H heads and head dimension D, attention output state is logically [R,L,H,D], mass is [R,L,H], and indices are [R,L,H,K_bucket] with valid counts/masks. Use FP32 mass and accumulation; source output storage precision is a measured accuracy/memory choice. Index width must cover the supported context. Prefill maps identify window-local rows and absolute positions separately. These are GPU tensors in accelerated paths.

A token's reference progresses through empty, source-ready, reusable and retired states. Source-ready requires stream ordering or an event establishing completed writes. Batch row numbers never identify persistent state. Position alone is also insufficient after rejection: a corrected token can reuse the same position. Reset state on token incarnation changes and reject stale execution generations.

Pipeline shifts move reference state with the surviving token, just as they move hidden, depth and candidate state. Snapshot/restore includes this state whenever the token can resume after its reference loop. A snapshot of hidden and KV alone cannot resume sparse attention correctly.

6. Composition with existing execution features

6.1 WCPB and fixed selection windows

WCPB may run (chunk A, loop 3) while later chunks are at loops 1 and 2. Global prompt top-k would require waiting for those later chunks and introduce a full prompt barrier. Use fixed selection windows instead, shared by ordinary, mixed, wavefront and PD execution.

Window boundaries are anchored to absolute prompt positions, not the current batch or prefix-cache hit. The final prompt window may be shorter. Initially, require scheduler chunks to align with the configured windows and contain whole windows; reserve enough hidden rows for them. Token-budget subdivisions may execute a window over several submissions, but selection for its next sparse loop waits for that window's complete scoring input. Report incompatible geometry at configuration time. The default window size remains a discussion item.

After active KV writes and inactive inheritance are ready, the existing per-depth frontier advances over the full logical window. Physical work count and logical completion count are different. Existing hidden-ring reclamation still waits for final-depth completion and dependent readers. Add the active maps and any old hidden rows needed for scoring to that lifetime and memory budget.

This windowed algorithm requires quality comparison against global selection. Changing the selection window is an algorithm change; regrouping the same windows into runtime batches should not change their selection decisions.

6.2 Mixed execution and asynchronous submissions

Keep the current mixed PRELUDE/RECURRENT selection rules. Sparse-prefill selection is local to its request/window; decode key selection is local to its token/head. Never run a top-k over unrelated requests simply because they share a batch.

Use a decode-first, active-prefill-second execution-row order. Build gather indices by stable partition and concatenation rather than a general sort on every submission. For example, [P0,D0,P1,P2,D1,P3] with active prefill rows P1 and P3 becomes [D0,D1 | P1,P3]. The split offset identifies the attention segments; compatible projections and MLP work can still share a batch. Preserve original absolute positions, request/token-state identities and a scatter map for results. Inactive prefill rows retain their prior state. Decode rows may require further dispatch by depth or reference/correction mode. Device-owned active selections and index construction remain on the GPU.

All token state and KV operations follow existing compute/boundary/copy stream dependencies. Do not read GPU positions, active counts or acceptance decisions back to the CPU to prepare the next asynchronous operation. Retain bounded buffers through ticket completion; CPU delivery of an older ticket cannot reset state already used by a newer submission.

6.3 Block speculation and token pipelines

With draft depth 2, drafting must retain loop-2 attention references for every verification input position as well as hidden and shallow KV. Verification then uses those references at loops 3 and 4. With draft depth 1, the verifier creates the reference when it executes loop 2. Classify verification rows as decode queries even when they are packed together; do not apply prompt token selection to them. Every row obeys its own causal length and source identity.

For a token pipeline, a request can simultaneously contain tokens at several depths. Reserve one reference slot per live token, not one per request. A rejected ancestor invalidates descendant references, candidate KV and pending writes; surviving slots shift together. Reuse is ordered after the last reader/writer. Existing legal preemption points remain; snapshots include surviving references and encoding state. Existing block/pipeline sampling and scheduling restrictions remain unless separately implemented and qualified.

Quantization adds a numerical rollback choice. An unverified candidate can change a K group's min/max and therefore the reconstructed values of accepted tokens. Removing the candidate does not undo that error. Reversible/private groups can preserve accepted values; retaining the quantization effect is an additional approximation to measure under Section 6.4. Rejected logical positions and stale pending writes must still be invalidated in either case.

6.4 Quantization timing and the quality/performance tradeoff

WCPB and speculation share the same quantization-timing problem. WCPB can read shallow KV while other work advances deeper windows; block verification can evaluate several positions before committing them. Encoding at different times can make these executions read different reconstructed values on the same prefix. Strict serial numerical equivalence is not a default integration requirement.

Memory safety remains mandatory: do not overwrite storage while GPU operations or transfers still use it. Order in-place conversion after those operations, or encode into separate storage and retire the old allocation after its final use. Readers must see a complete payload with matching parameters, maps and valid base/residual dependencies. This does not require every logical cache reference to disappear before encoding.

Evaluate two choices rather than mandating a version protocol:

• Execution-dependent encoding: convert at safe points in actual execution, without keeping extra versions solely to match serial execution. Measure additional numerical drift, task quality, speculative acceptance and runtime gains across batch sizes and schedules. This mode does not claim distributional equivalence to the originally defined serial target.
• Serial-aligned encoding: define representation visibility per query, splitting execution at encoding boundaries or retaining versions when necessary. Measure the additional storage, synchronization and lost batching against the stronger equivalence target; it is not assumed to be the preferred serving mode.

Different loops need not share quantization parameters. The choice concerns when readers switch representations, not whether KV may be quantized. Dependencies between encoded bases and residuals must remain well-defined in either mode.

A fixed completed-prompt encoding with generated KV kept in BF16 provides a simple comparison baseline. It saves no generated-history bytes and does not establish a lower prefill peak; reserve its full generation budget. Rolling and streaming encoding are candidates to evaluate against it, not blocked on strict serial equivalence. Include temporary BF16 storage and conversion overlap in peak memory accounting, and choose the policy from quality, throughput, latency and memory measurements.

6.5 PD handoff

Preserve stage roles: P produces prompt KV and final hidden; D performs CODA and local generation/speculation. Require an agreed model, logical layout, feature configuration and versioned encoding descriptor. The descriptor identifies bit width, group axes/sizes, depth representation, window policy and tail policy.

Extend NIXL registration and range descriptions to include base/residual payload, scales/minima, compact-position mappings, validity/inheritance metadata, BF16 tail and final hidden. Destination allocations bind logical entries to local addresses; source physical pointers are not portable cache identities. Reserve the required bytes for all regions before transfer.

Each exported range must have complete dependencies at the advertised depths. Unsealed groups can travel as an explicitly described BF16 region. Pin or version exported buffers until transport completion so tail flushes cannot mutate an in-flight source. D activates only after the required ranges and metadata are received and device-visible. Keep transfer generation, sequence, acknowledgement and cancellation leases; remote writes must terminate before destination reuse.

Ordinary prompt handoff does not transfer per-generated-token attention references: D creates those when it starts forwarding generated tokens. Migrating an already running speculative pipeline would require a richer snapshot and is outside this prompt-handoff protocol.

6.6 Prefix reuse and recovery

Prefix identity includes the model and algorithm/encoding configuration. Publish only completed selection windows whose token choices and stored representations are independent of a later suffix. A partially shared window cannot reuse its old selection: another suffix can change top-k even with causal attention. Quantization groups and inherited base dependencies must also be closed and immutable; complete physical pages alone are not sufficient evidence.

Snapshots include packed data, maps, encoding epochs, mutable tail and any live execution references. For WCPB, retain the existing rewind-to-completed-frontier policy and recompute incomplete windows deterministically. For pipeline recovery, restore surviving token state exactly. Preserve RNG and exit-policy state through the existing owners. Reject unsupported snapshot formats rather than restoring only dense fields and continuing with incomplete state.

6.7 CUDA Graphs and adaptive exit

Graphs need stable-address token/reference storage, bounded selection buffers, valid lengths and masked inactive rows. Extend graph keys for execution mode, encoding and capacity buckets where they change kernel structure. Capturing a specific batch's mask or source identity as a constant is invalid. An eager fallback may preserve the requested algorithm for unsupported shapes, with a diagnostic; it must not silently switch to dense model semantics.

Adaptive exit keeps the existing gate production and consumption timing. Sparse execution and quantization can change gate values and exit depths, so qualify their combined quality and latency separately. Requests exiting before loop 3 receive no late-loop attention savings, although source-statistics overhead may already have been paid. Measure that cost. Fixed-depth traces isolate execution savings from changes in exit behavior.

7. Configuration and resource accounting

Organize configuration into three independent feature groups. Each has a standard enabled switch and parameters belonging to that feature. All three switches default to false and do not require each other. These are proposed groups and fields, not an existing API; final names follow repository configuration conventions.

7.1 Token-sparse prefill: token_sparse_prefill

This group controls prompt token selection only. Rank within the window without reserving final-token slots. Validate the window against existing chunk settings as in Section 6.1, rather than introducing conflicting chunking rules.

7.2 Sparse decode attention: sparse_decode_attention

The initial algorithm uses the loop-2 reference for loops 3 and 4, as specified in Section 3.2. This is an algorithm contract, not an additional switch for unimplemented reference-loop or selection variants.

7.3 KV residual quantization: kv_residual_quantization

tail_size applies to rolling policies; fixed-prompt encoding leaves generated KV in BF16. Accepted policies and values must match implemented capabilities; candidate choices in this table are not claims of existing support.

7.4 Shared validation and resource accounting

Keep logical layout=last_exited separate from physical KV encoding, which is determined by the quantization group rather than another duplicate master switch. Resolve one immutable effective configuration at stage initialization, validate P/D agreement, and include it in diagnostics, snapshots, prefix identity and benchmark records. Do not expose an independent configuration system per runner.

The allocator must budget payload, quantization parameters, maps, inherited references, mutable tails, live token references, graphs and conversion scratch. Speculation increases live token/reference count beyond request count. Small residual streams may require partially filled groups; account for their physical padding rather than assuming ideal retention ratios. Resource queries should report actual format costs without making Scheduler inspect packed tensors.

As a payload-only illustration, four BF16 depth versions cost 4N * 16 bits per scalar channel. Retention [1,1,0.25,0.10] with INT4 costs 2.35N * 4 before metadata and tails, a ratio of about 6.81. This is not an end-to-end memory or speed claim. In particular, generated tokens may update every depth.

8. Feature integration matrix

Organize implementation and acceptance by the three FlashLoop features crossed with existing runtime capabilities, without prescribing sequential increments. Deliver the work in three implementation PRs, one per FlashLoop feature. Each PR covers its feature's core implementation and the applicable integration work in its column. Matrix cells divide collaboration and validation work within those PRs; they are not separate PR deliverables. Contributors are encouraged to collaborate on each feature PR because its integration scope spans many areas. Each cell names integration or validation work; all are pending targets, not claims of implemented support. Existing combination restrictions remain: listing mixed and speculation separately does not make their combination supported.

Also validate interactions among the three features: sparse attention reads the logical KV produced by token sparsity; token sparsity plus quantization stores compact residuals only for active tokens; sparse attention plus quantization reconstructs selected keys directly. Cover all eight switch combinations, including the all-off existing baseline and all-on configuration, under the existing legal execution modes above.

Implementation shares the existing execution stack, with explicit capability checks for unsupported combinations. A working kernel alone does not complete its integration cells. Apply Section 9 acceptance criteria; dense physical copies of inherited KV in a reference cannot substantiate packed-memory savings.

9. Validation and acceptance

Algorithm and runtime correctness

Compare implementation against an explicit FlashLoop reference on fixed inputs, then compare FlashLoop quality against dense Ouro. These answer different questions. Use declared BF16/FP32 tolerances rather than assuming cross-batch bitwise identity.

• Token selection: nested sets, budget rounding, stable ties, unchanged inactive hidden, nonconsecutive causal positions, and inherited state when a window's final token or the final prompt token is not selected.
• Attention: scalar worked example, multiple heads, inclusion of the current token, reference-loop identity, full retention and missing/stale references.
• KV: constant groups, metadata precision, compact maps, inactive inheritance, partial groups, tail rollover and bounded direct-reader error.
• Composition: fixed windows executed atomically, split, mixed and as WCPB; several requests and depths; padding; prefix hits at legal boundaries.
• Speculation: accepted prefixes, rejection at each candidate, same-position replacement, multiple outstanding rounds, pipeline shifts, and quantization boundaries crossed by a verification batch. Check each mode against a reference implementing its declared encoding policy. Compare verifier rows with serial execution on the same prefix: require numerical agreement for serial-aligned encoding, and measure the divergence for execution-dependent encoding.
• Lifecycle: EOS, cancellation, late completion, request-ID reuse, memory pressure, preemption/resume and CUDA Graph buffer reuse.
• PD: payload and metadata parity, prompt hidden parity, partial transfers, producer flush during transfer, cancellation and failed destination activation.

Extend the existing suites rather than creating a parallel runtime test harness: mixed execution, split prefill, KV, prefix growth, speculation, async speculation, async lifecycle, graphs and PD.

Quality and performance

Measure all three components independently and cumulatively. Start with four-loop Ouro, fixed prompt/output sets and dense execution using the same runtime optimizations. Include short and long contexts, low and high concurrency, and fixed-arrival mixed workloads. Compare PD with the same GPU count and P/D ratio. Record window size, retention schedule, encoding policy, backend and actual execution mode. Keep generated length/exit-policy changes visible.

Report prefill latency, TTFT, decode ITL/TPOT, throughput, tail latency, failed requests and peak allocated/reserved memory. Separate steady cache bytes from conversion peak, reference state, graph storage and transient buffers. For speculation, include acceptance rate and complete draft/verify/accept cost. For PD, include transferred bytes, handoff latency and temporary storage.

Follow the performance workflow: repeated unprofiled measurements first, then separate ops-only and full profiles. Attribute gains and overhead to selection, packing, dequantization, projection sizes, kernel launches, synchronization and transport. A reduction in FLOPs or payload bytes alone does not establish a runtime improvement.

Use held-out task evaluation and long-context checks, including perplexity and retrieval-sensitive cases. Report uncertainty and per-task changes; average scores can hide a regression. Establish acceptable quality changes before promoting a configuration to a recommended recipe. Publication speedups from FlashLoop are context for investigation, not acceptance targets for vllm-rlt.

10. Decisions for discussion

1. Selection-window default. Choose the initial size and scheduler alignment constraints from quality and occupancy measurements. Keep its semantics fixed across execution modes and include the window in prefix identity.
2. Rolling encoding policy. Compare execution-dependent and serial-aligned encoding, including the treatment of quantization effects from rejected candidates. Choose from measured quality, acceptance, throughput and memory; strict serial equivalence is optional. Fixed-prompt encoding is a baseline.
3. Quality budget. Agree on per-task degradation limits and uncertainty before selecting retention ratios, state precision and a deployment default.

References

• FlashLoop paper, especially Sections 4 and Appendix D.
• FlashLoop engine, reference-loop selection and decode correction.
• FlashLoop prefill and selection, compact active rows and nested budgets.
• FlashLoop cache and packing, reconstructed residuals and physical formats.
• Chipmunk, related cached sparse-delta attention and MLP computation across diffusion steps. Similarity of the update structure does not make its normalization or execution assumptions interchangeable with FlashLoop.

Source implementations guide the algorithm reference; vllm-rlt retains its own request, batching and resource lifetimes. Preserve applicable upstream notices when incorporating source code.

11. Ownership and progress

Use the cross-feature table and assignee columns from RFC #69, with the status markers from RFC #43. Track new FlashLoop implementation/integration, not completion of existing capabilities themselves. Section 8 defines each cell's scope.

Allocation: coordination comment. Shared support: @MinhaoLi0318 covers adaptive-exit evaluation across all three features; @0z5a helps with review and SM90 performance validation. KV quantization coordination, remaining storage/readers and unassigned integration cells remain open.

The implementation plan has exactly three feature PRs:

Collaboration is encouraged within each PR. Contributors can take different integration cells, contribute commits to the shared feature branch/PR, and review and validate each other's changes. Record the feature's coordinator(s) and collaborators above, and the contributors for each adaptation below. Coordinate shared interfaces across the three PRs; cross-feature fixes and combination validation belong in the relevant feature PRs rather than a separate integration PR per cell. PR numbering here does not impose a merge order.

• ⬜ Pending implementation; Claimed identifies assigned work without a reported implementation. 🚧 Implementation in progress or partially integrated; ✅ merged and validated for the stated scope.
• Claim a cell, feature column or integration row as collaborative work within the corresponding feature PR(s). Multiple contributors may share a cell; assignees from other RFCs are not assigned automatically.
• Link each feature PR once in the table above. Cells track adaptation progress, contributors and validation evidence; they do not require separate issues or PRs. Completion includes the supported scope and evidence.
• Include combined quality, performance and memory validation in the relevant claims. Standalone completion does not qualify every combination. Unassigned work is shown as —. Assignments refreshed on 2026-10-28; claims do not imply completed implementation or validation.