Repository navigation
feat: allocate physical KV pages in request-sized batches - #489
Draft
shipiyouniao wants to merge 3 commits into
Draft
shipiyouniao wants to merge 3 commits into
shipiyouniao wants to merge 3 commits into
Conversation
shipiyouniao
force-pushed
the
feature/batch-physical-page-allocation
branch
from
September 14, 2026 16:55
8a6a42b to
41d9de0
Compare
This was referenced Sep 22, 2026
shipiyouniao
force-pushed
the
feature/batch-physical-page-allocation
branch
from
September 24, 2026 11:26
41d9de0 to
a7c44ce
Compare
Closed
11 of 20 tasks
RixinLiu
self-requested a review
October 18, 2026 04:13
shipiyouniao
force-pushed
the
feature/batch-physical-page-allocation
branch
from
October 19, 2026 05:33
4273e9a to
4c2aabc
Compare
Co-authored-by: 石皮幼鸟 <2960474346@qq.com>
shipiyouniao
force-pushed
the
feature/batch-physical-page-allocation
branch
from
October 19, 2026 07:33
4c2aabc to
f76da25
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Inherits #574's tests through #455; the tests are not duplicated in this PR's own commit.
A request that needs several physical pages currently repeats the Python/native map callback for each page. This optional layer over #455 reserves resident IDs first and maps the missing IDs in one request-sized batch, then consumes fresh pages linearly.
Rebased onto the refreshed #455. Batched allocation now preserves main's logical block quota, packed allocation, shrink exclusions and lifecycle failure handling. Confirmed retained mappings keep their original batch through retry; resident IDs are restored separately on a failed fresh map. Unknown outcomes remain terminal. No old admission ticket or new fairness policy is introduced.
Dependency: #455; #488 is an independent sibling. Review this PR's own change against 2cc3240, rather than its cumulative main diff. The native extension must be rebuilt with the Python code.
Private validation at the final head: Python 3.11 CPU 1,553 passed / 15 skipped; rebuilt native extension on one GPU 104 passed / 3 two-GPU tests skipped. C++ checks, all-file pre-commit and strict MyPy targets 3.10-3.13 passed. Dedicated Codex review found no actionable regressions; its missing local PyTorch is covered by the private Linux runs. The updated head will run the Python 3.10-3.13 PR CI matrix.
Batching reduces callback count but can require a larger single grant. No new full-model throughput or multi-GPU TP/PP acceptance is claimed. Draft state remains unchanged.