Skip to content

feat: allocate physical KV pages in request-sized batches - #489

Draft
shipiyouniao wants to merge 3 commits into
ovg-project:mainfrom
shipiyouniao:feature/batch-physical-page-allocation
Draft

shipiyouniao wants to merge 3 commits into
ovg-project:mainfrom
shipiyouniao:feature/batch-physical-page-allocation

Conversation

@shipiyouniao

@shipiyouniao shipiyouniao commented Sep 14, 2026 •

Copy link
Copy Markdown
Collaborator

Inherits #574's tests through #455; the tests are not duplicated in this PR's own commit.

A request that needs several physical pages currently repeats the Python/native map callback for each page. This optional layer over #455 reserves resident IDs first and maps the missing IDs in one request-sized batch, then consumes fresh pages linearly.

Rebased onto the refreshed #455. Batched allocation now preserves main's logical block quota, packed allocation, shrink exclusions and lifecycle failure handling. Confirmed retained mappings keep their original batch through retry; resident IDs are restored separately on a failed fresh map. Unknown outcomes remain terminal. No old admission ticket or new fairness policy is introduced.

Dependency: #455; #488 is an independent sibling. Review this PR's own change against 2cc3240, rather than its cumulative main diff. The native extension must be rebuilt with the Python code.

Private validation at the final head: Python 3.11 CPU 1,553 passed / 15 skipped; rebuilt native extension on one GPU 104 passed / 3 two-GPU tests skipped. C++ checks, all-file pre-commit and strict MyPy targets 3.10-3.13 passed. Dedicated Codex review found no actionable regressions; its missing local PyTorch is covered by the private Linux runs. The updated head will run the Python 3.10-3.13 PR CI matrix.

Batching reduces callback count but can require a larger single grant. No new full-model throughput or multi-GPU TP/PP acceptance is claimed. Draft state remains unchanged.

@shipiyouniao
shipiyouniao force-pushed the feature/batch-physical-page-allocation branch from 4273e9a to 4c2aabc Compare October 19, 2026 05:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants