Skip to content

[Doc] 0.1.4: current release, PLE and KV disk-tier measurements - #42

Merged
Leonccaa merged 2 commits into
mainfrom
docs/0.1.4-measurements
Oct 27, 2026
Merged

Leonccaa merged 2 commits into
mainfrom
docs/0.1.4-measurements

Conversation

@Leonccaa

Copy link
Copy Markdown
Owner

Change

Updates the README and guides for 0.1.4 and adds the measurements taken after the release. No code changes.

  • README: 0.1.4 is the current release (install commands and image tag). The capacity table uses the recommended probabilistic DFlash2 settings: 1.37 million tokens (2.61× at 512K); the 1.38 million / 2.63× it showed before is the greedy-drafting value. A cached 32K-token document now comes back from CPU memory in 0.6 s, and from a local NVMe disk tier after an engine restart in 1.5 s, against 10.1 s recomputed. The experimental whole-table PLE disk option is described with its measured cost. "Next" lists the 0.1.5 plans. Thanks to the authors of NVLink results (V100-SXM2-32GB NVLink quad, TP4+DCP2, DFlash2) for 0.1.3, from a 9-model serving bench #38 and Nvfp4 Test Results #37.
  • Positioning: ReVolta is built for 4 × V100 32 GB; NVLink and one PCIe switch with P2P are the recommended and tested interconnects (README, docs/known-limits.md, .github/SUPPORT.md and the bug report form).
  • docs/known-limits.md: the whole-table PLE disk option measured on 0.1.4 on local NVMe and SATA SSDs, under 64 and 32 GiB limits, with and without a 32 GiB CPU KV pool: single-request decoding about 23–25% slower, a cold first token 1.7–2.2 times as long, a warm first token within 2–3% of the pinned path (5–8% with the CPU KV pool), and greedy output identical to the pinned path.
  • docs/kv-offload.md: with DFlash2, the drafter's cache block size differs from the target's, so the CPU pool and the disk tier give every cache group pages of the full block size: about 50 KB per token, about 2.5 times MTP4. The sizing rule for disk restores now gives both values. Adds the local NVMe results after a restart.
  • docs/performance.md: a 0.1.4 readback table (recomputed, from CPU memory, from NVMe after a restart) at 32K, 99K and 131K tokens; the earlier measurement stays below it.

Support matrix: NVLink changes from "community results, outside our release gate" to "recommended; tested with community results" in the README and .github/SUPPORT.md. Nothing else in the matrix changes.

Validation

  • Readback numbers: 4 × V100 32 GB PCIe, 0.1.4 with DFlash2 (recommended settings), Uncensored AWQ, E4M3 KV, 250 W, pinned PLE table, 7 GiB CPU pool and a 64 GiB disk budget on a local NVMe SSD (ext4). The page cache was verified empty before the restart (1,088 files, 27.8 GB, 0 resident). 11 of 11 documents were read back with the whole cacheable prefix and the same 64-token greedy output as the cold run.
  • DFlash2 page size: vllm/v1/kv_offload/cpu/spec.py uses the non-partitioned layout when the cacheable groups have different GPU block sizes. The disk pages written per document match it: 72 pages of 25.6 MB (1.84 GB) for 32K tokens, 249 pages (6.36 GB) for 131K.
  • PLE disk numbers: six arms in one window on 0.1.4 (pinned baseline; NVMe and SATA under 64 GiB; NVMe under 32 GiB; NVMe and SATA under 64 GiB with a 32 GiB CPU KV pool), 250 W, three 64-token greedy prompts identical to the baseline in every arm.
  • Capacity: 0.1.4 release gate logs (GPU KV cache size: 1,366,121 tokens, 2.61x with probabilistic DFlash2; MTP4 unchanged at 1,153,143).
  • pre-commit run --files on the six changed files (pre-commit 4.7.0): all hooks pass, including typos and markdownlint-cli2. The bug report form parses as YAML.

Attribution

AI-assisted: drafted with Claude Code. The commit is signed off under the Developer Certificate of Origin. A human review of every changed line is pending before merge.

🤖 Generated with Claude Code

Leonccaa and others added 2 commits October 27, 2026 11:30
Update the README and guides for 0.1.4 and add the measurements taken
after its release:

- README: 0.1.4 is the current release (install commands, image tag,
  capacity table with the recommended probabilistic DFlash2 pool of
  1.37M tokens / 2.61x), readback times from CPU memory and from a
  local NVMe disk tier after a restart, the measured cost of the
  experimental whole-table PLE disk option, 0.1.5 plans, and thanks
  to the authors of #38 and #37.
- Positioning: ReVolta is built for 4 x V100 32 GB; NVLink and one
  PCIe switch with P2P are the recommended and tested interconnects
  (README, known-limits, SUPPORT.md, bug report form).
- known-limits: whole-table PLE disk option measured on 0.1.4 (local
  NVMe and SATA, 64 and 32 GiB limits, with and without a 32 GiB CPU
  KV pool).
- kv-offload: DFlash2 needs about 50 KB per token in the CPU pool and
  on disk (about 2.5x MTP4); local NVMe results after a restart.
- performance: 0.1.4 readback table (recomputed, CPU memory, NVMe
  after a restart).

No code changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Leonccaa <166551845+Leonccaa@users.noreply.github.com>
The positioning keeps the scope of four or more GPUs, validated on
four, and adds the recommended and tested interconnects (NVLink, or
all GPUs under one PCIe switch with P2P).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Leonccaa <166551845+Leonccaa@users.noreply.github.com>
@Leonccaa
Leonccaa merged commit 39d8d3b into main Oct 27, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant