Repository navigation
[Doc] 0.1.4 follow-up: versions, host memory minimum, watchdog option, consistency fixes - #45
Merged
Merged
Conversation
A consistency review of the docs after the 0.1.4 release found stale versions and a few statements that no longer match the release notes or each other. - Versions: getting started, BUILDING and the DFlash2 page now describe 0.1.4 (install commands, image tag, recommended DFlash2 configuration and its KV pool: 1.37 million tokens, 2.61× at 512K with probabilistic drafting). - Host memory: the default pinned PLE table needs at least 80 GiB (smallest tested limit that passed memory stress, KV offload off), 128 GiB recommended; the experimental whole-table PLE disk option ran under 64 GiB and 32 GiB limits. README, support scope, configuration and known limits now say the same. - Positioning: four or more V100 32 GB GPUs, validated on four, in the support scope as in the README. - New: `--watchdog-config` in launch options; the 0.1.4 known issue for truncated `tool_use` on `/v1/messages`; the 0.1.4 speed statements from the release notes in performance; the first start after upgrading from 0.1.3. - Numbers: whole-table PLE cold first token 1.6–2.3 times (from the table); DFlash2 needs about 2.4 times MTP4's 21 KB per token in the CPU pool and on disk, so a 48 GiB pool holds about 1M tokens with DFlash2 (estimate); the 2.2–2.4 GB/s figure is the disk read rate inside the restore. - Links: a support matrix anchor in known limits; CONTRIBUTING explains that the large-files list is optional. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Leonccaa <166551845+Leonccaa@users.noreply.github.com>
The tested minimum is 32 GiB, with the experimental whole-table PLE disk option (0.1.4 served under a 32 GiB limit and greedy output matched the pinned path). 80 GiB is the minimum for the default pinned PLE table; 128 GiB stays recommended. README, support scope, configuration (two rows added to the memory table) and known limits now lead with the 32 GiB figure and state the speed cost of the disk option. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Leonccaa <166551845+Leonccaa@users.noreply.github.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change
A consistency review of the documentation after the 0.1.4 release (#42) found stale versions and statements that disagree with the release notes or with each other. Documentation only.
--watchdog-configin launch options (fields, defaults, where the dumps go; diagnostic only); the 0.1.4 known issue for a non-streaming/v1/messagesresponse cut inside a tool call; the 0.1.4 release-gate speed statements in performance; a note on the first start after upgrading from 0.1.3 with a kept compile cache.No effect on the support matrix itself.
Validation
All numbers come from the 0.1.4 release notes or from tables already in the docs; no new measurements. Watchdog fields and defaults were checked against
vllm/config/watchdog_config.pyandvllm/utils/watch_dog.py. pre-commit (markdownlint, typos) passes on the changed files; internal links and anchors in the changed files resolve.Attribution
The review and the edits are the work of a local model: Qwen3.8-Flash-Next (Uncensored AWQ g32) served by ReVolta 0.1.4 on our 4 × V100 server, working through the pi coding agent. It reviewed the documentation against the code and the release notes (25 findings), then made the edits. Claude Code checked every changed number against its source, corrected the host memory minimum after the owner's review and opened this PR. Commits are signed off under the DCO.
🤖 Generated with Claude Code