About ReVolta: why a separate repository, how it is developed, and what changed since 1Cat-vLLM v1.5.1 #1

This pinned issue is an informal overview of ReVolta, updated from time to time (last update: 2026-10-30). It covers:

1. Where ReVolta stands now and what is next.
2. How ReVolta is developed, and how to follow it.
3. Why ReVolta is a separate repository instead of a branch of 1Cat-vLLM.
4. What ReVolta 0.1.0 changed compared with 1Cat-vLLM v1.5.1, and what each release since then added.

Questions and corrections are welcome in the comments. Bug reports should go in separate issues, using the bug report form.

Current status (2026-10-30)

• Latest release: 0.1.5. Install with pip install revolta==0.1.5, or use the Docker image ghcr.io/leonccaa/revolta:0.1.5.
• New in 0.1.5: with DFlash2, the first reuse of a prompt hits the full cached prefix ([Bugfix][KV Cache] DFlash2: first prefix reuse hits the full cacheable prefix #48); the model loader reads index shards in subfolders and fails loudly when an indexed shard would stay unread ([Bugfix][Loader] Read index shards in subfolders; direct I/O U16/U32 #55, fixes [Bug][Loader] Index shards in subfolders are silently skipped; direct I/O rejects U16/U32 #54); large-integer tool arguments stay exact ([Bugfix][Tool Parser] Keep large integer arguments exact for number parameters #47); the startup report says not_applicable instead of contract_mismatch for deployments that never use the DFlash2 verifier ([Misc] SM70 accelerator report: say not_applicable for other models and drafting methods #43, reported in NVLink results (V100-SXM2-32GB NVLink quad, TP4+DCP2, DFlash2) for 0.1.3, from a 9-model serving bench #38). With the optional DFlash2 state replay on, the replay covers only the accepted prefix and single-request decoding is about 0.9% faster ([Perf] DFlash2 state replay: replay only the accepted prefix #44); on NVLink the MoE sum takes the PCIe-switch route ([Perf][Distributed] SM70 NVLink TP4: route sum2 as under a PCIe switch #49). The release gate added a 512K retrieval self-test (12 of 12 for DFlash2 and MTP4, NVLink results (V100-SXM2-32GB NVLink quad, TP4+DCP2, DFlash2) for 0.1.3, from a 9-model serving bench #38). Our own server runs 0.1.5 with DFlash2, probabilistic drafting and state replay off. Details: 0.1.5 release notes.
• Known issues in 0.1.5: CPU KV offload can leave /dev/shm/vllm_offload_*.mmap behind on a normal stop; start with --shutdown-timeout 30 or remove stale files before starting (CPU KV offload leaves /dev/shm/vllm_offload_*.mmap behind after a normal shutdown #51). On /v1/messages, a non-streaming response cut by max_tokens inside a tool call returns no tool_use block. DFlash2 remains workload-dependent (see Current gaps).
• 0.1.4 fixed an OffloadingConnector assertion exit with DFlash2 and CPU KV offload under long-context traffic ([Bugfix][KV Cache] Two-phase allocation for cross-group prefix hits; offload load boundary scan #36), restored prefix stores after a preempted request restarts ([Bugfix] Anthropic 4xx for client errors, and KV offload store cursor reset on restart #32), returned 4xx for client errors on the Anthropic endpoint ([Bugfix] Anthropic 4xx for client errors, and KV offload store cursor reset on restart #32), rejected INT8 KV with DFlash2 at startup ([Bugfix] Reject INT8 KV with DFlash2 at configuration time; opt-in watchdog stack dumps #33), made MTP4 at two requests about 3% faster on PCIe-switch systems ([Perf] SM70 PCIe TP4: avoid the generic sum2 pull kernel (MTP4 at 2-3 requests) #34), made DFlash2 state replay optional ([GDN] Replay: one-launch replay tables and a CUDA graph for the post-sampling replay #24) and recommended probabilistic DFlash2 drafting.
• 0.1.3 left more GPU memory for the KV cache with DFlash2 (1.38 million tokens at 512K, 8.7% more than 0.1.2, [Spec Decode] DFlash2: reserve its CUDA graph pool before sizing KV #4, [Spec Decode] DFlash2: shard the selector codebooks by vocabulary (on by default) #9, [Spec Decode] DFlash2: leave the verifier tail graphs off and reserve 1200 MiB #23), and fixed corrupt images returning 500 ([Multimodal] Reject corrupt images and audio with 400, not 500 #5), a stall with two adjacent multimodal items under MTP lookahead ([Bugfix][Spec Decode] Avoid zero-token MM rollback with EAGLE/MTP lookahead #6), memory growth of a busy shared-memory reader ([Bugfix][Distributed] Poll the shm notify socket from busy readers #7) and the Anthropic error.type values ([Frontend][Anthropic] Map error.type to the Anthropic error vocabulary #8).
• 0.1.2 added DFlash2 speculative decoding on V100 with the incoai drafter (1.23× on English code at one request), plus fixes for tool calling with thinking on, multimodal defaults, SSE keep-alive and prefix-cache resets.
• 0.1.1 fixed stop strings and the reasoning/content split with thinking on, streamed qwen3_coder string arguments as they are generated, returned 400 for invalid request parameters, and fixed 24 issues in streaming, prefix-cache resets, PLE memory, multimodal caching and startup checks. It was the first release on PyPI.
• Next: 0.1.6. Planned: scoring multi-request decode batches of the QSA indexer by request segments ([Perf][QSA] SM70: score multi-request decode batches by request segments #59: bitwise equal to the current scorer; four concurrent requests at 128K / 256K decode about 14–24% faster), and a fix for the CPU KV offload file left in /dev/shm (CPU KV offload leaves /dev/shm/vllm_offload_*.mmap behind after a normal shutdown #51).

How ReVolta is developed

Until 0.1.2, ReVolta was developed in a private repository, and each release was published here with its history rewritten. Since 2026-10-23, all development happens in this repository, so you can follow what changes and why:

• main: every change is a pull request, merged with a merge commit.
• release/0.1.x: frozen for release testing. The tested commit gets the tag v0.1.x, and the GitHub Release, the Docker image and the PyPI package are built from it.
• prod: the code that runs on our own 4 × V100 server. It changes only through a pull request from a release branch, and each deployment gets a tag deploy-YYYYMMDD-<version> (the first is deploy-20261023-v0.1.2).
• Checks on every pull request: no private addresses, local paths or credentials in the PR text, commit messages or added lines; every commit signed off; no added file over 1 MiB; pre-commit on the changed files; mypy for Python 3.12. GPU acceptance (4 and 8 concurrent requests, multimodal input, KV offload readback, long context) runs on our reference machine before each release; there is no GPU CI.
• Contributions are welcome: see CONTRIBUTING.md.

Why a separate repository

ReVolta is designed for four or more V100 32 GB GPUs and validated on four. We recommend, and test, the two interconnects that give the best speed: NVLink, and all GPUs under one PCIe switch with P2P. ReVolta began as the build behind our own daily service: Qwen3.8 Flash-Next (text, images and video) on such a server. 1Cat-vLLM is the base of that work, and we still contribute to it: 41 of our pull requests have been merged there, and 10 are open (2026-10-27). We opened a separate repository for these reasons:

• A focused target. 1Cat-vLLM supports many models, quantization formats and GPU setups. ReVolta targets one setup: four or more V100 32 GB GPUs (validated on four) serving Qwen3.8 Flash-Next. Some of our changes only make sense there: custom all-reduce for NVLink and PCIe-switch topologies, AWQ memory layouts tuned for this one model, and serving defaults tuned for TP4 + DCP2 with MTP4 or DFlash2. In a general project these changes would be opt-in switches, or they would not be accepted.
• Production settings in code. Before ReVolta, our service ran 1Cat-vLLM with a startup hook and many environment variables. The hook forced the topology checks and enabled the tuned kernels. In ReVolta these settings are code defaults. An automatic "ReVolta profile" applies them, and the startup log lists each value and where it came from. Explicit user settings still win.
• A fixed base and a strict gate. Every change must give the same greedy output as the running service, token for token. Where that is impossible, the change must come with a quality measurement. It must also pass a full serving gate: 4 and 8 concurrent requests, multimodal input, KV offload readback and 512K context. This works only if the base stays fixed. 1Cat-vLLM moves fast: its main has gained 1,311 commits since our fork point (2026-10-27). So ReVolta does not follow 1Cat-vLLM main. We port the upstream fixes that affect the code paths we run, and we review upstream about every two weeks.
• Our own release schedule. Upstream reviews take time, and that is normal for a project of this size. With our own repository, a fix ships when it passes our gate. We keep the upstream pull request open at the same time.
• A focused package. ReVolta ships an SM70-only wheel and Docker image under the name revolta, starting at version 0.1.0. We removed parts that are not needed at runtime: documentation sources, upstream CI, examples, benchmarks and platform build recipes (1,366 files changed, almost all deletions). Our own pull-request checks were added later. The runtime and native source trees are unchanged.

ReVolta does not replace 1Cat-vLLM. Most of the V100 kernel work in this repository comes from 1Cat-vLLM and its contributors (see NOTICE). When a ReVolta fix also applies upstream, we propose it there.

What changed in 0.1.0 since 1Cat-vLLM v1.5.1

Baseline

• Between v1.5.1 and the fork point: 0.1.0 includes 264 upstream commits, mainly 1Cat-vLLM's GGUF loader and lattice kernels, PLE DCP transport and mixed-prefill latency work.
• After the fork point: ReVolta adds 138 commits (git log --no-merges f551e0afea..v0.1.0).

The groups below list the main changes. Hashes refer to this repository. The release gate in the 0.1.0 release notes applies to 0.1.0 as a whole.

Topology and serving defaults

• Custom all-reduce over a PCIe switch (10cbb71).
  • Upstream disables custom all-reduce on PCIe-only TP4. Four V100s under one PCIe switch have working P2P, so ReVolta treats them as fully connected.
  • It limits custom all-reduce to small payloads and CUDA graphs.
  • It admits the multimodal Qwen3.8 class to the FP16 decode lane.
  • This mode replaces the startup hooks we used before.
• Interconnect policy (de450aa, c425cfc).
  • At startup, ReVolta chooses PCIe switch with P2P, PCIe without P2P, or NVLink, and it validates P2P again before use. It logs the choice as SM70 interconnect=....
  • NVLink is supported and tested (community results since 0.1.1, NVLink results (4× V100-SXM2, NV2 mesh) for 0.1.0, and the README 512K view script fails to load #2).
• ReVolta profile (dc609b9, 7920055). Serving defaults for SM70 QSA TP4, selected by the weight format. The startup log records each value's source, and the record also covers child processes.
• Custom all-reduce graph capture is VMM-safe (d450a43).
• Compile cache key (5153b2e). The SM70 GEMM LUT path no longer changes the compile cache key.

Decode and prefill speed

• Grouped small-M AWQ g32 MoE decode for Qwen3.8 TP4 (1d3c0e8, 4d931b3, f319521, e2be854). It handles up to 48 rows, has a split-4 W13 variant and configurable bands.

• Faster MTP verification:

  • greedy draft verification from the TP-local argmax (c671b95);
  • wide-M admission for 8-request MTP4 verification (1467b2d).

• GDN and PLE prefill:

  • GDN input without the packed copy (d3097d4);
  • channel-chunked PLE prefill short-conv (0b7b119).

• Prefill under decode load:

  • cap prefill tokens while decodes wait (7197213, 860c902, a866ed1);
  • grouped page4 route for QSA DCP prefill attention (53ddd21, [Perf][SM70] Use the grouped page4 route for QSA DCP prefill attention 1CatAI/1Cat-vLLM#828).

• 1Cat-vLLM pull requests merged after the fork point, ported because they are on our code paths:

  • batched HC and router: [Kernel] Enable FP32 batched HC without full-mesh NVLink 1CatAI/1Cat-vLLM#900, [Kernel] Fuse SM70 router top-k with Q8 activation quantization 1CatAI/1Cat-vLLM#1147;
  • expert alignment: [Kernel] Fuse small-batch expert alignment and FP32 restoration 1CatAI/1Cat-vLLM#907;
  • packing guards: [Bugfix][SM70] Skip precision-rejected router and shared packs 1CatAI/1Cat-vLLM#919;
  • GDN and MTP buffer reuse: [Kernel] Reuse GDN tail copies without materializing QKV 1CatAI/1Cat-vLLM#927, [Kernel] Read row-strided QKV in fused MTP recurrence 1CatAI/1Cat-vLLM#928, [Kernel] Reuse the MTP GDN verification output buffer 1CatAI/1Cat-vLLM#929, [Model] Reuse GDN tail copies on the default forward path 1CatAI/1Cat-vLLM#936;
  • PLE n-gram fusion: [Kernel] Fuse small-batch PLE n-gram preprocessing on SM70 1CatAI/1Cat-vLLM#914, [Kernel] Fuse PLE n-gram input preparation 1CatAI/1Cat-vLLM#988;
  • model-input preparation: [Core] Prepare independent model inputs before FULL graph metadata 1CatAI/1Cat-vLLM#955, [Core] Reuse asynchronous PLE metadata request indices 1CatAI/1Cat-vLLM#964;
  • QSA keys and selection: [Kernel] Reuse compressed QSA keys across SM70 verification queries 1CatAI/1Cat-vLLM#1127, [Kernel] Reduce exact QSA selection work for long contexts 1CatAI/1Cat-vLLM#1171.

  We also ported the still-open [Bugfix][QSA] Clamp the ring's cached RoPE positions before they index cos_sin_cache 1CatAI/1Cat-vLLM#1201 (RoPE position clamp in the QSA ring).

Memory

• 4-bit AWQ zero points in GPU memory (2dde305). Routed experts use 4.625 instead of 4.75 bits per weight. This frees about 0.45 GiB per GPU for the KV pool, about 95K more KV tokens in total (measured).
• M-RoPE YaRN cache sized at 1x without changing the frequencies (b7e52f5).
• Bounded scratch and graph memory:
  • prompt-logprobs memory (105a627, [Bugfix][MRv2] Bound prompt logprobs memory with small chunks and request limits 1CatAI/1Cat-vLLM#1225);
  • sampler fallback scratch (08750df, da9b2fa, 687068b, a5616c7, db93e33, 17fd992);
  • sampler buffer overflow rejected before submission (96ecdc5);
  • the MTP draft config no longer disables the target's graph reserve (7eaa858);
  • eager warmup cache released before draft graph capture (6b85d6d);
  • custom all-reduce buffers registered only after a successful capture (b8458c6).
• PLE host memory:
  • host share pinned at its exact size (d063888, [Bugfix][Qwen4Exp] Pin the PLE host share at its exact size 1CatAI/1Cat-vLLM#1199);
  • checks that the resident FP8 tables are complete (1f0d227, 8adaf15);
  • a warning when host memory is overcommitted within cgroup limits (ecc0ff0);
  • cascade merging is now opt-in (c13a69c).
• Optional startup memory budget log (d9af631).

Weight formats

• NVIDIA FP4-Local weights with packed NVFP4 PLE (a8f444b, a15a196).
• Standard NVFP4: raw E4M3 grouped decode (36f4321).
• NVFP4 vision tower on TP4: unaligned projections are now sharded across the four GPUs (faab1e5, 8fdd91a).
• FP4-Local QSA E4M3 KV scale pack (f074546).
• AWQ relabeling applies only to real AWQ checkpoints under TurboMind (561bf4e).

INT8 KV cache

• Shared page layout. One shared description of KV page layouts, with planar INT8 storage and a writer (b4681f9).
• Readers. INT8 page4 readers with explicit backend admission, compiled separately from the existing kernels (08a2018, f70d36e, af7addf).
• INT8 with offload and drafts (c26adab, c85c1ee, 8e64f15, e4a5b08, 554a7bd).
• Not supported: INT8 with the SSD tier is rejected at configuration time.

KV offload and prefix cache

• Filesystem tiers:
  • bounded tiers and validated hybrid cache modes (177d9d4);
  • the cache hash identity persists across restarts, and disk caches must have a size limit (45d3b9a);
  • an empty environment seed counts as unset (cd4040f).
• Asynchronous loads:
  • capacity reserved for KV load promotion (40a668a);
  • parked loads count against admission slots (6bef753);
  • load reservations adapted to hybrid state groups (60f736a, c1f43dc, 8f9258f).
• Ordering and reuse:
  • stores finish before a reused block is zeroed (a59bb7f, [Bugfix][KV Offload] Finish reused-block stores before MRv2 zeroes them 1CatAI/1Cat-vLLM#1232);
  • Mamba states restored where EAGLE GPU hits can reuse them (0571dab, [Perf][KV Offload] Restore Mamba states where EAGLE GPU hits can reuse them 1CatAI/1Cat-vLLM#1244);
  • prefix-cache skips honored at both lookup layers (0b90746).
• Block accounting:
  • packed DCP pages zeroed from the page base (b9a1990, [Bugfix][KV Cache] Zero packed DCP pages from the page base 1CatAI/1Cat-vLLM#1230);
  • local hits protected before ring and tail blocks are claimed (1e1e9f2);
  • the null KV block reserved during startup admission (5cc100f).

Correctness and API

• Speculative decoding:
  • NaN block maxima normalized before argmax (1213eef);
  • Mamba align blocks kept when drafts shrink (566cfc5);
  • output metadata trimmed after a stop string (ee77aca).
• OpenAI-compatible API:
  • the earliest completing stop string wins (04643ad);
  • every required tool call is streamed in each delta (d525a04);
  • tool calls keep their truncation reasons (79b722b);
  • logprobs are kept when parsers suppress deltas (f90b940);
  • streaming errors that occur before the first token are reported (2f8ebb3);
  • empty tool-call arrays are omitted (172c1eb);
  • top_logprobs: null is handled (4ba2a9a);
  • a grammar compilation failure affects only its own request (db055e2).
• Multimodal input:
  • image orientation and transparency handled (48f75d0);
  • remote media size bounded while streaming (2f7a660);
  • request budgets checked before images are decoded (ff4a1e4, 8c8a843);
  • every overlapping media item keyed in block hashes (f6c9ead);
  • recovery when the frontend and engine media caches drift apart (b1aa855).
• Startup validation:
  • unsupported draft KV precisions and QSA target formats rejected before loading (9c69cbe, 6559738);
  • the automatic model-runner fallback is explained in the log (ef40369).

Packaging, licenses and documentation

• Package. revolta 0.1.0, with debug symbols in a separate file (fc902ed, bb5117a, 9bed2e4, dca9e59, b0de8e1, e8eb1f5, 6208239).
• Trimmed repository (fc04ceb). SM70 kernel tests are now self-contained (8938434).
• Licenses. License notices and the provenance of the TurboMind, CUTLASS and SGLang-V100 code (b334451, 0f3514c, cf79f38, bcbbc5d).
• Usage statistics are never sent to the upstream server (20edbf9).
• Documentation. README, support scope and issue form (32470fc, 848ffc7, b02ee33).

Current gaps (0.1.5)

The supported scope, verified configurations and known limitations are in the README. The main gaps:

• V100 16 GB, TP2/TP8 and other models are not tested.
• With 2 to 8 concurrent requests, outputs are not bitwise reproducible from run to run: the tensor-parallel all-reduce sums in an order that depends on how requests are batched. A batch-invariant all-reduce is a goal for 0.2.
• INT8 KV decodes about 5–8% slower than E4M3.
• DFlash2 depends on the workload: short English and code prompts benefit most; Chinese text and long agent sessions often favour MTP4. With several requests at once DFlash2 pulls ahead: on our reference setup four and eight concurrent requests (512-token answers) ran about 20% and 9% faster than MTP4, while a replay of 90 agent sessions one request at a time put DFlash2 at 0.98 of MTP4, within run-to-run variation. INT8 KV is not supported with DFlash2, and DFlash2 has not been tested with preemption.
• Sleep level 2 (development mode) does not fit in host memory on the reference setup; sleep level 1 works.

Fixed since 0.1.0: stop strings with thinking on (0.1.1). ReVolta is on PyPI since 0.1.1.