From afd85aad93f74b22b46c2d85a2d120116299b1bf Mon Sep 17 00:00:00 2001 From: Leonccaa <166551845+Leonccaa@users.noreply.github.com> Date: Mon, 26 Oct 2026 12:12:18 -0700 Subject: [PATCH 1/2] [Doc] Add a test results issue form Community reports of measured results had no form: the only form is the bug report, and contributors cannot add labels in this repository. The new form applies the `test results` label, asks for the configuration fields the bug report already asks for, and asks how the numbers were measured. The README's Getting help section links to it. Co-Authored-By: Claude Opus 5.5 Signed-off-by: Leonccaa <166551845+Leonccaa@users.noreply.github.com> --- .github/ISSUE_TEMPLATE/test_results.yml | 102 ++++++++++++++++++++++++ README.md | 3 +- 2 files changed, 104 insertions(+), 1 deletion(-) create mode 100644 .github/ISSUE_TEMPLATE/test_results.yml diff --git a/.github/ISSUE_TEMPLATE/test_results.yml b/.github/ISSUE_TEMPLATE/test_results.yml new file mode 100644 index 0000000000..5a2c99620b --- /dev/null +++ b/.github/ISSUE_TEMPLATE/test_results.yml @@ -0,0 +1,102 @@ +name: Test results +description: Share speed, capacity or stability results measured on your hardware. Nothing needs to be broken. +title: "Test results: " +labels: ["test results"] +body: + - type: markdown + attributes: + value: | + Thank you for sharing results. Data from configurations we have not validated (NVLink, other GPU counts, V100 16 GB, other hosts) is especially useful. + + To compare with the release tables, see the method in [docs/performance.md](https://github.com/Leonccaa/ReVolta/blob/main/docs/performance.md#reference-system-mtp4). Any method is welcome; please describe the one you used so the numbers can be read correctly. + + For a problem or a crash, use the bug report form instead. Remove credentials, tokens and private paths from commands and logs. + - type: input + id: version + attributes: + label: ReVolta version and installation method + placeholder: 0.1.3, PyPI wheel (cp312) + validations: + required: true + - type: input + id: gpu + attributes: + label: GPU model, count, memory per GPU and power limit + placeholder: 4 x Tesla V100-SXM2 32 GB, 250 W + validations: + required: true + - type: dropdown + id: interconnect + attributes: + label: GPU interconnect + options: + - PCIe switch with P2P + - PCIe without P2P + - NVLink + - Other (describe in the notes) + validations: + required: true + - type: textarea + id: topology + attributes: + label: Output of `nvidia-smi topo -m` + render: text + validations: + required: true + - type: input + id: host-memory + attributes: + label: Host memory, container memory limit and /dev/shm size + placeholder: 128 GiB host, no container, 63 GiB shm + - type: input + id: checkpoints + attributes: + label: Target and draft checkpoints with revisions + placeholder: / @ ; drafter / @ + validations: + required: true + - type: dropdown + id: speculative + attributes: + label: Speculative decoding method + options: + - MTP + - DFlash2 + - None + - Other (describe in the notes) + validations: + required: true + - type: textarea + id: command + attributes: + label: Complete launch command and environment variables + description: Include --speculative-config and --kv-transfer-config, and every VLLM_*, NCCL_* and PYTORCH_* variable you set. + render: shell + validations: + required: true + - type: textarea + id: startup + attributes: + label: Startup log lines + description: The `ReVolta profile {...}` line, the `SM70 interconnect=` lines and the `GPU KV cache size` line. + render: text + validations: + required: true + - type: textarea + id: method + attributes: + label: How you measured + description: Prompts, response length, thinking on or off, temperature, number of concurrent requests, repetitions, and whether rates include the time to first token. + validations: + required: true + - type: textarea + id: results + attributes: + label: Results + description: Tables are welcome. Give the unit of every number. + validations: + required: true + - type: textarea + id: notes + attributes: + label: Stability, deviations from the README and other notes diff --git a/README.md b/README.md index c617263c1a..42c44f1d9c 100644 --- a/README.md +++ b/README.md @@ -109,7 +109,8 @@ command, relevant environment, model revision, host/container memory limits, offload settings, `nvidia-smi topo -m`, and the `ReVolta profile` / `SM70 interconnect=` log lines. Remove credentials. -Reports from NVLink V100 owners are especially welcome; thanks to +Reports from NVLink V100 owners are especially welcome; share measured results with the +[Test results](https://github.com/Leonccaa/ReVolta/issues/new?template=test_results.yml) form. Thanks to [@jttdev](https://github.com/jttdev) for sharing their results ([#2](https://github.com/Leonccaa/ReVolta/issues/2)). Report security issues through GitHub private vulnerability reporting. From 5265f239906ce1ecc4409dd237b3fcda40a56d2e Mon Sep 17 00:00:00 2001 From: Leonccaa <166551845+Leonccaa@users.noreply.github.com> Date: Mon, 26 Oct 2026 16:45:18 -0700 Subject: [PATCH 2/2] [Doc] Make the test results template a Markdown template for agents The issue form only works in the web UI. Many reports are now written by coding agents, which open issues through the API or the gh CLI; neither uses issue forms, and the label is not applied that way. A Markdown template can be read from the repository, filled in, and passed to the CLI. The template keeps the same sections, puts the instructions in HTML comments, starts with a key: value summary block, and says that the maintainers add the label when the issue is opened through the API. The README links to the template file. Co-Authored-By: Claude Opus 5.5 Signed-off-by: Leonccaa <166551845+Leonccaa@users.noreply.github.com> --- .github/ISSUE_TEMPLATE/test_results.md | 76 ++++++++++++++++++ .github/ISSUE_TEMPLATE/test_results.yml | 102 ------------------------ README.md | 4 +- 3 files changed, 79 insertions(+), 103 deletions(-) create mode 100644 .github/ISSUE_TEMPLATE/test_results.md delete mode 100644 .github/ISSUE_TEMPLATE/test_results.yml diff --git a/.github/ISSUE_TEMPLATE/test_results.md b/.github/ISSUE_TEMPLATE/test_results.md new file mode 100644 index 0000000000..a6c744a2ab --- /dev/null +++ b/.github/ISSUE_TEMPLATE/test_results.md @@ -0,0 +1,76 @@ +--- +name: Test results +about: Share speed, capacity or stability results measured on your hardware. People and coding agents can both fill this in. +title: "Test results: " +labels: ["test results"] +--- + + + +## Summary + +```text +revolta_version: +install: # PyPI wheel, Docker image tag, or source +gpus: # model x count, memory per GPU +power_limit_w: +interconnect: # PCIe switch with P2P, PCIe without P2P, NVLink, or other +host_memory: # host RAM, container limit, /dev/shm size +checkpoint: # owner/repository @ revision +drafter: # owner/repository @ revision, or none +speculative_method: # MTP, DFlash2, or none +draft_sample_method: +kv_cache_dtype: +cpu_kv_offload_gib: +max_model_len: +gpu_memory_utilization: +``` + +## Topology + + + +```text + +``` + +## Launch command and environment + + + +```bash + +``` + +## Startup log lines + + + +```text + +``` + +## How you measured + + + +## Results + + + +## Stability and notes + + diff --git a/.github/ISSUE_TEMPLATE/test_results.yml b/.github/ISSUE_TEMPLATE/test_results.yml deleted file mode 100644 index 5a2c99620b..0000000000 --- a/.github/ISSUE_TEMPLATE/test_results.yml +++ /dev/null @@ -1,102 +0,0 @@ -name: Test results -description: Share speed, capacity or stability results measured on your hardware. Nothing needs to be broken. -title: "Test results: " -labels: ["test results"] -body: - - type: markdown - attributes: - value: | - Thank you for sharing results. Data from configurations we have not validated (NVLink, other GPU counts, V100 16 GB, other hosts) is especially useful. - - To compare with the release tables, see the method in [docs/performance.md](https://github.com/Leonccaa/ReVolta/blob/main/docs/performance.md#reference-system-mtp4). Any method is welcome; please describe the one you used so the numbers can be read correctly. - - For a problem or a crash, use the bug report form instead. Remove credentials, tokens and private paths from commands and logs. - - type: input - id: version - attributes: - label: ReVolta version and installation method - placeholder: 0.1.3, PyPI wheel (cp312) - validations: - required: true - - type: input - id: gpu - attributes: - label: GPU model, count, memory per GPU and power limit - placeholder: 4 x Tesla V100-SXM2 32 GB, 250 W - validations: - required: true - - type: dropdown - id: interconnect - attributes: - label: GPU interconnect - options: - - PCIe switch with P2P - - PCIe without P2P - - NVLink - - Other (describe in the notes) - validations: - required: true - - type: textarea - id: topology - attributes: - label: Output of `nvidia-smi topo -m` - render: text - validations: - required: true - - type: input - id: host-memory - attributes: - label: Host memory, container memory limit and /dev/shm size - placeholder: 128 GiB host, no container, 63 GiB shm - - type: input - id: checkpoints - attributes: - label: Target and draft checkpoints with revisions - placeholder: / @ ; drafter / @ - validations: - required: true - - type: dropdown - id: speculative - attributes: - label: Speculative decoding method - options: - - MTP - - DFlash2 - - None - - Other (describe in the notes) - validations: - required: true - - type: textarea - id: command - attributes: - label: Complete launch command and environment variables - description: Include --speculative-config and --kv-transfer-config, and every VLLM_*, NCCL_* and PYTORCH_* variable you set. - render: shell - validations: - required: true - - type: textarea - id: startup - attributes: - label: Startup log lines - description: The `ReVolta profile {...}` line, the `SM70 interconnect=` lines and the `GPU KV cache size` line. - render: text - validations: - required: true - - type: textarea - id: method - attributes: - label: How you measured - description: Prompts, response length, thinking on or off, temperature, number of concurrent requests, repetitions, and whether rates include the time to first token. - validations: - required: true - - type: textarea - id: results - attributes: - label: Results - description: Tables are welcome. Give the unit of every number. - validations: - required: true - - type: textarea - id: notes - attributes: - label: Stability, deviations from the README and other notes diff --git a/README.md b/README.md index 42c44f1d9c..69c60dbbcf 100644 --- a/README.md +++ b/README.md @@ -110,7 +110,9 @@ offload settings, `nvidia-smi topo -m`, and the `ReVolta profile` / `SM70 interconnect=` log lines. Remove credentials. Reports from NVLink V100 owners are especially welcome; share measured results with the -[Test results](https://github.com/Leonccaa/ReVolta/issues/new?template=test_results.yml) form. Thanks to +[Test results](https://github.com/Leonccaa/ReVolta/issues/new?template=test_results.md) template +([`.github/ISSUE_TEMPLATE/test_results.md`](.github/ISSUE_TEMPLATE/test_results.md); agents can fill it in and +open the issue through the API). Thanks to [@jttdev](https://github.com/jttdev) for sharing their results ([#2](https://github.com/Leonccaa/ReVolta/issues/2)). Report security issues through GitHub private vulnerability reporting.