sources.lock.json

{
  "schema_version": 1,
  "sources": [
    {
      "id": "qwen3-tts-engine",
      "repository": "https://github.com/InferMatrixLab/qwen3-tts-engine",
      "ref": "main",
      "paths": [
        "recipes/qwen3-tts-ascend-910b1.yaml"
      ],
      "commit": "63620f6d906de103cc574f859124145998cc91a5",
      "files": [
        {
          "path": "recipes/qwen3-tts-ascend-910b1.yaml",
          "content": "schema_version: 1\nid: qwen3-tts-ascend-910b1\ntitle: Qwen3-TTS CustomVoice on one Ascend 910B1\nkind: standalone_engine\nmodel:\n  id: Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice\n  provider: Qwen\n  task: text_to_speech\nengine:\n  id: qwen3-tts-engine\n  name: qwen3-tts-engine\n  repository: https://github.com/InferMatrixLab/qwen3-tts-engine\n  ref: b4451682abac651f26046f8d27467de3f007e1e7\nruntime:\n  backend: ascend\n  precision: bf16\nhardware:\n  vendor: Huawei\n  model: Ascend 910B1\n  device_count: 1\nenvironment:\n  os: Linux (prepared Ascend environment; distribution not recorded)\n  requirements:\n  - One allocated Ascend 910B1; Python 3.12 with CANN and Triton Ascend already installed.\n  - 'Prepared environment: torch 2.10.0, torch_npu 2.10.0.post4, qwen-tts 0.1.1, transformers 4.57.3.'\n  - Preserve the CANN PYTHONPATH; the repository does not install the NPU driver or toolchain.\n  - Recorded ARM64 allocator workaround requires /usr/lib/aarch64-linux-gnu/libjemalloc.so.2.\n  - GitHub, Python package index and Hugging Face access are required for initial installation and model download.\ncommands:\n  install: |\n    git clone https://github.com/InferMatrixLab/qwen3-tts-engine.git\n    cd qwen3-tts-engine\n    git checkout b4451682abac651f26046f8d27467de3f007e1e7\n    python -m pip install --no-deps -e .\n    npu-smi info\n    python -c \"import torch, torch_npu, triton, qwen_tts; assert torch.npu.is_available()\"\n    export QWEN_TTS_MODEL=\"$PWD/models/Qwen3-TTS-12Hz-1.7B-CustomVoice\"\n    HF_HUB_OFFLINE=0 python - <<'PY'\n    import os\n    from huggingface_hub import snapshot_download\n    snapshot_download(repo_id=\"Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice\",\n                      revision=\"0c0e3051f131929182e2c023b9537f8b1c68adfe\",\n                      local_dir=os.environ[\"QWEN_TTS_MODEL\"])\n    PY\n  serve: |\n    export ASCEND_RT_VISIBLE_DEVICES=\"${ASCEND_RT_VISIBLE_DEVICES-0}\"\n    export OMP_NUM_THREADS=4\n    export HF_HUB_OFFLINE=1\n    export LD_PRELOAD=/usr/lib/aarch64-linux-gnu/libjemalloc.so.2\n    python - <<'PY'\n    import json\n    import os\n    import subprocess\n    import sys\n    from pathlib import Path\n    command = json.loads(Path(\"evidence/campaign-20261017/plan/continuous-direct-prompt-command.json\").read_text())\n    command[0] = sys.executable\n    command[command.index(\"--model\") + 1] = os.environ[\"QWEN_TTS_MODEL\"]\n    subprocess.run(command, check=True)\n    PY\n  request: |\n    curl --fail http://127.0.0.1:8099/health\n    python - <<'PY'\n    import base64\n    import json\n    import urllib.request\n    import wave\n    payload = {\"input\": \"Hello from the Ascend engine.\", \"voice\": \"Ryan\", \"language\": \"English\",\n               \"response_format\": \"pcm\", \"stream_format\": \"sse\", \"seed\": 42, \"max_new_tokens\": 2048}\n    request = urllib.request.Request(\"http://127.0.0.1:8099/v1/audio/speech\",\n        data=json.dumps(payload).encode(), headers={\"Content-Type\": \"application/json\"})\n    finished = False\n    sample_rate = None\n    with urllib.request.urlopen(request, timeout=1800) as response, wave.open(\"output.wav\", \"wb\") as wav:\n        wav.setnchannels(1)\n        wav.setsampwidth(2)\n        for line in response:\n            if not line.startswith(b\"data: \"):\n                continue\n            event = json.loads(line[6:])\n            if event[\"type\"] == \"speech.audio.error\":\n                raise RuntimeError(event[\"error\"])\n            if event[\"type\"] == \"speech.audio.delta\":\n                if sample_rate is None:\n                    sample_rate = event[\"sample_rate\"]\n                    wav.setframerate(sample_rate)\n                elif sample_rate != event[\"sample_rate\"]:\n                    raise RuntimeError(\"Sample rate changed during streaming\")\n                wav.writeframes(base64.b64decode(event[\"audio\"]))\n            elif event[\"type\"] == \"speech.audio.done\":\n                finished = True\n                print(\"Finish reason:\", event[\"finish_reason\"])\n    if not finished:\n        raise RuntimeError(\"Stream ended without completion\")\n    print(\"Saved output.wav\")\n    PY\n  benchmark: |\n    python -m qwen3_tts_engine.benchmark \\\n      --endpoint http://127.0.0.1:8099/v1/audio/speech \\\n      --model Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice \\\n      --engine independent --source-revision \"$(git rev-parse HEAD)\" \\\n      --workload evidence/campaign-20261017/plan/workload-128.jsonl \\\n      --output runs/local-c4 --concurrency 4 --requests 128 --warmup 4 --frame-rate 12.5\ndeployment:\n  status: unverified\n  last_verified_at: null\n  evidence_url: null\nacceptance:\n  verdict: INCONCLUSIVE\n  issue_url: null\n  evidence_url: null\nmaintainers:\n- amy-why-3459\nupdated_at: '2026-10-19'\nlimitations:\n- This catalog entry has not been executed on hardware during the 2026-10-19 documentation work.\n- The historical PASS covers measured engine e783a64b0fb4f9ade2ffb3980b3e1f74a00c9a6a and its frozen contract; it\n  does not qualify this deployment ref automatically.\n- 'Historical evidence: https://github.com/InferMatrixLab/qwen3-tts-engine/blob/b4451682abac651f26046f8d27467de3f007e1e7/evidence/campaign-20261017/acceptance.md'\n- Only this CustomVoice checkpoint, one 910B1 and the recorded BF16 configuration are represented. The separate\n  910B3 comparison is not validation of this recipe.\n- Keep the install and serve blocks in the same activated shell. Send the request from a second terminal after health\n  succeeds.\n- The endpoint streams base64 PCM over SSE; it does not return a WAV directly. The OS distribution and CANN version\n  were not fully frozen in the source documentation.\n",
          "sha256": "af7e11daba0719460932fe452d812aabf461d705ecf288b52c606382e7e3e070"
        }
      ]
    },
    {
      "id": "drama-3-single-b300-engine",
      "repository": "https://github.com/InferMatrixLab/drama-3-single-b300-engine",
      "ref": "codex-drama3-b300",
      "paths": [
        "recipes/drama3-b300-fp8.yaml"
      ],
      "commit": "c560a520913f51586687943ef0adb1ec764ef510",
      "files": [
        {
          "path": "recipes/drama3-b300-fp8.yaml",
          "content": "schema_version: 1\nid: drama3-b300-fp8\ntitle: Drama 3 native FP8 experts on one NVIDIA B300\nkind: standalone_engine\nmodel:\n  id: fishaudio/drama-3\n  provider: fishaudio\n  task: text_to_speech\nengine:\n  id: drama-3-single-b300-engine\n  name: drama-3-single-b300-engine\n  repository: https://github.com/InferMatrixLab/drama-3-single-b300-engine\n  ref: 58a1882a4d941f8d21bfce6be37d3d5e7c9fde80\nruntime:\n  backend: cuda\n  precision: fp8 experts / bf16 state and codec\nhardware:\n  vendor: NVIDIA\n  model: B300\n  device_count: 1\nenvironment:\n  os: Linux\n  requirements:\n  - One allocated NVIDIA B300, Python 3.12, uv, Git and curl.\n  - NVIDIA driver compatible with the locked CUDA 13 runtime; requirements-runtime.lock is installed by scripts/install_runtime.py.\n  - Fish Speech source is pinned to 541566932abbaebcb0a40e562a1f846738383f38; model revision is e0a9218aa48888270f06ef1ed9ea67622c0e17d2.\n  - Initial package installation and checkpoint download require network access.\ncommands:\n  install: |\n    git clone https://github.com/InferMatrixLab/drama-3-single-b300-engine.git\n    cd drama-3-single-b300-engine\n    git checkout 58a1882a4d941f8d21bfce6be37d3d5e7c9fde80\n    git clone https://github.com/fishaudio/fish-speech.git ../fish-speech\n    git -C ../fish-speech checkout 541566932abbaebcb0a40e562a1f846738383f38\n    python3 scripts/install_runtime.py --venv .venv --fish-source ../fish-speech\n    .venv/bin/hf download fishaudio/drama-3 \\\n      --revision e0a9218aa48888270f06ef1ed9ea67622c0e17d2 --local-dir models/drama-3\n  serve: |\n    set -a\n    . configs/benchmark-native-b300.env\n    set +a\n    CUDA_VISIBLE_DEVICES=\"${CUDA_VISIBLE_DEVICES-0}\" OMP_NUM_THREADS=8 .venv/bin/python -m drama3.server \\\n      --checkpoint models/drama-3 --fish-source ../fish-speech --slots 64 --port 18082\n  request: |\n    curl --fail http://127.0.0.1:18082/health\n    curl --fail-with-body http://127.0.0.1:18082/v1/audio/speech \\\n      -H 'Content-Type: application/json' \\\n      -d '{\"input\":\"Hello from Drama 3.\",\"stream\":true,\"response_format\":\"pcm\"}' \\\n      --output speech.pcm\n  benchmark: null\ndeployment:\n  status: unverified\n  last_verified_at: null\n  evidence_url: null\nacceptance:\n  verdict: INCONCLUSIVE\n  issue_url: null\n  evidence_url: null\nmaintainers:\n- david6666666\nupdated_at: '2026-10-19'\nlimitations:\n- This deployment ref has not been re-run on B300 during the catalog work; the Linux distribution is not frozen\n  in the quickstart.\n- 'Mixed precision is intentional: native FP8 experts, BF16 GDN state and BF16 codec. No all-FP8 claim is made.'\n- Output is 44.1 kHz mono signed 16-bit PCM. Allow model load, graph capture and codec warmup to finish before sending\n  requests.\n- Historical matched results use standalone 33f6d7237196520be9f457d86df161264cbb087e and include incomplete baseline\n  repetitions; they do not establish the all-concurrency 1.5x acceptance target.\n- 'Historical configuration, quality and raw evidence: https://github.com/InferMatrixLab/drama-3-single-b300-engine/blob/58a1882a4d941f8d21bfce6be37d3d5e7c9fde80/docs/current-results.md'\n- Built with Fish Audio. Fish Audio Community License and dependency licenses apply; retain NOTICE and third_party_licenses.\n",
          "sha256": "a0b646f096db508c241da9d1caf2bb79a9615197f6bbbe0d29133a76a73d1a51"
        }
      ]
    },
    {
      "id": "vllm-omni-mlx",
      "repository": "https://github.com/InferMatrixLab/vllm-omni-mlx",
      "ref": "main",
      "paths": [
        "recipes/qwen3-tts-mlx-m4-4bit.yaml"
      ],
      "commit": "4bf0a5ccea1d5bf364f62927bf015368879a68a1",
      "files": [
        {
          "path": "recipes/qwen3-tts-mlx-m4-4bit.yaml",
          "content": "schema_version: 1\nid: qwen3-tts-mlx-m4-4bit\ntitle: Qwen3-TTS CustomVoice 4-bit on Apple M4\nkind: standalone_engine\nmodel:\n  id: mlx-community/Qwen3-TTS-12Hz-1.7B-CustomVoice-4bit\n  provider: mlx-community\n  task: text_to_speech\nengine:\n  id: vllm-omni-mlx\n  name: vllm-omni-mlx\n  repository: https://github.com/InferMatrixLab/vllm-omni-mlx\n  ref: 210cc4e35e208b56d4146e7cc735952cde8ea24e\nruntime:\n  backend: mlx\n  precision: 4bit\nhardware:\n  vendor: Apple\n  model: M4 (16 GB unified memory)\n  device_count: 1\nenvironment:\n  os: macOS arm64\n  requirements:\n  - One Apple M4 with 16 GB unified memory, as documented in the Qwen3-TTS spike; exact macOS release is not recorded.\n  - Python 3.10+; the recorded spike used Python 3.13.5, mlx 0.32.3 and mlx-audio 0.5.7.\n  - Install the source [tts] extra; MLX requires Apple Silicon arm64 wheels.\n  - Initial source install and model load require network access and disk space for the model snapshot.\ncommands:\n  install: |\n    git clone https://github.com/InferMatrixLab/vllm-omni-mlx.git\n    cd vllm-omni-mlx\n    git checkout 210cc4e35e208b56d4146e7cc735952cde8ea24e\n    python3 -m venv .venv\n    source .venv/bin/activate\n    python -m pip install -e '.[tts]'\n  serve: |\n    vllm-omni-mlx serve mlx-community/Qwen3-TTS-12Hz-1.7B-CustomVoice-4bit --omni --api-key demo\n  request: |\n    curl --fail http://127.0.0.1:8000/health\n    curl --fail-with-body -H 'Authorization: Bearer demo' -H 'Content-Type: application/json' \\\n      -d '{\"input\":\"Hello from vllm omni mlx.\",\"voice\":\"vivian\"}' \\\n      http://127.0.0.1:8000/v1/audio/speech -o speech.wav\n  benchmark: null\ndeployment:\n  status: unverified\n  last_verified_at: null\n  evidence_url: null\nacceptance:\n  verdict: INCONCLUSIVE\n  issue_url: null\n  evidence_url: null\nmaintainers:\n- hsliuustc0106\nupdated_at: '2026-10-19'\nlimitations:\n- This recipe has not been re-run on Apple hardware during the 2026-10-19 catalog work.\n- M4/16 GB and the exact checkpoint are documented in the historical spike, which measured an earlier mlx-audio\n  adaptation rather than this deployment ref.\n- 'Source for the hardware scope: https://github.com/InferMatrixLab/vllm-omni-mlx/blob/210cc4e35e208b56d4146e7cc735952cde8ea24e/docs/spike-mlx-audio-qwen3-tts.md'\n- The dependency ranges and model download are not a fully locked macOS environment or checkpoint revision. Record\n  resolved versions, model snapshot and macOS release during remote validation.\n- The 4-bit label describes the quantized checkpoint; embeddings and vocoder may retain higher precision.\n- One model instance per process; generation is serialized. The demo API key is a local example; default host is\n  127.0.0.1.\n- No claim for every Apple Silicon SKU, ASR, speech-out chat or diffusion follows from this TTS recipe.\n",
          "sha256": "7556319ecd226660fef65bf8ac21b33e00fa3779a7a7f4ead1a158c1f710618e"
        }
      ]
    },
    {
      "id": "vllm-rlt",
      "repository": "https://github.com/InferMatrixLab/vllm-rlt",
      "ref": "main",
      "paths": [
        "recipes/ouro-1-4b-rlt-h20-3e-bf16.yaml"
      ],
      "commit": "326f7edb8580b6c729ba52ce5b15ed67e7ab649e",
      "files": [
        {
          "path": "recipes/ouro-1-4b-rlt-h20-3e-bf16.yaml",
          "content": "schema_version: 1\nid: ouro-1-4b-rlt-h20-3e-bf16\ntitle: Ouro-1.4B synchronous BF16 serving on one H20-3e\nkind: standalone_engine\nmodel:\n  id: ByteDance/Ouro-1.4B\n  provider: ByteDance\n  task: text_generation\nengine:\n  id: vllm-rlt\n  name: vllm-rlt\n  repository: https://github.com/InferMatrixLab/vllm-rlt\n  ref: 9d601ceaf9af63121c742df5f584467e23630a0b\nruntime:\n  backend: cuda\n  precision: bf16\nhardware:\n  vendor: NVIDIA\n  model: H20-3e\n  device_count: 1\nenvironment:\n  os: Linux\n  requirements:\n  - One allocated NVIDIA H20-3e; Linux, Git, curl, uv and Python 3.10+.\n  - CUDA-enabled PyTorch 2.5+ and a compatible NVIDIA driver; use the Triton attention backend.\n  - BF16 weights need roughly 3 GB before KV and runtime buffers; this first-request configuration bounds the KV\n    pool and active sequences.\n  - Initial installation and model download require package-index, GitHub and Hugging Face access.\ncommands:\n  install: |\n    git clone https://github.com/InferMatrixLab/vllm-rlt.git\n    cd vllm-rlt\n    git checkout 9d601ceaf9af63121c742df5f584467e23630a0b\n    uv venv --python 3.10 .venv\n    source .venv/bin/activate\n    uv pip install -e .\n    python - <<'PY'\n    from huggingface_hub import snapshot_download\n    snapshot_download(repo_id=\"ByteDance/Ouro-1.4B\",\n                      revision=\"574fa66cb8bf5abdc979642d01cf2b79b16bfab1\",\n                      local_dir=\"./artifacts/models/Ouro-1.4B\",\n                      allow_patterns=[\"*.json\", \"*.safetensors\", \"*.model\", \"*.txt\"])\n    PY\n  serve: |\n    CUDA_VISIBLE_DEVICES=\"${CUDA_VISIBLE_DEVICES-0}\" python -m vllm_rlt.entrypoints.serve \\\n      --model ./artifacts/models/Ouro-1.4B --served-model-name ouro \\\n      --device cuda --dtype bfloat16 --attention-backend triton \\\n      --num-blocks 64 --max-num-seqs 1 --host 127.0.0.1 --port 8000\n  request: |\n    curl --fail http://127.0.0.1:8000/health\n    curl --fail-with-body -sS http://127.0.0.1:8000/v1/completions \\\n      -H 'Content-Type: application/json' \\\n      -d '{\"model\":\"ouro\",\"prompt\":\"The capital of France is\",\"max_tokens\":32,\"temperature\":0,\"exit_threshold\":1.0,\"stream\":false}'\n  benchmark: null\ndeployment:\n  status: unverified\n  last_verified_at: null\n  evidence_url: null\nacceptance:\n  verdict: INCONCLUSIVE\n  issue_url: null\n  evidence_url: null\nmaintainers:\n- bjf-frz\nupdated_at: '2026-10-19'\nlimitations:\n- This exact deployment ref and first-request configuration have not been re-run on H20-3e during catalog work.\n- H20-3e BF16 hardware scope is taken from the historical README benchmark; those earlier runtime results are not\n  qualification of this ref or request.\n- 'Source and historical protocol: https://github.com/InferMatrixLab/vllm-rlt/blob/9d601ceaf9af63121c742df5f584467e23630a0b/README.md'\n- Uses default synchronous execution and exit_threshold 1.0 (fixed four loops). It does not enable asynchronous\n  scheduling, preemption, self-speculation or disaggregation.\n- Async execution with preemption and async self-speculation have documented correctness/lifecycle limitations.\n  The small KV pool is a first-request setting, not a throughput profile.\n- Linux distribution and resolved CUDA/PyTorch/Triton versions must be captured during remote validation. Preserve\n  scheduler-provided CUDA_VISIBLE_DEVICES instead of overriding it.\n",
          "sha256": "a672691cd055a1fa4b7dd3a2ce808d25a05baa63615b41f0d264c9ea93a7fb1f"
        }
      ]
    },
    {
      "id": "system1-omni",
      "repository": "https://github.com/InferMatrixLab/system1-omni",
      "ref": "main",
      "paths": [
        "recipes/lfm2-5-350m-system1-l40s-fp16.yaml"
      ],
      "commit": "733be7a6a47bace63b8acfd8d455f72cbc781c9b",
      "files": [
        {
          "path": "recipes/lfm2-5-350m-system1-l40s-fp16.yaml",
          "content": "schema_version: 1\nid: lfm2-5-350m-system1-l40s-fp16\ntitle: LFM2.5-350M FP16 choice scoring on one NVIDIA L40S\nkind: standalone_engine\nmodel:\n  id: LiquidAI/LFM2.5-350M\n  provider: LiquidAI\n  task: constrained_scoring\nengine:\n  id: system1-omni\n  name: system1-omni\n  repository: https://github.com/InferMatrixLab/system1-omni\n  ref: e66ff34996760a84ad56690f0ae4cf434e669e65\nruntime:\n  backend: cuda\n  precision: fp16\nhardware:\n  vendor: NVIDIA\n  model: L40S\n  device_count: 1\nenvironment:\n  os: Linux\n  requirements:\n  - One allocated NVIDIA L40S, Python 3.12, Git, curl and a stable Rust toolchain capable of Cargo resolver 3.\n  - Install src/models/lfm2/requirements.lock.txt (recorded torch 2.14.0 and transformers 5.17.0); a compatible\n    NVIDIA driver is required.\n  - The worker loads LiquidAI/LFM2.5-350M at 9e6c6ccf47cd318696e137d381a7ded8fe4df09f.\n  - Initial Rust/Python installation and checkpoint download require network access.\ncommands:\n  install: |\n    git clone https://github.com/InferMatrixLab/system1-omni.git\n    cd system1-omni\n    git checkout e66ff34996760a84ad56690f0ae4cf434e669e65\n    python3.12 -m venv .venv-lfm\n    .venv-lfm/bin/python -m pip install -r src/models/lfm2/requirements.lock.txt\n    source .venv-lfm/bin/activate\n    cargo build --release --locked\n  serve: |\n    # First terminal, from the repository root with .venv-lfm activated:\n    (cd src/models/lfm2 && CUDA_VISIBLE_DEVICES=\"${CUDA_VISIBLE_DEVICES-0}\" python worker.py \\\n      --candidate-batch-size 16 --device cuda --dtype float16 --host 127.0.0.1 --port 8000)\n    # Second terminal, from the same repository root after worker readiness:\n    OMNI_JEV_BIND=127.0.0.1:8080 OMNI_JEV_BACKEND_URL=http://127.0.0.1:8000 \\\n      ./target/release/omni-jev\n  request: |\n    curl --fail http://127.0.0.1:8080/health\n    curl --fail-with-body http://127.0.0.1:8080/v1/systemone \\\n      -H 'Content-Type: application/json' \\\n      -d '{\"model\":\"lfm2.5-350m\",\"state\":\"I was charged twice. Please refund the duplicate charge. No hurry.\",\"questions\":{\"refund\":{\"type\":\"choice\",\"instructions\":\"Does the customer ask for a refund?\",\"criteria\":{\"yes\":\"A refund is requested\",\"no\":\"No refund is requested\"}}}}'\n  benchmark: null\ndeployment:\n  status: unverified\n  last_verified_at: null\n  evidence_url: null\nacceptance:\n  verdict: INCONCLUSIVE\n  issue_url: null\n  evidence_url: null\nmaintainers:\n- prettygirlisnotme\nupdated_at: '2026-10-19'\nlimitations:\n- This deployment ref has not been re-run on L40S during catalog work; historical reports identify earlier Python\n  source by hashes and an earlier Rust frontend commit.\n- This path is a Python Transformers FP16 worker behind a Rust frontend, not native Rust/CUDA model execution.\n- Only text state and independent choice questions are supported by this worker. Confidence is normalized entropy\n  concentration, not calibrated correctness.\n- 'Historical exact hardware, evidence and boundaries: https://github.com/InferMatrixLab/system1-omni/blob/e66ff34996760a84ad56690f0ae4cf434e669e65/recipe/lfm2/README.md'\n- The Linux distribution and current Rust/CUDA runtime must be captured in remote validation. Keep the worker and\n  frontend running in separate terminals.\n- No repository-wide root license was found at this ref. Preserve component notices, the MIT RLCD reference notice\n  and the model license; repository-wide licensing remains a maintainer follow-up.\n",
          "sha256": "5b180ce2a2e4d1bd5166654e5f95fc3a17e5e377501a390d36020593d34a988b"
        }
      ]
    }
  ]
}