model-types: move per-model_type fixes into catalogue/model-types

AGENTS.md

@@ -23,7 +23,7 @@ llm-helper is one Go binary that manages inference servers (vLLM, mlx-lm, llama.
 
 ## Repository layout
 
-`internal/` holds the packages the binary builds from: `cli` for the commands, `control` for what the CLI and the web panel share, `web` for the panel, and the rest one package per feature. `catalogue/` holds the shipped data definitions (engines, recipes, hardware) with the Go that embeds them in `catalogue/catalogue.go`; each kind is read by the `internal/` package of the same name (`engine`, `recipe`, `hardware`), and `resolve` combines them with a model's card and config into launch settings. `docs/` is the shipped documentation, and `tools/recipediff` is a separate module.
+`internal/` holds the packages the binary builds from: `cli` for the commands, `control` for what the CLI and the web panel share, `web` for the panel, and the rest one package per feature. `catalogue/` holds the shipped data definitions (engines, recipes, model types, hardware) with the Go that embeds them in `catalogue/catalogue.go`; each kind is read by the `internal/` package of the same name (`engine`, `recipe`, `modeltype`, `hardware`), and `resolve` combines them with a model's card and config into launch settings. `docs/` is the shipped documentation, and `tools/recipediff` is a separate module.
 
 ## Essential commands
 
@@ -51,6 +51,7 @@ CI runs gofmt, vet, and the race tests on Linux and macOS, builds `linux/amd64`,
 - **On-disk compatibility:** config, profiles, and state outlive the binary. Changes to profile parsing, `EXTRA_ARGS` handling, or the service unit must still read files written by the previous release. Follow "Upgrade safety" in [RELEASING.md](RELEASING.md).
 - **Recipes:** each file in `catalogue/recipes/` is one recipe, named after its id, measured on real hardware and pinned by image digest. Do not invent or generate one. Record each deviation from upstream as data in the entry, not as a comment.
 - **Hardware:** each vendor, architecture and machine is one file in `catalogue/hardware/`, in the folder for its kind, named after its id. A machine needs a report in `catalogue/hardware/reports/` that detects it, and no two machines may match the same GPU. Do not match a GPU by its name in Go code: add a machine file.
+- **Model types:** what an architecture needs (files, flags, chat template fixes) is one file per HuggingFace `model_type` in `catalogue/model-types/`. Do not name a model type in Go; `TestNoGoCodeRepeatsAShippedModelType` fails if Go repeats what a shipped file says.
 - **Engines and sources:** an engine is one definition file in `catalogue/engines/`, named after its id, and a recipe's upstream is data that is read, never executed. Do not add code that branches on an engine's name: act on its capabilities, and put behavior a definition cannot express in a Go hook gated by a capability.
 - **Secrets:** keep API keys and tokens off command lines, out of dry-run output, and out of logs. Never commit `secrets.env`, `.env`, or a token.
 - **Dependencies:** prefer the standard library. Ask before adding a module. `tools/recipediff` is a separate module so its dependencies stay out of the CLI binary.

CHANGELOG.md

@@ -9,6 +9,31 @@ the fact.
 
 ## [Unreleased]
 
+### Added
+
+- Model types: what an architecture needs to run correctly is now data, one
+  file per HuggingFace `model_type` in `catalogue/model-types/`. `new` applies
+  the file for the repo's `model_type` and the profile's engine.
+  `llm-helper model-types list` and `show` print them, and a file in
+  `model-types.d/` in the config directory adds one or replaces a shipped one
+  without a rebuild.
+- `new` on a DeepSeek V4 repo (`deepseek_v4`: Flash, Pro, Flash Vision) adds
+  `--tokenizer-mode deepseek_v4` and the `deepseek_v4` tool-call and reasoning
+  parsers. The repos ship no chat template, so a profile without the tokenizer
+  mode could not serve chat requests.
+
+### Changed
+
+- gpt-oss profiles now carry `TIKTOKEN_ENCODINGS_BASE` themselves, and
+  `setup` no longer writes it into `config.env`. Existing configs keep the
+  line, and existing profiles serve as before.
+- `doctor` no longer reports whether the tiktoken vocab files are on disk,
+  and its JSON drops `harmony_vocab`. `new` fetches the files when a model
+  needs them.
+- The chat template fix for a system message after the first turn applies to
+  Qwen3.5 repos (`qwen3_5` and `qwen3_5_moe`, which includes Ornith), not to
+  any repo whose template contains the same text.
+
 ### Fixed
 
 - Profiles for Qwen3 hybrid models got `--reasoning-parser deepseek_r1` from

catalogue/catalogue.go

@@ -1,7 +1,8 @@
 // Package catalogue holds the definitions llm-helper ships as data, one JSON
 // file each, named by id: engines/ has the inference servers it can launch,
 // recipes/ the launch configurations that were run and measured on named
-// hardware, and hardware/ what is known about specific GPUs. Each folder's
+// hardware, model-types/ what every repo of a HuggingFace model_type needs on
+// each engine, and hardware/ what is known about specific GPUs. Each folder's
 // README says how to add one.
 package catalogue
 
@@ -13,7 +14,7 @@ import (
 // The hardware reports are test fixtures, read from disk by the tests, so
 // they are not embedded.
 //
-//go:embed engines/*.json recipes/*.json
+//go:embed engines/*.json recipes/*.json model-types/*.json
 //go:embed hardware/vendors/*.json hardware/architectures/*.json hardware/machines/*.json
 var files embed.FS
 
@@ -23,6 +24,9 @@ var Engines = sub("engines")
 // Recipes is every tested recipe shipped in the binary.
 var Recipes = sub("recipes")
 
+// ModelTypes is every model-type definition shipped in the binary.
+var ModelTypes = sub("model-types")
+
 // Hardware is every vendor, architecture and machine shipped in the binary,
 // each kind in a folder of its own.
 var Hardware = sub("hardware")

catalogue/model-types/README.md

@@ -0,0 +1,12 @@
+# Model types
+
+Each file here is one HuggingFace `model_type`, named after it: the value in a repo's `config.json`, such as `gpt_oss` in `openai/gpt-oss-120b`.
+It says what every repo of that type needs on each engine, so the finetunes and re-uploads of a model get it too.
+
+To add one, copy the closest file to `<model_type>.json`.
+Under `engines`, keyed by an engine id from [`catalogue/engines/`](../engines), it can list files to fetch (`files`), flags to add when the card did not name them (`args`), and chat template fixes (`chat_template_fixes`).
+Give each a `why`.
+A configuration for one specific repo that was run and measured belongs in [`catalogue/recipes/`](../recipes) instead.
+
+The fields are described in [docs/model-types.md](../../docs/model-types.md).
+To try one without rebuilding, put its file in `~/.config/vllm-helper/model-types.d/`; a file there with a shipped id replaces the shipped one.

catalogue/model-types/deepseek_v4.json

@@ -0,0 +1,28 @@
+{
+  "schema_version": 1,
+  "id": "deepseek_v4",
+  "display_name": "DeepSeek V4",
+  "why": "DeepSeek V4 (Flash, Pro and the Flash Vision preview) ships no chat template. It encodes chats in Python, which vLLM carries as its deepseek_v4 tokenizer mode, and writes tool calls and thinking in its own format.",
+  "engines": {
+    "vllm": {
+      "args": [
+        {
+          "flag": "--tokenizer-mode",
+          "values": ["deepseek_v4"],
+          "why": "The repos have no chat_template, so without this vLLM has no way to turn a chat request into a prompt."
+        },
+        {
+          "flag": "--tool-call-parser",
+          "values": ["deepseek_v4"],
+          "with": ["--enable-auto-tool-choice"],
+          "why": "vLLM refuses requests that let the model choose a tool unless a tool-call parser is set."
+        },
+        {
+          "flag": "--reasoning-parser",
+          "values": ["deepseek_v4"],
+          "why": "This separates the model's thinking from its answer."
+        }
+      ]
+    }
+  }
+}

catalogue/model-types/gpt_oss.json

@@ -0,0 +1,32 @@
+{
+  "schema_version": 1,
+  "id": "gpt_oss",
+  "display_name": "gpt-oss",
+  "why": "gpt-oss writes its tool calls and its thinking in OpenAI's harmony format, and harmony's tokenizer reads its vocab from disk rather than from the model repo.",
+  "engines": {
+    "vllm": {
+      "files": [
+        {
+          "dir": "tiktoken-encodings",
+          "env": "TIKTOKEN_ENCODINGS_BASE",
+          "base_url": "https://openaipublic.blob.core.windows.net/encodings/",
+          "names": ["o200k_base.tiktoken", "cl100k_base.tiktoken"],
+          "why": "Harmony's tokenizer reads these from the directory TIKTOKEN_ENCODINGS_BASE names. Without them the server starts healthy and fails every request with \"invalid tiktoken vocab file\"."
+        }
+      ],
+      "args": [
+        {
+          "flag": "--tool-call-parser",
+          "values": ["openai", "openai_gptoss", "gpt_oss"],
+          "with": ["--enable-auto-tool-choice"],
+          "why": "This model reports tool calls in its own format, and without a parser for it vLLM returns an empty message instead, which looks like the model ignoring the request. Measured on a DGX Spark serving gpt-oss-120b, where a tool request generated 47 tokens and returned nothing."
+        },
+        {
+          "flag": "--reasoning-parser",
+          "values": ["openai_gptoss", "gpt_oss", "openai"],
+          "why": "This separates the model's thinking from its answer."
+        }
+      ]
+    }
+  }
+}

catalogue/model-types/qwen3_5.json

@@ -0,0 +1,18 @@
+{
+  "schema_version": 1,
+  "id": "qwen3_5",
+  "display_name": "Qwen3.5",
+  "why": "Qwen3.5 and models built on it (Ornith among them) ship a chat template that raises an error on a system message after the first turn.",
+  "engines": {
+    "vllm": {
+      "chat_template_fixes": [
+        {
+          "contains": "System message must be at the beginning",
+          "match": "(\\{%-\\s*if\\s+not\\s+loop\\.first\\s*%\\}\\s*)\\{\\{-\\s*raise_exception\\('System message must be at the beginning\\.'\\)\\s*\\}\\}(\\s*\\{%-\\s*endif\\s*%\\})",
+          "replace": "${1}{# llm-helper: dropped instead of raising #}${2}",
+          "why": "This model's bundled template crashes if a system-role message isn't first in the conversation, which agent harnesses like Coder Agents send. The patched copy drops a misplaced system message instead."
+        }
+      ]
+    }
+  }
+}

catalogue/model-types/qwen3_5_moe.json

@@ -0,0 +1,18 @@
+{
+  "schema_version": 1,
+  "id": "qwen3_5_moe",
+  "display_name": "Qwen3.5 MoE",
+  "why": "Qwen3.5 and models built on it (Ornith among them) ship a chat template that raises an error on a system message after the first turn.",
+  "engines": {
+    "vllm": {
+      "chat_template_fixes": [
+        {
+          "contains": "System message must be at the beginning",
+          "match": "(\\{%-\\s*if\\s+not\\s+loop\\.first\\s*%\\}\\s*)\\{\\{-\\s*raise_exception\\('System message must be at the beginning\\.'\\)\\s*\\}\\}(\\s*\\{%-\\s*endif\\s*%\\})",
+          "replace": "${1}{# llm-helper: dropped instead of raising #}${2}",
+          "why": "This model's bundled template crashes if a system-role message isn't first in the conversation, which agent harnesses like Coder Agents send. The patched copy drops a misplaced system message instead."
+        }
+      ]
+    }
+  }
+}

docs/install.md

@@ -46,7 +46,6 @@ See [Hardware detection](./hardware.md).
 | `MODEL_BUDGET_GB`         | config.env  | Overrides the measured model budget. See [Memory budget](./memory.md).                                                                     |
 | `KV_CONTEXT_TARGET`       | config.env  | How many full-length requests each profile's KV cache is sized to hold. Default `4`.                                                       |
 | `METAL_WIRED_LIMIT_MB`    | config.env  | macOS only. GPU wired memory limit in MiB, applied at boot. Written by `llm-helper metal-limit --set`. See [Running on macOS](./macos.md). |
-| `TIKTOKEN_ENCODINGS_BASE` | config.env  | Where gpt-oss-style models find their tiktoken vocab files.                                                                                |
 | `UPDATE_CHANNEL`          | config.env  | Where `upgrade` looks for builds: `stable` or `dev`. Written when you pass `--channel`.                                                    |
 | `CODER_URL`               | config.env  | Coder deployment to sync with. See [Coder integration](./coder.md).                                                                        |
 | `CODER_AI_PROVIDER_ID`    | config.env  | The `openai-compat` provider to hang model entries off.                                                                                    |

docs/manifest.json

@@ -31,6 +31,11 @@
       "description": "Finding a model that fits this machine, and choosing a quant.",
       "path": "./models.md"
     },
+    {
+      "title": "Model types",
+      "description": "What a model's architecture needs to run correctly, and how to add or change it.",
+      "path": "./model-types.md"
+    },
     {
       "title": "Memory budget",
       "description": "How llm-helper decides what this machine can hold, and how to override it.",
@@ -155,6 +160,23 @@
           "description": "Show or raise how much unified memory macOS lets the GPU use",
           "path": "./reference/cli/metal-limit.md"
         },
+        {
+          "title": "llm-helper model-types",
+          "description": "List what specific model architectures need to run correctly",
+          "path": "./reference/cli/model-types.md",
+          "children": [
+            {
+              "title": "llm-helper model-types list",
+              "description": "List the model types in effect",
+              "path": "./reference/cli/model-types/list.md"
+            },
+            {
+              "title": "llm-helper model-types show",
+              "description": "Show a model type's definition",
+              "path": "./reference/cli/model-types/show.md"
+            }
+          ]
+        },
         {
           "title": "llm-helper new",
           "description": "Pull a new HuggingFace model, save it as a profile, and switch to it",

docs/model-types.md

@@ -0,0 +1,69 @@
+---
+title: Model types
+description: What a model's architecture needs to run correctly, and how to add or change it.
+---
+
+Some architectures need more than their model card says.
+gpt-oss needs vocab files on disk and parsers for its harmony format, and Qwen3.5's chat template raises an error on a system message after the first turn.
+llm-helper keeps this as data, one file per HuggingFace `model_type` in [`catalogue/model-types/`](https://github.com/coder/llm-helper/tree/main/catalogue/model-types).
+`new` and the panel read the repo's `config.json`, find the file for its `model_type`, and apply what it says for the engine the profile runs on.
+A GGUF repo is matched by the `model_type` of the model it was quantized from.
+
+A model type applies to every repo of that architecture, including finetunes and re-uploads.
+A launch configuration for one specific repo, run and measured on named hardware, is a [recipe](./recipes.md).
+
+`llm-helper model-types list` shows the model types in effect, and `llm-helper model-types show <id>` shows what one does.
+
+## Fields
+
+```json
+{
+  "schema_version": 1,
+  "id": "gpt_oss",
+  "display_name": "gpt-oss",
+  "why": "Why this architecture needs anything at all.",
+  "engines": {
+    "vllm": {
+      "files": [
+        {
+          "dir": "tiktoken-encodings",
+          "env": "TIKTOKEN_ENCODINGS_BASE",
+          "base_url": "https://openaipublic.blob.core.windows.net/encodings/",
+          "names": ["o200k_base.tiktoken", "cl100k_base.tiktoken"],
+          "why": "What fails without them."
+        }
+      ],
+      "args": [
+        {
+          "flag": "--tool-call-parser",
+          "values": ["openai", "openai_gptoss"],
+          "with": ["--enable-auto-tool-choice"],
+          "why": "What fails without it."
+        }
+      ],
+      "chat_template_fixes": [
+        {
+          "contains": "text that marks the broken template",
+          "match": "a regular expression",
+          "replace": "its replacement, with ${1} for groups",
+          "why": "What fails without it."
+        }
+      ]
+    }
+  }
+}
+```
+
+- `id` is the `model_type` value, and the file is named after it.
+- `engines` is keyed by engine id. An engine with no entry gets nothing added. `files` apply on any engine. `args` and `chat_template_fixes` are in vLLM's flag syntax, so they are only accepted for engines with the `vllm_args` capability.
+- `files` are fetched into `dir` under the data directory when a profile is created, unless already there. With `env` set, the profile gets that variable pointing at the directory.
+- `args` are added to the profile's `EXTRA_ARGS` unless the card already named the flag. `values` are tried in order and the first the installed engine registers is used; if none is registered, the flag is left out. `with` flags are added alongside.
+- `chat_template_fixes` apply when the repo's `chat_template.jinja` contains `contains`. The patched copy is saved under the data directory and passed with `--chat-template`. If the text is there but `match` does not match, nothing is patched and `new` says so.
+
+Each file is checked when it loads: an unknown field, an unknown engine, `args` or `chat_template_fixes` for an engine without `vllm_args`, a bad regular expression, a `dir` outside the data directory or a `base_url` that is not https is refused.
+
+## Changing one locally
+
+Put a file in `~/.config/vllm-helper/model-types.d/`, named after its `model_type`.
+A new id adds a model type, and a shipped id replaces the shipped file, so you can change what llm-helper does for an architecture without rebuilding.
+A file that fails to load is skipped, and `doctor` and `model-types list` say why.

docs/reference/cli.md

@@ -36,6 +36,7 @@ loaded at a time.
 | [`list`](cli/list.md)                 | List saved model profiles                                                                              |
 | [`logs`](cli/logs.md)                 | Show the server log, following by default on a terminal                                                |
 | [`metal-limit`](cli/metal-limit.md)   | Show or raise how much unified memory macOS lets the GPU use                                           |
+| [`model-types`](cli/model-types.md)   | List what specific model architectures need to run correctly                                           |
 | [`new`](cli/new.md)                   | Pull a new HuggingFace model, save it as a profile, and switch to it                                   |
 | [`node`](cli/node.md)                 | Register machines by HTTP URL and SSH target                                                           |
 | [`pool`](cli/pool.md)                 | Run a router in front of replicas serving the same model                                               |

docs/reference/cli/model-types.md

@@ -0,0 +1,19 @@
+---
+# Code generated by go run ./scripts/gendocs. DO NOT EDIT.
+title: "llm-helper model-types"
+description: "List what specific model architectures need to run correctly"
+---
+
+<!-- DO NOT EDIT | GENERATED CONTENT -->
+
+A model type is the model_type in a HuggingFace repo's config.json, such as gpt_oss.
+Its definition says what every repo of that type needs on each engine: files to
+fetch, flags to add, chat template fixes. 'new' applies it to the profile it writes.
+They ship in the binary, and a file in <config>/model-types.d adds one or replaces
+a shipped one with the same id, without a rebuild.
+
+## Usage
+
+```console
+llm-helper model-types
+```

docs/reference/cli/model-types/list.md

@@ -0,0 +1,26 @@
+---
+# Code generated by go run ./scripts/gendocs. DO NOT EDIT.
+title: "llm-helper model-types list"
+description: "List the model types in effect"
+---
+
+<!-- DO NOT EDIT | GENERATED CONTENT -->
+
+List the model types in effect: the shipped ones, and any added or replaced by
+files in <config>/model-types.d, which are shown with the file they came from.
+
+## Usage
+
+```console
+llm-helper model-types list [flags]
+```
+
+## Options
+
+### --json
+
+|      |        |
+| ---- | ------ |
+| Type | `bool` |
+
+print the model types as JSON