feat(server): add ft info and a pre-load memory preflight

docs/cli.md

@@ -7,6 +7,7 @@ ft <command> [args]
 | Command | Purpose |
 |---|---|
 | `ft serve` | Start the API server (OpenAI `/v1/*`, Anthropic `/v1/messages`, Responses) |
+| `ft info` | Forecast a model's GPU memory for a serve configuration, without loading it |
 | `ft shell` | Chat with a server in the terminal |
 | `ft ctl` | Query and manage a running server over HTTP |
 | `ft launch` | Configure and launch a coding agent against a server |
@@ -69,6 +70,7 @@ ft serve --model ... --gpu GPU-9e8d7c6b  # the same card by UUID (a unique prefi
 | Flag | Default | Meaning |
 |---|---|---|
 | `--memory-ratio` | 0.9 | Fraction of free VRAM the engine may use (weights + MoE cache + KV) |
+| `--skip-preflight` | off | Load the weights even when the pre-load memory forecast says the configuration cannot fit; see [ft info](#ft-info) |
 | `--num-pages` / `--num-tokens` | auto | KV capacity override in pages / tokens (mutually exclusive; auto sizes from VRAM left after weights and MoE cache) |
 | `--page-size` | 1 | KV page size; DSV4 forces 128, the TRTLLM backend needs 16/32/64, SWA models require 1 |
 | `--cache-type` | radix | `radix` (prefix reuse; SWA/GDN-aware variants picked automatically) or `naive` |
@@ -122,6 +124,65 @@ so a client can gate its attachment controls without reading the checkpoint conf
 | `--allowed-media-domains` | any | Comma-separated hostname allowlist for image URLs; requests for other domains are rejected with a 400. Empty allows any domain |
 | `--allowed-local-media-path` | off | Directory `file://` image refs may be read from; unset rejects local files |
 
+## ft info
+
+```bash
+ft info <model> [ft serve flags] [--json] [--gpu-memory-gib N | --gpu-free-gib N]
+```
+
+Forecasts what `ft serve <model> [flags]` would put on the GPU, and whether it
+fits, without reading a single weight: it resolves the configuration the way the
+engine does, builds the model on PyTorch's meta device, and sizes every pool
+with the engine's own sizing code. It runs in a few seconds, needs no CUDA, and
+takes every `ft serve` flag, so "what if" combinations can be tried before a
+multi-minute load:
+
+```bash
+ft info ~/models/Qwen3.8-27B-NVFP4 --text-model-only
+ft info Qwen/Qwen3-30B-A3B --gpu-memory-gib 24 --max-running-requests 1
+```
+
+The report lists:
+
+- the architecture, the format each layer type is stored in on the GPU, and the
+  checkpoint's dtypes;
+- the weights resident on the GPU per category (attention, linear attention,
+  MLP, routed experts, embeddings / lm_head, vision encoder, MTP heads, rope
+  tables) next to what the checkpoint stores, so a skipped vision tower
+  (`--text-model-only`), MTP heads the engine does not load, or offloaded experts
+  living in host RAM are visible;
+- the budget: `--memory-ratio` x free memory, minus weights, the MoE expert slot
+  cache, the GDN state pool (slots x bytes), and the KV cache (bytes per token x
+  tokens);
+- what lives outside the budget: the attention backend's workspace, the page
+  table, the CUDA graphs, the prefill activations of one `--max-prefill-length`
+  chunk, and `--vram-reserve-mb`; the "free after init" line is the number the
+  engine logs as `Free memory after initialization`;
+- a verdict -- `fits`, `tight` or `does not fit` -- with the max context and the
+  number of concurrent requests, and tips: flag changes with their predicted
+  effect, plus the cheapest combination that fits.
+
+Free memory comes from NVML for the local GPU (`--gpu` picks the card), minus
+the CUDA context the engine creates first. `--gpu-memory-gib N` plans for an
+empty card of N GiB instead, `--gpu-free-gib N` for N GiB free as `nvidia-smi`
+reports it. Without a GPU and without these flags the weights and per-token
+costs are still reported. `--json` prints the same data for tools; the exit code
+is 1 when the configuration does not fit, else 0.
+
+Weights come from the meta-built model, whose parameters are exactly what the
+loader fills (tensors the family skips are absent, re-quantized ones already
+have their stored size); the checkpoint column reads only the safetensors
+headers -- from the local directory, the Hugging Face cache, or the Hub through
+HTTP range requests. The load overhead (kernel modules, library handles), the
+prefill activations and the CUDA graph memory are estimates and are labelled so.
+
+`ft serve` (and the offline `LLM` class) runs the same forecast right before it
+loads the weights, against the free memory it has just measured. A configuration
+whose KV cache cannot be sized even before those estimates is refused within
+seconds, with the forecast and the tips in the log, instead of failing with
+`Not enough memory for KV cache` after the load. A tight one only logs a warning
+and continues. `--skip-preflight` loads regardless.
+
 ## ft shell
 
 ```bash

docs/quickstart.md

@@ -13,6 +13,14 @@ and MoE backends, cache sizes, tool-call and reasoning parsers — resolves from
 the checkpoint and the GPU; see [cli.md](cli.md) for the flags. The server is
 ready when the log reaches `API server is ready to serve on 127.0.0.1:1919`.
 
+To check first whether a model fits your GPU, and which flags to change if it
+does not, run `ft info` with the same arguments; it takes seconds and reads no
+weights:
+
+```bash
+ft info ~/models/Qwen3.6-35B-A3B
+```
+
 ## Send a request
 
 Check what is being served:

python/freetoken/cli.py

@@ -11,6 +11,7 @@ def _print_help(file: TextIO) -> None:
 
 Commands:
   serve       Start the FreeToken API server
+  info        Forecast a model's GPU memory for a serve configuration, without loading it
   shell       Chat with a FreeToken server in the terminal
   ctl         Query and manage a running FreeToken server
   daemon      Run the FreeToken supervisor (persistent engine service)
@@ -31,6 +32,17 @@ def _run_serve(argv: list[str]) -> int:
     return 0
 
 
+def _run_info(argv: list[str]) -> int:
+    import os
+
+    if "--verbose" not in argv:
+        # before the first freetoken logger exists: the level is read once
+        os.environ.setdefault("LOG_LEVEL", "WARNING")
+    from freetoken.server.info import main
+
+    return main(argv, prog="ft info")
+
+
 def _run_shell(argv: list[str]) -> int:
     from freetoken.shell import main
 
@@ -92,6 +104,7 @@ def _run_bench(argv: list[str]) -> int:
 
 COMMANDS = {
     "serve": "_run_serve",
+    "info": "_run_info",
     "shell": "_run_shell",
     "ctl": "_run_ctl",
     "daemon": "_run_daemon",

python/freetoken/engine/config.py

@@ -69,6 +69,8 @@ class EngineConfig:
     cuda_graph_max_bs: int | None = None
     page_size: int = 1
     memory_ratio: float = 0.9
+    # --skip-preflight: load the weights even when the pre-load memory forecast says the config cannot fit
+    skip_preflight: bool = False
     # Hybrid GDN models default to the HybridRadixCache (cross-request GDN-state prefix reuse);
     # `--cache-type naive` opts out. linear_state_cache_ratio sizes the GDN snapshot cache as
     # ceil(ratio * max_running_req) extra slots.

python/freetoken/engine/engine.py

@@ -76,6 +76,34 @@ def _startup_kv_budget(memory_ratio: float, init_free_memory: int, new_free_memo
     return int(memory_ratio * init_free_memory) - (init_free_memory - new_free_memory)
 
 
+def plan_moe_cache_auto(
+    config, pool_cls, *, baseline_free: int, weights_bytes: int, per_expert_bytes: int,
+    max_slots: int | None,
+) -> tuple[int, int, bool]:
+    """--moe-cache-auto's (moe_cache_size, num_pages, prefill_overlap) from the measured (or, for
+    the pre-load forecast, predicted) baseline and resident weights."""
+    from freetoken.engine.cache_budget import resolve_moe_cache_auto
+
+    cache_per_page, fixed_cache_size, page_tokens, min_reserve = pool_cls.kv_cost(config)
+    fixed_cache_size += state_pool_bytes(config)  # sibling GDN state pool, engine-summed
+    num_experts = config.model_config.num_experts
+    total_experts = config.model_config.num_moe_layers * num_experts
+    return resolve_moe_cache_auto(
+        baseline_free=baseline_free,
+        weights_bytes=weights_bytes,
+        memory_ratio=config.memory_ratio,
+        cache_per_page=cache_per_page,
+        fixed_cache_size=fixed_cache_size,
+        per_expert_bytes=per_expert_bytes,
+        num_experts=num_experts,
+        total_experts=total_experts,
+        prefill_overlap=config.moe_prefill_overlap,
+        kv_reserve_tokens=max(config.kv_reserve_tokens, min_reserve),
+        page_size=page_tokens,
+        max_slots=max_slots,
+    )
+
+
 def _page_table_width(max_seq_len: int, page_size: int) -> int:
     """Column count for the page table. ``_write_page_table`` writes WHOLE trailing pages, so the
     highest column touched is ``align_ceil(max_seq_len, page_size) - 1`` -- which the 32-alignment
@@ -357,6 +385,10 @@ def __init__(self, config: EngineConfig):
         set_rope_device(self.device)
         with torch.device("meta"), torch_dtype(config.dtype):
             self.model = create_model(config.model_config)
+        if not config.skip_preflight:
+            from .forecast import run_preflight
+
+            run_preflight(config, self.model, init_free_memory)
         self._load_weights(config)
         if config.active_encoders:
             from freetoken.models.blocks import SupportsMultimodal
@@ -592,24 +624,14 @@ def _resolve_auto_moe_cache_size(self, config: EngineConfig, banks, method=None)
         Pure glue over the Phase-1 budget policy; isolated here so it is unit-testable
         without a GPU. Reused by the Phase-2 runtime rebuild.
         """
-        from freetoken.engine.cache_budget import expert_bytes_per_slot, resolve_moe_cache_auto
+        from freetoken.engine.cache_budget import expert_bytes_per_slot
 
-        cache_per_page, fixed_cache_size, page_tokens, min_reserve = self._pool_cls.kv_cost(config)
-        fixed_cache_size += state_pool_bytes(config)  # sibling GDN state pool, engine-summed
-        num_experts = config.model_config.num_experts
-        total_experts = config.model_config.num_moe_layers * num_experts
-        return resolve_moe_cache_auto(
+        return plan_moe_cache_auto(
+            config,
+            self._pool_cls,
             baseline_free=self._baseline_free,
             weights_bytes=self._weights_bytes,
-            memory_ratio=config.memory_ratio,
-            cache_per_page=cache_per_page,
-            fixed_cache_size=fixed_cache_size,
             per_expert_bytes=expert_bytes_per_slot(banks.sources),
-            num_experts=num_experts,
-            total_experts=total_experts,
-            prefill_overlap=config.moe_prefill_overlap,
-            kv_reserve_tokens=max(config.kv_reserve_tokens, min_reserve),
-            page_size=page_tokens,
             max_slots=method.slot_limit() if method is not None else None,
         )