Report per-request prefill/decode timings in API responses (llama.cpp-style timings block) #813

Before you start

• I have read the Roadmap and this is not already planned there.
• I have read the FAQ.
• I have searched existing issues and found no duplicate.

Applies to

Engine

What problem does this solve

Proxies and dashboards that sit in front of OpenAI-compatible backends (llama-swap, Open WebUI) display per-request cache hits and prefill/decode throughput, but only read them from the llama.cpp timings object. Against FreeToken those columns stay empty: responses carry only usage.prompt_tokens / completion_tokens, and the cache hit only appears with --enable-cache-report. FreeToken's own numbers (/v1/stats aggregates) exist, but there is no per-request report in the API response.

Proposed solution

Attach a llama.cpp-shaped timings object next to usage in the response finish paths of the OpenAI, Responses (and optionally Anthropic) APIs:

"timings": {
  "prompt_n": 191412, "prompt_ms": 25100.0, "prompt_per_second": 149.7,
  "predicted_n": 376, "predicted_ms": 24700.0, "predicted_per_second": 15.2,
  "cache_n": 190569
}

All values are already available in the server layer: token counts and cached_tokens accumulate in the ack-consume loop (server/generation.py), and first-token/total timings match the per-request ttft_ms / duration_ms the request ring already records. Rates follow llama.cpp: prompt_per_second = (prompt_n - cache_n) / prompt_ms, predicted_per_second = predicted_n / predicted_ms. Engine and scheduler unchanged.

Suggested details:

• Streamed responses emit the block on the final chunk (with usage, per stream_options.include_usage).
• cache_n gated on --enable-cache-report to keep its sglang-parity semantics, or always on to match llama.cpp; either way document it in docs/cli.md.
• Draft counters (draft_n / draft_n_accepted for the MTP path) are out of scope here; a follow-up issue can cover them.

Alternatives considered

• vLLM-style metrics.time_to_first_token_ms / mean_itl_ms: also widely parsed, but one timings block covers token counts, rates and cache hits in the format the most tooling already reads.