[linhongyu510]

Problem

_fetch_custom_model_limit (used by resolve_max_tokens and get_model_context_window) receives the full model string (custom/tools-pool) but fills _custom_ctx_cache[base_url] with the bare ids the endpoint returns (tools-pool). if model in cache / cache.get(model) therefore miss on every call, and a miss is not remembered — so both callers issue GET /v1/models before each chat completion: 184 GETs for 47 completions in one long session on 3.5.87 (#183). Endpoints that do not report max_model_len/context_window (e.g. LiteLLM's max_input_tokens) are asked forever.

Fix

Two small changes in cheetahclaws/providers.py:

• Look the cache up by bare_model(model) so the prefixed caller key and the bare response id actually match.
• Memoize a miss as 0 (returned as None), so a model the endpoint does not describe is asked about once per process instead of before every call. Transient network failures are not memoized — a later successful fetch still works.

This matches the approach described by the issue reporter's attached patch; the implementation here is written independently, with test coverage added for the cache-hit, cross-form (bare vs prefixed), memoized-miss, and network-failure paths.

Tests

New tests/test_custom_provider_cache.py, 4 tests (all fail on main, pass here):

• repeated prefixed lookups hit the cache — 1 request total;
• a bare-id lookup and a prefixed lookup of the same model share 1 request;
• an unknown model is asked about once across 3 calls;
• a network failure is not memoized: a later success still fetches.

Verification: the 4 new tests pass; regression run of test_custom_provider_cache.py + test_litellm_provider.py + test_nim_provider.py + test_prompt_cache.py + test_skills.py = 92 passed. ruff check on providers.py reports the same 16 pre-existing findings as main — no new lint issues. No live endpoint was used; the HTTP layer is stubbed.

[chauncygu]

Merged — thanks for this. #182 had already been fixed on main by #188 while this was open, so I resolved the conflict on my side rather than asking you to: main's version is kept (it also accepts args: null and guards a non-string name), and from yours I took the error that names the keys the model sent, plus your four tests. Two of them were adjusted to match main — the fake sub-run now yields a real TextChunk, and the non-string case uses a boolean because numbers are coerced.

I'm closing #187 as superseded by #188. One note in case it's useful: calling bare_model() inside _fetch_custom_model_limit regresses vLLM-style ids — the stream path already passes the id with the prefix stripped, so Qwen/Qwen3-Coder gets cut down to Qwen3-Coder and misses. Main looks the id up as given first and only then strips a leading custom/.

Both were good catches. Sorry for the overlap.