[ryanManocha]

Implements item (3), capability discovery, from #375.

Extends the existing get_capabilities() contract from #385 rather than adding a
parallel entry point or a second schema, per the coordination in that thread.

What it adds

• features — three additive keys. instance_memory_limit reports True
  (landed in feat: add revisioned instance memory limits #414); operation_counters and runtime_reservation_reporting
  report False until Add KV cache operation observability #410 and item (4) land. Unlanded surfaces are reported
  False rather than omitted, so a consumer writes the detection once and the
  same code starts returning True when the surface ships.
• backends — allocator-level capabilities shared by every integration:
  KV pooling, elastic capacity, resize, trim, page size, the contiguous-layout
  default, and non_kv_memory_management: False to record the boundary agreed
  upthread (kvcached accounts for non-KV memory, never manages it).
• integrations — per-engine entries for vLLM and SGLang covering attention
  types, KV layouts, hybrid/linear state pooling, prefix caching, page-aware
  eviction, and worker IPC. These are real code-level distinctions today: only
  the vLLM shim accepts HYBRID_LINEAR through alloc_kv_cache(), and only vLLM
  evicts prefix-cache blocks page-aware. Reported statically from the shim
  contracts, so the record is queryable without importing torch or attaching to a
  running engine.
• operation_counters — empty list, enumerating the counters exposed once
  Add KV cache operation observability #410 lands.

Lifecycle/readiness is deliberately not included here: per the split agreed with
@rishabhsinha17, that key belongs to item (5) and lands with it.

Compatibility

No schema bump. New optional keys are additive, so schema_version stays
kvcached.observability.v1; consumers must treat a missing key as unsupported.
The rule is documented on get_capabilities(). No behavior changes and no new
state — the record is read-only and derived from existing contracts.

Tests

Five CPU-only contract tests added to tests/test_observability.py, following the
existing mocked-extension pattern:

• planned surfaces report False rather than being omitted;
• backend and integration records, including the vLLM/SGLang HYBRID_LINEAR split;
• enumerated snapshot fields match the dataclasses consumers actually receive;
• the whole record survives a JSON round-trip and is stable across calls;
• no private-field access anywhere in the record (recursive check).

tests/test_observability.py passes (14 tests), as do test_page_allocator_snapshot.py
and test_memory_limit_control.py. ruff, isort, codespell, and mypy pass via
pre-commit.

[rishabhsinha17]

Reviewing as the item-5 consumer: the lifecycle/readiness key from #375 hangs off this record, and the stacked PR for it is #478, so these are the things I hit while consuming the shape. Nothing here blocks; 1 and 2 are the ones worth a reply, the rest are optional.

1. Record shape works for the lifecycle key. features (kvcached/observability.py:197-212) is a flat map of bools with a comment per key and an explicit "unlanded surfaces report False" rule (:189-192). lifecycle_readiness: True slots in after runtime_reservation_reporting (:211) with the same shape, no parallel key, no schema bump. Later the phase vocabulary should be enumerable the way operation_counters is (:218), a top-level lifecycle_phases list; not for this PR.

2. "Queryable without importing torch" (docstring :147-149 and the PR body) is not true at package level. kvcached/__init__.py:14-22 imports torch unconditionally and raises ImportError without it, so import kvcached.observability needs torch installed or stubbed; the tests only run torch-free because tests/test_observability.py:12-13 puts a stub torch into sys.modules. Suggest "without importing an engine shim or attaching to a running engine", which is what the static record actually buys.

3. Static integrations claims, checked against the shims. All accurate today:

   • vllm attention_types (:154) match the guard at kvcached/integration/vllm/interfaces.py:316; sglang (:166) match kvcached/integration/sglang/interfaces.py:123.
   • kv_layouts: ["NHD"]: vllm rejects anything else (vllm/interfaces.py:319). sglang skips the layout check for MLA (sglang/interfaces.py:127, if not is_mla and kv_layout != "NHD"), so for MLA the argument is ignored rather than validated. One word in the comment ("validated") would cover it.
   • page_aware_eviction: vllm True via _page_aligned_victims and the page_aware flag (vllm/patches.py:699-771); sglang False is right, the RadixCache patch only caps evictable size (sglang/patches.py:1477-1541).
   • worker_ipc: True for both: vllm/interfaces.py:67,93 and sglang/interfaces.py:61 start the listener.
   • sglang hybrid_linear_state_pooling: True via alloc_mamba_states (sglang/interfaces.py:217).

   Drift risk: nothing ties :154/:166 to those guards. Cheapest fix is to hoist each guard's literal into a module constant (SUPPORTED_ATTENTION_TYPES) and add one test in the existing mocked-shim pattern (test_observability.py:325 already imports the vllm shim under stubs) asserting the record equals the constant; same for kv_layouts. Also, hybrid_linear_state_pooling means two different things (:156-158 same pool via HYBRID_LINEAR; :168-170 a separate entry point), so a consumer cannot tell from the bool whether mamba state appears in the pool snapshot; a note in the comment is enough for now.

4. backends values are import-time env reads. page_size_bytes and contiguous_layout_default (:137-138) come from kvcached.utils (utils.py:133, :177), which read KVCACHED_PAGE_SIZE_MB and the layout env at import. Correct in the engine process; a control plane importing kvcached in its own process gets its own env, not the engine's. KVCachePoolSnapshot.page_size_bytes (:92, :309) is the per-pool truth, so the docstring could point live consumers there.

5. No private-field leakage in the record. test_capabilities_need_no_private_field_access (:477-496) checks key names only, which is fine for a static record.

6. operation_counters lives twice: features.operation_counters: False (:209) and the top-level list (:218). Two places to flip when #410 lands; a one-line test that the list is non-empty iff the flag is True would keep them coupled. Optional.

7. Small test gaps: backends["page_size_bytes"] is only checked > 0 (:435-436), could assert equality with kvcached.utils.PAGE_SIZE; no test that integrations[*]["attention_types"] match the shim guards (see 3); no coupling test for 6.

[shipiyouniao]

Thanks for picking up item (3). I checked the implementation against the current vLLM/SGLang shims and ran tests/test_observability.py locally (14 passed). The additive get_capabilities() shape, missing-key compatibility rule, and JSON contract all look good.

Before this becomes a stable surface, I think two semantics should be tightened:

1. backends.page_size_bytes is the import-time value from this process, not an allocator capability and not necessarily the live engine's value when queried out of process. Please either name it default_page_size_bytes and document that scope, or leave runtime page size to KVCachePoolSnapshot.page_size_bytes. contiguous_layout_default already makes the distinction explicit.
2. hybrid_linear_state_pooling: true currently represents two operationally different contracts: vLLM places the state in the HYBRID_LINEAR pool, while SGLang uses a separate allocation entry point. A consumer cannot distinguish those shapes from the boolean. An additive machine-readable mode such as hybrid_linear_state_pooling_mode would keep feature detection useful without requiring consumers to parse comments.

I agree with the existing review that the package-level "without importing torch" wording should also be narrowed. I did not find another behavioral blocker.

[ryanManocha]

Thank you both, this was really helpful! I addressed everything in f651fdf and f2da050.

Page size — renamed to default_page_size_bytes and added a comment clarifying that it's this process's import-time env value, while live consumers should read KVCachePoolSnapshot.page_size_bytes. I kept it rather than dropping it since the default still seems worth reporting, and contiguous_layout_default next to it already sets that precedent.

Hybrid linear mode — added hybrid_linear_state_pooling_mode: "unified_pool" for vLLM and "separate_allocation" for SGLang. The boolean stays, so this is additive. I also updated the comments to make it clear whether that state shows up in the pool snapshot.

Torch wording — agreed on this. The tests were only passing torch-free because of the stub at test_observability.py:12-13. I reworded it to "without importing an engine shim or attaching to a running engine" and pointed to kvcached/__init__.py for the actual constraint. Updated the PR body as well.

Drift risk — both shims now define SUPPORTED_ATTENTION_TYPES / SUPPORTED_KV_LAYOUTS, and the guards read from those. test_reported_attention_types_match_the_shim_guards checks that the record stays in sync with both. I also tested this by adding a bogus attention type to the SGLang constant and confirmed the test catches it. The shim changes themselves are mechanical, with the same values and conditions as before.

Also added the counter coupling test (and renamed the list to operation_counter_names to make the two less ambiguous), the SGLang MLA layout caveat, and the PAGE_SIZE equality assertion.

I left the top-level lifecycle_phases list for later since I agree that fits better with item (5).

The record shape is unchanged by all of this, so lifecycle_readiness: True can still slot in after runtime_reservation_reporting as described for #478.

tests/test_observability.py is at 17 tests locally now.

[rishabhsinha17]

Re-checked f651fdf and f2da050 against my 09-02 review, point by point.

1. Page size. Renamed to default_page_size_bytes with the process-scope comment and the pointer to KVCachePoolSnapshot.page_size_bytes (observability.py:137-142), and the test now asserts equality with kvcached.utils.PAGE_SIZE plus absence of the old key (test_observability.py:439-440).
2. Hybrid mode. hybrid_linear_state_pooling_mode is additive next to the bool, unified_pool vs separate_allocation (observability.py:167, :184), the comments now say whether that state appears in the pool snapshot, and test_observability.py:456 pins both values.
3. Torch wording. Docstring now reads "without importing an engine shim or attaching to a running engine" with the kvcached/__init__.py caveat (observability.py:151-156). One leftover, the PR body still has the old "queryable without importing torch" line plus the pre-rename operation_counters bullet and the 14-test count; worth a quick edit since the body tends to become the squash message.
4. Drift coupling. Both shims hoist the guards into SUPPORTED_ATTENTION_TYPES / SUPPORTED_KV_LAYOUTS (vllm/interfaces.py:40-41, used at :322 and :325; sglang/interfaces.py:40-41, used at :130 and :134), the new guard conditions are value-identical to the old literals, and test_observability.py:573 imports each shim under stubs and asserts the record equals the constants for both attention types and kv layouts. The MLA layout caveat is recorded on both sides (observability.py:176-178, sglang/interfaces.py:38-39).
5. Counters. operation_counter_names rename plus the bool(names) == flag coupling test (observability.py:234, test_observability.py:476).

Ran tests/test_observability.py on f2da050 locally, 17 passed, and CI is green on 3.9 through 3.13. lifecycle_phases deferred to item (5) as agreed, and #478 still slots lifecycle_readiness in after runtime_reservation_reporting with no schema bump. Approving; the body wording is not blocking.

[shipiyouniao]

The capability semantics from my previous review are fixed: process-local defaults are named explicitly, hybrid pooling modes are distinguishable, and the record is tied to the shim guards. I re-ran the current head: 17 passed.

There is one integration-test blocker after today's #413 merge. A clean merge of this head with current main (cf46ed7) gives 16 passed / 1 failed in tests/test_observability.py. The new shim-loading fixture needs the same device-resolver stub already included in #478's rebased dependency commits. No allocator behavior change is needed. The PR body also still contains the older torch-free wording and counter-list name; those are documentation nits, not the blocker.

The blocking device-resolver fixture issue is fixed in edbb292. Validation against current main and CI passed. Dismissing my outdated change request; the author can review the appended fix.

[rishabhsinha17]

Checked the one-line stub, it matches the real resolve_gpu_device_index signature and the stubs #413 added to the other two fixtures, and on my side the merge with current main goes from 16 passed 1 failed (ImportError at vllm/interfaces.py:19) to 17 passed with it, so approving again at edbb292.