66 feat add context size trend diagram

SPEC.md

@@ -704,6 +704,7 @@ tool call's text is still stored, so older purged requests keep `file_path` NULL
 | `GET` | `/api/sessions` | List all sessions (newest first). Each session carries `status` (`active`\|`ended`\|`archived`) and `archived_at`. |
 | `GET` | `/api/sessions/{id}` | Get session detail + aggregated token stats for that session. 404 if missing. |
 | `GET` | `/api/sessions/{id}/lineage` | Complete session lineage graph. Returns exact/inferred continuation edges, diagnostic root-to-leaf paths, conservative conversation groups and membership, uncertainty diagnostics, timing, and per-edge context-delta summaries. Exact parents captured outside the selected session are external nodes. Each node also carries `conversation_code` (`C<n>` / `AUX`, as on request cards; `null` for external nodes) and `conversation_previous_request_id` / `conversation_next_request_id`, the neighbouring requests in its conversation in session order (`null` for auxiliary and external nodes; neighbours are not proven parent/child links). |
+| `GET` | `/api/sessions/{id}/trend` | Per-request metric values for the session trend chart: a `metrics` registry (`id`, `label`, `unit`, `estimated`, `empty_hint`) and one `series` per conversation (plus auxiliary), each with `points` (`request_id`, `session_seq`, `ordinal`, `time` = `started_at` else `timestamp`, `context_fidelity`, `purpose`, `values`). Metrics: `context_estimated`, `cache_hit_pct` (`cache_read / provider_input`, as request detail), `ttft_ms`, `duration_ms`; a value is `null` when unavailable. A request appears in one series only; ids from other sessions are dropped. 404 if missing. |
 | `PATCH` | `/api/sessions/{id}` | Rename a session. Body: `{ "name": "string" }`. 422 if blank, 404 if missing. |
 | `POST` | `/api/sessions/{id}/end` | End a session. 404 if missing. |
 | `POST` | `/api/sessions/{id}/archive` | Archive an ended session (one-way; repeatable): removes raw payloads and unreferenced block text, keeps all analysis data. Returns `{ session, freed: { requests, request_body_bytes, content_rows, content_bytes }, space: { auto_vacuum: "incremental"\|"none", reclaimed_bytes, free_bytes_remaining, note }, already_archived }`. 404 if missing, 409 if the session is active or an archive of it is already running. Broadcasts `session_archived` over the WebSocket. |

contextspy/analysis/accounting.py

@@ -0,0 +1,26 @@
+# Copyright 2026 Rimantas Zukaitis
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Small shared derivations over provider-reported usage."""
+from __future__ import annotations
+
+
+def cached_share_pct(cache_read_tokens: int | None, provider_input_tokens: int | None) -> float | None:
+    """Share of provider-reported input tokens served from the prompt cache, as a percentage.
+
+    ``None`` when the provider did not report input tokens (or reported zero). Single definition
+    used by the request detail ("Cached share") and the session trend chart.
+    """
+    if provider_input_tokens is None or provider_input_tokens <= 0:
+        return None
+    return (cache_read_tokens or 0) / provider_input_tokens * 100

contextspy/analysis/activity.py

@@ -6,7 +6,7 @@
 
 Derived at read time from the persisted ``source_key`` (nothing is stored), so the tables below
 can be refined without a migration. Vocabulary: read, search, edit, vcs, test, command, web,
-orchestration, mcp, other. See plans/wi0-data-foundation.md section 10.
+orchestration, mcp, other. See plans/archive/wi0-data-foundation.md section 10.
 """
 from __future__ import annotations
 

contextspy/analysis/block_hotspots.py

@@ -7,7 +7,7 @@
 Hot spots rank the blocks of a scope (a conversation or a whole session) by the visible tokens they
 add up to over all the requests that carry them. Nothing here needs block content: labels come from
 stored structure (type, tool, source key, file path, provider item type), so everything works after a
-session was archived. See plans/hot-spots.md.
+session was archived. See plans/archive/hot-spots.md.
 """
 from __future__ import annotations
 

contextspy/analysis/block_occurrences.py

@@ -8,7 +8,7 @@
 an ordered list of requests (a whole session, one conversation, or a single request). Presence is
 reported as runs of consecutive positions in that order, so a block that is in every request of a
 conversation is one run even when other conversations' requests are interleaved in session order.
-Token figures are visible-block tokens, never provider totals. See plans/info-panel.md.
+Token figures are visible-block tokens, never provider totals. See plans/archive/info-panel.md.
 """
 from __future__ import annotations
 

contextspy/analysis/paths.py

@@ -8,7 +8,7 @@
 by any extractor (structured tool arguments, shell commands, patch headers) goes through it.
 Today it validates and trims. If paths ever become a privacy problem (shared databases, exports),
 obfuscation (basename only, a salted hash, a setting) is a change inside that function plus a
-re-derivation migration, nothing else. See plans/file-paths.md.
+re-derivation migration, nothing else. See plans/archive/file-paths.md.
 
 Only a path that a known read/edit tool targets is read; no other argument (command text, URLs,
 patterns, patch bodies, secrets) is ever returned from this module.

contextspy/analysis/purpose.py

@@ -7,7 +7,7 @@
 The baseline is structural and works for every provider: it looks at the last input message
 (trailing user text vs trailing tool results) and at what the response contained. Agent-specific
 detectors (compaction, housekeeping side calls, ...) can be registered; none ships yet because
-they need captures from each agent. See plans/wi0-data-foundation.md section 5.
+they need captures from each agent. See plans/archive/wi0-data-foundation.md section 5.
 """
 from __future__ import annotations
 

contextspy/analysis/sources.py

@@ -5,7 +5,7 @@
 """What produced a block: a compact, provider-neutral ``source_key``.
 
 Keys are persisted at capture time (``blocks.source_key``) because tool-call arguments are
-purged by archive/retention. See plans/wi0-data-foundation.md section 4.
+purged by archive/retention. See plans/archive/wi0-data-foundation.md section 4.
 
 Grammar: bare keys ``system``, ``user``, ``assistant``, ``reasoning``, ``other``; otherwise
 ``<namespace>:<name>``:
@@ -17,7 +17,7 @@
 
 Parsers record program names. The only argument ever read is the file a known read/edit tool
 targets, stored separately in ``blocks.file_path`` (``analysis/paths.py`` is the single choke point);
-command text, URLs, patterns and patch bodies are never stored. See plans/file-paths.md.
+command text, URLs, patterns and patch bodies are never stored. See plans/archive/file-paths.md.
 """
 from __future__ import annotations
 

contextspy/api/routers/sessions.py

@@ -92,6 +92,16 @@ def get_session_lineage(session_id: str):
         }
 
 
+@router.get("/sessions/{session_id}/trend")
+def get_session_trend(session_id: str):
+    """Per-request metric values (context size, cache hit, TTFT, latency) per conversation."""
+    with get_db() as db:
+        if not crud.get_session(db, session_id):
+            raise HTTPException(status_code=404, detail="Session not found")
+        with db.begin_nested():
+            return crud.get_session_trend(db, session_id)
+
+
 @router.get("/sessions/{session_id}/lineage/revision")
 def get_session_lineage_revision(session_id: str):
     """Small polling response so diagnostics reloads only when evidence changes."""

contextspy/db/compaction.py

@@ -7,7 +7,7 @@
 SQLite never returns freed pages to the filesystem on its own: deleting block contents or
 nulling request bodies (retention, session archive) only grows the file's free list. ``VACUUM`` rebuilds
 the file without it, and switching the database to *incremental* auto-vacuum lets later deletions shrink
-the file online (``PRAGMA incremental_vacuum``). See plans/db-compact.md.
+the file online (``PRAGMA incremental_vacuum``). See plans/archive/db-compact.md.
 
 Everything here uses the stdlib ``sqlite3`` module directly and runs with the maintenance lock held, so
 the server cannot be running. ``VACUUM`` is atomic: an interruption leaves the original file intact.

contextspy/db/crud.py

@@ -25,7 +25,7 @@
 from sqlalchemy.orm import Session as OrmSession, load_only
 
 from contextspy.analysis.blocks import BlockType, Direction
-from contextspy.db import session_lineage_service
+from contextspy.db import session_lineage_service, trend_service
 from contextspy.db.models import BlockContent, BlockRecord, Request, Session, ToolStat
 
 if TYPE_CHECKING:
@@ -766,6 +766,12 @@ def get_timeline(
     return sorted(buckets.values(), key=lambda x: x["bucket"])
 
 
+def get_session_trend(db: OrmSession, session_id: str) -> dict:
+    """Per-request metric series grouped by conversation for the session trend chart."""
+    graph = get_session_lineage_graph(db, session_id)
+    return trend_service.build_session_trend(db, session_id, graph)
+
+
 # ---------------------------------------------------------------------------
 # Sessions summary (for dashboard timeline table)
 # ---------------------------------------------------------------------------

contextspy/db/hotspots_service.py

@@ -12,7 +12,7 @@
 (``FROM hs_scope s CROSS JOIN blocks b ON b.request_id = s.id``): left to itself SQLite scans
 ``idx_blocks_content_hash`` for a ``GROUP BY content_hash`` and is 15-100x slower. A test asserts the plan.
 Block identity is the content hash; input blocks only (output blocks reappear as input in the next request,
-so counting both would double count). See plans/hot-spots.md.
+so counting both would double count). See plans/archive/hot-spots.md.
 """
 from __future__ import annotations
 
@@ -155,7 +155,7 @@ def get_session_hotspots(
     block_type: str | None = None, source: str | None = None, in_context: str = "all",
     limit: int = 25, offset: int = 0,
 ) -> dict | None:
-    """The ranking described in plans/hot-spots.md; None when the session does not exist."""
+    """The ranking described in plans/archive/hot-spots.md; None when the session does not exist."""
     if db.get(Session, session_id) is None:
         return None
     resolved = scope_for_session(

contextspy/db/migrations.py

@@ -988,7 +988,7 @@ def _v9_match_json_paths(stored: list, parsed: list) -> dict[int, str]:
 
 
 def _migrate_to_v9(db: OrmSession) -> None:
-    """Backfill request purpose, block ``source_key`` and block ``json_path`` (plans/wi0-data-foundation.md section 8)."""
+    """Backfill request purpose, block ``source_key`` and block ``json_path`` (plans/archive/wi0-data-foundation.md section 8)."""
     _backfill_classification(db, "v9", json_paths=True)
 
 
@@ -997,7 +997,7 @@ def _migrate_to_v10(db: OrmSession) -> None:
 
     Same phase A as v9 (a database upgraded straight from before v9 has already done it, so this finds
     nothing to do). File paths come from retained tool-call content only; purged tool calls keep
-    ``file_path`` NULL, so historical coverage is partial by nature (plans/file-paths.md).
+    ``file_path`` NULL, so historical coverage is partial by nature (plans/archive/file-paths.md).
     """
     _backfill_classification(db, "v10", json_paths=False)
 

contextspy/db/models.py

@@ -28,6 +28,7 @@
 )
 from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column, relationship
 
+from contextspy.analysis.accounting import cached_share_pct
 from contextspy.analysis.activity import activity_for
 from contextspy.analysis.tokenizer import TOKENIZER_ID
 
@@ -232,11 +233,7 @@ def to_dict(self, include_raw: bool = True) -> dict:
             if provider_input is not None and provider_input > 0
             else None
         )
-        cached_share = (
-            (self.cache_read_tokens or 0) / provider_input * 100
-            if provider_input is not None and provider_input > 0
-            else None
-        )
+        cached_share = cached_share_pct(self.cache_read_tokens, provider_input)
         d["context_accounting"] = {
             "visible_input_tokens": visible_input,
             "provider_input_tokens": provider_input,
@@ -355,7 +352,7 @@ class BlockRecord(Base):
     # compact JSON array (e.g. ["messages",3,"content",1]); NULL when unknown.
     json_path: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
     # The file a read/edit tool call (and its result) targets; written only via analysis/paths.py.
-    # Kept after archive. See plans/file-paths.md.
+    # Kept after archive. See plans/archive/file-paths.md.
     file_path: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
 
     def to_dict(

contextspy/db/session_archive.py

@@ -7,7 +7,7 @@
 Archiving removes what is large and re-derivable only from the original traffic (the five request/response
 body columns and the ``block_contents`` text no other session needs) and keeps everything the analysis views
 use: block rows (hash, type, category, tokens, source key, JSON path), lineage and conversation data.
-See plans/session-archive.md.
+See plans/archive/session-archive.md.
 
 Concurrency: SQLite serialises writers, and ContextSpy's capture path keeps writing while an archive runs. Every
 write here is therefore a short transaction. The content cleanup is the delicate part: it must never delete content

contextspy/db/trend_service.py

@@ -0,0 +1,127 @@
+# Copyright 2026 Rimantas Zukaitis
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Per-request metric series for the session trend chart.
+
+Every number the chart plots is computed here (AGENTS.md: analysis lives in Python); the UI only
+formats and lays out. A metric is one entry in ``METRICS``: adding one needs no client change
+beyond unit formatting.
+"""
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Any, Callable
+
+from sqlalchemy import select
+from sqlalchemy.orm import Session as OrmSession, load_only
+
+from contextspy.analysis.accounting import cached_share_pct
+from contextspy.db.models import Request
+
+
+@dataclass(frozen=True)
+class Metric:
+    id: str
+    label: str
+    unit: str  # "tokens" | "percent" | "ms"
+    estimated: bool
+    empty_hint: str | None
+    compute: Callable[[Request], float | int | None]
+
+
+METRICS: tuple[Metric, ...] = (
+    Metric("context_estimated", "Context size (estimated)", "tokens", True, None,
+           lambda r: r.tokens_total_input),
+    Metric("cache_hit_pct", "Cache hit %", "percent", False,
+           "Provider did not report cache usage",
+           lambda r: cached_share_pct(r.cache_read_tokens, r.provider_input_tokens)),
+    Metric("ttft_ms", "TTFT", "ms", False,
+           "No streamed responses (TTFT needs streaming)", lambda r: r.ttft_ms),
+    Metric("duration_ms", "Latency", "ms", False,
+           "No completed responses", lambda r: r.duration_ms),
+)
+
+_COLUMNS = (
+    Request.id, Request.session_id, Request.session_seq, Request.timestamp, Request.started_at,
+    Request.purpose, Request.context_fidelity, Request.tokens_total_input,
+    Request.provider_input_tokens, Request.cache_read_tokens, Request.ttft_ms, Request.duration_ms,
+)
+
+
+def _point(r: Request, ordinal: int) -> dict[str, Any]:
+    when = r.started_at or r.timestamp
+    return {
+        "request_id": r.id,
+        "session_seq": r.session_seq,
+        "ordinal": ordinal,  # 1-based position in the session; Request-axis fallback for null seq
+        "time": when.isoformat(),
+        "context_fidelity": r.context_fidelity,
+        "purpose": r.purpose,
+        "values": {m.id: m.compute(r) for m in METRICS},
+    }
+
+
+def _sort_key(r: Request) -> tuple:
+    return (r.session_seq is None, r.session_seq or 0, r.started_at or r.timestamp, r.id)
+
+
+def build_session_trend(db: OrmSession, session_id: str, graph: dict) -> dict[str, Any]:
+    """One series per conversation group (plus auxiliary) of ``graph``, one point per request.
+
+    Ids that are not rows of this session (external lineage parents) are dropped; a request listed
+    in several groups is plotted only in the first one; a session request in no group falls into
+    the auxiliary series.
+    """
+    rows = list(db.execute(
+        select(Request).options(load_only(*_COLUMNS, raiseload=True))
+        .where(Request.session_id == session_id)
+    ).scalars().all())
+    ordered = sorted(rows, key=_sort_key)
+    by_id = {r.id: r for r in ordered}
+    position = {r.id: i for i, r in enumerate(ordered)}
+
+    def members(group: dict, claimed: set[str]) -> list[Request]:
+        ids = [rid for rid in group["request_ids"] if rid in by_id and rid not in claimed]
+        claimed.update(ids)
+        return [by_id[rid] for rid in sorted(ids, key=position.__getitem__)]
+
+    claimed: set[str] = set()
+    series: list[tuple[dict, bool, list[Request]]] = []
+    for group in graph["conversations"]:
+        series.append((group, False, members(group, claimed)))
+    aux_group = graph.get("auxiliary")
+    aux_members = members(aux_group, claimed) if aux_group else []
+    leftover = [by_id[rid] for rid in sorted(by_id.keys() - claimed, key=position.__getitem__)]
+    aux_all = sorted(aux_members + leftover, key=lambda r: position[r.id])
+
+    out = [
+        {"key": g["key"], "label": g["label"], "auxiliary": aux, "request_count": len(reqs),
+         "points": [_point(r, position[r.id] + 1) for r in reqs]}
+        for g, aux, reqs in series
+    ]
+    if aux_all:
+        out.append({
+            "key": aux_group["key"] if aux_group else f"session:{session_id}:auxiliary",
+            "label": aux_group["label"] if aux_group else "Auxiliary requests",
+            "auxiliary": True, "request_count": len(aux_all),
+            "points": [_point(r, position[r.id] + 1) for r in aux_all],
+        })
+    return {
+        "session_id": session_id,
+        "metrics": [
+            {"id": m.id, "label": m.label, "unit": m.unit, "estimated": m.estimated,
+             "empty_hint": m.empty_hint}
+            for m in METRICS
+        ],
+        "series": out,
+    }

docs/changelog.md

@@ -11,6 +11,16 @@
 - Two defaults changed: payloads are **no longer deleted after 7 days** (see *Retention and archive*), and new
   databases use incremental auto-vacuum.
 
+### Session trend chart
+
+- The session Summary gets a second chart next to *Token usage over time*. It plots a metric **per request**. Pick **Context size
+  (estimated)**, **Cache hit %** (same figure as *Cached share* in request detail), **TTFT** or **Latency**, plotted by
+  request number. The chart shows the latest 100 requests; pan back with the slider, the arrow buttons or by dragging,
+  and use **Latest** to return (it follows new requests while you stay at the end).
+- One line per conversation, with a checkbox to show or hide each; the auxiliary group starts hidden. The tooltip flags requests whose
+  context is only partly captured (their context size is under-reported). Click a point to open the request.
+- The selected metric is kept in the page URL. The data comes from `GET /api/sessions/{id}/trend`.
+
 ### Hot spots
 
 - **New session view: Hot spots.** Ranks what the context window keeps carrying: the blocks (tool definitions, big