feat(pflash): unified view with the SSD session cache (stacked on #746)

[Graffioh]

Stacked on #746 (head eca7265bd). #746's branch lives on a fork, so this PR targets main and its diff also shows #746's commits. This PR's own commits are eca7265bd..HEAD, listed below. Merge it together with #746: it is the PFlash mechanism the PFlash v2 post describes and measures. The SSD session cache it builds on is already in main (#839).

What it does

Unified view, on by default with a trained head. PFLASH_UNIFIED=1 joins #746's release profile, so a server started with the head and probe GGUFs serves chats and agent sessions with one mechanism. PFLASH_UNIFIED=0 keeps #746's chat view.

• What gets paged. Only the newest turn's long uncached content: the user messages since the last reply, or the outputs of the last tool calls (every tool by default, PFLASH_REGION_TOOLS). It needs at least --prefill-threshold (16K) droppable tokens and lines to cut at.

• How it is cut. At line starts. A note says the content was shortened, and a marker says how many lines were left out. The request probe's pins (the question it finds and its recency pin) stay whole. History is never paged again.

• Later turns. They continue the stored view token for token, so the prefix cache keeps working.

• Recall. A new user question brings back left-out lines as excerpts at the start of its message, through a need-based gate: nothing comes back when the lines the view does not hold carry less than θ = 0.075 of the question's score mass (PFLASH_RECALL_GATE).

• Hand-back. Requests the view cannot serve well go to the chat view and are served exactly as without the flag:

  • a cold uncached history;
  • an unreadable turn structure;
  • a remote drafter;
  • a scorer that gives no per-candidate scores (legacy, split, fold).

  A hand-back or a raw fallback ends the conversation's unified views, so no later turn revives a shortened history.

On the SSD session cache (--kv-cache-dir, #839):

• Admission counts the disk. The restorable prefix is the deeper of the RAM snapshot and the deepest disk entry, in both views. It is an index lookup and reads no file.
• Shortened conversations survive a restart and eviction. Views are kept in <kv-cache-dir>/.pflash-views, written when they leave RAM and at shutdown, and read on a RAM miss. A prompt served from a view gets a disk KV key: the tokens it served.
• Limits. At most 256 views. The cache's TTL applies, counted from a view's last write and checked at every lookup.
• Fail-closed. Every view file is checked in full before use, and one that does not load is deleted. Views of another model, template or context are never read.

With the disk cache off, decisions, records and log lines are unchanged.

R9700 checks

Setup:

• One AMD Radeon AI PRO R9700, --kv-cache-dir with defaults.
• Each restart check: shutdown (SIGTERM, flush), restart, next request.
• Workloads are built from in-tree docs.

After a restart	This PR	Without the disk work	Full prefill, cold
Raw chat, 27.3K tokens	0.99 s, served raw, 27,277 positions from disk	11.08 s, handed back and compressed whole	29.07 s
Agent session, 45.1K tokens	2.49 s, 44,721 positions from disk; answer and tool calls byte-identical to full prefill	2.58 s	56.22 s
Chat with a 27.3K paste paged to 7.8K, then a question	4.69 s: view read from disk, 7,809 positions restored, recall brought back 864 tokens	10.89 s, handed back and compressed whole	29.10 s

Every chat answer was correct in every arm.

Release-profile default check. A launch with only the model files (head, probes, drafter) installs PFLASH_UNIFIED=1, the release threshold and curve, and gate 0.075. Its served prompt is byte-identical to a launch with the release environment set explicitly.

Final build (ac7ba4526), rebased onto #746. The paged chat and its post-restart question serve byte-identical prompts to the build before the rebase:

• t1: 7,816 tokens served from 27,272;
• t2: 8,794 tokens, with the view and 7,809 positions read from disk;
• answer: correct.

TTFT was 10.4 s for t1 and 4.4 s for t2.

Known limit

The unified view does not keep #746's query pin (PFLASH_QUERY_PIN). The drafter computes that pin inside its own selection, and this view replaces that selection, so the pin is never applied here. The post's numbers were measured this way.

On the post's public R9700 runs (six sets plus LongCodeQA), the pinned text was cut in 19 of 228 paged requests. In none of those 19 did this view get wrong an answer that full prefill got right. Most of the cut pins were not the question: in LongCodeQA they land on the repository header, license or code of conduct. Passing the pin through to the view is a follow-up.

Tests

test_server_unit (HIP gfx1201 build, HIP_VISIBLE_DEVICES=-1): 994 passed, 0 failed, 6 skipped. They cover:

• Unified view: paging, recall and the gate; hand-backs, including selections without candidate scores; probe pins kept; raw fallback ending the view; drafter failure with and without PFLASH_FAIL_CLOSED.
• Release profile: it installs the unified view.
• Disk: the restorable prefix rule, a restart with the history on disk in both views, the Full-scope-only rule, and a shortened chat continued across a restart.
• Disk view store: restart, prefixes, forks, cap, lineages, identity, broken and inconsistent files, TTL at startup and at lookup, partial write.

Commits

• a69a9cdea feat(pflash): format-agnostic region module for the unified view (stage 1)
• 44469d925 feat(pflash): PFLASH_UNIFIED serves one view/recall mechanism for chat and agent turns
• 0d36150e9 fix(pflash): unified view review fixes (FlowKV fence, re-read holds, reset view, single widening)
• 82fa2b5a0 feat(pflash): need-based recall gate and fresh-minus-held recall source for the unified view
• 6e5e469c7 feat(pflash): PFLASH_RECALL_TRACE_UNITS records each region's units for an offline recall replay
• df0e9c69c fix(server): recall gate review fixes (fresh any-overlap, NaN fail-closed, new-region drops, served deficit)
• e87ac2000 fix(server): unified view takes every user message since the last reply as the newest turn
• 678a85284 feat(server): unified view release defaults, hand-back to the chat view
• 55c923fc3 feat(pflash): admission counts the disk session cache, in the unified view too
• 6755e53b8 feat(pflash): shortened conversations survive a restart and eviction on the SSD cache
• 9e88f91c1 feat(pflash): the release profile serves the unified view
• 3ffd0fa6e fix(pflash): unified view review fixes before it ships as the release default
• ac7ba4526 fix(pflash): unified view keeps views until a hand-back is served, and checks repeats and disk views

🤖 Generated with Claude Code

[davide221]

CI failed on 5278f730

Windows build fails: a new file reads a Linux-only file-timestamp field

The Windows CI build fails to compile one of the new PFlash source files. It reads a file-modification-time field that exists only on Linux and macOS, not in the Windows C runtime. The change under test is at fault, and it is a small portability fix in one function.

How to fix: Use st_mtime instead of st_mtim on Windows in pflash_file_identity()

1. In server/src/pflash/pflash_session_offload.cpp, wrap the mtime read in pflash_file_identity() in #if defined(_WIN32) (use st_mtime * 1e9) / #else (keep st_mtim).
2. Rebuild the failing target: cmake --build server/build --target luce_common.
3. Run the affected unit tests: ./server/build/test_server_unit --filter PFlashSessionOffload* (or push and let CI re-run).

_{windows compile error: non-portable stat member · high confidence · run}

Details and evidence

Failed: Build Windows (MSVC + CUDA, library + server targets) › Build Windows targets

Job Build Windows (MSVC + CUDA, library + server targets), step Build Windows targets: FAILED: CMakeFiles/luce_common.dir/src/pflash/pflash_session_offload.cpp.obj with

server\src\pflash\pflash_session_offload.cpp(294): error C2039: 'st_mtim': is not a member of 'stat'

pflash_file_identity() (server/src/pflash/pflash_session_offload.cpp:288-296, added by this PR) builds a file identity from struct stat and reads s.st_mtim.tv_sec / s.st_mtim.tv_nsec. st_mtim is a POSIX extension; MSVC's struct stat (ucrt sys/stat.h) exposes only st_mtime (whole seconds), so the file compiles on Linux/HIP but not on Windows. Nothing else in the tree uses st_mtim (the other C++ call sites use st_mtime, e.g. server/src/server/http_server.cpp:2385). The unit test for this function (server/test/unit/test_pflash_session_offload.cpp:397-408) only checks that the identity changes when the file changes — the rewrite also changes the size — so second-resolution mtime on Windows keeps the test and the identity semantics intact.

Other things to try

• Alternative: compute the identity from std::filesystem::last_write_time instead of struct stat, which is portable but needs a duration cast to keep the mtime_ns format stable.
• If more POSIX-only stat fields appear later, consider moving a portable file_mtime_ns() helper into server/src/common/platform_io.cpp, which already centralizes platform differences.

Patch (1 file)

Not applied anywhere. To try it, save the diff as fix.patch and run git apply fix.patch.

diff --git a/server/src/pflash/pflash_session_offload.cpp b/server/src/pflash/pflash_session_offload.cpp
index 5d15b523..8b913d1c 100644
--- a/server/src/pflash/pflash_session_offload.cpp
+++ b/server/src/pflash/pflash_session_offload.cpp
@@ -289,9 +289,15 @@ std::string pflash_file_identity(const std::string & path) {
     if (path.empty()) return "-";
     struct stat s {};
     if (::stat(path.c_str(), &s) != 0) return path + "|missing";
+    // st_mtim is POSIX-only; MSVC's struct stat has st_mtime (whole seconds).
+#if defined(_WIN32)
+    const long long mtime_ns = (long long) s.st_mtime * 1000000000LL;
+#else
+    const long long mtime_ns = (long long) s.st_mtim.tv_sec * 1000000000LL + (long long) s.st_mtim.tv_nsec;
+#endif
     std::ostringstream out;
     out << path << '|' << (unsigned long long) s.st_dev << ':' << (unsigned long long) s.st_ino << '|'
-        << (long long) s.st_size << '|' << (long long) s.st_mtim.tv_sec * 1000000000LL + (long long) s.st_mtim.tv_nsec;
+        << (long long) s.st_size << '|' << mtime_ns;
     return out.str();
 }

Prompt for a coding agent

A GitHub Actions run failed in Luce-Org/lucebox. Please fix it.

- Pull request #840 (branch `codex/pflash-unified-ssd`), commit 5278f7305f152dacccecb9c479614d6b8311dc16
- Workflow: CI (https://github.com/Luce-Org/lucebox/actions/runs/38299539297)
- Failed: Build Windows (MSVC + CUDA, library + server targets) › Build Windows targets

Diagnosis:
Job **Build Windows (MSVC + CUDA, library + server targets)**, step *Build Windows targets*: `FAILED: CMakeFiles/luce_common.dir/src/pflash/pflash_session_offload.cpp.obj` with

```
server\src\pflash\pflash_session_offload.cpp(294): error C2039: 'st_mtim': is not a member of 'stat'
```

`pflash_file_identity()` (server/src/pflash/pflash_session_offload.cpp:288-296, added by this PR) builds a file identity from `struct stat` and reads `s.st_mtim.tv_sec` / `s.st_mtim.tv_nsec`. `st_mtim` is a POSIX extension; MSVC's `struct stat` (ucrt `sys/stat.h`) exposes only `st_mtime` (whole seconds), so the file compiles on Linux/HIP but not on Windows. Nothing else in the tree uses `st_mtim` (the other C++ call sites use `st_mtime`, e.g. server/src/server/http_server.cpp:2385). The unit test for this function (server/test/unit/test_pflash_session_offload.cpp:397-408) only checks that the identity changes when the file changes — the rewrite also changes the size — so second-resolution mtime on Windows keeps the test and the identity semantics intact.

Proposed fix: Use st_mtime instead of st_mtim on Windows in pflash_file_identity()
1. In `server/src/pflash/pflash_session_offload.cpp`, wrap the mtime read in `pflash_file_identity()` in `#if defined(_WIN32)` (use `st_mtime * 1e9`) / `#else` (keep `st_mtim`).
2. Rebuild the failing target: `cmake --build server/build --target luce_common`.
3. Run the affected unit tests: `./server/build/test_server_unit --filter PFlashSessionOffload*` (or push and let CI re-run).

Suggested patch (a starting point):
```diff
diff --git a/server/src/pflash/pflash_session_offload.cpp b/server/src/pflash/pflash_session_offload.cpp
index 5d15b523..8b913d1c 100644
--- a/server/src/pflash/pflash_session_offload.cpp
+++ b/server/src/pflash/pflash_session_offload.cpp
@@ -289,9 +289,15 @@ std::string pflash_file_identity(const std::string & path) {
     if (path.empty()) return "-";
     struct stat s {};
     if (::stat(path.c_str(), &s) != 0) return path + "|missing";
+    // st_mtim is POSIX-only; MSVC's struct stat has st_mtime (whole seconds).
+#if defined(_WIN32)
+    const long long mtime_ns = (long long) s.st_mtime * 1000000000LL;
+#else
+    const long long mtime_ns = (long long) s.st_mtim.tv_sec * 1000000000LL + (long long) s.st_mtim.tv_nsec;
+#endif
     std::ostringstream out;
     out << path << '|' << (unsigned long long) s.st_dev << ':' << (unsigned long long) s.st_ino << '|'
-        << (long long) s.st_size << '|' << (long long) s.st_mtim.tv_sec * 1000000000LL + (long long) s.st_mtim.tv_nsec;
+        << (long long) s.st_size << '|' << mtime_ns;
     return out.str();
 }
```

Check out that commit, confirm the diagnosis, make the fix (adjust it if the diagnosis is wrong), and re-run the failing step locally before pushing.

_{Reply with /ci-fix and what to change to get a revised diagnosis.}