Skip to content

feat: retain compaction memory and evidence in the project blackboard - #355

Merged
Ed1s0nZ merged 16 commits into
AIPentest:mainfrom
Sycun:codex/blackboard-context-memory
Oct 16, 2026
Merged

Ed1s0nZ merged 16 commits into
AIPentest:mainfrom
Sycun:codex/blackboard-context-memory

Conversation

@Sycun

@Sycun Sycun commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Context compaction now keeps a retrievable continuation summary on the project blackboard and preserves independent pre-compaction evidence snapshots. The host adds the actual snapshot path and history directory to both the continuation context and blackboard body. Existing summary sections 1–7 and raw-user-input collection remain; section 8 adds only tool-evidence keywords and key progress, with no extra model call or standalone index files.

Automatic note/context-* working records are scoped to their source conversation, including its agents, planners and executors. Initial injection, post-compaction refresh and default get/list/search exclude other conversations and unknown origins. Explicit include_history=true enables historical reference within the same authorized project; source/time labels and relationship filtering keep those records from appearing as current work. Ordinary shared project facts retain their behavior, and the system checkpoint prefix is protected from tool overwrites. SQLite checks the current project binding in the same statement as the checkpoint upsert, preventing stale runs from writing after rebinds.

Generated user-role summaries and skills retain their provenance across classic/native conversion and the actual persisted-trace resume boundary. Recovery carries only allowlisted origin fields and the host's structured original-user ledger, rerenders that ledger under current budgets, and drops old system instructions. Assistant/tool text or a generated summary cannot supply human-ledger entries by imitating text tags. Internal metadata stays out of provider request JSON. Chinese ADK regression tests exercise three rounds of compaction, persistence, recovery and recompaction, plus the real database loader, multipart text recovery and the actual native before-model system-message merge. Canonical ledger metadata takes precedence over legacy text during both classic and native merging. Generated PlanExecute templates have separate allowlisted provenance and carry the human ledger from unflattened inputs before executor/replanner rewriting. Eino’s deterministic original-user section is populated from real human entries rather than template plan/tool text; role/source labels remain in working input and snapshots. Context-overflow and transient restarts preserve only the canonical human ledger while discarding old instructions. Formatting-aware budgets retain a bounded first long human input instead of dropping it completely. This cannot reconstruct provenance already discarded by older traces.

Snapshots use transcript-<first 5 MD5 hex characters>-<timestamp>.txt, with nanoseconds and timezone. Full-content comparison handles retry deduplication and short-prefix collisions; prior SHA-256 filenames remain valid. Snapshots are atomically written and retained for the conversation-artifact lifecycle. A failed compatibility transcript.txt update no longer hides a successfully saved immutable snapshot; failed snapshot creation still produces no success hint. The timestamp is first-save time, not a strict successful-compaction counter.

Checkpoint bodies replace the previous summary per conversation/agent scope; full bodies and raw snapshots remain outside automatic injection. Brief labels preferentially reuse section 8's existing evidence clues, with a legacy fallback and a 200-character cap. Index budgets include framing, omission notices and relationship overviews. A successful empty index build removes the stale old block while preserving other system context and the user ledger; database errors retain the old block. New key writes reject surrounding whitespace, while SQL scope filters handle existing Unicode-padded keys without migrating them. Loaded historical records with an unchanged ID/project/exact key remain editable, preserving relationship references. project.context_checkpoints defaults to enabled and can disable automatic blackboard checkpoints independently of snapshots.

Attack-chain analysis now labels generated summaries, skills and plan templates by their persisted provenance instead of presenting them as new human input. The host's canonical original-user ledger remains available as historical scope; old system instructions are omitted, and graph nodes still require actual tool records. A collapsible sidebar displays only metadata-marked summaries from the current conversation's latest saved trace, including exact snapshot references and full-text copy. Cached graphs also expose current context without changing nodes, counts, filters or exports. The notice distinguishes current context from an older cached graph. Summaries render as plain text; this adds no snapshot file-read API or cross-conversation blackboard query. Request ownership and sequence guards prevent stale responses, errors or polling from overwriting another view. Busy regeneration no longer deletes the cache before acquiring its lock, and cached single-node graphs return empty edge arrays.

Validation on the PR's independent source and dependencies: go test ./..., go build ./..., go test -race ./internal/agent ./internal/attackchain ./internal/app ./internal/database ./internal/handler ./internal/multiagent ./internal/project ./internal/projectprompt, node --test web/static/js/*.test.cjs, and git diff --check passed. Desktop and mobile dark-theme sidebar behavior was checked with synthetic context in a local browser fixture. Added Context memory CI directly runs those regression suites on Ubuntu, macOS and Windows. All six Context memory and Self-update checks passed for commit 2b76b313. Tests cover actual resume provenance, forged-ledger rejection, historical scope/pagination/edges/authorization, atomic project binding, snapshot retention/deduplication/collisions/failures, classic/native middleware, legacy padded keys, reduced-budget refresh and evidence-label selection. A production-helper test recovers a synthetic Cookie absent from both the 2001-line tool-output preview and summary via real reduction, snapshot persistence, resume and segmented filesystem reads in classic/native paths. Chat/approval interactions and API routes retain their behavior; the attack-chain sidebar adds the summary/retrieval entry described above. Design and boundaries are documented in docs/zh-CN/blackboard-context-memory.md.

@Sycun
Sycun marked this pull request as ready for review October 11, 2026 16:32
Sycun added 13 commits October 12, 2026 00:33
…retrieval

Carry allowlisted summary origin and host user ledger across the trace recovery boundary. Keep immutable snapshot links after alias failures, scope legacy padded keys, remove stale indexes under small budgets, reuse retrieval clues for labels, and add cross-platform memory CI.
… legacy edits

Mark generated execution prompts, populate deterministic user sections from actual human entries, preserve canonical ledger on overflow/transient restarts, fit long human entries under their formatted budget, and keep unchanged historical keys editable. Add real reduction-to-snapshot Cookie recovery and production entry-point regression tests.
…oard

Attack chain: failure text goes to textContent, so opt out of i18next
interpolation escaping that rendered upstream errors as &#x2F;/&quot;;
give regenerate its own message key; keep the summary card expanded across
a 409 refresh; probe for a cached graph before parsing last_react_input so
polling no longer re-reads a multi-megabyte column; hide the card in the
node-details exclusive state and bound it on narrow screens; drop the dead
hardcoded Chinese title from the API payload.

Blackboard: stop mirroring note/context-* keys into the fact body links
section, which get_project_fact returns verbatim without scope filtering;
single-line category and confidence in index rows so a newline cannot inject
a fake row; make the path overview fail closed on an empty index set.

Provenance: reuse fitUserIntentLedgerText when sanitizing a ledger entry so
a single entry cannot exceed its cap by the three ellipsis runes.

The .cjs harness now escapes interpolated values like real i18next, so the
existing XSS test also pins the escapeValue opt-out.
A node claiming something ran must cite a tool_call_id the host issued in
the trajectory. A citation absent from the trajectory is invented, so the
node is dropped along with its edges; a missing citation counts as invented
only when every real ID survived budget fitting, so a trimmed payload cannot
silently empty a graph. Judging is skipped where the evidence cannot be
enumerated: the process-detail supplement dumps raw event JSON and the
messages-table fallback carries no call IDs. Verified citations land in node
metadata rather than tool_execution_id, which is foreign-keyed to
tool_executions.

Also close two smaller holes: fact text can no longer carry the host index
delimiters, which would close the block early and spill the remaining index
into the system prompt; and the console fact endpoints reject the reserved
note/context-* prefix, matching the MCP tool layer.
@Ed1s0nZ
Ed1s0nZ merged commit 3c0d57f into AIPentest:main Oct 16, 2026
6 checks passed
@Sycun
Sycun deleted the codex/blackboard-context-memory branch October 16, 2026 15:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants