Skip to content

feat(memory): enforce subject-bound atomic extraction - #2078

Merged
hidb4ai merged 18 commits into
oceanbase:masterfrom
beatenevo:feat/enforce-memory
Oct 28, 2026
Merged

hidb4ai merged 18 commits into
oceanbase:masterfrom
beatenevo:feat/enforce-memory

Conversation

@beatenevo

@beatenevo beatenevo commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Which issue or RFC does this PR close?

Closes #2066.

Rationale for this change

Atomic Memory extraction could preserve imperative or second-person wording from historical evidence, causing durable preferences and constraints to be represented as instructions for a future agent.

This change strengthens the extraction contract so retained memories are subject-bound declarative propositions, while task-specific procedures remain outside Atomic Memory.

What changes are included in this PR?

  • Version the coding and conversation Atomic Memory extraction prompts to .v2.
  • Require preferences, decisions, and constraints to identify their subject explicitly.
  • Prevent instruction-like evidence from becoming commands or second-person memory.
  • Exclude task-specific procedures and troubleshooting workflows unless they independently establish a durable fact, decision, or constraint.
  • Preserve literal commands, code, paths, configuration keys, and quoted factual values.
  • Keep the new rules in invariant instructions so custom prompts cannot remove them.
  • Add a versioned bilingual extraction evaluation with deterministic retention and instruction-leakage scoring.
  • Add safe live-run, replay, reporting, and pinning commands.
  • Include a pinned content-free live-run report and run manifest.
  • Add evaluation tests to the default pytest collection.

The pinned live-run report used:
evaluation/memory/atomic_extraction/reports/atomic-extraction-20261019-03/report.md

  • Provider/model: deepseek / deepseek:deepseek-flash
  • Temperature: 0
  • Max tokens: 1024
  • Timeout: 120s
  • Max requests: 2
  • Overall: 9/9 passed
  • Factual retention: 6/6
  • False rejections: 0
  • Instruction leakage: 0

Are there any user-facing changes?

Atomic Memory extraction now produces subject-bound declarative statements instead of instruction-shaped memories.

There are no public API or schema changes. Existing Atomic Memory records are not rewritten or migrated, and search, ranking, reconciliation, lifecycle, and context preparation behavior remain unchanged.

How was this change tested?

  • make check
  • uv run pytest evaluation/memory/atomic_extraction/tests
    • 24 passed
  • uv run pytest tests/builtin/artifacts/atomic_memory tests/builtin/artifacts/prompt tests/builtin/artifacts/memory
    • 208 passed, 3 skipped
  • Replayed the pinned live run successfully with status=passed.
  • Verified the pinned dataset, prompt identities, and report remain consistent with the current contract.

AI usage statement

OpenAI Codex (GPT-5) was used to assist with implementation, tests, validation, and review fixes. DeepSeek deepseek:deepseek-flash was used only for the documented opt-in live generation evaluation.

@AlexStocks AlexStocks added the 1.4.1 v1.4.1 release tracking label Oct 20, 2026

@hireview001 hireview001 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Resolve merge conflicts with the target branch.

@hidb4ai hidb4ai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The scorer still certifies instruction-bearing runs and rejects a valid attributed preference. See the inline finding and the follow-up in the existing attribution thread.

@hidb4ai hidb4ai added 1.5.0 v1.5.0 release tracking and removed 1.4.1 v1.4.1 release tracking labels Oct 24, 2026

@hidb4ai hidb4ai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The finite-attribution exemption can suppress a later independent directive, allowing an instruction-bearing run to be pinned as leak-free. See the inline reproduction.

@hireview001 hireview001 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hidb4ai hidb4ai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed c1485da1: the reported scoring regressions are fixed, and replay/pin rejects the prior instruction-bearing outputs while preserving valid attributed preferences. No remaining blocking findings within the reviewed scope. This change adds evaluation tooling without new HTTP/MCP/SDK business APIs, database tables, or migrations.

@hidb4ai
hidb4ai merged commit d1e5aef into oceanbase:master Oct 28, 2026
74 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

1.5.0 v1.5.0 release tracking

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(memory): enforce declarative Atomic Memory extraction and family routing

4 participants