Skip to content

feat(memory): persist versioned Source evidence and neutral Atomic Memory projection - #2004

Open
MaoMengww wants to merge 8 commits into
oceanbase:masterfrom
MaoMengww:feature/1975-source-evidence-declarations
Open

MaoMengww wants to merge 8 commits into
oceanbase:masterfrom
MaoMengww:feature/1975-source-evidence-declarations

Conversation

@MaoMengww

@MaoMengww MaoMengww commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Which issue or RFC does this PR close?

Closes #1975.

Implements the Source authority and verification prerequisite in RFC #1652, while preserving the Source access and lineage_only semantics in RFC #1458.

Rationale for this change

Source generation eligibility does not describe the authority or verification of its material. Atomic Memory needs explicit, versioned evidence declarations that remain inspectable after registration changes, restarts, and projection rebuilds.

This change records those facts without introducing trust scores, ranking bonuses, or additional write authority.

What changes are included in this PR?

  • Add MemoryEvidenceDeclaration and exact Source material bindings through the existing Source Definition registry.
  • Accept declarations only from trusted native adapters or deployment-owned configuration for exact remote Definition manifest identities.
  • Retain historical Definition manifests, Source declaration bindings, and evidence snapshots keyed to exact Atomic Memory revisions.
  • Add an authorized Python evidence-read API and reuse the existing projection rebuild path to verify retained historical material.
  • Resolve missing, invalid, or unverifiable declaration records to neutral untrusted / unknown values. Missing or corrupt authoritative Source material and database failures remain errors.
  • Add the additive p0005 evidence schema migration for SQLite and MySQL-compatible backends, without backfilling historical trust.
  • Add compact SQLite/OceanBase acceptance scenarios covering revision history, write rollback, registration drift, cross-Scope isolation, missing-snapshot neutrality, synthetic model trust claims, and unchanged recall consumers.

Are there any user-facing changes?

This adds optional Source Definition evidence declarations and a Python API for inspecting an exact Atomic Memory revision's evidence.

It does not change Atomic Memory bodies or content hashes, Artifact revision identities, HTTP/OpenAPI search schemas, public search scores, or citation identities. It introduces no quality ranking or new generation eligibility rules.

Persistence gains additive evidence tables. Existing Sources and Memory revisions without accepted declarations remain neutral; current configuration does not retroactively attest them.

How was this change tested?

  • make check: passed, including lock consistency, pre-commit checks, generated guidance checks, and type checking.
  • make contract-test: 52 passed.
  • make opendal-test: 8 passed; connector lint, formatting, and type checks passed.
  • Full suite with doctests and the required DSH configuration runtime, using Python 3.12.13 / SQLite 3.50.4:
    • 5294 passed, 385 skipped, 0 failures.
    • Command: uv run --frozen --python 3.12 python -m pytest -n 6 --doctest-modules --require-dsh-config-runtime
  • Focused Source, persistence, migration, lifecycle, and recall regressions: 128 passed.
  • Real OceanBase CE 4.4.2.1 evidence acceptance: 3 passed, 0 skipped.
  • Wheel and source distribution builds passed; the wheel contains the new evidence migration resources.

The initial full run under the local Python 3.11 / SQLite 3.37.2 environment exposed an existing Topic Memory vector-prefilter compatibility failure. A standalone SQLite reproducer isolated the cause: sqlite-vec's rowid IN KNN prefilter requires SQLite 3.38 or newer. The original test and full suite passed with the newer SQLite runtime, without changing code, assertions, or skip conditions.

The complete GitHub Actions Python-version matrix has not been run locally.

AI usage statement

AI assistance was used through Oh My Pi with gpt-6.1sol for implementation, test design and consolidation, code inspection, and validation analysis.

Implement issue01 of the Source evidence declarations specification. Bind trusted native and remote declarations to exact Source material and retain per-revision Atomic Memory snapshots in the existing transaction.

Preserve neutral fallback, exact identities, scope authorization and public retrieval contracts. Add versioned p0004 migrations and SQLite persistence regressions.

Refs oceanbase#1975. Full suite: 5255 passed, 431 skipped, 7 failed; introduced observability adapter failures fixed and its file passes 18 tests. Topic Memory vector failure also reproduces at the unchanged baseline.
Cover persisted declarations, scope isolation, historical neutrality, synthetic
model trust claims, and unchanged recall consumers on SQLite and OceanBase.
Preserve the published p0004 artifact handle revision and introduce evidence as p0005. Retain both schema intents, recompute exact SQLite fingerprints, and update MySQL migration head expectations.

@hidb4ai hidb4ai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address the two native Source evidence issues below before merging. Both reproduce through public SQLite flows and break evidence inspection and projection rebuild after successful Memory generation.

Comment thread src/powercontext/builtin/persistence/sources.py
Comment thread src/powercontext/builtin/persistence/sources.py Outdated

@hidb4ai hidb4ai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Native Source evidence inspection and projection rebuild pass the input-alias and model-default scenarios. Remote observation ingestion still fails for serialization aliases; please address the inline P2 before merging.

Comment thread src/powercontext/sources/observations.py

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(memory): add versioned Source evidence declarations and neutral lifecycle projection

2 participants