Add frozen semantic-memory benchmark admission and hosted fixture controls

docs/plans/2026-10-16-semantic-memory-task1.md

@@ -0,0 +1,19 @@
+# Semantic memory Task 1: public fixture increment
+
+This implements Task 1 of the [existing frozen experiment plan](https://github.com/d6g8k5htny-coder/main/blob/b3bc164f4608e783e4c987022931915f04fa5c2d/docs/plans/2026-10-08-semantic-memory-experiment.md#task1-freeze-the-corpus-adversarial-events-and-measurement-protocol). The original 47,248-byte plan, SHA-256 `e4a8f1ab196b95727f44267ccb608925da2ee7f039232a7621b19c952f61f17b`, remains unchanged at that source. Its later tokenizer, chunking, retrieval, measurement, review and installation gates still apply. This dated addendum avoids copying its historical relative preparation-plan links into a branch that does not carry the preparation files.
+
+The active source pickup is [Main #333, comment 6205877540](https://github.com/d6g8k5htny-coder/main/pull/333#issuecomment-6205877540). The existing two-vector prerequisite and [original receipt 6086667695](https://github.com/d6g8k5htny-coder/main/pull/333#issuecomment-6086667695) remain delivered. This increment does not carry the encoder-preparation workflow or recommission its model acquisition.
+
+The input contract is 32 public synthetic documents, at most 384 KiB of extracted text, eight development questions, 40 held-out questions with eight in each of five strata, and 20 preselected generation cases with four per stratum. The seed is `20261008`; the materialized query, arm and profile orders are frozen before any ranking. Gold source/revision identities, exact half-open UTF-8 byte spans and answerability annotations remain in evaluator-only records. Corpus and request projections use closed field allowlists. A source sentence containing the requested fact is legitimate evidence; evaluator annotations and answerability cues must not enter requests.
+
+The unchanged author packet documents are retained as `experiments/semantic_memory/TASK1-CONTRACT.json` and `experiments/semantic_memory/TASK1-SOURCE-README.md`. The contract's payload inventory uses the original author's packet-relative names; it is not a claim that its README was the repository-root README. The hosted workflow separately captures the complete selected Git source inventory under the actual publication paths. Neither document belongs to the admitted corpus or request projections.
+
+The three external methods are `SemanticMemoryFixtureContract.test_frozen_case_inventory_and_hashes`, `test_gold_never_enters_index_or_prompt`, and `test_deep_and_confusable_spans_are_real` in `tests/test_semantic_memory_benchmark.py`. Both normal and optimized discovery must exercise all three without skips. The first hosted event must retain their intended missing-validator failures before product code is authored. A successor must validate the same frozen inputs, refuse malformed or unsafe inputs, preserve raw UTF-8 identities, and produce deterministic oracle-free corpus/request projections. The trusted caller controls output destinations; the product CLI has no source-selected file, network or execution destination.
+
+The [C282 source-contract handoff](https://github.com/d6g8k5htny-coder/main/pull/333#issuecomment-6205553088), delivered on 16 October 2026, reports W's current source SHA-256 as `654cdefecb16fdc5a6620cb241b393aee76296a4adcebe05357350396a90baa6`, superseding the original plan's W observation for a future integration baseline. It reports C unchanged at `e201b1cb05d33cd8d4eff3b5361e70715cc93a7afa2fee44a2d9ea92d6727b8d`. These are attributed source-read observations, not a new installed-state check or an assignment of semantic consumer ownership. Future integration must preserve W's guidance/cache and 4096/768 contract and C's separate 12000-byte message, 12288-input, 16384-context and 1024-output contract.
+
+C282 also distinguishes an incoming JSON representation hash from the hash of derived `body + newline + title` text. Task 1 pins that distinction with synthetic Unicode controls. All spans address exact extracted UTF-8 bytes; character positions, normalized strings and representation hashes cannot substitute. Deep evidence must lie after the original 12,000-character boundary while retaining its independent byte offset.
+
+Authoring and inert source inspection do not establish hosted RED or GREEN. Original run identities, logs, discovered method IDs, source/fixture hashes and native artifact bytes must be retained separately. The fixture lane runs on a disposable hosted runner in the project's restricted pinned Python container, with finite deadlines, bounded resources, source mounted read-only and no network inside the execution container. Original failures remain evidence.
+
+Task 1 does not evaluate tokenization, the final 128-chunk cap, approximately 500-token chunking, embeddings, ranking, recall, generation quality, cost savings, peer fairness or installed consumers. No model call, private corpus, installed-worker modification, mathematical acceptance or scientific-status authority is part of this increment. The full eight-part architecture goal remains active and incomplete. Scientific effect: **NONE**.

experiments/semantic_memory/BENCHMARK-ADMISSION-v1.json

@@ -0,0 +1,208 @@
+{
+  "cli": {
+    "argument_closure": "Exactly one fixtures value; exactly one report only for verify-report; no report for prepare-report/admit-benchmark; no abbreviations/help/arbitrary command/hash/ref/env authority.",
+    "commands": [
+      "prepare-report --fixtures ROOT",
+      "verify-report --fixtures ROOT --report REPORT",
+      "admit-benchmark --fixtures ROOT"
+    ],
+    "path": "tools/semantic_memory_experiment.py",
+    "refusal": "exit1; empty stdout; fixed opaque stderr without traceback",
+    "success": "exit0; canonical JSON+LF on stdout only; empty stderr"
+  },
+  "dated_source_record": "2026-10-17",
+  "evidence_limits": [
+    "Synthetic engineering benchmark only; no retrieval/vector storage/model/tokenization/quality/cost/fairness/installation or scientific promotion.",
+    "Existing dated contract/README/plan statuses remain historical; runtime and source review are distinct."
+  ],
+  "frozen_contract": {
+    "bytes": 20550,
+    "disposition": "PRESERVED_HISTORICAL_V1_SOURCE_CONTRACT; no original entry rewritten",
+    "path": "experiments/semantic_memory/TASK1-CONTRACT.json",
+    "sha256": "8a92308bbb54e1ab01c9cf3d6d9d5755fdd6fabc87e5f8214e350870ccf99f75"
+  },
+  "kind": "PROSPECTIVE_TASK1_REPORT_V2_BENCHMARK_ADMISSION_EXTENSION",
+  "prospective_scope": {
+    "all_fixture_schemas_and_raw_frozen_inputs": "UNCHANGED",
+    "replaces_only": [
+      "cli.commands",
+      "schemas.prepare-report",
+      "structural report recomputation in verify-report",
+      "admitted_corpus now requires strict frozen benchmark admission"
+    ],
+    "runtime_status": "Captured and mounted documentation under the current workflow; not read/executed by product/CLI admission logic, not admission authority and not a candidate bypass. Actual source-bound run outcomes remain separate records.",
+    "v1_reports": "The new CLI emits/recomputes report version2. Prior closed version1 reports are historical and fail exact verify-report recomputation; generate a fresh version2 structural report. This extension does not retroactively reinterpret old receipts."
+  },
+  "schema_version": 1,
+  "schemas": {
+    "admit-benchmark": {
+      "benchmark_admission": {
+        "admitted": true,
+        "benchmark_id": "semantic-memory-synthetic-20261008-v1",
+        "semantic_sha256": "8d28c546b7871916adac363f56dc58204785093686ba919f14f5a4c79cefd04f"
+      },
+      "closed_keys": [
+        "schema_version",
+        "scientific_effect",
+        "scientific_status_authority",
+        "fixture_manifest_sha256",
+        "files",
+        "counts",
+        "seed",
+        "query_order",
+        "baseline_definitions",
+        "gates",
+        "corpus_projection",
+        "request_projections",
+        "validation",
+        "pending",
+        "benchmark_admission"
+      ],
+      "corpus_projection_item_closed_keys": [
+        "source_id",
+        "revision_id",
+        "source_sha256",
+        "extraction_sha256",
+        "extractor_version",
+        "text_sha256",
+        "text_bytes",
+        "content_ref",
+        "observed_at",
+        "policy_epoch",
+        "access_state",
+        "exclusion_state",
+        "supersession_state",
+        "text"
+      ],
+      "counts_closed_keys": [
+        "documents",
+        "text_bytes",
+        "development",
+        "heldout",
+        "by_stratum",
+        "generation"
+      ],
+      "pending": {
+        "chunk_count": null,
+        "installed_consumers": "NOT_EVALUATED",
+        "quality_cost_fairness": "NOT_EVALUATED",
+        "retrieval": "NOT_EVALUATED",
+        "tokenizer_parity": "NOT_EVALUATED"
+      },
+      "request_profiles": [
+        "W",
+        "C"
+      ],
+      "request_projection_keys": [
+        "query",
+        "exact_source_id",
+        "exact_ref",
+        "task_scope",
+        "profile"
+      ],
+      "schema_version": 2,
+      "scientific_effect": "NONE",
+      "scientific_status_authority": false,
+      "validation": {
+        "deep": "passed",
+        "gold": "passed",
+        "inventory": "passed",
+        "representation": "passed"
+      }
+    },
+    "prepare-report": {
+      "benchmark_admission": {
+        "admitted": false,
+        "reason": "STRUCTURAL_VALIDATION_ONLY"
+      },
+      "closed_keys": [
+        "schema_version",
+        "scientific_effect",
+        "scientific_status_authority",
+        "fixture_manifest_sha256",
+        "files",
+        "counts",
+        "seed",
+        "query_order",
+        "baseline_definitions",
+        "gates",
+        "corpus_projection",
+        "request_projections",
+        "validation",
+        "pending",
+        "benchmark_admission"
+      ],
+      "corpus_projection_item_closed_keys": [
+        "source_id",
+        "revision_id",
+        "source_sha256",
+        "extraction_sha256",
+        "extractor_version",
+        "text_sha256",
+        "text_bytes",
+        "content_ref",
+        "observed_at",
+        "policy_epoch",
+        "access_state",
+        "exclusion_state",
+        "supersession_state",
+        "text"
+      ],
+      "counts_closed_keys": [
+        "documents",
+        "text_bytes",
+        "development",
+        "heldout",
+        "by_stratum",
+        "generation"
+      ],
+      "pending": {
+        "chunk_count": null,
+        "installed_consumers": "NOT_EVALUATED",
+        "quality_cost_fairness": "NOT_EVALUATED",
+        "retrieval": "NOT_EVALUATED",
+        "tokenizer_parity": "NOT_EVALUATED"
+      },
+      "request_profiles": [
+        "W",
+        "C"
+      ],
+      "request_projection_keys": [
+        "query",
+        "exact_source_id",
+        "exact_ref",
+        "task_scope",
+        "profile"
+      ],
+      "schema_version": 2,
+      "scientific_effect": "NONE",
+      "scientific_status_authority": false,
+      "validation": {
+        "deep": "passed",
+        "gold": "passed",
+        "inventory": "passed",
+        "representation": "passed"
+      }
+    }
+  },
+  "strict_reference": {
+    "API": "admitted_corpus calls the same admit_benchmark gate, returning corpus rows only on strict success.",
+    "authority": "Reviewed product literal, never a supplied report/manifest/hash/environment/path. Recompute from the same validated manifest/documents/raw byte snapshot used for the returned projection; no second fixture read.",
+    "benchmark_id": "semantic-memory-synthetic-20261008-v1",
+    "canonical_json": "UTF8 json.dumps(sort_keys=True,ensure_ascii=False,indent=2,allow_nan=False)+LF; arrays ordered; no Unicode/newline normalization",
+    "compatibility": "Formatting/key order of JSON and an inert sentinel-only change preserve semantic identity after raw transport descriptors are coherently rebound. Structural safety, duplicate-key/type/inventory/hash/span checks and decoded sentinel leak scan still precede fingerprinting. Coherent corpus/query/answer/order/span changes may remain structurally valid but cannot be admitted as this frozen original.",
+    "emitted_report_schema_version": 2,
+    "preimage": {
+      "benchmark_id": "trusted literal above",
+      "corpus": "manifest source order of content_ref, actual captured byte length and SHA256",
+      "development_cases": "complete parsed development-cases.json",
+      "heldout_cases": "complete parsed heldout-cases.json",
+      "manifest": "complete parsed manifest except case_files raw transport descriptors",
+      "oracle": "complete parsed heldout-oracle.json except oracle_only_sentinel",
+      "schema_version": 1
+    },
+    "reference_preimage_schema_version": 1,
+    "semantic_sha256": "8d28c546b7871916adac363f56dc58204785093686ba919f14f5a4c79cefd04f",
+    "verify_report": "Recomputes the complete canonical structural v2 report with admitted=false. Supplied admitted=true cannot confer strict admission."
+  }
+}