From c7a26f917d320aa757e2768419f2d0dd2313703d Mon Sep 17 00:00:00 2001 From: AdaWorldAPI Date: Sun, 25 Oct 2026 14:40:30 +0100 Subject: [PATCH 01/10] docs(D-DRAV-1): specify address-first DR and demand-driven value paths --- .../d-dr-address-first-value-demand-v1.md | 149 ++++++++++++++++++ 1 file changed, 149 insertions(+) create mode 100644 .claude/plans/d-dr-address-first-value-demand-v1.md diff --git a/.claude/plans/d-dr-address-first-value-demand-v1.md b/.claude/plans/d-dr-address-first-value-demand-v1.md new file mode 100644 index 000000000..b2152d50f --- /dev/null +++ b/.claude/plans/d-dr-address-first-value-demand-v1.md @@ -0,0 +1,149 @@ +# D-DRAV-1 — address-first DR: independent map and value lifetimes, demand-driven hydration + +**Status:** PROPOSAL / DOCUMENTATION-ONLY AMENDMENT (2026-10-25). No Rust, wire/ABI, storage format, optimizer rule, numeric method, benchmark, CI gate, or production implementation is introduced. This is a reviewable hypothesis and falsification plan, **not** an authorization to implement a new runtime. + +**Scope and precedence.** Specializes [D-DRF-1](d-dr64-focus-lease-foreign-cache-v1.md) (focus residency, optional foreign gather, writeback), [D-RMAP-1A](d-rmap-separate-tables-thought-palettes-v1.md) (RelationshipMap ownership and attested binding), [D-NFC](d-native-first-dependency-correction-v1.md) / [D-NFR](d-native-repr-contract-v1.md) (native-first representation), and the [Quack storage portability ruling](../board/entries/2026-10-05-quack-storage-portability.md). None of their guards is superseded. D-DRF-1's present lab *implementation* packages an 8 KiB map and one gathered facet in a single Focus; that coupling is a probe choice, **not** the required production lifetime. + +## 1. Thesis: an address map is a useful result before any value gather + +DR64/DR256 should be considered an **independently reusable, sealed relational address structure**, not merely an intermediate buffer on the way to a local value register. Once global-to-local coordinates and typed relationships have been resolved, many subsequent operations can consume the map, masks, index permutations, or admitted relation slices without re-reading original values. + +Four things must not be conflated: + +1. **Topology / address semantics:** which typed source, target, predicate, and ordinal are related in a specified revision and scope. The DR64 u16 map is 4,096 × 2 B = **8 KiB**; the DR256 u16 map is 65,536 × 2 B = **128 KiB**. These are not 512-byte SoA row copies. +2. **Relationship support:** optional Boolean masks, slice views, adjacency/permutation-like descriptors or a typed DR64³ relation. Presence/support does not automatically prove truth, multiplicity, SQL NULL status, evidence independence, or numerical magnitude. +3. **Values:** canonical SoA rows and any disposable gathered facets / derived numerical artifacts. These may be needed once, repeatedly, or not at all for a particular operation. +4. **Physical location:** a storage-version-pinned translation from canonical identity/ordinal to addressable memory or storage extents. A 16-bit DR ordinal is **not** an NVMe logical block address or a globally stable physical row identifier. + +The CSR/adjacency analogy is about **amortizing a sealed indexing structure**: resolve an irregular relationship once, then use it many times. It does not require converting every DR map into CSR, allocating a parallel graph, or replacing existing RelationOrder/index owners. Arithmetic coordinates (e.g. fixed-stride base + ordinal × row width) can be cheaper than following a stored permutation **only where a witnessed layout actually permits that formula**. + +**Not a free-lunch claim:** inexpensive ordinal arithmetic does not remove cache misses, TLB pressure, page faults, decompression, network access, I/O amplification, or the cost of proving that a relationship/support view is complete. + +## 2. Separate the lifetimes, not the semantic owners + +| Representation or resource | Retain when | Drop / recompute when | Existing authority | +| --- | --- | --- | --- | +| Sealed map / sorted relation index | The versioned binding is reused | Its actual readers end, budget policy evicts it, or a different revision is requested | RelationshipMapStore / original owner | +| FocusBinding, palette, scope and index witness | Required to interpret local ordinals | The binding is retired; a new binding must be attested before reuse | D-RMAP-1A BIND, scope/Thought-Git witnesses | +| Optional Boolean support/DR slices | A complete, typed relation-only operation reuses them | Dependency changes or their derived-cache lease ends | Existing derived relation/view owner | +| Gathered i32/16-byte facets or other local payload | A costed, admitted repeated numerical path needs them | Last dependent reader ends; values are safely rehydratable | Canonical SoA owner + DRF derived cache | +| Canonical source snapshot / row-address seal | Any borrowed reader, descriptor, historical answer or pending non-detached writer needs it | **Only** after all source-dependent uses are detached and replay/retention rules permit | Existing source owner and persistence | +| Dirty local mask and sparse change set | Uncommitted typed changes still need reduction | Owned, reconciled payload is handed off under existing failure semantics | Existing transient work / CycleWriter | + +**Map-only** means no *resident copied facet* is required for this operation; it does **not** mean a sealed map can be used after its source/binding witness expires, nor that historically required source snapshots can be deleted. A map may remain in memory while a previously materialized value cache is independently evicted. Re-fetching values must revalidate the source version, scope, index seal, projection and source lifetime. + +Hardware caches are not explicit objects to forcibly flush. Releasing a disposable software buffer is different from invalidating CPU cache lines. + +## 3. Address mapping versus value residency: optional execution shapes + +The logical query, typed relation, exact source revision, contributor closure and demanded result remain constant. BIND/Quack may choose among already-admitted capabilities; a physical winner does **not** silently create a new legality rule. + +| Shape | Reads | Candidate win | Cost / limitation | +| --- | --- | --- | --- | +| **MAP** | Map + typed masks/index support; no canonical value payload | Relationship-only operations; repeated joins/composition from complete support | Building/maintaining support; incomplete closure requires expansion, fallback or refusal | +| **BORROW** | Sealed map → live borrowed SoA facet | Few uses, resident or local rows; avoid derived value copy | Repeated irregular DRAM/cache/TLB access | +| **GATHER** | One admitted pre-fold gather → compact disposable facet → repeated kernels | Many value-dependent reuses | Cold gathering, allocation, memory budget, focus switching, changed values | +| **COMPILED** | Versioned weight/codebook/math artifact and necessary runtime inputs | Reuse a validated mathematical transformation without rehydrating original weight format | Build/calibration and identity gate; compiled value information still exists | +| **STORAGE** | Map → witnessed physical locator → async/batched pages/extents → local operand | Cold source data with reusable index, useful read ordering / batching | NVMe page/read amplification, queue depth, decompression, faults, reorder and restoration overhead | + +These are *planning candidates*, not new public types, mandatory allocations, extra storage APIs or universal DR runtime paths. In particular, DR64³ is a native *relational* focus proposal, not a requirement to copy 64³ numerical cells. + +**Prefetch has two independent jobs:** anticipate/bind the next focus map; and optionally fetch SoA value lines for an upcoming materialization or demand. A map-only operation does not owe value prefetch merely because its address map is resident. Conversely a numerical operation cannot skip required operands merely because the topology is known. + +## 4. Dirty-driven updates, immutable roots and asynchronous handoff + +**Case A: only payload values changed and the relationship/index does not depend on those values.** Retain the sealed relationship map; stage changed local positions, map them back through the **same attested original revision**, coalesce aliases by canonical destination and operation, and submit only the owned dirty-row images (or a separately admitted sparse payload). Do not rebuild unrelated address maps or transfer unchanged rows. + +**Case B: value-derived relationship/support changed.** A support/map cache depends on its inputs. Track the relevant field/source/version dependency, invalidate affected derived views and compute a **successor sealed revision** (incremental delta only if closure and index-maintenance semantics prove it complete). Reusing yesterday's support because only "values" changed would be stale. + +**Case C: the topology, physical index, source schema, palette/scope, or row addressing changed.** Rebind/reseal or refuse. Compaction can preserve stable logical identities while changing physical addresses; raw u16 equality is not sufficient. A change in access rights can invalidate authorization even with unchanged bytes. + +For every case: + +- Dirty-row reduction is typed (overwrite order, Boolean union, additive/bag, explicit conflict/refusal), not arbitrary last-writer-wins. Duplicate DR slots targeting one canonical row are reconciled once; unrelated fields survive. +- Keep **one authority for durable writes**, via existing BatchWriter/CycleWriter/WalSink. "Fire and forget" applies **only after successfully transferring an owned, detached payload** to the queue. Successful enqueue is not durability. Queue-full/error retains ownership; ambiguous acknowledgement replays the **same frozen batch**. +- A descriptor that still points to live SoA cannot release its source lease merely because it has been enqueued. Historical/replay retention is separate from transient cache residency. +- Proposed incremental map maintenance must account for deletion, inverse indexes, joins, negative dependencies and cross-focus contributor closure; "only dirty rows" is insufficient when a changed fact has nonlocal effects. + +## 5. NVMe / persistent addressing is a separate, measured physical layer + +The conceptual path is: + +~~~text +sealed DR coordinate + -> canonical SoA ordinal / stable ID (source + branch + version + index seal) + -> layout-specific row/page/fragment/extent resolver + -> resident memory, mmap/page cache or explicit read plan + -> optional local DR facet, preserving original slot/result semantics +~~~ + +For an actually contiguous, fixed-width 512-byte SoA layout, an address may be computed from **base + ordinal × 512 + facet offset**. For Lance fragments, compressed columns, reordered rows, compaction, or another backend, the mapping may require a versioned index and page/fragment lookup. No raw-LBA shortcut is assumed. S3/object storage remains artifact hydration, **not** a per-pixel random-access replacement. + +Benchmark cold-storage access using **actual I/O requested and completed**, not only nominal 512-byte field width: count physical bytes/pages/extents, reads, page faults, decompression, batching, queue depth, sorted-vs-original visitation, stable slot permutation restoration and throughput/latency including p95/p99. Sorting reads by physical address is an optional plan; if result order, bag multiplicity, floating-point reduction order or side effects cannot be preserved, it must not be applied. + +## 6. ONNX/TFLite value-demand frontier + +An external ONNX/TFLite node graph can be decoded to native operator identities, dependency order, shapes, quantization/calibration and a witness of inputs/weights. After admission and native lowering, operation classes have different *runtime value demands*: + +| Work class | May run without original source payload? | Boundary | +| --- | --- | --- | +| Topology, known shape/coordinate transform, typed dependency scheduling, complete Boolean support/relational composition | **Yes**, for those operations | Complete typed contributors and scope are essential; Boolean support is not numeric truth | +| Relationally gated numerical operation | **Sometimes** avoid loading noncontributors | A zero/presence mask is not automatically safe under IEEE NaN, ±Inf, signed zero, SQL K3, bag or evidence semantics | +| Precompiled model weights / palette LUT / factored math | **Possibly** omit fetching original weight bytes during execution | Compiled artifact carries necessary information; full model/layer/weight/codebook/calibration/snapshot identity and independent numeric/quality gates | +| Runtime activations, generic MatMul/Gemm/Softmax, value-dependent predicates, model updates | **No generic value-free claim** | Actual numerical input/weight information must still be supplied or equivalently represented | +| Candidate plasticity/learning output | May update a separate *candidate* relation overlay | Not certified truth or automatic base-map mutation; commit only with admitted provenance and independent approval | + +Thus "no longer need the values" means **either the operation is genuinely value-independent, or the required numerical information exists in a proven alternative representation**. Mathematical method compilation does not make arbitrary numeric inputs disappear. ONNX/TFLite remain interchange/conformance leaves; do not put a second graph interpreter or storage owner in them. + +## 7. Cost hypotheses and falsification, not a cache rule + +For a fixed logical workload, report separately: + +\[ +T_{\mathrm{total}} = +T_{\mathrm{bind\,+\,map}} + +T_{\mathrm{locate}} + +T_{\mathrm{hydrate}} + +T_{\mathrm{execute}} + +T_{\mathrm{reduce\,+\,handoff}} +\] + +Report initial and rebind costs, total wall time, allocation/bytes resident, actual source reads, and amortized **per-map** reuse (N=1/8/64/256). This is an accounting decomposition, **not** a model asserting these terms are always independent or additive in overlapped pipelines. A compute-only warm number cannot decide a full-run storage plan. + +[D-DRF-1's A1/A2/A3](d-dr64-focus-cache-falsifiers-v1.md) keep their original meanings: **A1** = distinct dirty canonical rows / distinct rows read; **A2** = facet reads served from resident cache / all facet reads after cold build; **A3** = relationship-only learning steps / all learning steps. **A1 and A3 are NOT RUN** without a representative write/learning stream. Map-only eligibility measured on a synthetic fixture is *not* an A3 observation. The cache-exception lapse rule remains undecided for the intended workload. + +**Falsifier matrix; pre-register fixtures and independent oracles before measurement:** + +| ID | Control / negative test | Required outcome | +| --- | --- | --- | +| AV1 | Build sealed map, release every gathered value cache, then run a complete map-only Boolean query | Same typed result and provenance as the independent scalar oracle; no value fetch on the map-only timed path | +| AV2 | Force value-dependent operation after cache eviction | Rehydration from the **exact** source revision or explicit refusal, never stale/invented values | +| AV3 | Change source numeric field that *does not* affect map/support, then change a field that *does* | First case preserves eligible topology; second invalidates derived support and seals/rebinds the successor | +| AV4 | Compaction, map/palette ABA, dataset/branch collision, ACL revocation | Stale physical/scope binding refuses even if ordinal, shape or map bytes match | +| AV5 | Aliased slots update different fields; then conflicting same-field updates | Typed deterministic coalesce preserving both fields or explicit conflict; one durable canonical destination image where admitted | +| AV6 | Queue-full, delayed WAL and lost acknowledgement | Ownership not lost; no early lease release; retry identical frozen batch; enqueue never reported as durable | +| AV7 | Same query via MAP/BORROW/GATHER, including out-of-focus contributors, NULL/UNKNOWN, Bag counts and evidence roots | Semantic parity where legal; otherwise explicit typed expansion/fallback/refusal | +| AV8 | Same map and values with sequential versus seeded shuffled visits and physical-order batched reads | Explain latency by access counts, address reuse and page distribution; identical ordered result under declared algebra | +| AV9 | Verified 4 KiB versus 2 MiB backing; distinct lines/offset histogram held fixed | Measure translation hypothesis independently of skewed line/cache-set distributions; no TLB causality claim from wall time alone | +| AV10 | ONNX/TFLite topology-only nodes, compiled weights and generic value-dependent nodes, including changed weight with same shape | Fetch only truly necessary values; stale compiled artifact refuses; compare numeric/semantic oracle and bounded approximations separately | +| AV11 | Deliberately omit one dependency contributor outside local DR64 focus | Expand by a legal witnessed mapping or refuse, never silently treat absence as FALSE/zero | +| AV12 | Sparse relation mutation that changes a transitive dependency or inverse index | Incremental map update matches full rebuild, or explicitly refuses and rebuilds; unchanged-index shortcut must fail | + +The latest [#1774 TLB lab](https://github.com/AdaWorldAPI/lance-graph/pull/1774) provides **locality evidence only**: balanced within-page offsets remove an earlier artificial 512→1,024-page jump; warm gathers get slower above roughly 2,048 pages at 4,096 distinct lines. The original 1,024-versus-65,536-row P2 gap also changed the number of distinct lines. Huge pages, hardware counters and a causal split were not verified there. None of this proves that direct NVMe reads are cheap or that map-only learning is common. + +## 8. Minimal staged next steps (not implementation authorization) + +1. **Doc review and ownership admission:** have D-RMAP-1A / D-DRF-1 owners confirm map-versus-value lifetime separation, source snapshot obligations and legal borrowed/address-only bindings. Do not modify the no-foreign-gather firewall, H14c/H14d, CE64, or Quack semantic owners in this stage. +2. **Map-only falsifier:** extend the standalone DRF lab with AV1/AV2/AV3/AV7/AV11, comparing to independent scalar semantics. Account for allocations and value reads, not just CPU cycles. +3. **Address translation vs I/O:** physically pin source layouts and benchmark borrowed resident, gathered, mmap-backed and explicit batched-read paths on identical map ordinals, with original/result permutation integrity checks. Require a genuine NVMe-capable host before claims about NVMe. +4. **Workload-driven admission:** collect real per-thought value demand, per-map reuse, changed-row counts and learning-step eligibility; report A1/A2/A3 honestly and compare whole-run P2 A/B/C with map-only as a distinct, semantically eligible case. +5. **External numeric conformance:** separately test whether native ONNX/TFLite operations are topology-only, value-dependent, or compiled-artifact reusable; no auto-learning rule or new runtime from this document. + +**Decision to postpone:** whether address-first/map-only becomes a runtime physical-plan capability, and whether a separate gather cache remains valuable for the motivating 64-thought foreign-key workload. The architecture can describe both possibilities now; only admitted workload evidence should choose. + +## Source crosswalk + +- [D-DRF-1 focus lease, cache exception and writeback](d-dr64-focus-lease-foreign-cache-v1.md) and [falsifier ledger](d-dr64-focus-cache-falsifiers-v1.md). +- [D-RMAP-1A RelationshipMap/RelationOrder and BIND owner](d-rmap-separate-tables-thought-palettes-v1.md) and [D-NFC native dependency direction](d-native-first-dependency-correction-v1.md). +- [D-DR3 holographic focused relation semantics](d-dr3-holo-onnx-s3a-s3c-v1.md) and [S3d physical-plan comparisons](d-onnx-relalg-s3d-physical-shootout-v1.md). +- [Quack storage portability ruling](../board/entries/2026-10-05-quack-storage-portability.md) and [#1774 diagnostic](https://github.com/AdaWorldAPI/lance-graph/pull/1774). From 733ba5a4764b2e3ce2bc42210848a6dc971b5287 Mon Sep 17 00:00:00 2001 From: AdaWorldAPI Date: Sun, 25 Oct 2026 14:41:00 +0100 Subject: [PATCH 02/10] docs: crosslink address-first DR review from existing owners --- .claude/plans/d-dr64-focus-lease-foreign-cache-v1.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.claude/plans/d-dr64-focus-lease-foreign-cache-v1.md b/.claude/plans/d-dr64-focus-lease-foreign-cache-v1.md index dca54ab2b..dbd415cc4 100644 --- a/.claude/plans/d-dr64-focus-lease-foreign-cache-v1.md +++ b/.claude/plans/d-dr64-focus-lease-foreign-cache-v1.md @@ -4,6 +4,8 @@ **Canonical authority:** D-RMAP-1A (separate RelationshipMap tables and thought palettes), D-NFC (native-first dependency direction), D-NFR (representation status), D-MBX persistence, and the Rust-owned SoA. This note specializes their **focus residency / cache / writeback / optional learning** gap; it does not replace their semantic ownership. +> **2026-10-25 address/value separation proposal:** [D-DRAV-1](d-dr-address-first-value-demand-v1.md) examines retaining the sealed DR map while independently evicting gathered value facets, map-only relationship execution, witnessed physical storage lookup, and value-demand-based ONNX/TFLite lowering. It is a proposed physical-plan/lifetime refinement; it does **not** loosen this plan's source lease, foreign-gather admission, dirty-writeback, or semantic constraints. + ## 0. Operator decision and thesis 1. **One DR64 focus is the common case.** It is not a mandate to allocate DR256 or a complete cubic tensor. From a21ef46f8dd0e51e0787769acf87a662cf3dc527 Mon Sep 17 00:00:00 2001 From: AdaWorldAPI Date: Sun, 25 Oct 2026 14:41:03 +0100 Subject: [PATCH 03/10] docs: crosslink address-first DR review from existing owners --- .claude/plans/d-rmap-separate-tables-thought-palettes-v1.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.claude/plans/d-rmap-separate-tables-thought-palettes-v1.md b/.claude/plans/d-rmap-separate-tables-thought-palettes-v1.md index 38116717f..e6142aa98 100644 --- a/.claude/plans/d-rmap-separate-tables-thought-palettes-v1.md +++ b/.claude/plans/d-rmap-separate-tables-thought-palettes-v1.md @@ -6,6 +6,8 @@ > > **Operator ruling for this proposal:** RelationshipMaps live in separate explicit tables, expose a zero-copy map of existing indexed values, and are not stored inside RelAlg. Each thought selects a *palette of global RelationshipMaps* using another independent table. Nothing is crammed into CausalEdge64. +> **Related physical-lifetime study (2026-10-25):** [D-DRAV-1](d-dr-address-first-value-demand-v1.md) treats the attested relationship/address map as independently reusable from optional gathered payload values, and compares map-only, borrowed, materialized and physical-storage execution. RelationshipMapStore, BIND, Quack and source ownership remain as defined here; this is **not** a new mapping authority or backend implementation. + ## 1. Ownership contract | Owner | Owns | Must not own | From f67b8edfecd7139b9359e0e7193a0cf2900f916f Mon Sep 17 00:00:00 2001 From: AdaWorldAPI Date: Sun, 25 Oct 2026 14:41:46 +0100 Subject: [PATCH 04/10] docs(D-DRAV-1): use GitHub-renderable math block --- .claude/plans/d-dr-address-first-value-demand-v1.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/.claude/plans/d-dr-address-first-value-demand-v1.md b/.claude/plans/d-dr-address-first-value-demand-v1.md index b2152d50f..b98a93d6d 100644 --- a/.claude/plans/d-dr-address-first-value-demand-v1.md +++ b/.claude/plans/d-dr-address-first-value-demand-v1.md @@ -99,14 +99,14 @@ Thus "no longer need the values" means **either the operation is genuinely value For a fixed logical workload, report separately: -\[ +$ T_{\mathrm{total}} = -T_{\mathrm{bind\,+\,map}} + +T_{\mathrm{bind+map}} + T_{\mathrm{locate}} + T_{\mathrm{hydrate}} + T_{\mathrm{execute}} + -T_{\mathrm{reduce\,+\,handoff}} -\] +T_{\mathrm{reduce+handoff}} +$ Report initial and rebind costs, total wall time, allocation/bytes resident, actual source reads, and amortized **per-map** reuse (N=1/8/64/256). This is an accounting decomposition, **not** a model asserting these terms are always independent or additive in overlapped pipelines. A compute-only warm number cannot decide a full-run storage plan. From 39adf9306d6db335dd5ae8fc5c621a5b96d8f091 Mon Sep 17 00:00:00 2001 From: AdaWorldAPI Date: Sun, 25 Oct 2026 14:58:24 +0100 Subject: [PATCH 05/10] docs(board): register D-DRAV-1 proposal and evidence boundary --- ...26-10-25-d-drav-1-address-first-value-demand.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 .claude/board/entries/2026-10-25-d-drav-1-address-first-value-demand.md diff --git a/.claude/board/entries/2026-10-25-d-drav-1-address-first-value-demand.md b/.claude/board/entries/2026-10-25-d-drav-1-address-first-value-demand.md new file mode 100644 index 000000000..ea7ebcd23 --- /dev/null +++ b/.claude/board/entries/2026-10-25-d-drav-1-address-first-value-demand.md @@ -0,0 +1,14 @@ +# D-DRAV-1 — address-first DR separates map residency from value demand + +**Date:** 2026-10-25 · **Status:** PROPOSED, documentation only / no runtime authorization · **Plan:** `.claude/plans/d-dr-address-first-value-demand-v1.md` · **PR:** [#1775](https://github.com/AdaWorldAPI/lance-graph/pull/1775) (draft) + +- **Question:** should a sealed DR64/DR256 address/relationship map remain useful after optional gathered value facets have been released, and when does a typed operation actually need source values? +- **Thesis:** map/address, Boolean support, canonical/derived values and storage physical locators are distinct objects with different costs and leases. An 8 KiB DR64 u16 map is not a 2 MiB materialized SoA. MAP (complete relationship-only support), BORROW, GATHER, COMPILED and STORAGE are **hypothetical physical execution choices**, not new semantic authorities, shipped paths or a mandate to gather. +- **Authority:** D-RMAP-1A owns RelationshipMap/RelationOrder and BIND; D-DRF-1 owns focus lease, admitted materialization and dirty writeback; D-NFC/D-NFR keep ONNX/TFLite as downstream interoperability leaves; Quack retains native query semantics. No CE64 changes, foreign-gather firewall relaxation, storage backend/ABI, numeric equivalence claim or new executor. +- **Dirty changes:** unchanged relationship topology may reuse a pinned map; value-derived relationship support requires dependency invalidation/resealing, including remote/inverse/transitive effects. Alias-safe typed per-row coalescing hands detached owned work to existing CycleWriter; enqueue is **not** durable publication. +- **I/O:** sealed logical ordinal is not a physical NVMe LBA. Storage plans must count page/extent reads, batching, read amplification, hydration, reordering and restoration to DR-local semantics; no nanosecond claims for cold random I/O. +- **Evidence boundary:** #1774 shows page-locality cost after cache-line/offset controls and verified 4 KiB mapping; shuffled visits preserve the qualitative rise. Huge-page arm, counters, physical NVMe and decomposition of the older 1,024-vs-65,536-row gap are still absent. That probe says nothing about how frequently real thought/learning operations are value-free. +- **Pre-registered:** AV1–AV12 are **QUEUED / NOT RUN** in STATUS_BOARD; AV1 and AV2 are the cheapest independent-lab candidates. A1/A3 remain **NOT RUN** without an actual workload, so D-DRF-1's scoped cache-exception lapse rule is **not decided**. +- **Revisit when:** independent MAP-vs-BORROW/GATHER oracle parity, invalidation and rehydration tests, realistic thought demand, and physical I/O results can choose a strategy without changing logical results. + +**Review questions:** independent map/value leases; value-derived relation invalidation; full-address witnesses on compaction and ACL changes; whether the five physical candidates fit existing BIND/Quack ownership. \ No newline at end of file From deb2259f7a3a444034b506219a2412c046a2bbbd Mon Sep 17 00:00:00 2001 From: AdaWorldAPI Date: Sun, 25 Oct 2026 14:58:40 +0100 Subject: [PATCH 06/10] docs(D-DRAV-1): render math and cite verified 4 KiB/shuffle probe --- .claude/plans/d-dr-address-first-value-demand-v1.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/.claude/plans/d-dr-address-first-value-demand-v1.md b/.claude/plans/d-dr-address-first-value-demand-v1.md index b98a93d6d..929b83005 100644 --- a/.claude/plans/d-dr-address-first-value-demand-v1.md +++ b/.claude/plans/d-dr-address-first-value-demand-v1.md @@ -99,15 +99,14 @@ Thus "no longer need the values" means **either the operation is genuinely value For a fixed logical workload, report separately: -$ +```math T_{\mathrm{total}} = T_{\mathrm{bind+map}} + T_{\mathrm{locate}} + T_{\mathrm{hydrate}} + T_{\mathrm{execute}} + T_{\mathrm{reduce+handoff}} -$ - +``` Report initial and rebind costs, total wall time, allocation/bytes resident, actual source reads, and amortized **per-map** reuse (N=1/8/64/256). This is an accounting decomposition, **not** a model asserting these terms are always independent or additive in overlapped pipelines. A compute-only warm number cannot decide a full-run storage plan. [D-DRF-1's A1/A2/A3](d-dr64-focus-cache-falsifiers-v1.md) keep their original meanings: **A1** = distinct dirty canonical rows / distinct rows read; **A2** = facet reads served from resident cache / all facet reads after cold build; **A3** = relationship-only learning steps / all learning steps. **A1 and A3 are NOT RUN** without a representative write/learning stream. Map-only eligibility measured on a synthetic fixture is *not* an A3 observation. The cache-exception lapse rule remains undecided for the intended workload. @@ -129,7 +128,7 @@ Report initial and rebind costs, total wall time, allocation/bytes resident, act | AV11 | Deliberately omit one dependency contributor outside local DR64 focus | Expand by a legal witnessed mapping or refuse, never silently treat absence as FALSE/zero | | AV12 | Sparse relation mutation that changes a transitive dependency or inverse index | Incremental map update matches full rebuild, or explicitly refuses and rebuilds; unchanged-index shortcut must fail | -The latest [#1774 TLB lab](https://github.com/AdaWorldAPI/lance-graph/pull/1774) provides **locality evidence only**: balanced within-page offsets remove an earlier artificial 512→1,024-page jump; warm gathers get slower above roughly 2,048 pages at 4,096 distinct lines. The original 1,024-versus-65,536-row P2 gap also changed the number of distinct lines. Huge pages, hardware counters and a causal split were not verified there. None of this proves that direct NVMe reads are cheap or that map-only learning is common. +The latest [#1774 TLB lab](https://github.com/AdaWorldAPI/lance-graph/pull/1774) provides **locality evidence only**: balanced within-page offsets remove an earlier artificial 512→1,024-page jump; warm gathers get slower above roughly 2,048 pages at 4,096 distinct lines. The original 1,024-versus-65,536-row P2 gap also changed the number of distinct lines. For the source row buffer, the lab reports `/proc/self/smaps` KernelPageSize 4 kB and AnonHugePages 0 kB, and its seeded shuffled-visit arm retains the qualitative flat-then-rise pattern. The verified 2 MiB-page arm, hardware counters, and the causal split remain outstanding. None of this proves that direct NVMe reads are cheap or that map-only learning is common. ## 8. Minimal staged next steps (not implementation authorization) From f897875ececc7f754b3770af27a09abf9ffe0bcb Mon Sep 17 00:00:00 2001 From: AdaWorldAPI Date: Sun, 25 Oct 2026 14:59:10 +0100 Subject: [PATCH 07/10] docs(board): queue D-DRAV AV1-AV12 on STATUS_BOARD --- .claude/board/STATUS_BOARD.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index f875d9e37..5ae4344eb 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -1,3 +1,23 @@ +## D-DRAV — address-first DR map and demand-driven value residency (2026-10-25) + +**Status: PROPOSAL, DOCS ONLY (#1775 draft).** Plan: [d-dr-address-first-value-demand-v1.md](../plans/d-dr-address-first-value-demand-v1.md). Ledger: [2026-10-25-d-drav-1-address-first-value-demand.md](entries/2026-10-25-d-drav-1-address-first-value-demand.md). Specializes D-DRF-1/D-RMAP-1A without a new owner, executor, ABI, storage backend or admission rule. MAP/BORROW/GATHER/COMPILED/STORAGE are candidate physical paths; A1/A3 remain **NOT RUN**, the cache-exception lapse condition undecided. #1774 is a separate locality probe, not proof of cheap NVMe random access. + +| D-id | scope | status | gate / falsifier | +|---|---|---|---| +| **D-DRAV-1** | documentation-only proposal: independent map/value leases, dirty support invalidation, physical storage addressing and native-first ONNX/TFLite value demand | Proposed, docs only (#1775 draft) | source/ACL/index seals, no value-free numerical overclaim, no runtime changes | +| **D-DRAV-AV1** | map-only after gathered values evicted | Queued (NOT RUN; not authorized) | independent scalar Boolean oracle, provenance and zero value reads | +| **D-DRAV-AV2** | value-dependent operation after cache eviction | Queued (NOT RUN; not authorized) | rehydrate exact witnessed snapshot or refuse stale access | +| **D-DRAV-AV3** | dirty value-only vs value-derived relationship change | Queued (NOT RUN; not authorized) | only dependent support invalidated; successor map/reseal or refusal | +| **D-DRAV-AV4** | compaction, ABA, cross-dataset and ACL collision | Queued (NOT RUN; not authorized) | stale mapping / scope use refuses even at identical ordinal | +| **D-DRAV-AV5** | duplicate-target dirty edits, conflicts and disjoint facets | Queued (NOT RUN; not authorized) | typed deterministic reduction, no lost fields, conflict refusal | +| **D-DRAV-AV6** | queue-full, delayed WAL, ambiguous acknowledgement | Queued (NOT RUN; not authorized) | owned payload and lease safety; same frozen batch retry | +| **D-DRAV-AV7** | MAP / BORROW / GATHER semantic parity | Queued (NOT RUN; not authorized) | preserve Bag, SQL K3, evidence and focus completeness; else refuse | +| **D-DRAV-AV8** | same ordinals in visit and storage-ordered I/O variants | Queued (NOT RUN; not authorized) | same ordered results; count reads, bytes, reorder costs | +| **D-DRAV-AV9** | verified 4 KiB vs 2 MiB backing | Queued (NOT RUN; not authorized) | offset/line controls; TLB causality not inferred from time alone | +| **D-DRAV-AV10** | ONNX/TFLite value-demand and compiled artifact witnesses | Queued (NOT RUN; not authorized) | topology-only vs runtime numerical dependency, independent oracle | +| **D-DRAV-AV11** | missing contributor outside focused relationship view | Queued (NOT RUN; not authorized) | admitted expansion or explicit refusal; never silently zero | +| **D-DRAV-AV12** | sparse mutation affecting inverse/transitive relationships | Queued (NOT RUN; not authorized) | incremental maintenance equals complete rebuilt oracle or fallback | + ## D-SAG-6 — Rust Codegraph → Mathematics → Logic → Tracert (2026-10-24) **Status: S0 PROPOSAL / DOCS ONLY.** Plan: [d-sag-6-rust-codegraph-math-logic-tracert-v1.md](../plans/d-sag-6-rust-codegraph-math-logic-tracert-v1.md). Agent prompt: [d-sag-6-rust-codegraph-logic-tracert-claude-code.md](../prompts/d-sag-6-rust-codegraph-logic-tracert-claude-code.md). Coordination: [EXO-50](https://linear.app/exo-red/issue/EXO-50/agent-message-bus-gpt-xgrok-ccsession-id). Continuation of D-SAG / D-DRMC / Ascent-BYODS, **not** a revived SoftwareGraph, separate registry, runtime or new proof authority. Parallel #1759 only reconciles existing board status. From 2e50dc9dc60ad7e6cbf22c70418ef1456be7f8a0 Mon Sep 17 00:00:00 2001 From: AdaWorldAPI Date: Sun, 25 Oct 2026 14:59:47 +0100 Subject: [PATCH 08/10] docs(board): prepend D-DRAV-1 to integration plans --- .claude/board/INTEGRATION_PLANS.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/.claude/board/INTEGRATION_PLANS.md b/.claude/board/INTEGRATION_PLANS.md index 21d4ffc58..614473efa 100644 --- a/.claude/board/INTEGRATION_PLANS.md +++ b/.claude/board/INTEGRATION_PLANS.md @@ -1,3 +1,11 @@ +## 2026-10-25 — D-DRAV-1 address-first DR: independently reusable map, demand-driven values → `.claude/plans/d-dr-address-first-value-demand-v1.md` + +**Status:** PROPOSAL / DOCUMENTATION ONLY ([#1775](https://github.com/AdaWorldAPI/lance-graph/pull/1775), draft), not a runtime, ABI, optimizer, storage-backend or ONNX/TFLite implementation. **Board entry:** [D-DRAV-1](entries/2026-10-25-d-drav-1-address-first-value-demand.md). **Falsifiers:** AV1–AV12 QUEUED / NOT RUN in [STATUS_BOARD.md](STATUS_BOARD.md); A1/A3 remain NOT RUN on actual workloads and D-DRF-1's scoped foreign-cache exception is undecided. + +The sealed DR64/DR256 address/relationship map is an independently useful, version/scope-witnessed structure, not a mandatory prelude to copying all referenced SoA values. Optional local facet caches can be evicted separately; topology-only relationship execution may avoid value fetches where contributor completeness and typed semantics are proven. Five *physical candidates* (MAP / BORROW / GATHER / COMPILED / STORAGE) retain D-RMAP-1A's map/index authority, D-DRF-1's lease and typed writeback contract, native Quack semantics, the no-foreign-gather firewall and existing CycleWriter durability boundaries. Dirty-value changes retain unaffected topology; value-derived support or physical index changes demand appropriate invalidation, successor sealing or refusal. Numeric ONNX/TFLite kernels do not become value-free merely because topology is mapped. + +**Evidence:** #1774 has independently counted cache lines and offsets, smaps-verified 4 KiB backing and a seeded shuffled-order arm; it does not prove TLB causality, huge-page benefit, NVMe economics or map-only learning frequency. First proposed lab controls: AV1/AV2 against a scalar oracle, subject to separate approval. + ## 2026-10-24 — D-SAG-6 Rust Codegraph → Mathematics → Logic → Tracert → `.claude/plans/d-sag-6-rust-codegraph-math-logic-tracert-v1.md` **Status:** S0 PROPOSAL / DOCUMENTATION ONLY, no code, benchmarks, new compiler, proof certification or source rewrite. **Prompt:** `.claude/prompts/d-sag-6-rust-codegraph-logic-tracert-claude-code.md`. **Handoff:** [Linear EXO-50](https://linear.app/exo-red/issue/EXO-50/agent-message-bus-gpt-xgrok-ccsession-id). From 0c28c47a10a7696528d64d161d4065ce0a848ba7 Mon Sep 17 00:00:00 2001 From: AdaWorldAPI Date: Sun, 25 Oct 2026 15:00:18 +0100 Subject: [PATCH 09/10] docs(board): refresh generated entry index for D-DRAV-1 --- .claude/board/entries/README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.claude/board/entries/README.md b/.claude/board/entries/README.md index baf6fd483..ebad65568 100644 --- a/.claude/board/entries/README.md +++ b/.claude/board/entries/README.md @@ -25,7 +25,7 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately opposite directions; the stranding this convention prevents shows up in exactly one of them, never both. -472 entries, 2026-08-06 .. 2026-10-25. +473 entries, 2026-08-06 .. 2026-10-25. | date | entry id | finding | file | |---|---|---|---| @@ -35,6 +35,7 @@ exactly one of them, never both. | 2026-10-25 | `D-DRF-W1` | | [2026-10-25-drf-w1-writeback.md](2026-10-25-drf-w1-writeback.md) | | 2026-10-25 | `D-DRF-G-FACET` | | [2026-10-25-drf-focuskey-facet.md](2026-10-25-drf-focuskey-facet.md) | | 2026-10-25 | `D-DRF-CHURN` | | [2026-10-25-drf-churn-policy.md](2026-10-25-drf-churn-policy.md) | +| 2026-10-25 | `D-DRAV-1` | | [2026-10-25-d-drav-1-address-first-value-demand.md](2026-10-25-d-drav-1-address-first-value-demand.md) | | 2026-10-24 | `tanh-precision-symmetric-i8` | | [2026-10-24-tanh-precision-symmetric-i8.md](2026-10-24-tanh-precision-symmetric-i8.md) | | 2026-10-24 | `D-DRF-P2` | | [2026-10-24-drf-p2-breakeven.md](2026-10-24-drf-p2-breakeven.md) | | 2026-10-24 | `D-DRF-P1` | | [2026-10-24-drf-p1-focus-lease-probe.md](2026-10-24-drf-p1-focus-lease-probe.md) | From 59bd0104b4a0f177f357561392b8e7d453332a2d Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 25 Oct 2026 17:11:09 +0000 Subject: [PATCH 10/10] board: regenerate entries index row for D-DRAV-AV1 (duplicate id) Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01G22yT6htkcdyXsihxxXdrg --- .claude/board/entries/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.claude/board/entries/README.md b/.claude/board/entries/README.md index 44bc675bd..465b9bd68 100644 --- a/.claude/board/entries/README.md +++ b/.claude/board/entries/README.md @@ -36,7 +36,7 @@ exactly one of them, never both. | 2026-10-25 | `D-DRF-TLB` | | [2026-10-25-drf-tlb-probe.md](2026-10-25-drf-tlb-probe.md) | | 2026-10-25 | `D-DRF-G-FACET` | | [2026-10-25-drf-focuskey-facet.md](2026-10-25-drf-focuskey-facet.md) | | 2026-10-25 | `D-DRF-CHURN` | | [2026-10-25-drf-churn-policy.md](2026-10-25-drf-churn-policy.md) | -| 2026-10-25 | `D-DRAV-1` | | [2026-10-25-drav-av1-membership.md](2026-10-25-drav-av1-membership.md) | +| 2026-10-25 | `D-DRAV-AV1` | | [2026-10-25-drav-av1-membership.md](2026-10-25-drav-av1-membership.md) | | 2026-10-25 | `D-DRAV-1` | | [2026-10-25-d-drav-1-address-first-value-demand.md](2026-10-25-d-drav-1-address-first-value-demand.md) | | 2026-10-24 | `tanh-precision-symmetric-i8` | | [2026-10-24-tanh-precision-symmetric-i8.md](2026-10-24-tanh-precision-symmetric-i8.md) | | 2026-10-24 | `D-DRF-P2` | | [2026-10-24-drf-p2-breakeven.md](2026-10-24-drf-p2-breakeven.md) |