From 12333ab9cfe0ba761efc9cb3b23214c129b3cff1 Mon Sep 17 00:00:00 2001 From: AdaWorldAPI Date: Wed, 4 Nov 2026 07:48:34 +0100 Subject: [PATCH 1/7] docs(D-DRIVER-ZC-0): preregister zero-copy driver membrane and virtual bus algebra --- .../d-driver-zc-virtual-bus-contract-v1.md | 197 ++++++++++++++++++ 1 file changed, 197 insertions(+) create mode 100644 .claude/plans/d-driver-zc-virtual-bus-contract-v1.md diff --git a/.claude/plans/d-driver-zc-virtual-bus-contract-v1.md b/.claude/plans/d-driver-zc-virtual-bus-contract-v1.md new file mode 100644 index 000000000..56e4f1781 --- /dev/null +++ b/.claude/plans/d-driver-zc-virtual-bus-contract-v1.md @@ -0,0 +1,197 @@ +# D-DRIVER-ZC-0: Virtual zero-copy access driver + address-preserving bus algebra + +> **2026-11-04 · PROPOSAL / PRE-REGISTRATION / DOCS ONLY.** No runtime, public driver API, format, compiler rewrite, socket, second source identity authority, SIMD opcode, benchmark or theorem proof is shipped here. This document refines, **without superseding**, [D-DRISA-0](d-drisa-remote-register-shader-v1.md) and the operator's [zero-copy lens law](../knowledge/zero-copy-lens-law.md). D-DRISA-1 (#1955) and D-DRISA-3 (#1956) are now **merged lab evidence**, not future stages. +> +> **Operator north star:** Apply the *ndarray SIMD polyfill architectural move* to **data access**. Instead of trying to audit roughly 17,000 independently written calls, require admitted consumers to pass through **one narrow, typed virtual driver/lens boundary**. A single witnessed DR address supplies local tile or admitted foreign/global-to-local access, the driver borrows canonical SoA data once under a source lease, and existing algebra reads the same address through logical XY/XZ/XX/YY/ZZ buses without duplicating value columns or constructing row objects. Remote calls request *data-local execution*, not remote Rust references. +> +> **Tracking:** [EXO-154](https://linear.app/exo-red/issue/EXO-154) owns this narrower zero-copy/bus research program; [EXO-153](https://linear.app/exo-red/issue/EXO-153) owns its broader D-DRISA execution program; [EXO-50](https://linear.app/exo-red/issue/EXO-50) is append-only message coordination. [Staged PR charter](d-driver-zc-staged-pr-charter-v1.md), [falsifiers](d-driver-zc-falsifier-matrix-v1.md), [board entry](../board/entries/2026-11-04-d-driver-zc-0-virtual-bus-contract.md). + +## 1. New architectural ruling proposed, NOT granted + +**Driver = address-preserving execution membrane, not an ISA or new ALU.** All admitted read paths should use one capability-bound contract with five conceptual steps: + +1. **BIND:** admit a DR Frame / source incarnation / tile-local position and class/scope witness. A single costly global→local bind is amortized *outside the hot fold*. +2. **PROJECT:** produce typed virtual *views* onto demanded canonical SoA lanes/relationships; never create a second row/column owner in the admitted hot path. +3. **STREAM:** consume the same source address across tile-sized operations; preserve masks' coordinate universe; reuse the bound frame without per-method foreign lookup. +4. **EXECUTE:** existing Quack/RelAlg/R2IL → mask-risc → ndarray SIMD. Abakus carry words are an optional physical realization, **not** mandatory ALU. +5. **RELEASE:** close source, frame, scope and scratch lifetimes; return scalar/compact witnessed mask or explicit admitted egress. + +This is **a proposed facade and type/lifetime discipline**, not a statement that a new `Driver` trait already exists. Names below are *conceptual operation names*, not Rust/API signatures. + +~~~text +logical caller (DT64/ClassView/Quack, demanded fields, scope) + | + v +single logical address: (source, DR Frame, tile, local, facet, layout) + | + v ++--------------------------------------------------------+ +| VIRTUAL ACCESS DRIVER [research contract, not code] | +| BIND -> PROJECT -> STREAM -> EXECUTE -> RELEASE | +| - one coordinate/layout authority, a validated lease | +| - zero implicit duplicate canonical payload | +| - no row-object/tuple construction within hot fold | +| - local or data-owner remote execution location | ++--------------------------------------------------------+ + | | | + XY/XZ XX/YY/ZZ existing ALU + lens lens mask-risc/ndarray + \ | / + same canonical borrowed SoA tile/source owner + | + scalar / mask / result receipt / refusal +~~~ + +### 1.1 What the comparison with ndarray actually says + +| SIMD polyfill architecture | Proposed zero-copy driver architecture | +| --- | --- | +| One mathematical operation expressed once | One logical witnessed address and requested operation expressed once | +| ndarray::simd hides scalar/AVX2/AVX-512/NEON lowering | Driver hides admitted local, tiled and data-local foreign source-owner access | +| Per-backend kernels, same vector semantics | Per-source binding implementation, same relationship/value semantics | +| SIMD fidelity and performance verified by test oracles | Coordinate, source-layout, lifetime, no-owned-payload and bus-composition verified by independent oracles | +| Callers cannot choose a private ISA implementation by accident when using facade | **Migrated** callers cannot choose another payload ownership path by accident when only the sealed capability is exposed | + +**Important limitation:** A facade does not magically make preexisting public raw accessors inaccessible, nor can ordinary Rust types prevent all possible copying of public scalar values. Enforce the law at an owned, testable boundary, phase migration, and do not claim that 17,000 historical callers are already proven clean. + +## 2. Non-negotiable meaning of "zero copy" + +The existing [zero-copy lens law](../knowledge/zero-copy-lens-law.md) says the lens **is** the read floor, the array is already a projection, and there are no implicit payload-copy escape hatches. This program respects that operator ruling. + +### 2.1 Three proof scopes, never conflated + +| Claim | What would prove it | What it does NOT mean | +| --- | --- | --- | +| **ZC-A: resident borrowed execution** | Source-owner borrows canonical buffer; no new owned value/record column is made on admitted hot path; alias/pointer + instrumentation evidence | No scratch/result bits; no CPU loads/stores; no cache misses | +| **ZC-B: virtual layout** | Multiple typed bus projections share same source buffer/address binding; no physical XY/XZ/XX/YY/ZZ arrays or row object replicas | All pairwise results for arbitrary N are O(1) | +| **ZC-C: remote data-local operation** | Remote client sends typed descriptor; source-owning worker resolves and executes; no payload values sent to caller except explicit authorized egress | RDMA, network zero-copy, universal no disk I/O, low latency by definition | + +**Accounted boundaries, not hidden exceptions:** storage page fetch / decompression / Arrow batch, Lance `RowAddrTreeMap` S-INGRESS mask translation, network framing, and explicitly demanded results may move/copy bytes. They are **not** reclassified as "zero-copy hot execution." A remote request that must return an entire result population necessarily transmits an output proportional to its result size. For storage-backed "Thinking Hard Drive" claims, count all page reads, cache fills, decompression, ingestion copies and results. + +### 2.2 Enforceable guarantee vs slogan + +A usable guarantee requires all of: +- The source-locating operation resolves to **one** canonical owner, with a lease valid for the entire fold and a layout witness minted/checked **by that owner**. +- Only a **sealed, typed view/capability** for the requested fields and scope crosses from the access membrane to execution. Runtime forbids indiscriminate full-column gather. +- Binds, lease acquisitions and foreign hops are **explicitly accounted**, so they cannot hide inside an apparently pure arithmetic fold. +- A bounded borrowed view can be used immediately or through an executor callback without allowing an owned dataset replica by default. Internal source storage may be packed/strided; neither mandates a copy. +- Existing ndarray/mask-risc executes computation. The driver **does not** implement a new elementwise loop, hash index, relation store, serializer or ALU. +- All **migrated reachable** production paths go through the membrane; bypasses fail compile or a required CI policy. Unmigrated legacy paths are marked EXCLUDED/NOT-YET-GUARANTEED, never silently certified. + +**False proof:** Counting only allocator calls or testing pointer equality on one demonstration is insufficient to prove an entire call graph has no copies. A stack copy, source decoder, output serializer, a call to `to_vec()` in another module, and a hidden second cache are independent failure modes. + +## 3. The virtual bus: five requested views, one physical owner + +Let a canonical **same-identity** SoA, under a sealed layout witness `W`, expose typed x,y,z lanes: `S=(X[],Y[],Z[])`. For a fixed, already-bound ordinal `i`: + +```text + Bus_XX(i) := (read X[i], read X[i]) + Bus_XY(i) := (read X[i], read Y[i]) + Bus_XZ(i) := (read X[i], read Z[i]) + Bus_YY(i) := (read Y[i], read Y[i]) + Bus_ZZ(i) := (read Z[i], read Z[i]) + # YZ optional symmetry control; not required as a sixth stored bus +``` + +These are **view descriptors with typed operand roles**, not automatically numeric products or persistent arrays. The higher-level meaning of XX, XY etc may be a relation, projection, orientation, quadratic form, variance component, or grammar edge; that interpretation belongs to its *existing semantic owner*, not to this driver. Same-name pair XX may require only **one physical lane load** reused in two operand roles. + +For a requested operator `F` and a bounded population mask `M_W`: + +\[ +\operatorname{Fold}_{W,M}(F,\text{Bus}_{ab}) = +\bigoplus_{i \in M_W} F(S_a[i], S_b[i]). +\] + +The core implementation can use two borrowed operand lanes and one existing mask-reduction operator; it **does not** have to build `Vec<(a,b)>` or `Vec` before folding. + +**Foreign XY is a different law.** If `Y` refers to another identity, dataset, source version or a relationship edge, this is **not** `Y[i]` by definition. Its global→local hop must be admitted under EXO-46/140, bound and witnessed once, potentially executed at the source owner; arbitrary cross-source Cartesian products are not free. + +### 3.1 Compositional lens fusion + +A strong target is to run a chain such as `XY → XZ → ZZ` while retaining its address basis: + +- **Static semantic composition:** Quack/DT64 lowers operator composition into one typed program and one source/frame lease where lawful. Virtual outputs can remain scratch masks/planes rather than owned rows. +- **Repeated shared-address input:** X from XY and XZ need not be read from a different store or copied into separate value vectors; the same borrowed X lane participates in both. +- **Fusable program:** keep intermediate predicates in existing mask-risc scratch under its epoch. Do **not** claim a scratch slot survives across calls; EXO-153 M3 is open. +- **Barrier:** if the next step needs a different global ID, joins different owners, changes frame/scope, or demands an actual returned row object, it must execute an admitted bind/egress transition or refuse. + +### 3.2 Complexity: the precise O(1) claim + +A *constant-size lens descriptor* and **fixed number of resident SoA field accesses at an already-resolved local ordinal** can be O(1) in population size. This excludes: finding the global ID in a set of tiles, storage-page reads, decompressing an on-disk representation, runtime fan-out, computing all pairwise relations or emitting N outputs. A query over K selected identities generally needs at least Ω(K) work or a separately proven aggregate shortcut. + +The virtual buses avoid **physical representation expansion**, not the laws of physics or information-theoretic output bounds. The strong test is no duplicate canonical storage and address-preserving composition, with exact oracle parity. + +## 4. Small driver contract, not a new ISA + +### 4.1 Conceptual operations (not promised Rust symbols) + +| Operation | Required input | Result / ownership | Failure | +| --- | --- | --- | --- | +| `bind` | DR frame/map version, source-incarnation and layout witness, scope, selected IDs or co-addressed mask | One bounded, source-valid binding; no copied values | stale frame, wrong source, layout mismatch, unauthorized scope | +| `project` | ClassView basis and demanded field positions/roles `(a,b)` | borrowed, typed local view for the leased source; no new value owner | absent/unknown/ambiguous/unauthorized field | +| `stream` | already-bound population/relationship mask, tile/field demands | caller-owned scratch and source-borrowed lanes, tile-by-tile | dimension mismatch, expired lease, unsupported foreign map | +| `execute` | approved Quack/RelAlg/R2IL/mask-risc plan and authorized terminal | existing ndarray execution, no private engine | unsupported opcode, invalid truth mode, overflow/NULL policy | +| `release` | lease/owner token, scratch scope, optional result receipt | no outstanding reads beyond owner lifetime | stale or reused handle refused | + +There must be exactly one **logical** caller contract; local and remote are **two admissible implementations** under the same semantics, not two competing APIs. + +- **Local:** expose an in-process borrowed view with a lifetime no longer than the source snapshot. Use `LaneRef::Strided` when appropriate, no forced contiguous value gathering. +- **Remote:** never transmit a raw pointer or Rust reference. A typed descriptor addresses the source owner, which validates source/frame/scope and borrows locally before evaluation. Cross-process command and result messages are costed, not falsely called zero-copy. +- **Storage:** the source owner may need actual disk I/O. Full DR→Lance integration remains `BLOCKED_D4` and `S_INGRESS_COPY` under EXO-114/#1633. A page is not automatically the same dimension as a DR local tile. +- **Legacy:** a `PSCustomObject`-like consumer object is a projection capability / lazy *view*, not a bag of copied properties. If a consumer explicitly asks for a value object, that is result egress requiring separate intent/owner. + +### 4.2 Witness tuple (design only) + +```text +W = ( + canonical_source_identity, + source_incarnation + value_revision + storage/index_seal, + DR_map_identity + Frame.issue/route_version + tile/ordinal_basis, + source-owned layout/field-schema/facet digest, + class_id + ClassView.field_basis + demanded fields, + scope/permission/principal/tenant + expiry, + selected relation_semiring (B / N), truth_mode, method_sig_rev, + driver-capability version + execution owner +) +``` + +A local FNV TileMap digest helps compare layout; **it is not collision-resistant remote authentication**. A remote descriptor sent by a client is untrusted until the source-owning executor checks its own versioned snapshot and authorization. A value-only write may leave coordinate validity unchanged while invalidating value-derived predicates. A different SoA incarnation can reset its local numeric revision and still be stale. Historical frame replay is valid for old source values **only if the matching historical source snapshot exists**. + +## 5. New "Thinking Hard Drive" claim, framed for proof instead of marketing + +**Proposed theorem schema, NOT a mathematical theorem or a shipped result**: + +Given a finite witnessed source with one canonical SoA layout, a finite bounded set of pure admitted operations, valid source/frame/scope leases, and a driver whose implementation borrows the source lanes, an independently specified oracle and a virtual bus driver evaluate the *same relation* while **the resident executor constructs no secondary owned canonical value/row representation**. + +The proof has two independent obligations: + +1. **Extensional parity:** identical results, including Boolean vs Bag, NULL/UNKNOWN, absent vs zero, tail bits, multi-tile and foreign/source refusals. Same address universe and timestamps. +2. **Representation bound:** resident execution's *newly owned canonical payload bytes* = **0**; scratch/selection/result memory is separately counted, not prohibited. Prove no unauthorized value reconstruction inside the admitted graph and measure actual I/O and network boundaries separately. + +**Counterexample:** A graph that computes `XY` from one frame and `XZ` from a different population, then silently uses the same local ordinal, may allocate zero bytes **yet be wrong**. Zero copies alone do not prove valid cognition. Likewise a physical source-page read can dominate latency despite zero payload copies. A theorem can only cover the declared execution path and cost model. + +This is the precise research connection to the "thinking hard drive": persistent data should supply *the operands of computation in place* (when resident and lawful), rather than first becoming a second reconstructed object world. The disk is not doing CPU arithmetic; data-local computation still uses ordinary processors, memory and storage I/O. + +## 6. Evidence, current status and relation to existing programs + +| Earlier source | Verified / measured evidence | What D-DRIVER-ZC does NOT inherit | +| --- | --- | --- | +| [Zero-copy lens law](../knowledge/zero-copy-lens-law.md) | Operator law: no implicit copy escape hatch; ClassView/array is projection; specific transparent-stride examples | No blanket performance proof for every source/storage layout | +| [#1955](https://github.com/AdaWorldAPI/lance-graph/pull/1955) merged | D-DRISA-1 lab: borrowed in-memory canonical SoA, frame/layout/source checks, route once, no owned value arrays in register file | No virtual XY/XZ bus, no disk or remote guarantee | +| [#1956](https://github.com/AdaWorldAPI/lance-graph/pull/1956) merged | D-DRISA-3 lab: carry-ALU semantics exact, but 1.5–4.8× slower than native alternatives on eight measured shapes | No mandate for a carry-based ALU, no new public ISA | +| [#1950](https://github.com/AdaWorldAPI/lance-graph/pull/1950) merged | Source mask read and ingress costs with user-provided row IDs; prepared-bind Q≥2 includes copied `gather_lane` values | No end-to-end DR→Lance D4 or zero-copy prepared receipt | +| [#1949](https://github.com/AdaWorldAPI/lance-graph/pull/1949) merged docs | Optional source locator receipts and dynamic ClassView lens proposal | No production location-lease owner | +| [#1953](https://github.com/AdaWorldAPI/lance-graph/pull/1953) merged docs | D-DRISA remote-instruction contract, 57 falsifier pre-registration | No automatically authorized remote endpoint | +| [EXO-104](https://linear.app/exo-red/issue/EXO-104) | Existing `tuples_over_mask` issue: row copies, enumeration, sorting, latency violation | A tuple oracle is not a production implementation | +| [EXO-105](https://linear.app/exo-red/issue/EXO-105) | Reference executor/public-API licensing decision open | Do not mutate frozen mask-risc source from this docs plan | +| [EXO-109](https://linear.app/exo-red/issue/EXO-109) | Oracle independence decision | Reference implementation may not reuse the candidate's algorithm | + +**Open admission decisions:** EXO-46 (counted global→focus binding), EXO-89 (RelationshipMap), EXO-151 (DR focus/lease), EXO-140 (foreign semijoin/gather), EXO-121 (ClassView/field contract), EXO-114 (ordered source), D4/S-INGRESS. Source-owner layout witness and result-mask continuation also require ownership decisions from [D-DRISA M1/M3](d-drisa-decision-memos-v1.md). + +**Deferred by operator instruction:** Any audit or consolidation of migrated ladybug-rs/cognitive-shader-driver, P64, StreamDTO, BusDTO or PerturbationDTO. The eventual goal of **one canonical surface** is recorded, but the current plan does not inspect or choose among those residues. No new per-op web/REST/gRPC surface is authorized. + +## 7. Acceptance gate for the docs and first executable probe + +Read [staged PR charter](d-driver-zc-staged-pr-charter-v1.md) and [independent falsifier matrix](d-driver-zc-falsifier-matrix-v1.md). Passing ZC-1 means **one existing SoA source, one framed binding, at least five distinct virtual bus views, no owned value replicas, exact independent results and deliberate disable tests**. A passing zero-allocation microbenchmark is insufficient. + +**Explicit NO-GO**: new universal hash table, full-field gather as default, hidden owned `Vec` of source values, fake FNV cryptographic source proof, repeated per-call foreign lookup, mixing schema/source/scope witnesses, unadmitted cross-source joins, unaccounted disk/network copies, new SIMD backend or independent cognitive/shader driver. From 41f8a8afac9fa6d6cd3c554788c0bea3d701856f Mon Sep 17 00:00:00 2001 From: AdaWorldAPI Date: Wed, 4 Nov 2026 07:50:04 +0100 Subject: [PATCH 2/7] docs(D-DRIVER-ZC-0): stage seven independent zero-copy bus and firewall PR gates --- .../plans/d-driver-zc-staged-pr-charter-v1.md | 144 ++++++++++++++++++ 1 file changed, 144 insertions(+) create mode 100644 .claude/plans/d-driver-zc-staged-pr-charter-v1.md diff --git a/.claude/plans/d-driver-zc-staged-pr-charter-v1.md b/.claude/plans/d-driver-zc-staged-pr-charter-v1.md new file mode 100644 index 000000000..fb51dbfb7 --- /dev/null +++ b/.claude/plans/d-driver-zc-staged-pr-charter-v1.md @@ -0,0 +1,144 @@ +# D-DRIVER-ZC: staged PR charter for the virtual access membrane + +> **2026-11-04. STATUS: PROPOSED, PR0 DOCS ONLY.** This is a plan for independently reviewed, *future* executable PRs. Nothing here authorizes a new production driver/ISA, a global index, or a network listener. [Master contract](d-driver-zc-virtual-bus-contract-v1.md); [falsifier matrix](d-driver-zc-falsifier-matrix-v1.md); [EXO-154](https://linear.app/exo-red/issue/EXO-154); [EXO-153](https://linear.app/exo-red/issue/EXO-153); EXO-50 is communication, not admission. +> +> **Relationship to D-DRISA:** Stages D-DRISA-0 (#1953), D-DRISA-1 (#1955) and D-DRISA-3 (#1956) are **merged**. This plan does **not** duplicate local register or Abakus ALU tests. Stages D-DRISA-2 / 4 remain separate async/remote owner projects; our stages ZC-4/5 are *integration gates*, not competing drivers. + +## Staging and dependency graph + +~~~mermaid +flowchart TD + Z0["ZC-0: contract, ownership, proof matrix"] --> Z1["ZC-1: local sealed borrow membrane"] + Z1 --> Z2["ZC-2: five virtual bus lenses and fusion"] + Z1 --> Z3["ZC-3: address reuse and foreign bind"] + Z2 --> Z4["ZC-4: async descriptor integration only"] + Z3 --> Z4 + Z4 --> Z5["ZC-5: storage-backed/Lance attestation"] + Z2 --> Z6["ZC-6: migrated caller firewall and CI"] + Z3 --> Z6 + Z5 --> Z7["ZC-7: Thinking Hard Drive proof verdict"] + Z6 --> Z7 + G1{"EXO-46/89/151/140"} -.-> Z3 + G2{"D-DRISA-2/4 & security"} -.-> Z4 + G3{"D4 + S-INGRESS + EXO-114"} -.-> Z5 + G4{"class/semantic owners"} -.-> Z6 +~~~ + +Stages labelled GATED only proceed as decision memos until owner authorization. Do not open stacked implementation PRs before reviewing their prerequisites. Docs PR ZC-0 can be merged only after CI, reviewers and governance agree; **no auto-merge**. + +## ZC-0: docs, status board, EXO map (this PR) + +**Ownership:** operator / architecture review. **Files:** master, this staged charter, independent falsifier matrix, board entry, board status/integration/index and Linear EXO-154 / EXO-50 handoff. + +**Acceptance:** +- The existing zero-copy law remains in force, including no implicit copy below a size threshold. +- Dynamic scope and virtual bus semantics separated from physical layout and from user-level object egress. +- All identifiers labelled logical/hypothetical; no public Rust type, opcode, layout, wire protocol or F### presumed. +- Precise distinction among local source borrowing, disk/page ingest, and remote data-local computation. +- Existing D-DRISA-1/3 evidence attributed correctly, existing owner gates preserved; no investigation of the legacy DTO/shader-consolidation stack. +- Generated board entry index diff preserves all existing curated findings and passes repository check. + +**Exit:** reviewed architecture, no runtime. + +## ZC-1: smallest local no-copy borrow membrane + +**Status:** FUTURE LAB, no public API or source rewrite. **Owner review:** DR/DT64, ClassView, mask-risc and source layout owner. + +**Goal:** Consume #1955 `RegisterFile` on a borrowed source-owned SoA; define a *scoped internal access membrane* which can project a typed pair of fields at one witnessed DR ordinal. Do not add a new hash table, helper row cache, generic `Vec`, duplicated fields or new mask-risc executor. Prefer extending the existing excluded `drisa-register-probe` fixture to avoid a new crate. + +**Positive tests:** one tile DR64, one DR256, 256 tiles (65,536 identities), demanded 2 of 64 fields, same physical buffer pointers as source, borrowed slices/strides, 0 new owned canonical bytes, mask-only COUNT/EXISTS without source value reads; compare direct reference oracle by global ID. + +**Anti-evasion tests:** intentionally call a view materializer (must be unavailable under the admitted interface or rejected by a lint/compile-fail fixture), request unknown/unauthorized field, change source incarnation with numeric revision reset, permute the source layout, escape a borrowed slice beyond the lease lifetime (compile-fail). A separate in-process bad/legacy arm may materialize solely as an independent reference oracle, never an admitted execution route. + +**Measured:** cold bind cost, warm fold, route calls (0 after reuse), allocations and payload copy counters (heap and stack where instrumentation can detect), source bytes read, source pointer alias, scratch/result bytes reported separately. Do **not** claim universal no-copy because one compiled scope passed. + +**Go:** precise guarantee limited to migrated local consumer path. **Stop:** if raw source exposes uncontrolled owned-copy access to that path; report coverage gap, do not bypass with unguarded `unsafe`, `transmute` or a "fast Vec" path. + +## ZC-2: five virtual buses and composition without materialization + +**Status:** FUTURE LAB; depends ZC-1. **Owner review:** Quack/RelAlg and DT64 method semantics. + +**Goal:** Prove `XX, XY, XZ, YY, ZZ` each project and fold on the SAME canonical X/Y/Z SoA rows without building five physical arrays or rows. `YZ` may be included as symmetric negative/positive control. Do not assign any fixed semantic meaning (product, covariance, adjacency) to the bus labels without a typed operator declared by the semantic owner. + +**Fixture:** three deliberately non-interchangeable field lanes with differing values (include zeros/negative/missing), 64/256/multi-tile, source-layout permutation, 2–3 typed methods, input population mask. Compare: +- direct per-row scalar oracle by global ID and declared field operators; +- admitted one-lens folds; +- two-step `XY → XZ`, three-step `XY → XZ → ZZ` compiled into existing mask-risc scratch / native algebra and returned as terminal; +- materializing reference implementation **outside timed/admitted path** to catch semantic mismatch, counted as B1. + +**Representation proof:** no new owned `Vec<(X,Y)>`, `Vec` or 2D bus table; X remains one source buffer and one coordinate binding across XY and XZ. Explicitly test XX/YY/ZZ roles do not double-load a physically identical field just to fill two logical slots. + +**Negative controls:** force the same ordinal from two different frames; delete an edge connecting a foreign Y; accidentally swap X and Y; vary NULL/Bag semantics; cross-call scratch epoch; hidden full-field eager gather; violate tail bits. + +**Go:** five bus views and multi-step composition pass independent oracles with a stable source owner and zero intermediate source-value representation. **Stop:** if composition forces an owned value table or drops frame/source/scope; refuse as `NOT_FUSABLE` rather than copying. + +## ZC-3: one address binding, local tiles and foreign/remote source-owner hop + +**Status:** read-only decision memo **BLOCKED EXO-46/89/151/140** for production foreign navigation. Local co-addressed tile proof may run with no new foreign authority. + +**Question:** Can an already-resolved address be reused across multiple virtual buses and methods while all requests retain one source/layout/permission witness? Compare: +- co-addressed input already in tile/local form; +- bind once on first call, use Q=1/2/8/16/64/256; +- per-query global→local routing (#1955 failed 256-tile gate); +- FOREIGN case when legally admitted, resolved on source owner and reused under scope with no new global index. + +**Exact cost:** cold routing, per-call source lookup, tiles visited, bytes, allocations, total p50/p95 and negative misprediction/eviction; Q=1 must not be hidden. Include mismatched foreign source, expired foreign source lease, source compaction, wrong field schema and cross-tenant refusal. + +**Proposed owner preference:** co-addressed masks plus existing `MapTable` / `Frame` first. No independent hash map or trie by default; a versioned index is a **separate EXO-46 operator decision** only if the first alternative cannot meet a measured requirement. + +**Stop:** neither name coincidence nor matching local ordinal authorizes cross-source relationship truth. A remote pointer never crosses the process boundary. + +## ZC-4: async source-owner driver bridge, NOT a second async engine + +**Status:** **GATED D-DRISA-2 / 4, security/transport owner**. This plan provides *consumer compatibility tests*, not a competing driver. No examination or reorganization of the legacy Ladybug/P64/StreamDTO/BusDTO/PerturbationDTO/cognitive-shader-driver stack in this stage. + +**Positive:** take the same witnessed `bind/project/execute` semantics from ZC-1/2; send a small typed method, source/frame/layout/scope reference into an existing, authorized **data-owner** loopback runner; source owner validates and borrows its own values, returns COUNT/EXISTS/mask scalar. Asynchronous queue, cancellation and invalidation tests remain D-DRISA-2's responsibility; here only require that a submitted virtual-bus lens does not acquire a new payload owner. + +**Negative:** forged caller-supplied layout digest (FNV insufficient), stale source while queued, scope revoke, early release, foreign owner mismatch, oversized result, secret value bytes on the request wire. In-process "no wire payload" is not a remote performance result. + +**Go:** driver lens semantics survive the async descriptor; do not assert interprocess/no-payload transfer until D-DRISA-4 runs an actual transport counter. **Stop:** second execution engine, public per-op endpoint or remote raw pointer. + +## ZC-5: persistent source / Thinking Hard Drive physical proof + +**Status:** **BLOCKED D4/S-INGRESS, EXO-114, source/lifetime owner**. + +**Goal:** Test a source-backed tile whose canonical value planes reside in disk-backed storage and can be selectively read without reconstructing per-row objects. The CPU still computes; source I/O still occurs. Attest DR frame → correct Lance _rowid under ONE existing physical authority only, and count copied row-mask ingress. Reuse #1633 / #1950 source read arms. + +**Fixture:** 1M/10M rows, 1 or 16 demanded columns, 0.1/1/10/50/95% selection, clustered/scattered, page-cache warm/cold and disk-cold; fragment split/compaction, stable row-ID toggle, value-only update, remote request. Project XY/XZ/ZZ and compare independent stored-row oracle. Never equate range_count×4096 with pages touched. + +**Go:** the admitted resident borrowed portion adds **0 secondary canonical payload bytes**, while physical read and ingress costs are accurately reported. **Stop:** missing D4, reading wrong identity, row ID = row address without witness, or claiming disk reads are zero-copy computation. + +## ZC-6: access firewall and incremental legacy migration + +**Status:** FUTURE POLICY LAB + operator ownership. This is the **17,000-call** risk reduction program, not a claim that 17,000 call sites have been examined or are all reducible. + +**Phase 1:** static *reachable access-surface* census from chosen admitted consumer roots. Classify canonical borrowed reads, explicit result egress, source-owner decode, old raw direct access, tuple materialization, reference-only oracle, and unknown dynamic/plugin paths. Do **not** scan whole repo merely to create a giant list: use Kythe/Codegraph if already ready, else a narrow static AST/call graph with completeness caveats. + +**Phase 2:** seal canonical source fields and expose only the validated capability in a selected migrated module; add Rust UI compile-fail tests for value-owner escape, raw reference past lifetime, direct constructor/copy bypass and unauthorized field projection. Use lints/checks for forbidden `Vec`, `to_vec`, `collect`, `clone` only *inside admitted hot-path modules* and manually classify false positives (cloning an instruction descriptor or a result scalar isn't source duplication). + +**Phase 3:** prohibit calling an oracle implementation as a normal hot execution path (EXO-105/109 unresolved); preserve independent scalar or tuple oracle in test/reference scope. Do not edit frozen mask-risc without owner authorization. + +**Phase 4:** expand migrated coverage incrementally. Report explicit numerator/denominator and roots; only then claim e.g. "all 37 reachable read paths behind this facade" instead of "all 17,000 calls." Negative fixture deliberately inserts a copied field buffer into a migrated path, and CI must fail. + +**Go:** local compile-time/CI coverage is credible within declared roots, with an independent runtime copy/materialization probe. **Stop:** no completeness analysis, caller can bypass private boundary, or a linter flags all `clone` as wrongdoing. + +## ZC-7: proposed "Thinking Hard Drive" theorem / engineering verdict + +**Status:** FUTURE INDEPENDENT REVIEW, no performance or mathematical theorem certified now. **Dependencies:** enough ZC-1/2/5/6 data; EXO-154, EXO-153, semantic owners. + +**Claim categories:** +1. **Semantics:** same witnesses, exact Boolean/Bag/Kleene/absence parity and typed refusals. +2. **Representation:** same canonical source address and zero newly owned canonical *value payload* per admitted resident fold, no virtual bus replicas. +3. **Economics:** whole-query throughput, cold/hot, disk/page bytes, ingress, network bytes and output size; a requested N-row output costs Ω(N). +4. **Enforcement:** migrated reachable call graph cannot bypass the tested facade within the stated threat model. +5. **Scope of theorem:** bounded operator set and layout conditions, not proof of O(1) all-pairs computation or exemption from physical I/O. + +Report `PASS_LOCAL_ONLY`, `PASS_SOURCE_BACKED`, `FAIL_SEMANTIC`, `FAIL_COPY`, `BLOCKED_OWNER` or `NOT_COMPARABLE`. New production adoption or a consolidated driver identity is a **separate operator ruling**. + +## PR discipline + +- Open one PR per stage; ZC-0 is docs-only. Use current main, no accidental stacking or auto-merge. Keep independent falsifier and deliberate-disable logs, code refs, SHA, and board status. Research data can live on a pinned branch with source links. +- No new ClassId, F###, mask-risc opcode, ndarray backend, hash table, global row-id map, source value clone or direct production driver module in the initial probes. +- Keep D-DRISA staging and ownership intact: do not start a second async engine. If D-DRISA-2/4 later ships, ZC-4 may be a *compatibility* probe only. +- EXO-50 messages are append-only coordination and cannot admit architecture changes. +- Keep migrated cognitive-shader-driver/P64/StreamDTO/BusDTO/PerturbationDTO investigation **explicitly deferred**, per operator request. From 6df97727cf32464faa1cf763bef8d7e69c617785 Mon Sep 17 00:00:00 2001 From: AdaWorldAPI Date: Wed, 4 Nov 2026 07:51:29 +0100 Subject: [PATCH 3/7] docs(D-DRIVER-ZC-0): preregister 47 independent bus, copy, foreign and theorem falsifiers --- .../plans/d-driver-zc-falsifier-matrix-v1.md | 117 ++++++++++++++++++ 1 file changed, 117 insertions(+) create mode 100644 .claude/plans/d-driver-zc-falsifier-matrix-v1.md diff --git a/.claude/plans/d-driver-zc-falsifier-matrix-v1.md b/.claude/plans/d-driver-zc-falsifier-matrix-v1.md new file mode 100644 index 000000000..4ed7266d7 --- /dev/null +++ b/.claude/plans/d-driver-zc-falsifier-matrix-v1.md @@ -0,0 +1,117 @@ +# D-DRIVER-ZC independent falsifier matrix: virtual buses, source borrow, foreign bind, storage and enforcement + +> **2026-11-04 · ALL TESTS BELOW ARE PROPOSED, NOT RUN.** Results from #1955, #1956 and #1950 are explicitly identified as *predecessor* lab evidence; they do not pass these new virtual-bus/call-graph/storage obligations. A matching answer alone cannot certify a no-copy path. [Master](d-driver-zc-virtual-bus-contract-v1.md) · [PR charter](d-driver-zc-staged-pr-charter-v1.md) · [EXO-154](https://linear.app/exo-red/issue/EXO-154). + +## Method and independent reference + +**Source fixture:** Independently generated table keyed by global ID, with three uncorrelated typed x/y/z values, missing/unknown/present-zero cases, relation edges, explicit snapshots and scope. Separate oracle enumerates identities and applies declared semantic operators; it **never** calls the candidate driver, reuses the candidate binding, asks `TileMap::local` for oracle answers, or receives candidate-created derived masks. + +**Disable protocol:** Every cited deliberate violation must yield **an observable different answer or typed refusal**, not merely "test still passes" or a panic. For refusal tests, a positive control under a valid witness must succeed. Anti-vacuity: nonzero and unequal expected outputs where a bad result could otherwise escape. + +**Physical evidence:** source buffer identity/pointers + address source ownership, no-value-vector build in candidate code, source read/copy counters and alias invariants, allocations (including stack-copy caveat), scratch/result and mask ingress separated, number of tiles/ids touched, peak resident bytes, page-cache/disk separately, p50/p95/p99 with variance, Q=1/2/8/16/64/256. + +**Notation:** `V` = bus algebra, `Z` = resident no-copy, `F` = foreign route/identity, `S` = async/storage, `E` = enforcement/economics. Tests below do **not** supersede D-DRISA's R/A/N/C/P pre-registration. + +## V. Virtual bus semantics and composition (V1–V12) + +| ID | Independent fixture / expected | Disable falsifier and refusal | PR | +| --- | --- | --- | --- | +| V1 | Bound same-identity XY view: x=2,y=7,z=13, independently oracle-defined operator F(x,y) | swap x/y deliberately: noncommutative F changes answer | ZC-2 | +| V2 | XZ view on same IDs and same source X; expected differs from XY due to independent Z values | substitute Y lane for Z → wrong answer | ZC-2 | +| V3 | XX view reads one canonical X twice as logical operands | physical duplicate X vector allocated or wrong alias → fail | ZC-2 | +| V4 | YY and ZZ on different fields, distinct negative, zero and positive values | alias wrong lane or mix field identities → fail | ZC-2 | +| V5 | All XX/XY/XZ/YY/ZZ on same sealed DR64 and DR256 populations | silently use local ordinal from different frame → divergent oracle/refusal | ZC-2 | +| V6 | XY and XZ share the same source X pointer and witnessed tile mapping, Q≥2 | create different copied X arrays → fail representation checker | ZC-2 | +| V7 | Two-step XY→XZ program with intermediate mask held in existing in-program scratch, output COUNT | force object-row intermediate or stale scratch epoch → fail | ZC-2 | +| V8 | Three-step XY→XZ→ZZ and independent scalar expression evaluated by global ID | materialize intermediate or reorder step/frame → fail | ZC-2 | +| V9 | YZ symmetric control, foreign Y excluded from local same-identity rule | accidental treat foreign id as Y[local_i] → typed refusal | ZC-2 | +| V10 | ClassView field mapping through physical permutation, predicate_iri not ordinal/label | ordinal or name shortcut silently selects wrong column → fail | ZC-1/2 | +| V11 | Boolean support vs Bag multiplicity on one bus, Kleene UNKNOWN retained for absent value | force Boolean collapse/UNKNOWN=false → oracle divergence | ZC-2 | +| V12 | Requested N outputs vs aggregate scalar over N identities | false O(1) output claim disproved by counted bytes; explicit egress required | ZC-2/7 | + +## Z. Resident no-copy membrane and lease (Z1–Z10) + +| ID | Independent fixture / expected | Disable / failure state | PR | +| --- | --- | --- | --- | +| Z1 | Existing #1955 Soa under known source/frame, two demanded borrowed lanes, direct pointer alias, exact fold | swap to owned `Vec`: CI/anti-copy instrumentation must detect | ZC-1 | +| Z2 | Map-only COUNT and EXISTS after value buffers unmap/unavailable | any canonical value-lane read → forbidden; valid answer remains | ZC-1 | +| Z3 | 3 demanded fields of 64, differing physical order and field basis | full 64-field gather or forbidden field → fail | ZC-1 | +| Z4 | Same DR frame and source layout, one value-only write | map-only valid; derived predicate stale until new value witness | ZC-1 | +| Z5 | New SoA incarnation resets numeric revision to 0 | numeric revision match alone must not grant stale read | ZC-1 | +| Z6 | Same IDs but permuted source tile local order | without source-owner layout witness, wrong oracle; with witness refuse | ZC-1 | +| Z7 | Source buffer released while a candidate view escapes callback | Rust UI compile-fail / lifetime proof, no use-after-free | ZC-1 | +| Z8 | A stack-based copy of value tuples and a heap owned column both attempted inside admitted module | heap allocation count alone must not falsely certify; policy/lint rejects both | ZC-1/6 | +| Z9 | An eager value gather happens inside fake "prefetch" stage | count canonical duplicate bytes / explicit no-copy refusal | ZC-1/3 | +| Z10 | Candidate returns scalar/mask while scratch/result allocator is active | allowed scratch/result bytes separate from value-copy counter; do not falsely fail | ZC-1 | + +## F. Local/foreign addressing and bind amortization (F1–F9) + +| ID | Independent fixture / expected | Deliberate break / status | PR | +| --- | --- | --- | --- | +| F1 | Co-addressed DR tile/local population, 64/256/multi-tile, repeat Q=256 | any hidden `MapTable::locate` per method fails route counter | ZC-3 | +| F2 | Global IDs pre-bound once to 256 DR256 tiles using #1955 route, oracle by IDs | route per call loses time; honest ≥1ms FAILED-GATE if applicable | ZC-3 | +| F3 | Two populations include same global ID at different local ordinals | join on local position → wrong answer/refusal | ZC-3 | +| F4 | One legal foreign relationship with independently enumerated edges and source-owner permit | invalid/unclassified `EqU32Via`/gather hop refused under EXO-140 | ZC-3 BLOCKED | +| F5 | Same foreign source/frame and reused binding for Q=1/2/16/256 | copy values to fake "accelerator" cache → forbidden | ZC-3 BLOCKED | +| F6 | Foreign source compaction under original DR coordinate frame | stale row location/source index seal → source-lease refusal | ZC-3 BLOCKED | +| F7 | Foreign source scope revoked while owner still hosts data | scope mismatch fails without returning old mask | ZC-3 BLOCKED | +| F8 | Source index/mapping receipt evicted but underlying relation unchanged | relation answer invariant; admitted rebind or refusal, no lost truth | ZC-3 BLOCKED | +| F9 | Two owners have identical integer tile/local coordinates | never join identities merely because coordinates coincide; refusal | ZC-3 BLOCKED | + +## S. Async, disk, source isolation (S1–S8) + +| ID | Independent fixture / expected | Disable and decision | PR | +| --- | --- | --- | --- | +| S1 | In-process async worker receives typed bound query, source unchanged | same scalar oracle; no remote/payload claim from loopback | ZC-4 gated | +| S2 | Value changes while a descriptor is queued; map-only vs value-scoped methods | value operation stale refusal; map-only may remain valid | ZC-4 gated | +| S3 | Caller supplies matching FNV digest for foreign layout, executor knows actual source | reject caller's untrusted "authority"; source-owning witness wins | ZC-4 gated | +| S4 | Two-process request contains only method/witness; source owner computes and returns COUNT | packet/transport counter detects any unexpected source field payload | ZC-4 gated | +| S5 | Real Lance storage 1M/10M, selected 1/16 fields, 0.1–95% distributed rows | D4 missing must produce BLOCKED_D4, not accept handwritten row IDs as integrated truth | ZC-5 blocked | +| S6 | Fragment compaction, stable row-id option, source revision and local frame mapping | forged `_rowid` or wrong page silently accepted → fail | ZC-5 blocked | +| S7 | Page cache warm vs disk cold, clustered vs scattered; physical byte counter | cannot claim disk-level no copy from resident-only borrow evidence | ZC-5 blocked | +| S8 | Partial/missing source pages, mid-execution cancel and error | typed I/O error or indeterminate, **never** zero count or empty mask | ZC-5 blocked | + +## E. Enforcement and economic proof (E1–E8) + +| ID | Independent test / expected | Negative control / no-go | PR | +| --- | --- | --- | --- | +| E1 | Compile-fail: attempted source buffer ownership through the sealed migrated consumer | bad direct accessor must fail or be policy rejected | ZC-6 | +| E2 | Compile-fail: borrowed value view escapes source lease or wrong lifetime | UAF path rejected at compile time; no `unsafe` bypass | ZC-6 | +| E3 | Compile/policy failure: inject `to_vec` field copy into one selected hot-path module | CI fails; do not flag legitimate descriptor cloning | ZC-6 | +| E4 | Enumerate reachable data-access sites from declared caller roots and classify migrated/excluded/unknown | missing dynamic edge means INCOMPLETE coverage, not "17k proved" | ZC-6 | +| E5 | Reference tuple oracle independently computes global-ID truth | test oracle equals production implementation by sharing function → fail independence | ZC-6/EXO-109 | +| E6 | Same query B0 local borrowed vs B1 client payload fetch vs B2 source-local instruction, full total time and bytes | equal/greater B2 end-to-end time: no speedup claim; report why | ZC-7 | +| E7 | Same fixed-address XY/XZ operation before/after lens fusion; count payload copies and scratch | zero allocations but wrong frame or hidden copies → fail; memory metrics separate | ZC-7 | +| E8 | Whole program output N rows, or all pairs N², while claim promises O(1) time | output lower bound contradiction -> reject broad theorem claim | ZC-7 | + +**Total:** 12 V + 10 Z + 9 F + 8 S + 8 E = **47 new proposed gates**. None executed in this documentation PR. + +## Stage / evidence crosswalk + +| Current evidence | Reuse here | Still unresolved | +| --- | --- | --- | +| #1955 merged local register | Existing SoA borrowed planes, layout and scope refusal, route-once | ZC-1 enforceable membrane, five bus projections, historical-call coverage | +| #1956 merged ALU | k-of-n / Bag / Kleene correctness, measured native advantage | No virtual-bus / remote speedup inference | +| #1949 and #1951 | D-DRAM locator proposal and `U24x5` census | EXO-46/89/151 lease owner, no public Abakus type | +| #1950 / #1633 | selective source reading, known ingress copy | D4 and actual source-owner witness | +| Zero-copy lens law | operator invariant, extant transparent view precedents | no automatic full-repo enforcement or physical I/O guarantee | +| EXO-104/105/109 | precise tuple/rollback/oracle risks | open owner decisions; no automatic mask-risc modifications | + +## Reproducibility record required on every future PR + +~~~text +GATE: D-DRIVER-ZC/ +PR/COMMIT: URL + immutable head SHA +OWNER: owner and outstanding EXO decisions +PROGRAM: ClassView fields, method, bus roles, semiring, NULL mode +SOURCE: canonical source incarnation / source-version / DR frame / tile digests +ORACLE: independent global-ID scalar reference, source-independent +POSITIVE: exact result, permitted scope +DISABLE: exactly one deliberate failure with expected divergence/refusal +COPY: owned canonical payload bytes created; scratch/result/ingress separately +COST: Q, source page/disk bytes, rpc bytes, allocations, p50/p95/p99, cold/warm +SCOPE: local | source-backed | two-process | NOT-RUN +VERDICT: PASS | FAIL | PARTIAL | BLOCKED_OWNER | NOT-COMPARABLE +~~~ + +**No-go**: a test that proves only equal answers, only zero heap allocations, or only a fast microkernel cannot certify the driver zero-copy law or the "Thinking Hard Drive" theorem. The driver cannot itself certify a caller excluded from the migrated/closed access surface. From a6df40b97bda96c6dfca85906e0e9f2558bdc99e Mon Sep 17 00:00:00 2001 From: AdaWorldAPI Date: Wed, 4 Nov 2026 07:52:12 +0100 Subject: [PATCH 4/7] board(D-DRIVER-ZC-0): record virtual access membrane and zero-copy bus gates --- ...1-04-d-driver-zc-0-virtual-bus-contract.md | 44 +++++++++++++++++++ 1 file changed, 44 insertions(+) create mode 100644 .claude/board/entries/2026-11-04-d-driver-zc-0-virtual-bus-contract.md diff --git a/.claude/board/entries/2026-11-04-d-driver-zc-0-virtual-bus-contract.md b/.claude/board/entries/2026-11-04-d-driver-zc-0-virtual-bus-contract.md new file mode 100644 index 000000000..566a80b0d --- /dev/null +++ b/.claude/board/entries/2026-11-04-d-driver-zc-0-virtual-bus-contract.md @@ -0,0 +1,44 @@ +# D-DRIVER-ZC-0 — virtual zero-copy driver, address-preserving XY/XZ/XX/YY/ZZ bus + +**Date:** 2026-11-04. **Status:** ARCHITECTURE / FALSIFIER PRE-REGISTRATION ONLY; no new source pointer/driver/API/codec/ISA/remote server and no numeric performance result. + +**Operator:** Apply the `ndarray::simd` polyfill pattern to DR's **data access**, not by auditing approximately 17,000 calls one by one. One admissible `BIND → PROJECT → STREAM → EXECUTE → RELEASE` membrane over witnessed source-owned DR coordinates should make implicit canonical value copying impossible for **migrated sealed consumer paths**. It should expose typed virtual buses `XX, XY, XZ, YY, ZZ` over *one* canonical SoA rather than storing additional bus arrays or constructing `PSCustomObject`-like rows. Foreign global→local navigation happens only under owner admission, and can amortize once per witnessed frame. Local borrowing and data-owner remote execution are separate realizations, not an exported Rust pointer. + +**Plans:** [D-DRIVER-ZC master](../../plans/d-driver-zc-virtual-bus-contract-v1.md) · [staged PR charter](../../plans/d-driver-zc-staged-pr-charter-v1.md) · [47 falsifier rows](../../plans/d-driver-zc-falsifier-matrix-v1.md). + +**Linear:** [EXO-154](https://linear.app/exo-red/issue/EXO-154) owner issue; [EXO-153](https://linear.app/exo-red/issue/EXO-153) existing D-DRISA stages; [EXO-50](https://linear.app/exo-red/issue/EXO-50) append-only agent handoff. + +## Exact reuse + +- [Zero-copy lens law](../../knowledge/zero-copy-lens-law.md) remains operator ruling. No hidden source value array, tuple enumeration or row reconstruction in the admitted fold. An array is already a ClassView projection; source stride may be borrowed where layout permits. +- [#1955](https://github.com/AdaWorldAPI/lance-graph/pull/1955) **MERGED LAB**: sealed DR register with local owned test-SoA, scope/frame/layout/source witnesses, borrow semantics, route-once. It does not prove a virtual bus or remote/disk execution. +- [#1956](https://github.com/AdaWorldAPI/lance-graph/pull/1956) **MERGED LAB**: Abakus carry-word correctness for k-of-n/Kleene/Bag, but 1.5–4.8× slower than native arms in eight measured shapes; carry is not a forced driver ALU. +- [#1950](https://github.com/AdaWorldAPI/lance-graph/pull/1950) **MERGED LAB**: real Lance mask ingress costs; prepared-binding timings include copied value lanes, and DR→Lance `BLOCKED_D4`. +- [#1953](https://github.com/AdaWorldAPI/lance-graph/pull/1953) **MERGED DOCS**: D-DRISA async register research architecture and stages; new plan does not build a second async runtime. + +## Staged PRs (NONE of ZC-1..ZC-7 opened here) + +| D-id | Deliverable | Status / gate | +| --- | --- | --- | +| D-DRIVER-ZC-0 | master architecture, 47 falsifiers, staged charter, board/EXO map | THIS DOCS PR, no runtime | +| D-DRIVER-ZC-1 | local borrowed-lens membrane over merged D-DRISA-1 | FUTURE LAB, source/DT64 owners | +| D-DRIVER-ZC-2 | five virtual bus roles with address-preserving lens composition | FUTURE LAB, Quack/DT64 semantic owners | +| D-DRIVER-ZC-3 | co-addressed/reused tiles; foreign global→local amortization | LOCAL LAB possible; FOREIGN BLOCKED EXO-46/89/151/140 | +| D-DRIVER-ZC-4 | async source-owner descriptor **compatibility** (not new driver) | GATED D-DRISA-2/4, security owner | +| D-DRIVER-ZC-5 | physical Lance source / disk-backed no-owned-intermediate proof | **BLOCKED D4/S-INGRESS**, EXO-114 | +| D-DRIVER-ZC-6 | typed read access firewall, compile-fail and incremental caller coverage | FUTURE scoped policy lab; EXO-104/105/109 | +| D-DRIVER-ZC-7 | "Thinking Hard Drive" representation/semantic/economic verdict | FUTURE REVIEW, not theorem certified | + +## Critical guarantees and refusals + +- **Resident ZC-A:** zero *newly owned source value payload bytes* in migrated admitted hot execution; legitimate scratch/selection/result bytes counted separately. A one-time source I/O/decode still costs data movement. +- **Virtual ZC-B:** no physical bus arrays for XY/XZ/XX/YY/ZZ; an already-bound fixed-size row lens costs O(1) loads in row count, but arbitrary foreign joins, K-row folds and output are not universally O(1). +- **Remote ZC-C:** source owner performs local borrow/compute; remote client sends typed instruction and receives a measured result, not a raw pointer. **No network/RDMA zero-copy claim.** +- **No universal historical certification:** the ~17k calls are operator context, not an enumerated call graph. CI can enforce a sealed migrated scope; bypass coverage remains a separate measurement. +- **Theorem remains PROPOSED:** parity + owned-payload representation bound + scope/layout/lifetime validity + exact physical I/O accounting. A zero-allocation wrong-answer path is a failure. + +**Owner gates:** [EXO-46](https://linear.app/exo-red/issue/EXO-46), [EXO-89](https://linear.app/exo-red/issue/EXO-89), [EXO-151](https://linear.app/exo-red/issue/EXO-151), [EXO-140](https://linear.app/exo-red/issue/EXO-140), [EXO-121](https://linear.app/exo-red/issue/EXO-121), [EXO-114](https://linear.app/exo-red/issue/EXO-114), [EXO-104](https://linear.app/exo-red/issue/EXO-104), [EXO-105](https://linear.app/exo-red/issue/EXO-105), [EXO-109](https://linear.app/exo-red/issue/EXO-109). No owner decision is made by these plans. + +**Deferred:** no audit or consolidation now of cognitive-shader-driver, former ladybug-rs, P64, StreamDTO, BusDTO, PerturbationDTO. Eventual one-canonical-surface choice belongs to a separate operator decision. + +**STOP:** copying canonical payload values in the admitted hot path; inventing another hash map or row authority; foreign/local ordinal confusion; relabeling physical disk/network copies as free; new public ISA or "GPU shader"; completing full caller coverage without a defensible reachability census; automatic merge. From 47bac8b64cb63ab4b3462388714df1fcddd65e10 Mon Sep 17 00:00:00 2001 From: AdaWorldAPI Date: Wed, 4 Nov 2026 07:53:01 +0100 Subject: [PATCH 5/7] board(D-DRIVER-ZC-0): map driver, bus and theorem gates to EXO-154 --- .claude/board/STATUS_BOARD.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index ae8ca70d0..7ca5be70d 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -1,3 +1,22 @@ +## D-DRIVER-ZC — one zero-copy access membrane and virtual bus (2026-11-04) + +**Status:** D-DRIVER-ZC-0 PROPOSED / DOCS ONLY. [Master](../plans/d-driver-zc-virtual-bus-contract-v1.md) · [staged PR charter](../plans/d-driver-zc-staged-pr-charter-v1.md) · [47 independent falsifiers](../plans/d-driver-zc-falsifier-matrix-v1.md) · [entry](entries/2026-11-04-d-driver-zc-0-virtual-bus-contract.md) · [EXO-154](https://linear.app/exo-red/issue/EXO-154) · [EXO-153](https://linear.app/exo-red/issue/EXO-153) · [EXO-50](https://linear.app/exo-red/issue/EXO-50). **No driver implementation or production access rule shipped.** + +**Updated predecessor status:** [D-DRISA #1953](https://github.com/AdaWorldAPI/lance-graph/pull/1953) merged docs; [#1955](https://github.com/AdaWorldAPI/lance-graph/pull/1955) merged local borrowed register lab; [#1956](https://github.com/AdaWorldAPI/lance-graph/pull/1956) merged ALU semantics lab, carry 1.5–4.8× slower than native in eight measured shapes. Earlier D-DRISA entries are historical plans, not the current PR state. + +| D-id | Stage | Status / owner | +| --- | --- | --- | +| D-DRIVER-ZC-0 | One witnessed bind/project/stream/execute/release access contract, bus roles, proof matrix | THIS DOCS PR, proposed | +| D-DRIVER-ZC-1 | Local sealed borrowed-lens membrane, compile-fail misuse probes | FUTURE LAB, DT64/source/mask-risc review | +| D-DRIVER-ZC-2 | Typed XX/XY/XZ/YY/ZZ views and in-program composition, no owned bus arrays | FUTURE LAB, Quack/DT64 | +| D-DRIVER-ZC-3 | Bind-once across local tiles; admitted foreign global→local reuse | LOCAL LAB; FOREIGN **BLOCKED EXO-46/89/151/140** | +| D-DRIVER-ZC-4 | D-DRISA-2/4 async compatibility only, no parallel driver | GATED source-owner/security/transport | +| D-DRIVER-ZC-5 | Lance disk-backed source + truthful I/O/copy accounting | **BLOCKED D4/S-INGRESS**, EXO-114 | +| D-DRIVER-ZC-6 | Sealed migrated caller firewall with explicit reachability coverage | FUTURE policy lab, EXO-104/105/109 | +| D-DRIVER-ZC-7 | Thinking Hard Drive semantic, representation and whole-query verdict | FUTURE independent review; no theorem proof claimed | + +**Law:** existing [zero-copy lens doctrine](../knowledge/zero-copy-lens-law.md) prohibits implicit source payload materialization; local borrow only with live source/layout/scope witness. Existing ndarray SIMD facade is the model: one admitted access vocabulary, multiple implementations without bypassing semantics. Virtual buses are typed same-address views, not stored arrays. O(1) is limited to a fixed number of already-resolved resident row reads, never arbitrary K-row output, all-pairs joins, storage I/O or RPC. Legacy ~17,000-call premise is operator context, not audited coverage. Do **not** inspect/consolidate migrated ladybug/P64/StreamDTO/BusDTO/PerturbationDTO/cognitive-shader-driver yet. + ## D-DRISA — DR remote register + Abakus cognitive shader (2026-11-04) **Status: D-DRISA-0 PROPOSED / DOCS ONLY.** [Master architecture](../plans/d-drisa-remote-register-shader-v1.md) · [staged PR charter](../plans/d-drisa-staged-pr-charter-v1.md) · [57 falsifiers](../plans/d-drisa-falsifier-matrix-v1.md) · [board entry](entries/2026-11-04-d-drisa-0-remote-register-contract.md) · [EXO-153 owner tracking](https://linear.app/exo-red/issue/EXO-153) · [EXO-50 append-only handoff](https://linear.app/exo-red/issue/EXO-50). **No remote driver, API, copied payload cache, new map or performance measurement is implemented by this plan.** From 99af47d4cac2839096838494054574995aff04cd Mon Sep 17 00:00:00 2001 From: AdaWorldAPI Date: Wed, 4 Nov 2026 07:53:08 +0100 Subject: [PATCH 6/7] board(D-DRIVER-ZC-0): map driver, bus and theorem gates to EXO-154 --- .claude/board/INTEGRATION_PLANS.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.claude/board/INTEGRATION_PLANS.md b/.claude/board/INTEGRATION_PLANS.md index 3b2142fb4..1140a6872 100644 --- a/.claude/board/INTEGRATION_PLANS.md +++ b/.claude/board/INTEGRATION_PLANS.md @@ -1,3 +1,7 @@ +## 2026-11-04 — D-DRIVER-ZC-0: virtual access driver instead of 17,000 per-call copy audits + +**Status:** PROPOSED / DOCS ONLY. [Master](../plans/d-driver-zc-virtual-bus-contract-v1.md) · [staged PR charter](../plans/d-driver-zc-staged-pr-charter-v1.md) · [47 falsifiers](../plans/d-driver-zc-falsifier-matrix-v1.md) · [board entry](entries/2026-11-04-d-driver-zc-0-virtual-bus-contract.md) · [STATUS_BOARD](STATUS_BOARD.md) D-DRIVER-ZC · [EXO-154](https://linear.app/exo-red/issue/EXO-154) / [EXO-153](https://linear.app/exo-red/issue/EXO-153) / [EXO-50](https://linear.app/exo-red/issue/EXO-50). **Architecture move:** apply ndarray SIMD polyfill's facade-and-parity law to a single local/foreign/data-owner *address-access membrane*. Five virtual XX/XY/XZ/YY/ZZ lenses share one witnessed canonical SoA address; typed ClassView and Quack/mask-risc/ndarray execute without default copied values/row reconstruction. BIND→PROJECT→STREAM→EXECUTE→RELEASE; strict local borrow vs disk ingress vs remote message distinction; no new ISA/ALU/hash index or driver runtime. Co-addressed selections and once-per-frame routing first; foreign hops remain EXO-46/89/151/140-gated; D4/S-INGRESS and oracle-lifetime owners open. A 47-case matrix separates semantic results, representation bytes, reachability enforcement and end-to-end economics. #1953/#1955/#1956 already merged; the 8 ZC stages are independent, only ZC-0 documented now. "Thinking Hard Drive" is a proposed restricted theorem schema, not an assertion that disks perform computation or every query takes O(1). **Explicit defer:** no cognitive-shader-driver/Ladybug/P64/StreamDTO/BusDTO/PerturbationDTO stack investigation or consolidation during this PR. + ## 2026-11-04 — D-DRISA decision memos after the local register proof → `.claude/plans/d-drisa-decision-memos-v1.md` **Status:** PROPOSED, owned by this session (EXO-153 stages 1 and 3). **Board:** [STATUS_BOARD](STATUS_BOARD.md) D-DRISA-1-LAB, D-DRISA-3-LAB, D-DRISA-M1..M7; [entry](entries/2026-11-04-d-drisa-1-register-proof.md). Architecture: #1953 (D-DRISA-0, a GPT-session plan adopted after review). Seven owner questions the local proof leaves open: a layout witness on the SoA (the first missing connection), a global id → tile index or co-addressed selections, result-register lifetime, source-location receipt, DR → Lance authority, the carry words as ALU, async loopback. From f45cefbe55293ad90b95cbf5ded2dc5f01a4187b Mon Sep 17 00:00:00 2001 From: AdaWorldAPI Date: Wed, 4 Nov 2026 07:53:42 +0100 Subject: [PATCH 7/7] board(D-DRIVER-ZC-0): regenerate entries index without curated finding loss --- .claude/board/entries/README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.claude/board/entries/README.md b/.claude/board/entries/README.md index b4a3364b0..ab4badb3c 100644 --- a/.claude/board/entries/README.md +++ b/.claude/board/entries/README.md @@ -25,10 +25,11 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately opposite directions; the stranding this convention prevents shows up in exactly one of them, never both. -575 entries, 2026-08-06 .. 2026-11-04. +576 entries, 2026-08-06 .. 2026-11-04. | date | entry id | finding | file | |---|---|---|---| +| 2026-11-04 | `D-DRIVER-ZC-0` | | [2026-11-04-d-driver-zc-0-virtual-bus-contract.md](2026-11-04-d-driver-zc-0-virtual-bus-contract.md) | | 2026-11-04 | `D-DRISA-3` | | [2026-11-04-d-drisa-3-alu-semantics.md](2026-11-04-d-drisa-3-alu-semantics.md) | | 2026-11-04 | `D-DRISA-1` | | [2026-11-04-d-drisa-1-register-proof.md](2026-11-04-d-drisa-1-register-proof.md) | | 2026-11-04 | `D-DRISA-0` | | [2026-11-04-d-drisa-0-remote-register-contract.md](2026-11-04-d-drisa-0-remote-register-contract.md) |