RFC: Stabilizing includes / nested materialization

[KyleAMathews]

RFC: Stabilizing includes / nested materialization

Status: active — state-aware relationship fuzzing in review; audited issue inventory and transition-history coverage next
Scope: bug fixes and internal refactors only — no new public API surface, no behavior changes beyond fixing verified bugs.
Working model: small PRs from current main; reduced expected-failure seeds land before the architectural slice that fixes them.

Progress:

• Base recompute-oracle harness merged in test(db): add includes recompute oracle #1716.
• Oracle hardening merged in test(db): tighten includes oracle failure checks #1717: assertion-specific known failures, stable converted puts, and repeated optimistic rollback coverage.
• Reusable trace runner merged in test(db): extract reusable trace runner #1718: driver/projection split, same-turn checkpoints, cleanup-error preservation, synchronous assertion typing, and a full-row batch driver.
• The expanded scalar-materialization driver found a new deterministic stale nested-reference failure; its reduced expected-failure seed landed in test(db): extract reusable trace runner #1718.
• State-equivalence coverage expansion merged in test(db): expand includes oracle state coverage #1719: generated full-row batches at depths 1–4, guaranteed connected paths, reorder and single-row rekey/reparent coverage, flat array/concat projections, and three reduced expected-failure classes.
• Controlled readiness/timing drivers merged in test(db): add controlled includes oracle drivers #1720, including exact expected-failure checkpoints and obsolete-demand coverage.
• State-aware relationship-transition fuzzing in test(db): fuzz visible includes relationship transitions #1722: transition-only and stateful paired cases across every depth/target level, connected scalar activity, explicit key-separation invariants, and reduced expected-failure traces for every observed rekey and reparented-subtree boundary.
• Audited every issue and PR opened since this RFC on 2026-07-08; added the relevant reports below and recorded adjacent work owned by other RFCs.
• Shared observer feat(db): shared live-query observer + migrate all five adapters (RFC #1623 step 3) #1642, layout revision feat(db): ordered snapshot / layout-revision contract (RFC #1623 phase 4) #1669, and window controller feat(db): shared live-query window controller (RFC #1623 phase 5) #1675 merged in the parallel live-query platform track; use those contracts for publication/revision coverage rather than adding adapter-local state.
• Follow-up oracle coverage: after the runtime fixes, fuzz updates on the affected subtree by level relative to the transition; add a state-aware transition-history grammar for second relationship changes plus insert/delete interleavings. Add joined-alias correlation, initially-null correlation plus transition controls, and duplicate-key join multiplicity.
• Controlled drivers for publication/revision, ownership/lifecycle, and deterministic work counters, including layered live queries (Updating a parent source row empties materialize() fields in live queries #1713) and correlated-join pushdown (Join inside a correlated include ignores the subquery's where filter — scans the whole source collection #1709).
• Architectural PR 2: opaque plan identities and the multiplicity-aware shared transition reducer.

1. What's happening

The includes system (subquery-in-select, toArray(), materialize()) has produced a steady
stream of correctness bugs: silently misrouted data, dropped children, stale sort order, broken
adapter reactivity, and permanent loading states. Every claim below was verified against current
main with a red/green test (tests live on this branch, in describe('cluster-verification …')
blocks appended to existing test files).

#	Claim	Verified	Evidence
#1454	Duplicate alias in sibling includes silently misroutes data	RED — confirmed, worse than reported: the issues include is fully replaced by tag rows, real issues lost, nested comments empty	packages/db/tests/query/includes.test.ts (cluster-verification, claim A)
#1444	orderBy in an include ignored after optimistic update on the child collection	RED — confirmed: child values propagate, re-sort does not	same file, claim B
#1510	Live query stuck loading forever when a subquery's inner collection is cold on-demand and the outer produces zero rows	RED — confirmed: allCollectionsReady() never true because per-row lazy loadSubset never fires	packages/db/tests/query/includes-lazy-loading.test.ts (#1510 block)
#1533	Progressive sync: nested toArray children skip the fast-path snapshot	RED — confirmed: lazy alias ⇒ includeInitialState: false ⇒ the only requestSnapshot fires per parent row, after the progressive buffering window closed	same file, #1533 block (paired passing baseline for the direct query)
#1571 (part 1)	Solid: toArray include updates never reach the rendered data store	RED — confirmed, stronger than reported: even an untracked re-read of data is stale; the state map and underlying collection row do update	packages/solid-db/tests/useLiveQuery.test.tsx (#1571 block)
#1571 (part 2)	Initially-empty include starts null and never becomes reactive	Not reproduced: field is an empty child Collection from first render and populates on insert (caveat: Collection instances aren't Solid-reactive by design)	same file
#1495	Sync-confirmed child update misclassified as insert, crashes duplicate-key diagnostics	Fixed on main by merged #1600 (has() reclassification + config.utils guard)	includes.test.ts, claim C (green)
#1501	3-level nested toArray drops children when correlation keys overlap across parent groups	Fixed on main by merged #1607 (fan-out routing + snapshot reseeding)	includes.test.ts, claim D (green control)
#1488	On-demand observer reuse loses row ownership; cleanup deletes rows still in use	Not reproducible on main: the early-return shape exists, but atomic observer+ownership cleanup and ownership re-registration on subscribers:change compensate; likely fixed since the reported version	packages/query-db-collection/tests/query.test.ts (#1488 block, green)

New reports since this RFC was first drafted:

• innerJoin drops a live-query result when one of many children is deleted #1703 — deleting one of several matching joined rows deletes the collapsed live-query row.
  This is a P2 multiplicity failure outside the include wrapper: several differential rows can map
  to one public key, so the reducer must retain net multiplicity and publish a delete only when the
  count crosses zero. Add a direct join driver and reduced expected-failure trace before PR 2.
• materialize() subquery resolves to [] when its correlation predicate targets a joined alias #1704 / PR fix: correlated subquery resolves empty when correlation targets a joined alias #1705 — a correlated materialized subquery is empty when its predicate targets a
  joined alias. This is direct evidence for P1's scoped source identity: the compiler assumes the
  correlation ref belongs to the subquery's main from alias. Keep fix: correlated subquery resolves empty when correlation targets a joined alias #1705's initial and incremental
  tests; it may land as a narrow fix, while PR 2 removes the alias assumption structurally.
• materialize(q…findOne()) resolves to null, not undefined, when the correlation key is itself null #1706 / PR fix: resolve materialize(findOne()) to undefined when the correlation key is null #1707 — materialize(findOne()) leaks the internal null placeholder when the
  correlation key is null. The typed result is undefined. Treat fix: resolve materialize(findOne()) to undefined when the correlation key is null #1707 as an independent narrow
  semantic fix. The oracle narrows the current failure to rows first observed with a null key; matched→null and unmatched non-null cases are green controls.
• Join inside a correlated include ignores the subquery's where filter — scans the whole source collection #1709 — a join inside a correlated include defeats source-side predicate pushdown. Results
  are correct, but work scales with the whole child collection. Add a deterministic work counter
  beside Poor performance for nested includes #1634's benchmark; do not assume the P3 rewrite fixes this compiler/optimizer defect.
• Updating a parent source row empties materialize() fields in live queries #1713 — a parent-only update empties materialized fields in a live query layered over another
  live query. This is the core, adapter-free publication repro for Nested includes are undefined on next render after collection.update #1635. PR fix(db): materialize includes into parent rows before commit #1684 attempted the
  same pre-commit materialization fix but closed unmerged. Keep Updating a parent source row empties materialize() fields in live queries #1713 open and use it as P4's primary
  red/green gate.
• 3-level nested materialize is dropped when the middle level is shared by more than one parent row #1685 / PR Fix #1685 #1686 — three nested materialize(findOne()) levels drop the deepest row for one
  root when distinct middle rows point to the same shared row. RED — confirmed: the first
  recursive consumer deletes the shared row's INCLUDES_ROUTING stamp, so the second cannot
  register the next routing edge. This is another P3 shared-routing failure. PR Fix #1685 #1686 defers stamp
  cleanup until the outermost flush; land it as a narrow stopgap with the exact scalar regression,
  then delete that lifetime bookkeeping in PR 3.
• Discovered by test(db): extract reusable trace runner #1718 — nested scalar reference replacement stays stale. Updating a middle
  row from sharedId: 1 to a missing sharedId: 2 updates the visible foreign key but retains the
  old shared row in the existing incremental query. A fresh query over the same collections matches
  recomputation, isolating the fault to incremental nested routing/materialization. The reduced
  five-step expected-failure seed now lives in the scalar-materialization driver. Treat this as
  another P3 routing-lifetime failure, not an isolated symptom fix.
• Discovered by test(db): expand includes oracle state coverage #1719 — reinserted parents retain an obsolete shared route. When two parents share a correlation key, deleting and reinserting one under a new key can leave the old route alive; a later child for the old key is materialized under the reinserted parent. The reduced expected-failure trace isolates another P3 route-lifetime failure.
• Discovered by test(db): expand includes oracle state coverage #1719 — intra-batch child hand-off leaves aggregate membership stale. When a full-row batch moves multiple children between correlation buckets, array materialization and concat(toArray(...)) can both retain old membership instead of applying the whole hand-off. These are two projections of one batch-routing defect, covered by one reduced scenario under both materializations.
• Discovered by test(db): fuzz visible includes relationship transitions #1722 — rekey detachment and moved-subtree reactivity share a two-descendant boundary. Rekeying a visible row diverges immediately when at least two included levels remain below it. Reparenting is initially correct, but sequential scalar updates through the next two descendant levels leave deeper values stale. Reduced traces cover rekey at depth/level (3,1), (4,1), and (4,2), plus the corresponding three reparent/update-chain shapes; generated post-transition activity stays on the unaffected branch until the runtime fix turns those traces green.
• Nested includes are undefined on next render after collection.update #1635 — nested includes come back undefined on the next render after a parent
  collection.update() (Electric + React), self-healing on forced re-render. Updating a parent source row empties materialize() fields in live queries #1713 reduces this to
  an adapter-free layered live query and shows the bad placeholder persists until a child changes.
  Publication-contract class: this strengthens the case for PR 4 below over per-adapter shims.
• Poor performance for nested includes #1634 — useLiveQuery with multi-level includes costs up to ~100ms per run. A performance
  dimension this RFC's correctness scope doesn't target directly, but the per-flush bookkeeping
  that PR 3 deletes is the likely hot path; PR 3 should carry a benchmark for this repro.
• Ref count misassumption #1631 and PR Fix/live query record drop on subset unmount #1656 — two more ownership-loss reports in query-db-collection
  (eager-refcount vs hasListeners disagreement; persisted-owner baseline overwritten on insert).
  The specific on-demand store entries are removed while queries still use them when reusing an existing observer #1488 path did not reproduce (see appendix), but these show the ownership-loss
  failure mode is real via other mechanisms — the lifecycle-bookkeeping critique stands. Merged
  PRs fix(query-db): clean up empty ownership sets #1672, test(query-db-collection): characterize cancellation and subset cleanup #1673, and test: cover late query readiness rejection #1681 fixed bounded ownership/listener cleanup cases and added lifecycle
  tests; they do not prove the remaining reports fixed.

Reviewed but owned elsewhere: #1662 is already evidence E13 in loadSubset RFC #1657; #1698 is
an Electric/sync ingestion defect under RFC #1659; #1708 is a general join-index selection issue;
#1712 asks for a new remote-join feature; and #1721 changes React hook state ownership under the
live-query platform work. They are not gates for this includes RFC.

Status audit (2026-08-12): #1495 and #1501 are correctly closed after merged fixes #1600 and
#1607. #1635 and #1571 correctly remain open because their proposed fixes #1684 and #1604 closed
without merging. #1704 and #1706 correctly remain open while #1705 and #1707 are open. No issue
was closed as part of this audit.

Why these keep happening

The bugs are not independent. The includes system compiles each include into its own child D2
pipeline (sound), but then reconstructs include semantics in a ~2,300-line imperative output layer
(packages/db/src/query/live/collection-config-builder.ts) using alias maps, child collection
registries, correlation routing indexes, pending-change buffers, and in-place parent-row mutation.
Correctness rests on identities that are only implicit:

1. An alias is not a source identity. Sibling subqueries legitimately reuse lexical names, but
   the compiler flattens all includes aliases into one namespace, so { i: issues } and
   { i: tags } share one D2 input (Duplicate alias in sibling includes silently breaks nested children #1454).
2. A correlation key is not a parent identity. Multiple parents can subscribe to the same
   correlated child set, and a shared result row's routing metadata must outlive every recursive
   consumer (3-level nested materialize is dropped when the middle level is shared by more than one parent row #1685). A destructively-drained shared buffer can't represent that fan-out
   (3-level nested toArray: shared buffer in createPerEntryIncludesStates drops children when correlation keys overlap across parent groups #1501/fix(db): propagate changes through nested toArray includes at depth 3+ #1457 — patched by fix(db): nested toArray includes drop children when sibling groups share a correlation key (#1501) #1607, but the shared-state design remains).
3. Differential multiplicity is not CRUD intent. Several differential rows may also map to
   one public key (innerJoin drops a live-query result when one of many children is deleted #1703), so deleting one contributor must not delete the result while another
   remains. A (-1,+1) pair must become one atomic replacement including its order metadata. Today "insert vs update" is decided per call site —
   three near-copies of the accumulator exist (parent/child/nested), and the child copy retained a
   stale orderByIndex (TanStack DB "includes" ignores orderBy after optimistic update #1444). The landed fix: reconcile duplicate live query child inserts #1600 fix decides by checking collection.has(key)
   mid-flush, which works but keeps classification dependent on whatever state exists at flush time.
4. Object identity is not result revision. flushIncludesState mutates parent rows in place
   and force-emits through changesManager.emitEvents(events, true) to defeat the collection's own
   deepEquals suppression. React's version-bump mostly tolerates this; Solid's reconcile does
   not (Include value updates break with solidjs #1571), Nested includes are undefined on next render after collection.update #1635 suggests React has its own window, and each future adapter needs its own
   workaround.
5. Source-collection readiness is not query readiness. Readiness is a global boolean over all
   involved collections; lazy children that were never demanded (fix(db): live query stuck loading when subquery-in-select inner is cold on-demand #1510) or progressive children
   whose fast-path window is timing-dependent (Progressive sync: nested toArray subqueries skip the fast-path snapshot #1533) fall through it.

2. Design direction (all internal)

Five internal principles, each of which converts a bug class into an invariant. No public API is
added or changed.

• P1 — Opaque plan identities. The compiler assigns every include node and source a generated
  ID; user aliases are resolved lexically per subquery scope and never used as runtime keys. Joined
  aliases remain valid correlation targets (materialize() subquery resolves to [] when its correlation predicate targets a joined alias #1704). Invariant: alpha-renaming any subquery alias
  cannot change results.
• P2 — One transition reducer. A single reduction boundary turns a batch of weighted D2 tuples
  into net per-key transitions {key, countBefore, countAfter, before?, after?, orderBefore?, orderAfter?}. It retains multiplicity when several rows collapse to one public key (innerJoin drops a live-query result when one of many children is deleted #1703),
  publishes membership changes only when the count crosses zero, and applies replacements through
  two idempotent ops (set(key, after, orderAfter) / delete(key)). Value and order tuple live in
  the same versioned entry, so a replacement updates both atomically. Used by root live queries and
  all include levels. Invariant: no code path decides insert-vs-update by inspecting store state
  mid-batch, and one contributor cannot delete a multiply-supported result.
• P3 — Correlated relation operator. Replace the shared nested buffers / routing indexes /
  cumulative snapshots with one reusable internal structure: buckets keyed by
  (includeNodeId, correlationTuple) holding an ordered keyed relation, plus subscriber edges.
  Child deltas update a bucket once and fan out to every subscribed parent; a newly subscribed
  parent receives the bucket snapshot; removing a parent removes only its edge. Nesting recurses
  through the same operator — depth 3 is not a separate code path from depth 1. Internal child
  Collections (with their gcTime: 0 and config.utils hazards) shrink to this lightweight
  relation, keeping a Collection facade only where the API already promises one (bare
  subquery-in-select).
• P4 — Publication by replacement. When an include value changes, publish a shallow-copied
  parent row with a new include array/value (structural sharing for unchanged fields) through the
  normal update path. Layered live queries must observe the materialized value on parent-only
  updates (Updating a parent source row empties materialize() fields in live queries #1713), not an internal placeholder. Reference change ⇔ value change, for every adapter.
  Deletes the force-emit hack, the Solid clone shim, and the in-place/deepEquals tension.
• P5 — Demand-relative readiness. Internally, a live query is ready when every currently
  demanded source subset has settled its initial snapshot. An empty outer demands nothing from the
  child, so the child is vacuously ready (fix(db): live query stuck loading when subquery-in-select inner is cold on-demand #1510). A nested progressive child requests its
  correlated subset through the same snapshot path a direct query uses (Progressive sync: nested toArray subqueries skip the fast-path snapshot #1533). This is a
  reorganization of existing readiness bookkeeping, not a new status API.

Explicitly out of scope (would be new features): a query.explain() API, public demand/lease
APIs, per-include loading-status fields, new materialization modes or helpers. Dev-mode internal
assertions (throw on duplicate routing registration, orphaned buffer entries, child writes with no
registered parent) are in scope — today's failure mode is silent data corruption.

3. Proposed PR series

Sequencing rationale — oracle first. The obvious order (fix the five verified bugs, then build
the safety net) repeats the pattern that produced this cluster: each fix validated only by its own
repro test — that is exactly how "fix depth 2 (#1457), discover depth 3 (#1501), fix that (#1607)"
happened. Instead, the oracle harness is the first sequenced work, and the state-correctness bugs
are fixed against it. Writing the naive recompute evaluator also forces the semantics questions
(optimistic child update + orderBy, empty-include representation, optimistic+confirm convergence)
to be settled once, in a reference implementation, rather than implicitly across five PR reviews.

The bugs split into two property classes, which is why there are two tracks:

• State-equivalence bugs (Duplicate alias in sibling includes silently breaks nested children #1454 misrouting, TanStack DB "includes" ignores orderBy after optimistic update #1444 stale sort, 3-level nested materialize is dropped when the middle level is shared by more than one parent row #1685 dropped nested row, innerJoin drops a live-query result when one of many children is deleted #1703
  multiplicity, materialize() subquery resolves to [] when its correlation predicate targets a joined alias #1704 joined-alias correlation, materialize(q…findOne()) resolves to null, not undefined, when the correlation key is itself null #1706 null-key materialization — and the whole
  phase 2–3 refactor): detectable by
  incremental(query, history) === recompute(query, state). These wait for the oracle and are
  fixed against it, except for explicitly labeled narrow stopgaps already under review.
• Liveness, timing, and adapter bugs (fix(db): live query stuck loading when subquery-in-select inner is cold on-demand #1510 never-ready, Progressive sync: nested toArray subqueries skip the fast-path snapshot #1533 fast-path window, Include value updates break with solidjs #1571 Solid
  data store): invisible to a state-equivalence oracle — the converged state is correct; what is
  wrong is when it becomes available or which layer sees it. Gating these already-reviewed
  community PRs on harness-building adds no confidence and delays users, so they proceed in
  parallel.

Track A — parallel, not oracle-gated (community PRs, own regression tests)

• A1 — Readiness for undemanded lazy children (fixes fix(db): live query stuck loading when subquery-in-select inner is cold on-demand #1510). Land open PR fix(db): live query stuck loading when subquery-in-select inner is cold on-demand #1510: skip lazy
  aliases in allCollectionsReady; the isLoadingSubset gate still holds the query while per-row
  loads are in flight. Gate: the fix(db): live query stuck loading when subquery-in-select inner is cold on-demand #1510 verification tests (liveness assertions, bounded wait).
• A2 — Publication shim superseded. PR fix(solid-db): clone rows in syncDataFromCollection so reconcile sees in-place mutations #1604 closed unmerged; PR feat(db): shared live-query observer + migrate all five adapters (RFC #1623 step 3) #1642 has merged the shared
  createLiveQueryObserver across all five adapters. Do not revive a Solid-only clone. Put the
  publication fix at the shared collection/observer boundary in PR 4, gated by Include value updates break with solidjs #1571, Nested includes are undefined on next render after collection.update #1635, and
  the adapter-free layered-query repro Updating a parent source row empties materialize() fields in live queries #1713.
• A3 — Progressive fast path for nested children (fixes Progressive sync: nested toArray subqueries skip the fast-path snapshot #1533). When the child collection is
  in progressive mode, request the correlated snapshot at subscription setup (or on first
  parent-key batch) through the same requestSnapshot path a direct query uses, instead of the
  per-row lazy tap that fires after the buffering window closes
  (packages/db/src/query/live/collection-subscriber.ts:116-119,
  packages/db/src/query/compiler/index.ts:544-578). Fold in PR Test/progressive nested fastpath bug #1532's draft tests. Needs a
  small design note: the electric adapter's snapshot window (isBufferingInitialSync) vs late
  loadSubset. Gate: Progressive sync: nested toArray subqueries skip the fast-path snapshot #1533 verification test green (timing assertion), baseline stays green.

Sequenced track

1. Recompute-oracle property harness — merged in test(db): add includes recompute oracle #1716, hardened in test(db): tighten includes oracle failure checks #1717.
   The landed test-only harness compares real public include queries at depths 1–4 against an
   independent full-recompute model after each generated action. It includes scalar-materialization
   and metamorphic properties plus deterministic known-failure cases for Duplicate alias in sibling includes silently breaks nested children #1454, TanStack DB "includes" ignores orderBy after optimistic update #1444, 3-level nested materialize is dropped when the middle level is shared by more than one parent row #1685,
   confirmed reordering, and parent correlation-key changes.

   Post-merge fuzzing immediately found a generator-normalization counterexample. test(db): tighten includes oracle failure checks #1717 made known
   failures accept only assertion mismatches, normalized every generated or converted put, fixed
   the shared sync mock so rejected mutations do not suppress later optimistic actions, and added a
   repeated-rollback regression. Required CI is green.

   The landed green generator deliberately keeps existing relationship keys and positions stable
   where broader mutation would enter known-broken classes. It also does not guarantee a connected
   observation at every selected depth. Those are explicit limits of the foundation, not claims of
   complete coverage; the next test-only PR below closes them before the transition reducer lands.

Reusable trace runner — merged in #1718. The oracle now separates lifecycle and mutations
(drivers) from observation and recomputation (projections). The runner preserves same-turn
checkpoints for synchronous mutations, awaits genuinely asynchronous hooks, keeps trace failures
primary when cleanup also fails, and rejects async assertions at compile time. Structural-history,
scalar-materialization, and deterministic full-row batch drivers validate the boundary. Extending
the scalar driver immediately found the nested-reference replacement failure recorded above.

State-equivalence driver expansion — merged in #1719. The structural and materialization projections now cover generated full-row and multi-change batches, connected observations at depths 1–4, row reorders, single-row rekeying/reparenting, and flat array/concatenated materialization. Three assertion-specific expected-failure classes guard nested scalar redirects, obsolete shared routes after parent reinsertion, and intra-batch child hand-offs. Fixes turn these seeds green; unrelated runtime errors still fail.

In review — state-aware relationship-transition fuzzing (#1722). The generator now selects existing visible rows, enumerates every query depth and target level, varies identities and old/new correlation keys, and pairs a transition-only baseline with stateful connected scalar activity before and after the transition. Every forced transition must change recomputation at its exact checkpoint. The generated green corpus keeps post-transition activity on the unaffected branch; explicit expected-failure traces pin every observed rekey boundary and every reparented-subtree update-chain boundary.

Controlled readiness and timing drivers — merged in #1720. Deterministic lifecycle/checkpoint drivers now cover an empty outer query with an undemanded lazy child (#1510), progressive nested-child snapshot timing (#1533), and obsolete-demand cancellation. Temporal assertions record readiness and snapshot events at exact checkpoints rather than comparing only converged rows.

Next oracle follow-ups. Once the relationship runtime defects turn the reduced traces green, add an affected-subtree matrix over update level relative to the transitioned row. Separately, add a state-aware transition-history grammar that can issue a second relationship change and interleave inserts/deletes around a visible transition; basic hand-written controls are green, but this broader history space is not yet fuzzed. The next state-oracle slice adds joined-alias correlation (#1704), initially-null correlation with transition controls (#1706), and collapsed join multiplicity (#1703). Later controlled slices cover publication/revision (#1571/#1635/#1713), ownership/lifecycle (#1488/#1631/#1656), and deterministic work counters for #1634 and #1709.

These controlled slices gate the later architectural work that touches each class; they do not block starting the state-correctness refactor once its seed inventory is in place. Narrow production fixes land early only when urgent and independent of machinery the sequenced refactor will replace.
2. Opaque node/source IDs + shared multiplicity-aware transition reducer (fixes #1454, #1444, #1703, and #1704 structurally).

• Compiler assigns generated IDs to include nodes and sources; all runtime maps
  (collectionByAlias, routing, lazy-target resolution) key by ID; lexical aliases resolve per
  scope — this fixes Duplicate alias in sibling includes silently breaks nested children #1454 without alias mangling. Also replace computeRoutingKey's
  JSON.stringify([correlationKey, parentContext]) with one canonical structural-key encoder.
• Extract the single per-key transition reducer (P2) and route parent, child, and nested outputs
  through it — value and order tuple replaced atomically, which fixes TanStack DB "includes" ignores orderBy after optimistic update #1444 in all three
  accumulator sites by deleting them; also removes the has(key)-based reclassification from
  fix: reconcile duplicate live query child inserts #1600 (its tests remain and must stay green).
• Fallback: if this PR's timeline stretches, open PRs fix: duplicate alias in sibling includes silently breaks nested children #1455 (scoped inputs via __inc_N_alias
  mangling) and fix(db): includes orderBy ignored after optimistic update on child collection #1496 (third copy of the order-index fix) can land first as stopgaps — with the
  oracle from PR 1 now validating their completeness — and be deleted here.
• Gate: oracle harness (including the previously known-failing seeds), the alpha-renaming
  property, and the entire existing includes suite.

3. CorrelatedRelation replaces nested buffers/routing (P3). Introduce the operator with
   bucket state, subscriber edges, snapshot-on-subscribe, and non-destructive fan-out. Run in
   shadow mode first (tests compare it against the legacy materializer over the oracle workloads),
   then swap nestedSetups / drainNestedBuffers / updateRoutingIndex /
   createPerEntryIncludesStates over to it and delete the legacy path. Internal child state
   becomes the lightweight relation; the Collection facade remains only for bare
   subquery-in-select includes. Removes the gcTime: 0 workaround and the internal-collection
   config.utils hazard class, plus Fix #1685 #1686's deferred routing-stamp cleanup. Also the likely fix for
   the Poor performance for nested includes #1634 performance report — carry a benchmark based on that repro (deep-nested includes,
   target well under the reported ~100ms).
4. Copy-on-write publication (P4, fixes Include value updates break with solidjs #1571 structurally; expected to fix Nested includes are undefined on next render after collection.update #1635/Updating a parent source row empties materialize() fields in live queries #1713).
   Shallow-copy parent rows on include change with structural sharing; publish through the normal
   update path; delete emitEvents(events, true) and the hand-cloned prev/next; remove the Solid
   shim from A2 and verify the cross-adapter conformance suite
   (packages/solid-db/tests/conformance.test.tsx et al., from merged test: useLiveQuery conformance suite across all five framework adapters (RFC #1623) #1636) passes for
   React/Solid/Vue/etc. Coordination: if feat(db): shared live-query observer + migrate all five adapters (RFC #1623 step 3) #1642's shared createLiveQueryObserver has landed,
   implement the publication contract at the observer's snapshot boundary — one place instead of
   five adapters; add a red/green repro for Nested includes are undefined on next render after collection.update #1635 first. Perf gate: A/B bench before/after — rows
   are already double-cloned today for the forced event, so this is likely neutral-to-better.
5. Demand-relative readiness (P5, subsumes A1, fixes Progressive sync: nested toArray subqueries skip the fast-path snapshot #1533's class). Consolidate
   allCollectionsReady, lazy-alias exclusions, isLoadingSubset, and progressive snapshot
   delivery behind one internal demand model: readiness = all currently-demanded subsets settled.
   A1's exclusion list and A3's special-casing collapse into it. The oracle harness gains
   liveness/timing assertions here (bounded readiness; subset-before-full-sync) so this property
   class is fuzzed too, not just unit-tested.

Ongoing

• Dev-mode invariant assertions land opportunistically inside PRs 2–4 (duplicate routing
  registration, orphaned buckets at flush end, alias-keyed runtime lookups, unbalanced weighted
  batches). Each converts a silent-corruption mode into a thrown error in development builds.

4. Non-goals / rejected approaches

• No new public APIs (explain, loading-status fields, demand/lease surface, new helpers).
• No alias mangling (beyond the fix: duplicate alias in sibling includes silently breaks nested children #1455 fallback, if taken), no additional per-depth buffers or
  flush sub-passes, no per-adapter cloning beyond the temporary A2 shim, no growing the
  readiness-exclusion list beyond A1's stopgap. Each of these closes one issue while making the
  state machine harder to reason about — PRs 2–5 exist to delete them.

5. Risks

• PR 3 is the big one. Shadow mode, the landed oracle, and the generalized trace runner are the
  mitigation; it must not land until that runner is in place.
• PR 4 changes result-object identity guarantees (rows are replaced, not mutated). This is the
  documented expectation adapters already assume; the conformance suite plus the react/solid/vue
  adapter tests are the gate. Any user code depending on in-place mutation of live-query rows was
  already broken by deepEquals suppression semantics.
• A3 touches the electric adapter's sync window; it needs an e2e test in
  packages/electric-db-collection/e2e (PR Test/progressive nested fastpath bug #1532's draft e2e test is a starting point).
• Oracle-first delays the Duplicate alias in sibling includes silently breaks nested children #1454/TanStack DB "includes" ignores orderBy after optimistic update #1444 fixes relative to landing fix: duplicate alias in sibling includes silently breaks nested children #1455/fix(db): includes orderBy ignored after optimistic update on child collection #1496 immediately. The
  fallback in PR 2 caps that delay: if the structural fix stalls, the stopgap PRs land
  oracle-validated instead.
• The shared observer from RFC RFC: Live Query Framework Bindings Platform #1623 is now merged in feat(db): shared live-query observer + migrate all five adapters (RFC #1623 step 3) #1642. PR 4 must put the publication
  contract at the shared collection/observer boundary, not add per-adapter clones.

Appendix: relationship to reviewed PRs

PR	Current state and disposition
#1455 (duplicate alias)	Open; held for PR 2's structural fix, fallback only if PR 2 stalls
#1496 (orderBy after optimistic update)	Open; held for PR 2's reducer under the same fallback rule
#1510 (readiness)	Open; may land as A1, then be subsumed by PR 5
#1532 (progressive nested test)	Open; fold its tests into A3
#1604 (Solid clone)	Closed unmerged; do not revive after merged shared observer #1642
#1642 (shared live-query observer)	Merged; PR 4 now targets the shared collection/observer boundary
#1656 (record drop on subset unmount)	Closed unmerged; ownership family remains tracked by #1631/#1488
#1660 (gcTime 0 falsy default)	Merged; independent fix complete
#1672/#1673/#1681 (ownership lifecycle)	Merged narrow fixes/tests; remaining ownership reports still need their own pass
#1684 (pre-commit materialization)	Closed unmerged; #1713 is the current canonical repro
#1686 (#1685 routing-stamp lifetime)	Open narrow P3 stopgap; removed by PR 3
#1705 (joined-alias correlation)	Open narrow P1 fix; retain its regressions and remove the alias assumption in PR 2
#1707 (null correlation key)	Open independent semantic fix; retain its regressions in the oracle corpus
#1607/#1600/#1580	Merged; their tests remain gates

Issue #1488 (observer-reuse ownership) did not reproduce on main as reported; however, #1631 and
PR #1656 demonstrate the same ownership-loss failure mode through different mechanisms, so the
query-db-collection ownership/refcount lifecycle deserves its own focused pass (single
acquisition path that always registers ownership; leases over incidental bookkeeping) rather than
a per-symptom fix — tracked separately from this RFC.
Issue #1505 is closed; its underlying concern (include fields transiently unmaterialized, types
don't admit it) is addressed by PR 4's always-attached include values.