Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions docs/multiplayer-schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -2602,3 +2602,15 @@ persists, so KA-11 does not require a layout bump or migration.
An opt-in `dynamic_groups` catalog entry expands a saved row-count controller into indexed widget identities. These use the existing name-keyed registers; no op kind or on-document key changes. The new catalog hash requires coordinated reader adoption. Existing documents retain their original catalog and are re-minted to adopt the new layout.

Count writes change the projected layout, retaining hidden registers and applying fresh-row defaults only at read time. They do not delete rows or links. Explicit `widgets_values_form` declarations remain authoritative; supplied group metadata adds row-count bound validation (`min` to the allocation bound) and rejects a count change requiring a new declaration before mutation. This validation applies to the addressed occurrence in top-level, interior, and promoted writes. A count outside that range is rejected `dynamic_group_count_out_of_range`, whether it arrives as a `set_widget` value or as a saved count in an `add_node` or `insert_workflow` payload (including its definitions); a count change against a declared form is rejected `dynamic_group_form_mismatch`. Both are new rejection codes, so callers can count DynamicGroup refusals without parsing messages. A row-field name that no group row can address keeps `unknown_widget`, as any other unknown widget name does. Without group metadata, the package cannot identify count fields; compatible producer/catalog/reader admission remains the caller's responsibility. See [DynamicGroup catalog support](dynamic-groups.md) for the producer contract, bounds, and fixture provenance.

## Amendment A27 — 2026-11-01 — Lamport floor reads past companion rows; opt-in lenient read (no schema change)

§1.6 says `observedDocCounter()` validates every tuple and that a malformed entry fails the entire scan. The applier itself writes two `__stamps` rows that are not stamp tuples and carry no counter:
- `["grow_request", to_node, grow_id, family]`: an autogrow connect's requested slot shape, `[name, widget | null, isInputcount]`.
- `["interior_route", instance_id, incarnation]`: a deleted subgraph instance's retained definition id, a string.

So one autogrow connect or one subgraph-instance delete made the floor unreadable for every later producer admission on that document.

`observedDocCounter()` now passes over `__stamps` rows whose key begins with either tag and whose value has the shape the applier stores there (a string for `interior_route`; an array whose first element is not a number for `grow_request`, whose rows are `[name, widget, isInputcount]` with the grow's widget stored as sent). A companion-keyed row holding anything else is read as an ordinary row, so a counter can never hide behind a companion-looking key. The tags are `GROW_REQUEST_TAG` and `INTERIOR_ROUTE_TAG` in `doc.ts`, shared with the applier's key builders, and are matched on the canonical `JSON.stringify` key prefix. Every other row is validated exactly as §1.6 says, so the default read still fails closed. `__clock_reservations` rows are never skipped. The counters those operations carry are still counted through their own stamps (`["grow", …]`, `["node", …]`).

A producer that must keep minting rather than refuse may call `observedDocCounter(doc, { skipMalformed: true })`. It returns `{ counter, skipped }`: the maximum over the readable counters of both ledgers, and the number of malformed rows (companion rows are not counted). A malformed row whose first element is still a valid counter raises the floor: over-estimating the floor is safe, under-estimating it would let the producer mint under a counter already in the document. A caller that sees `skipped > 0` should log it. An unreadable `__clock_reservations` root counts as one skipped entry in this mode, since the winning counters live in `__stamps`; an unreadable schema or `__stamps` root still throws. The strict read and `DocDerivedLamportClockStore` refuse both. `DocDerivedLamportClockStore` keeps the strict read.
6 changes: 4 additions & 2 deletions src/applier.ts
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,8 @@ import {
resolveDefinition,
stampsMap,
widgetStorageOf,
GROW_REQUEST_TAG,
INTERIOR_ROUTE_TAG,
} from "./doc.js";
import {
declaresWidgetForm,
Expand Down Expand Up @@ -2173,7 +2175,7 @@ interface InteriorResolution {
}

function interiorRouteKey(instanceId: string, incarnation: string): string {
return JSON.stringify(["interior_route", instanceId, incarnation]);
return JSON.stringify([INTERIOR_ROUTE_TAG, instanceId, incarnation]);
}

function resolveInteriorNode(
Expand Down Expand Up @@ -3083,7 +3085,7 @@ type GrowRequest = [string, string | null, boolean];

/** `__stamps` key holding a grow's own request, companion to its `["grow", ...]` stamp. */
function growRequestKey(toNode: unknown, growId: unknown, family: string): string {
return JSON.stringify(["grow_request", String(toNode), String(growId), family]);
return JSON.stringify([GROW_REQUEST_TAG, String(toNode), String(growId), family]);
}

/**
Expand Down
93 changes: 81 additions & 12 deletions src/clock.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import * as Y from "yjs";
import { ROOT_CLOCK_RESERVATIONS, ROOT_STAMPS } from "./doc.js";
import { GROW_REQUEST_TAG, INTERIOR_ROUTE_TAG, ROOT_CLOCK_RESERVATIONS, ROOT_STAMPS } from "./doc.js";
import { assertReadableSchema } from "./schema-version.js";
import type { Actor } from "./types.js";

Expand Down Expand Up @@ -29,13 +29,14 @@ const documentTransactionTails = new WeakMap<Y.Doc, Promise<void>>();
* counter: the Y.Doc is the caller-owned lineage snapshot and this store's
* transaction queue is the serialization boundary.
*
* Both ledgers are validated rather than silently skipped. Malformed state
* Both ledgers are validated rather than silently skipped; only `__stamps`
* companion rows, which hold no counter, are passed over. Malformed state
* must fail closed before a producer mints an unsafe counter.
* Incarnation-qualified target keys all belong to this document lineage, so
* old node lives remain part of the observed floor (DQ-11 / ADR-ORDERING-0021). A
* successful transaction writes `__clock_reservations` so the next transaction
* observes it even before the producer's semantic op is applied. `__stamps`
* remains exclusively the semantic write-target ledger exposed by readStamps.
* remains the semantic write-target ledger exposed by readStamps.
*/
export class DocDerivedLamportClockStore implements LamportClockStore {
public constructor(private readonly doc: Y.Doc) {}
Expand Down Expand Up @@ -70,32 +71,100 @@ export class DocDerivedLamportClockStore implements LamportClockStore {
}
}

// Counterless `__stamps` rows (doc.ts companion tags), matched on the canonical
// JSON.stringify key prefix the applier writes AND on the value shape the
// applier stores there, so a counter under a companion-looking key is never
// silently dropped from the floor.
const GROW_REQUEST_PREFIX = `["${GROW_REQUEST_TAG}",`;
const INTERIOR_ROUTE_PREFIX = `["${INTERIOR_ROUTE_TAG}",`;

function isCompanionRow(key: string, value: unknown): boolean {
if (key.startsWith(INTERIOR_ROUTE_PREFIX)) return typeof value === "string";
// applier GrowRequest: [name, widget, isInputcount]. The applier stores the
// grow's widget field as sent, so only the leading position is load-bearing:
// a row whose first element is not a number cannot carry a counter.
if (key.startsWith(GROW_REQUEST_PREFIX)) return Array.isArray(value) && typeof value[0] !== "number";
return false;
}

// A malformed row still raises a lenient floor when its first element is a
// valid counter: over-estimating the floor is safe, under-estimating it lets a
// producer mint under a counter already in the document.
function salvageCounter(value: unknown): number | undefined {
if (!Array.isArray(value)) return undefined;
const counter: unknown = value[0];
return typeof counter === "number" && Number.isSafeInteger(counter) && counter >= 0 ? counter : undefined;
}

/** Result of a lenient floor read: the maximum valid counter and how many rows were unreadable. */
export interface LenientDocCounter {
counter: number | undefined;
skipped: number;
}

/**
* Maximum counter across winning stamps and reservations in a current-schema
* document. Missing ledgers are empty; missing/unsupported schema is refused.
* Companion rows in `__stamps` carry no counter and are skipped.
* Reads neither write structs nor create absent roots.
*
* By default a malformed row throws (fail closed, ADR-021). With
* `skipMalformed`, a producer that must keep minting gets the maximum over the
* readable counters instead (a malformed row whose first element is a valid
* counter still raises it), plus the count of malformed rows (companion rows
* are not counted). A caller that sees `skipped > 0` should log it. An
* unreadable reservations root counts as one skipped entry; an unreadable
* schema or `__stamps` root still throws.
*/
export function observedDocCounter(doc: Y.Doc): number | undefined {
export function observedDocCounter(doc: Y.Doc): number | undefined;
export function observedDocCounter(doc: Y.Doc, options: { skipMalformed: true }): LenientDocCounter;
export function observedDocCounter(
doc: Y.Doc,
options?: { skipMalformed: true },
): number | undefined | LenientDocCounter {
assertReadableSchema(doc, "observedDocCounter");
let maximum: number | undefined;
let skipped = 0;
for (const name of [ROOT_STAMPS, ROOT_CLOCK_RESERVATIONS]) {
const root = doc.share.get(name);
if (root === undefined) continue;
// Snapshot roots arrive as AbstractType, not Y.Map. Type only an existing
// root; reject sequence content before getMap can hide it in a map view.
if (root._start !== null) throw new TypeError(`${name} root contains non-map content`);
const ledger = doc.getMap<unknown>(name); // throws for a different concrete Y type
let ledger: Y.Map<unknown>;
try {
if (root._start !== null) throw new TypeError(`${name} root contains non-map content`);
ledger = doc.getMap<unknown>(name); // throws for a different concrete Y type
} catch (error) {
// Lenient mode passes over an unreadable reservations root as one skipped
// entry. Winning counters live in __stamps, whose root must stay readable.
if (options?.skipMalformed !== true || name !== ROOT_CLOCK_RESERVATIONS) throw error;
skipped++;
continue;
}
ledger.forEach((value, key) => {
if (!Array.isArray(value) || value.length !== 3 || typeof value[1] !== "string" ||
typeof value[2] !== "string" || value[2].length === 0) {
throw new TypeError(`${name} contains a malformed stamp tuple`);
if (name === ROOT_STAMPS && isCompanionRow(key, value)) return;
let counter: number | undefined;
try {
counter = ledgerRowCounter(name, key, value);
} catch (error) {
if (options?.skipMalformed !== true) throw error;
skipped++;
counter = salvageCounter(value);
if (counter === undefined) return;
}
if (name === ROOT_CLOCK_RESERVATIONS) validateReservation(key, value[1], value[2]);
const counter = validateLamportCounter(value[0], name === ROOT_STAMPS);
maximum = maximum === undefined ? counter : Math.max(maximum, counter);
});
}
return maximum;
return options?.skipMalformed === true ? { counter: maximum, skipped } : maximum;
}

function ledgerRowCounter(name: string, key: string, value: unknown): number {
if (!Array.isArray(value) || value.length !== 3 || typeof value[1] !== "string" ||
typeof value[2] !== "string" || value[2].length === 0) {
throw new TypeError(`${name} contains a malformed stamp tuple`);
}
if (name === ROOT_CLOCK_RESERVATIONS) validateReservation(key, value[1], value[2]);
return validateLamportCounter(value[0], name === ROOT_STAMPS);
}

function validateReservation(key: string, producer: string, storedKey: string): void {
Expand Down
6 changes: 5 additions & 1 deletion src/doc.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@
* ├── Y.Map '__applied' — op_id → sha256 of the canonical op payload
* │ (idempotency + op_id-reuse detection, §4
* │ amendment A8; a legacy `1` is a pre-A8 record)
* ├── Y.Map '__stamps' — write-target key → [base_version, actor, op_id] (§4)
* ├── Y.Map '__stamps' — write-target key → [base_version, actor, op_id] (§4),
* │ plus counterless grow_request / interior_route rows
* ├── Y.Map '__link_state' — normalized link id → durable descriptor (§1.5)
* └── Y.Map '__clock_reservations' — producer identity → reserved counter tuple (§1.6)
*/
Expand Down Expand Up @@ -278,6 +279,9 @@ export const ROOT_META = "meta";
export const ROOT_APPLIED = "__applied";
export const ROOT_STAMPS = "__stamps";
export const ROOT_LINK_STATE = "__link_state";
/** `__stamps` companion-row key tags: counterless rows written beside a stamp. */
export const GROW_REQUEST_TAG = "grow_request";
export const INTERIOR_ROUTE_TAG = "interior_route";
/** Created lazily by a successful clock admission, never exported from the entrypoint. */
export const ROOT_CLOCK_RESERVATIONS = "__clock_reservations";

Expand Down
3 changes: 2 additions & 1 deletion src/read.ts
Original file line number Diff line number Diff line change
Expand Up @@ -472,7 +472,8 @@ export function readApplied(doc: Y.Doc): Readonly<Record<string, unknown>> {
* The LWW ledger as plain frozen data: write-target key →
* `[base_version, actor, op_id]` (schema §3/§4). This is where per-op actor
* attribution is durably observable, and it is what a cross-language
* conformance harness compares — no projection contains it.
* conformance harness compares — no projection contains it. It also carries
* counterless `grow_request` / `interior_route` companion rows.
*/
export function readStamps(doc: Y.Doc): Readonly<Record<string, unknown>> {
assertSnapshotReadable(doc, "readStamps");
Expand Down
Loading
Loading