Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions docs/15-costs-and-margins.md
Original file line number Diff line number Diff line change
Expand Up @@ -203,6 +203,50 @@ within a live instance. Async handlers can interleave across awaits. Those
flags are admission controls, not persisted crash recovery or authority to
delete data. Repository CAS and persisted authority remain necessary.

The coordination tests pause actual Git HTTP and alarm requests at an R2
root read. Competing Git requests return 429 without reading R2. While HTTP
holds `graspBusy`, signed event writes and HTTP management are refused and
an alarm defers its storage work. While an alarm holds `graspControls`, a
signed repository-state event still enters purgatory. This is intentional
evidence of a narrower admission fence, not proof of exclusive maintenance.
That event changes SQL authority without an R2 operation in the paused
fixture; it does not demonstrate an existing Git publication or data-loss bug.
Both successful and failed R2 reads release their flags. Consuming the
completed HTTP response performs no more R2 reads in this fixture.

| Entry point | Existing coordination | Limit before collection |
|---|---|---|
| Git advertisement, fetch and push | `grasp` sets `graspBusy` before its first await and rejects either occupied flag | Live-instance admission only; no persisted reader lease |
| Accepted state and promotion | Run inside the HTTP fence or alarm controls; promotion rechecks candidates after reads | An immutable read session caches bytes but acquires no lock |
| Non-GET HTTP management | `Relay.fetch` holds `graspControls` while the route runs and refuses `graspBusy` | Other controls and event writes can still interleave |
| Signed events, imports and pulls | `acceptAny` or `writeGate` refuses `graspBusy` | `graspControls` alone does not refuse event writes |
| Alarm, PR expiry and pending promotion | Alarm defers during `graspBusy`, otherwise holds `graspControls` | Alarm entry does not refuse an already active control operation |
| Whole-relay deletion | Management or lease-expiry alarm calls `teardown` | Teardown has no internal lock and deletes the whole slug prefix; it is not a metadata collector |
| Direct repository helper calls | Production callers are the HTTP handler and `graspTick` | `gitRepository` and `checkpoint()` acquire no host fence themselves; test calls bypass admission |

The current HTTP handler constructs its response bytes before the read
session closes. A future streaming path would need to preserve coordination
until its last storage read. None of these in-memory flags coordinates an
independent R2 client, another object ID sharing a prefix, or recovery after
the owning instance is replaced. CAS protects root publication, but does not
prevent deletion of an object that a reader or unpublished writer still
needs. Removing a repository announcement also leaves its WAL and quota
rows behind; visibility is not a storage-liveness test.

Before metadata collection, a complete inventory needs explicit object,
byte and operation budgets and must refuse an incomplete mark. The host
also needs a proven ownership and recovery protocol covering active
readers, unpublished writes, administrative deletion and maintenance.
Budget exhaustion, missing or corrupt dependencies and changed roots must
prevent a deletion decision. Stable before/after roots alone are insufficient:
a reader may hold an older root, or a publisher may have uploaded objects
without publishing its new root yet. This remains a collection blocker;
the tests introduce no collector or runtime coordination change.

`npm test -- test/object/grasp-coordination.test.ts` reproduces the paused
HTTP/alarm checks. The checkpoint inventory above remains an isolated,
quiescent fixture rather than an online maintenance API.

Provider request accounting separates HTTP/RPC/alarm invocations from
incoming WebSocket messages, which have a 20:1 billing ratio. Socket frames
do not become Worker requests. DO duration counts overlapping work once per
Expand Down
106 changes: 106 additions & 0 deletions test/object/grasp-coordination.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
// Real HTTP and alarm reads pause at R2 to expose the live instance's
// admission fences. These tests establish no crash-safe collection lease.
import { env, runInDurableObject } from "cloudflare:test";
import { describe, expect, it } from "vitest";
import { generateSecretKey } from "nostr-tools/pure";
import { npubEncode } from "nostr-tools/nip19";
import { KIND_REPO, KIND_REPO_STATE } from "../../src/kinds.ts";
import { gitRepositoryPath } from "../../src/grasp-policy.ts";
import { ev, pk, rpc } from "../helpers/relay.ts";
import { WS } from "../helpers/ws.ts";

describe("GRASP coordination", () => {
it.each([
{ mode: "http", outcome: "success" }, { mode: "http", outcome: "failure" },
{ mode: "alarm", outcome: "success" }, { mode: "alarm", outcome: "failure" },
])("a paused $mode read holds admission until $outcome", async ({ mode, outcome }) => {
const host = `grasp-coordination-${mode}-${outcome}.bind.ws`;
const owner = generateSecretKey();
await rpc(host, owner, "claim");
await rpc(host, owner, "applypreset", "grasp");
const path = gitRepositoryPath(npubEncode(pk(owner)), "coordination");
const announcement = ev(owner, KIND_REPO, "", [
["d", "coordination"], ["clone", `https://${host}${path}`],
["relays", `wss://${host}`], ["maintainers", pk(owner)],
]);
const connection = await WS.connect(host);
expect((await connection.ok(announcement)).ok).toBe(true);
connection.ws.close();
const state = ev(owner, KIND_REPO_STATE, "", [["d", "coordination"], ["refs/heads/main", "a".repeat(40)]]);
await runInDurableObject(env.RELAY.getByName(host.split(".")[0]), async (relay) => {
const bucket = relay.media;
let entered!: () => void;
let release!: () => void;
const atRead = new Promise<void>((resolve) => { entered = resolve; });
const gate = new Promise<void>((resolve) => { release = resolve; });
let gets = 0;
let otherOperations = 0;
let held = false;
const observed = new Proxy(bucket, { get(target, name) {
if (name === "get") return async (key: string, options?: R2GetOptions) => {
gets++;
if (!held && key.endsWith("/root.json")) {
held = true;
entered();
await gate;
if (outcome === "failure") throw new Error("injected R2 read failure");
}
return target.get(key, options);
};
const value = Reflect.get(target, name, target);
return typeof value === "function" ? (...args: unknown[]) => {
otherOperations++;
return Reflect.apply(value, target, args);
} : value;
} });
Object.defineProperty(relay, "media", { value: observed, configurable: true });
const request = () => new Request(`http://${host}${path}/info/refs?service=git-upload-pack`);
const active = mode === "http" ? relay.fetch(request()) : relay.alarm();
try {
await atRead;
expect(relay.graspBusy).toBe(mode === "http");
expect(relay.graspControls).toBe(mode === "alarm" ? 1 : 0);
const before = gets;
const otherBefore = otherOperations;
const refused = await relay.fetch(request());
expect(refused.status).toBe(429);
await refused.arrayBuffer();
expect(gets).toBe(before);
const admission = await relay.acceptAny(state, relay.virtualConn(host, pk(owner)));
expect(gets).toBe(before);
expect(otherOperations).toBe(otherBefore);
if (mode === "http") {
expect(admission.ok).toBe(false);
expect(admission.msg).toContain("Git transaction in progress");
const management = await relay.fetch(new Request(`http://${host}/`, { method: "POST", body: "{}" }));
expect(management.status).toBe(429);
await management.arrayBuffer();
await relay.alarm();
expect(gets).toBe(before);
expect(relay.graspBusy).toBe(true);
expect(relay.graspControls).toBe(0);
} else {
// Controls blocks Git admission, but event writes can still alter
// authority while alarm work awaits R2. It is not a general lock.
expect(admission.ok).toBe(true);
expect(relay.sql.exec("SELECT id FROM events WHERE id=?", state.id).toArray()).toHaveLength(1);
expect(relay.graspControls).toBe(1);
}
release();
const response = await active;
expect(relay.graspBusy).toBe(false);
expect(relay.graspControls).toBe(0);
const completedGets = gets;
if (response) {
expect(response.status).toBe(outcome === "failure" ? 503 : 200);
await response.arrayBuffer();
expect(gets).toBe(completedGets);
}
} finally {
release();
await active;
Reflect.deleteProperty(relay, "media");
}
});
});
});