diff --git a/docs/15-costs-and-margins.md b/docs/15-costs-and-margins.md index c67f599..659e5c2 100644 --- a/docs/15-costs-and-margins.md +++ b/docs/15-costs-and-margins.md @@ -103,7 +103,8 @@ Prices go up when bitcoin falls. When it rises, do nothing until a line has sat ## Accounting coverage, checked Sept. 4, 2026 -The following inventory describes the host with ntig 0.1.1. Observing an +The following inventory describes the host with ntig 0.2.0 and unmigrated +format-1 repositories. Installing the library does not migrate roots. Observing an expense does not authorize a new customer charge. `/fuel` and the Health tab expose aggregate tenant meters, not every operation the host pays for. @@ -136,6 +137,18 @@ automatic reconciliation queue or orphan collector exists. Counting PUT bytes again as stored bytes would double-charge existing media accounting. Future temporary packs also need reservations while both copies exist. +For an explicitly migrated format-2 repository, ref advertisements read two +R2 objects, the root and manifest. This saves legacy history and pack reads +on that path; full loads and Git transfers still read all stored packs. +Each new commit also retains a manifest and receipt-index path nodes beside +its record and optional pack. Ref-only workloads can pass 128 transactions, +but their metadata and reservation rows continue to grow. All of these keys +use the same pre-PUT storage reservation. CAS-loser objects remain accounted +for, and an ambiguous PUT can leave a reservation larger than physical +storage. Neither receipt lookup nor checkpointing reclaims old bytes. +Operation counts are host costs, not a new tenant charge; lower read costs +alone do not establish a margin after metadata writes and retained storage. + `graspBusy`, `graspControls` and the job round's `working` flag fence work within a live instance. Async handlers can interleave across awaits. Those flags are admission controls, not persisted crash recovery or authority to @@ -219,7 +232,7 @@ These are proposed operating boundaries, not new allowances or charges. ## Proposed maintenance contract -ntig 0.1.1 has no resumable optimization runner. No maintenance job, budget +ntig 0.2.0 has no resumable optimization runner. No maintenance job, budget setting, wake schedule, format migration or garbage collection is enabled by this contract. Existing ref cleanup, retention and alarms keep their current roles. An eventual integration has these boundaries: diff --git a/docs/22-grasp-01-git-hosting.md b/docs/22-grasp-01-git-hosting.md index c6bf571..b6358c6 100644 --- a/docs/22-grasp-01-git-hosting.md +++ b/docs/22-grasp-01-git-hosting.md @@ -121,11 +121,23 @@ The hard limits are: |---|---:|---| | Relay | 16 repositories, 320 MiB Git storage | accepted repositories and billed retained Git objects across the relay | | Repository | 4 MiB per pack, 16 MiB packed history | one upload and the aggregate packed history read into memory | -| Repository | 128 transactions, 1,024 refs, 4,096 objects | WAL history, ref maps and verified object graphs | - -Checkpoints, compaction, orphan collection, large-pack streaming and -production capacity measurements remain follow-up work. Once a repository -reaches its transaction limit, all further Git writes are refused, including +| Repository | 1,024 refs, 4,096 objects | ref maps and verified object graphs | +| Format 1 | 128 transactions | legacy WAL replay | +| Format 2 | 128 unique packs, 2 MiB manifest | explicitly migrated repository metadata | + +ntig 0.2.0 reads both formats, but installing it leaves existing roots alone +and new repositories still use format 1. Bindws exposes no checkpoint action +or scheduled migration. The backend's explicit `checkpoint()` upgrade is +one-way: every reader and writer must support format 2 before an isolated +repository is migrated. Versions 0.1.x cannot read the upgraded root. +Migration retains old data and receipts; it does not reclaim storage. + +Format-2 ref advertisements read the root and manifest without downloading +packs. They retain the same accepted HEAD and hidden PR rules. Full clone, +fetch and push still load the stored packs; selective object reads, +compaction, orphan collection, large-pack streaming and production capacity +measurements remain follow-up work. Once a format-1 repository reaches its +transaction limit, all further Git writes are refused, including the timed cleanup transaction. An expired unknown PR ref therefore stays hidden when cleanup cannot commit, while immutable WAL history keeps its old object bytes billed until the relay is torn down. The same retention behavior diff --git a/package-lock.json b/package-lock.json index 009d5ac..565b989 100644 --- a/package-lock.json +++ b/package-lock.json @@ -9,7 +9,7 @@ "dependencies": { "@noble/hashes": "^2.4.0", "nostr-tools": "^2.25.1", - "ntig": "file:vendor/ntig-0.1.1.tgz" + "ntig": "file:vendor/ntig-0.2.0.tgz" }, "devDependencies": { "@cloudflare/vitest-pool-workers": "^0.22.0", @@ -2852,9 +2852,9 @@ "license": "MIT" }, "node_modules/ntig": { - "version": "0.1.1", - "resolved": "file:vendor/ntig-0.1.1.tgz", - "integrity": "sha512-8Sd4ZrGVmI2ydPLvZxiQyncSp4hld3PBjpXlKgUbyf4X1mOfOMrFMJSb+vDUxm8MBgWz6M8RwhNhsc+E0XNSsA==", + "version": "0.2.0", + "resolved": "file:vendor/ntig-0.2.0.tgz", + "integrity": "sha512-t2/XOHMfFs4nom7ZHnvpaoOHc1ZZRyNThocoYhVWAr76t6UY33zBgngMy6tB3YafAdxfUp8nR7CjXMtKmljyHg==", "license": "MIT", "dependencies": { "@scure/base": "2.4.0", diff --git a/package.json b/package.json index 9be3035..bc70f4b 100644 --- a/package.json +++ b/package.json @@ -25,7 +25,7 @@ "dependencies": { "@noble/hashes": "^2.4.0", "nostr-tools": "^2.25.1", - "ntig": "file:vendor/ntig-0.1.1.tgz" + "ntig": "file:vendor/ntig-0.2.0.tgz" }, "devDependencies": { "@cloudflare/vitest-pool-workers": "^0.22.0", diff --git a/src/grasp.ts b/src/grasp.ts index b7ce4c8..e5d50ec 100644 --- a/src/grasp.ts +++ b/src/grasp.ts @@ -94,6 +94,7 @@ function authorizedRepository(relay: Relay, repo: RepositoryAnnouncement, wal: W }); return { load: () => authorized.load(), + loadRefs: () => authorized.loadRefs ? authorized.loadRefs() : authorized.load(), commit: async (request: CommitRequest) => { const receipt = await authorized.commit(request); for (const update of request.updates) { diff --git a/src/nip11.ts b/src/nip11.ts index 2cf2d3d..1d0c4c9 100644 --- a/src/nip11.ts +++ b/src/nip11.ts @@ -68,7 +68,7 @@ export function nip11(relay: Relay, host: string) { if (relay.settings.isLeased() && p.lease) doc.lease = { expires_at: p.lease.until, holder: p.lease.holder || undefined, claim_url: host ? "https://" + host + "/" : undefined }; if (featureOn(p, "grasp") && p.reads === "open") { doc.supported_grasps = ["GRASP-01"]; - doc.repo_acceptance_criteria = "Repository announcements name this service in clone and relays; the relay write rule, bans, proof of work and fuel apply. At most 16 repositories, 4 MiB per pack, 16 MiB packed history and 128 Git transactions per repository."; + doc.repo_acceptance_criteria = "Repository announcements name this service in clone and relays; the relay write rule, bans, proof of work and fuel apply. At most 16 repositories, 4 MiB per pack, 16 MiB packed history and 128 Git transactions per unmigrated format-1 repository. Explicitly migrated format-2 repositories retain at most 128 unique packs."; if (p.writes !== "open" || p.blockedWords.length) doc.curation = "The relay's configured write rule and blocked topics apply to Nostr repository and collaboration events."; } if (featureOn(p, "sites")) doc.nsites = { diff --git a/test/object/grasp-checkpoint.test.ts b/test/object/grasp-checkpoint.test.ts new file mode 100644 index 0000000..16d714c --- /dev/null +++ b/test/object/grasp-checkpoint.test.ts @@ -0,0 +1,208 @@ +// Checkpoint migration is exercised through bindws' repository store. The +// fixture is isolated to one Durable Object and never touches a production key. +import { env, runInDurableObject } from "cloudflare:test"; +import { describe, expect, it } from "vitest"; +import { generateSecretKey } from "nostr-tools/pure"; +import { npubEncode } from "nostr-tools/nip19"; +import { encodePack, type WalRecord } from "ntig"; +import { KIND_REPO } from "../../src/kinds.ts"; +import { repository } from "../../src/grasp-state.ts"; +import { gitRepository } from "../../src/grasp.ts"; +import type { RepositoryAnnouncement } from "../../src/grasp-policy.ts"; +import { ev, pk, rpc } from "../helpers/relay.ts"; +import { WS } from "../helpers/ws.ts"; + +const encoder = new TextEncoder(); +const oid = async (type: string, data: Uint8Array) => { + const header = encoder.encode(`${type} ${data.length}\0`); + const input = new Uint8Array(header.length + data.length); + input.set(header); input.set(data, header.length); + return Array.from(new Uint8Array(await crypto.subtle.digest("SHA-1", input)), (b) => b.toString(16).padStart(2, "0")).join(""); +}; + +async function fixturePack() { + const blob = encoder.encode("checkpoint fixture\n"); + const blobID = await oid("blob", blob); + const tree = new Uint8Array([...encoder.encode("100644 README\0"), ...Uint8Array.from(blobID.match(/../g)!, (part) => parseInt(part, 16))]); + const treeID = await oid("tree", tree); + const commit = encoder.encode(`tree ${treeID}\nauthor ntig 1 +0000\ncommitter ntig 1 +0000\n\ncheckpoint\n`); + const commitID = await oid("commit", commit); + return { commitID, pack: await encodePack([{ type: "blob", data: blob }, { type: "tree", data: tree }, { type: "commit", data: commit }]) }; +} + +const announcement = (sk: Uint8Array, host: string, identifier: string) => ev(sk, KIND_REPO, "", [ + ["d", identifier], + ["clone", `https://${host}/${npubEncode(pk(sk))}/${identifier}.git`], + ["relays", `wss://${host}`], + ["maintainers", pk(sk)], +]); + +describe("GRASP checkpoint integration", () => { + it("migrates an isolated legacy repository and retains indexed receipts", async () => { + const host = "grasp-checkpoint.bind.ws"; + const owner = generateSecretKey(); + const identifier = "checkpoint"; + await rpc(host, owner, "claim"); + await rpc(host, owner, "setpolicy", { features: { grasp: true } }); + const connection = await WS.connect(host); + const event = announcement(owner, host, identifier); + expect((await connection.ok(event)).ok).toBe(true); + connection.ws.close(); + + const fixture = await fixturePack(); + const measurements = await runInDurableObject(env.RELAY.getByName(host.split(".")[0]), async (relay) => { + const repo = repository(relay, pk(owner), identifier) as RepositoryAnnouncement | null; + expect(repo).not.toBeNull(); + const wal = await gitRepository(relay, repo!); + const first = await wal.commit({ id: "checkpoint-initial", updates: [{ name: "refs/heads/main", old: null, new: fixture.commitID }], pack: fixture.pack }); + const legacy: WalRecord[] = []; + legacy.push((await wal.lookupRecord!(first.id))!); + let prior = fixture.commitID; + for (let i = 1; i < 128; i++) { + const receipt = await wal.commit({ id: `checkpoint-legacy-${i}`, updates: [{ name: `refs/tags/legacy-${i}`, old: null, new: prior }] }); + legacy.push((await wal.lookupRecord!(receipt.id))!); + } + expect((await wal.load()).sequence).toBe(128); + const before = relay.sql.exec<{ n: number; bytes: number }>("SELECT count(*) AS n, coalesce(sum(size),0) AS bytes FROM grasp_objects WHERE owner=?", pk(owner)).one(); + + const migration = await wal.checkpoint(); + expect(migration).toEqual({ sequence: 128, changed: true }); + expect(await wal.checkpoint()).toEqual({ sequence: 128, changed: false }); + const migrated = await wal.load(); + expect(migrated.checkpoint?.manifestHash).toMatch(/^[0-9a-f]{64}$/); + expect(migrated.records).toHaveLength(1); + expect(migrated.refs["refs/heads/main"]).toBe(fixture.commitID); + expect(await wal.lookupRecord!(legacy[0]!.id)).toMatchObject({ id: legacy[0]!.id, sequence: 1 }); + + // Format 2 allows more than the legacy 128-record ceiling. Ref-only + // commits exercise manifest/index growth without creating more packs. + for (let i = 128; i < 260; i++) { + await wal.commit({ id: `checkpoint-v2-${i}`, updates: [{ name: `refs/tags/v2-${i}`, old: null, new: prior }] }); + } + const replay = await wal.commit({ id: first.id, updates: [{ name: "refs/heads/main", old: null, new: fixture.commitID }], pack: fixture.pack }); + expect(replay).toMatchObject({ id: first.id, sequence: 1, replayed: true }); + const rebuilt = await gitRepository(relay, repo!); + const cold = await rebuilt.load(); + expect(cold.sequence).toBe(260); + expect(cold.refs["refs/tags/v2-259"]).toBe(fixture.commitID); + const advertised = await rebuilt.loadRefs!(); + expect(advertised.sequence).toBe(260); + expect(advertised.refs["refs/heads/main"]).toBe(fixture.commitID); + expect(await rebuilt.lookupRecord!("checkpoint-legacy-127")).toMatchObject({ sequence: 128 }); + expect(await rebuilt.lookupRecord!("missing-checkpoint-receipt")).toBeNull(); + + const after = relay.sql.exec<{ n: number; bytes: number }>("SELECT count(*) AS n, coalesce(sum(size),0) AS bytes FROM grasp_objects WHERE owner=?", pk(owner)).one(); + const keys = relay.sql.exec<{ key: string; size: number }>("SELECT key, size FROM grasp_objects WHERE owner=? ORDER BY key", pk(owner)).toArray(); + const categories = keys.reduce>((counts, { key }) => { + const category = key.split("/").at(-2) ?? key; + counts[category] = (counts[category] ?? 0) + 1; + return counts; + }, {}); + return { before, after, keyCount: keys.length, categories, sequence: cold.sequence }; + }); + + console.info("checkpoint storage measurement", { + beforeObjects: measurements.before.n, + afterObjects: measurements.after.n, + beforeBytes: measurements.before.bytes, + afterBytes: measurements.after.bytes, + retainedKeys: measurements.keyCount, + sequence: measurements.sequence, + }); + // The accounting adapter retains every immutable manifest/index/receipt + // write. This is intentionally observational: no successful PUT sum is + // treated as net storage, and the test records the retained SQL totals. + expect(measurements.after.n).toBeGreaterThan(measurements.before.n); + expect(measurements.after.bytes).toBeGreaterThan(measurements.before.bytes); + expect(measurements.keyCount).toBeGreaterThan(128); + expect(measurements.sequence).toBe(260); + }); + + it("retains a reservation when an immutable checkpoint write is ambiguous", async () => { + const host = "grasp-checkpoint-ambiguous.bind.ws"; + const owner = generateSecretKey(); + await rpc(host, owner, "claim"); + const fixture = await fixturePack(); + await runInDurableObject(env.RELAY.getByName(host.split(".")[0]), async (relay) => { + const objects = new Map(); + const operations = { get: 0, getBytes: 0, put: 0, putBytes: 0, putAck: 0, head: 0, headBytes: 0 }; + let writes = 0; + let throwManifest = false; + let failedManifestKey = ""; + let failedManifestBytes = 0; + const previous = { ...operations }; + const measure = (phase: string) => { + const delta = Object.fromEntries(Object.keys(operations).map((key) => [key, operations[key as keyof typeof operations] - previous[key as keyof typeof operations]])); + Object.assign(previous, operations); + console.info("checkpoint R2 operation measurement", { phase, ...delta }); + }; + const bucket = { + get: async (key: string) => { + operations.get++; + const item = objects.get(key); + if (item) operations.getBytes += item.bytes.length; + return item ? { size: item.bytes.length, etag: item.etag, arrayBuffer: async () => item.bytes.slice().buffer } : null; + }, + put: async (key: string, value: ArrayBufferView, options: { onlyIf: { etagMatches?: string; etagDoesNotMatch?: string } }) => { + operations.put++; + operations.putBytes += value.byteLength; + const current = objects.get(key); + if (options.onlyIf.etagDoesNotMatch === "*" && current) return null; + if (options.onlyIf.etagMatches !== undefined && (!current || current.etag !== options.onlyIf.etagMatches)) return null; + const bytes = new Uint8Array(value.buffer, value.byteOffset, value.byteLength).slice(); + const item = { bytes, etag: `fake-${++writes}` }; + objects.set(key, item); + if (throwManifest && key.includes("/manifests/")) { + failedManifestKey = key; + failedManifestBytes = bytes.length; + throw new Error("lost manifest acknowledgement"); + } + operations.putAck++; + return { etag: item.etag }; + }, + head: async (key: string) => { + operations.head++; + const item = objects.get(key); + if (item) operations.headBytes += item.bytes.length; + return item ? { size: item.bytes.length } : null; + }, + }; + const fakeRelay = { + slug: "checkpoint-ambiguous", + media: bucket, + sql: relay.sql, + fuelStatus: () => ({ outOfFuel: false }), + settings: { limitsOf: () => ({ cap: 0 }) }, + store: { authorBytes: () => 0 }, + } as unknown as typeof relay; + const repo = { id: "a".repeat(64), owner: pk(owner), identifier: "ambiguous", clone: [], relays: [], maintainers: [pk(owner)], private: false }; + const wal = await gitRepository(fakeRelay, repo); + await wal.commit({ id: "ambiguous-initial", updates: [{ name: "refs/heads/main", old: null, new: fixture.commitID }], pack: fixture.pack }); + measure("legacy commit"); + await wal.checkpoint(); + measure("format1 to format2 checkpoint migration"); + await wal.commit({ id: "measured-v2-ref", updates: [{ name: "refs/tags/measured", old: null, new: fixture.commitID }] }); + measure("format2 ref-only commit"); + await wal.loadRefs(); + measure("format2 loadRefs"); + await wal.load(); + measure("format2 full load"); + await wal.lookupRecord("ambiguous-initial"); + measure("format2 old receipt lookup"); + throwManifest = true; + await expect(wal.commit({ id: "ambiguous-next", updates: [{ name: "refs/tags/ambiguous", old: null, new: fixture.commitID }] })).rejects.toThrow(); + measure("ambiguous manifest PUT"); + expect(failedManifestKey).toContain("/manifests/"); + const retained = relay.sql.exec<{ size: number }>("SELECT size FROM grasp_objects WHERE key=?", failedManifestKey).one(); + expect(retained.size).toBe(failedManifestBytes); + expect((await wal.load()).refs["refs/tags/ambiguous"]).toBeUndefined(); + throwManifest = false; + await expect(wal.commit({ id: "ambiguous-next", updates: [{ name: "refs/tags/ambiguous", old: null, new: fixture.commitID }] })).resolves.toMatchObject({ replayed: false }); + expect((await wal.load()).refs["refs/tags/ambiguous"]).toBe(fixture.commitID); + const reserved = relay.sql.exec<{ n: number; bytes: number }>("SELECT count(*) AS n, coalesce(sum(size),0) AS bytes FROM grasp_objects WHERE owner=? AND key LIKE '%/manifests/%'", pk(owner)).one(); + console.info("ambiguous checkpoint reservation", reserved); + expect(reserved.n).toBeGreaterThan(0); + expect(reserved.bytes).toBeGreaterThan(0); + }); + }); +}); diff --git a/test/object/grasp.test.ts b/test/object/grasp.test.ts index 53f5250..5962d81 100644 --- a/test/object/grasp.test.ts +++ b/test/object/grasp.test.ts @@ -5,6 +5,8 @@ import { afterEach, describe, expect, it } from "vitest"; import { generateSecretKey } from "nostr-tools/pure"; import { npubEncode } from "nostr-tools/nip19"; import { KIND_REPO, KIND_REPO_STATE, KIND_GIT_PR, KIND_GIT_PR_UPDATE } from "../../src/kinds.ts"; +import { gitRepository } from "../../src/grasp.ts"; +import { repository as storedRepository } from "../../src/grasp-state.ts"; import { gitRepositoryPath } from "../../src/grasp-policy.ts"; import { ev, info, pk, rpc, sleep } from "../helpers/relay.ts"; import { WS } from "../helpers/ws.ts"; @@ -29,15 +31,15 @@ const objectID = async (type: string, data: Uint8Array) => { value.set(header); value.set(data, header.length); return Array.from(new Uint8Array(await crypto.subtle.digest("SHA-1", value)), (byte) => byte.toString(16).padStart(2, "0")).join(""); }; -const receivePack = (path: string, host: string, old: string | null, next: string | null, ref: string, pack?: Uint8Array) => SELF.fetch(`http://${host}${path}/git-receive-pack`, { +const receivePack = (path: string, host: string, old: string | null, next: string | null, ref: string, pack?: Uint8Array, requestId?: string) => SELF.fetch(`http://${host}${path}/git-receive-pack`, { method: "POST", - headers: { "content-type": "application/x-git-receive-pack-request" }, + headers: { "content-type": "application/x-git-receive-pack-request", ...(requestId ? { "X-Git-Request-Id": requestId } : {}) }, body: concat(packet(`${old ?? "0".repeat(40)} ${next ?? "0".repeat(40)} ${ref}\0report-status\n`), flush(), ...(pack ? [pack] : [])), }); // receiveSettled waits for transient alarm contention before testing the Git response. -const receiveSettled = async (path: string, host: string, old: string | null, next: string | null, ref: string, pack?: Uint8Array) => { +const receiveSettled = async (path: string, host: string, old: string | null, next: string | null, ref: string, pack?: Uint8Array, requestId?: string) => { for (let attempt = 0; attempt < 20; attempt++) { - const response = await receivePack(path, host, old, next, ref, pack); + const response = await receivePack(path, host, old, next, ref, pack, requestId); if (response.status !== 429) return response; await response.arrayBuffer(); await new Promise((resolve) => setTimeout(resolve, 10)); @@ -45,8 +47,8 @@ const receiveSettled = async (path: string, host: string, old: string | null, ne throw new Error("Git transaction did not become available"); }; -async function tinyRepository() { - const blob = encoder.encode("hello from grasp\n"); +async function tinyRepository(content = "hello from grasp\n") { + const blob = encoder.encode(content); const blobID = await objectID("blob", blob); const tree = new Uint8Array([...encoder.encode("100644 README\0"), ...Uint8Array.from(blobID.match(/../g)!, (part) => parseInt(part, 16))]); const treeID = await objectID("tree", tree); @@ -250,6 +252,109 @@ describe("GRASP", () => { late.ws.close(); }); + it("replays a hidden PR correction after checkpointing and more than 128 later transactions", async () => { + const host = "grasp-pr-checkpoint.bind.ws"; + const owner = generateSecretKey(); + const first = await tinyRepository(); + const next = await tinyRepository("corrected PR\n"); + await rpc(host, owner, "claim"); + await rpc(host, owner, "applypreset", "grasp"); + const c = await WS.connect(host); + expect((await c.ok(repository(owner, host, "correction"))).ok).toBe(true); + expect((await c.ok(ev(owner, KIND_REPO_STATE, "", [["d", "correction"], ["HEAD", "ref: refs/heads/main"], ["refs/heads/main", first.commitID]]))).ok).toBe(true); + c.ws.close(); + const path = gitRepositoryPath(npubEncode(pk(owner)), "correction"); + expect(await (await receiveSettled(path, host, null, first.commitID, "refs/heads/main", first.pack)).text()).toContain("ok refs/heads/main"); + const pr = ev(owner, KIND_GIT_PR, "", [["a", `30617:${pk(owner)}:correction`], ["c", next.commitID]]); + const ref = `refs/nostr/${pr.id}`; + expect(await (await receiveSettled(path, host, null, first.commitID, ref)).text()).toContain(`ok ${ref}`); + const publisher = await WS.connect(host); + expect((await publisher.ok(pr)).ok).toBe(true); + publisher.ws.close(); + const hidden = await SELF.fetch(`http://${host}${path}/info/refs?service=git-upload-pack`); + expect(await hidden.text()).not.toContain(ref); + + const stub = env.RELAY.getByName("grasp-pr-checkpoint"); + await runInDurableObject(stub, async (relay) => { + relay.graspBusy = true; + try { + const wal = await gitRepository(relay, storedRepository(relay, pk(owner), "correction")!); + expect((await wal.checkpoint()).changed).toBe(true); + } finally { relay.graspBusy = false; } + }); + const metadata = await runInDurableObject(stub, async (relay) => { + const bucket = relay.media; + const keys: string[] = []; + const observed = new Proxy(bucket, { get(target, name) { + if (name === "get") return (key: string, options?: R2GetOptions) => { keys.push(key); return target.get(key, options); }; + const value = Reflect.get(target, name, target); + return typeof value === "function" ? value.bind(target) : value; + } }); + Object.defineProperty(relay, "media", { value: observed, configurable: true }); + try { + const response = await relay.fetch(new Request(`http://${host}${path}/info/refs?service=git-upload-pack`)); + expect(response.status).toBe(200); + return { body: await response.text(), keys }; + } finally { Reflect.deleteProperty(relay, "media"); } + }); + expect(metadata.keys).toHaveLength(2); + expect(metadata.keys[0]).toMatch(/\/root.json$/); + expect(metadata.keys[1]).toContain("/manifests/"); + expect(metadata.body).toContain("symref=HEAD:refs/heads/main"); + expect(metadata.body).not.toContain(ref); + const requestId = "hidden-pr-correction"; + expect(await (await receiveSettled(path, host, null, next.commitID, ref, next.pack, requestId)).text()).toContain(`ok ${ref}`); + const sequence = await runInDurableObject(stub, async (relay) => { + relay.graspBusy = true; + try { + const wal = await gitRepository(relay, storedRepository(relay, pk(owner), "correction")!); + for (let i = 0; i < 130; i++) await wal.commit({ id: `later-${i}`, updates: [{ name: `refs/tags/later-${i}`, old: null, new: first.commitID }] }); + const snapshot = await wal.load(); + expect(snapshot.records).toHaveLength(1); + expect(snapshot.records[0].id).not.toBe(requestId); + return snapshot.sequence; + } finally { relay.graspBusy = false; } + }); + const retried = await runInDurableObject(stub, async (relay) => { + const bucket = relay.media; + const counters = { gets: 0, getBytes: 0, puts: 0, heads: 0 }; + const observed = new Proxy(bucket, { get(target, name) { + if (name === "get") return async (key: string, options?: R2GetOptions) => { + counters.gets++; + const object = await target.get(key, options); + counters.getBytes += object?.size ?? 0; + return object; + }; + if (name === "put" || name === "head") return (...args: unknown[]) => { + if (name === "put") counters.puts++; else counters.heads++; + return Reflect.apply(target[name], target, args); + }; + const value = Reflect.get(target, name, target); + return typeof value === "function" ? value.bind(target) : value; + } }); + Object.defineProperty(relay, "media", { value: observed, configurable: true }); + try { + const response = await relay.fetch(new Request(`http://${host}${path}/git-receive-pack`, { + method: "POST", + headers: { "content-type": "application/x-git-receive-pack-request", "X-Git-Request-Id": requestId }, + body: concat(packet(`${"0".repeat(40)} ${next.commitID} ${ref}\0report-status\n`), flush(), next.pack), + })); + return { body: await response.text(), counters }; + } finally { Reflect.deleteProperty(relay, "media"); } + }); + expect(retried.body).toContain(`ok ${ref}`); + expect(retried.counters.puts).toBe(0); + console.info("checkpoint old PR correction HTTP retry", JSON.stringify({ sequence, ...retried.counters })); + await runInDurableObject(stub, async (relay) => { + const wal = await gitRepository(relay, storedRepository(relay, pk(owner), "correction")!); + expect((await wal.load()).sequence).toBe(sequence); + }); + const visible = await SELF.fetch(`http://${host}${path}/info/refs?service=git-upload-pack`); + const body = await visible.text(); + expect(body).toContain(`${next.commitID} ${ref}`); + expect(body).toContain("symref=HEAD:refs/heads/main"); + }); + it("removes the public Git contract when reads become restricted", async () => { const host = "grasp-restricted.bind.ws"; const owner = generateSecretKey(); diff --git a/vendor/README.md b/vendor/README.md index 7449651..3e8a250 100644 --- a/vendor/README.md +++ b/vendor/README.md @@ -1,24 +1,37 @@ # Vendored dependencies -`ntig-0.1.1.tgz` is the Git engine and conditional R2 WAL used by GRASP. +`ntig-0.2.0.tgz` is the Git engine and conditional R2 WAL used by GRASP. The lockfile pins the archive integrity. `ntig-source.tar.gz` preserves the complete source and tests for the same build; neither artifact depends on a sibling checkout or a public package registry publication. | Fact | Value | |---|---| -| Source commit | `e70e3b3adfdf827ed0899acbe4dd556687c3aaef` | +| Source commit | `f6346eb54ce12f54b1c5a18fdd4d3e16220b4feb` | +| Package SHA-256 | `dcfba370518014a0dace1960e13782748965d2479d46d264fdbe0cf0745b153b` | +| Source archive SHA-256 | `ab5552da34bd2dbf3e751c794949d891a5fca8ec1c17ea8d5d845b1636009eff` | | Source branch | `main` in [FelineStateMachine/ntig](https://github.com/FelineStateMachine/ntig) | | License | MIT, included in both archives | | Library build | `npm ci`, then `npm run build:library` in the extracted source | | Package build | `npm pack` in the extracted source | | Backend checks | `npm run check`, `npm run format:check` | +Rebuilding the preserved source with Node 26.8.1 and npm 11.19.0 produces +identical tar entries and metadata. The uncompressed tar SHA-256 is +`2c0acfc94c300c59ba73cac4c83185ea0b1b7f939cecf565cfc2f69eb4952fad`; +gzip bytes can differ across toolchains. The supplied package checksum above +and the lockfile pin the exact installed archive. + The source's `docs/INTEGRATION.md` describes the exported contracts, failure semantics, cost counters and current limits. Backend work stays in that repository; a dependency update replaces both archives and the lockfile integrity after its checks and bindws's integration checks pass. -The backend's 128-transaction limit also stops timed ref deletion. Bindws -hides expired unknown PR refs even when physical cleanup cannot advance. -Retained packs and orphan writes remain charged until relay teardown. +New and existing repositories remain in format 1, whose 128-transaction +limit also stops timed ref deletion. Bindws hides expired unknown PR refs +even when physical cleanup cannot advance. Installing 0.2.0 does not migrate +roots. The source's `docs/CHECKPOINTS.md` describes the explicit one-way +`checkpoint()` upgrade: every reader and writer must be upgraded first, +and 0.1.x cannot read a format-2 root. Bindws exposes no migration endpoint +or scheduled migration. Retained packs, receipts, manifests and orphan +writes remain charged until relay teardown. diff --git a/vendor/ntig-0.1.1.tgz b/vendor/ntig-0.1.1.tgz deleted file mode 100644 index a3f4d7d..0000000 Binary files a/vendor/ntig-0.1.1.tgz and /dev/null differ diff --git a/vendor/ntig-0.2.0.tgz b/vendor/ntig-0.2.0.tgz new file mode 100644 index 0000000..672faa0 Binary files /dev/null and b/vendor/ntig-0.2.0.tgz differ diff --git a/vendor/ntig-source.tar.gz b/vendor/ntig-source.tar.gz index 9193470..7271375 100644 Binary files a/vendor/ntig-source.tar.gz and b/vendor/ntig-source.tar.gz differ