Repository navigation
Add operator-only Convex function to clear app and Better Auth tables - #43
Conversation
There was a problem hiding this comment.
Important
The reset is correct in shape and the test is strong, but it cannot clear a deployment that holds real bookmark data: batchSize bounds each mutation by document count, not bytes, and collections/backups embed whole snapshot trees.
Reviewed changes
convex/maintenance.ts(new) — addsclearAll, aninternalActiongated onconfirm: v.literal('DELETE_ALL_DATA'), that walks every table inconvex/schema.tsthrough a newclearAppBatchinternalMutationand then all ten Better Auth component tables throughcomponents.betterAuth.adapter.deleteMany, returning per-table deletion counts.tests/maintenance.test.ts(new) — oneconvex-testend-to-end test that seeds app rows and 101backups/ 101rateLimitrows, asserts the exact counts, pins the reported key set againstschema.tablesplusbetterAuth.schema.tables, checks every table is empty afterwards, and asserts a rerun reports all zeros.docs/auth-setup.md— new "Clearing a deployment" section with thenpx convex run maintenance:clearAll '{"confirm":"DELETE_ALL_DATA"}'command, deployment-selection flags, the destructive-scope list, and the "stop clients from writing" / "rerun to finish" guidance.convex/_generated/api.d.ts— registers the newmaintenancemodule in the generated API surface, correctly sorted betweenhttpandsync.
⚠️ A batch of 100 snapshot documents can blow Convex's 16 MiB transaction read limit
clearAppBatch deletes up to batchSize documents per mutation, and collections / backups each embed a complete bookmark tree, so the batch is bounded by count while the limit is by bytes. This fails deterministically on the same documents, which means the documented "run the same command again to finish" recovery never converges.
Technical details
# Bound the reset batch by bytes, not document count
## Affected sites
- convex/maintenance.ts:62 — `ctx.db.query(table as TableNames).take(batchSize)` reads up to 100 whole documents in one mutation.
- convex/maintenance.ts:7 — `const batchSize = 100;`
- packages/model/index.ts:65 — `MAX_NODES = 2000`, and `validateTree` allows up to 600 KB of `JSON.stringify(nodes)` per snapshot, so a single `collections`/`backups` document can be ~0.6 MB.
- convex/sync.ts:92, convex/sync.ts:274 — both `backups` inserts store the full `nodes` array, and nothing prunes the table, so the oldest snapshots (the largest) sit in the first batch.
- docs/auth-setup.md:73 — "if it is interrupted, run the same command again to finish" is not true for a limit error; the same first 100 documents fail again every time.
- tests/fixtures/auth.ts:7 — `convexTest(schema, modules)` with no `transactionLimits`, so no test in the repo can currently observe a Convex transaction limit.
## Reproduction
Convex caps a transaction at 16 MiB of data read (docs.convex.dev/production/state/limits);
16,777,216 / 100 documents is ~168 KB average per document, roughly 600 bookmarks.
Under `convexTest({ schema, modules, transactionLimits: true })` with the Better Auth
component registered, inserting 100 `backups` of 700 nodes each (~206 KB per document,
well inside the app's own 2,000-node / 600 KB cap) and then calling
`internal.maintenance.clearAppBatch` with `{ table: 'backups' }`:
clearAppBatch threw: Error: Read too much data in a single function execution
(limit: 16777216 bytes). This is a Convex limit: https://docs.convex.dev/production/state/limits
## Blast radius
`collections`, `devices` and `operations` are cleared before `backups`, so the reset aborts
with the first three tables already gone — a half-cleared deployment, and the operator is
left with the first 100 backup documents undeletable by this tool.
## Required outcome
- `clearAppBatch` must stop taking documents once the transaction is close to Convex's
16 MiB read budget, so it still makes progress on any mix of tiny and 600 KB documents
and still returns a non-zero count while rows remain.
- Add a regression test that observes the limit. It needs
`convexTest({ schema, modules, transactionLimits: true })`; the shared fixture in
`tests/fixtures/auth.ts` builds the instance without limits, so the test either needs
its own instance or the fixture needs an option.
- Correct the recovery sentence in `docs/auth-setup.md:73` so it does not promise that a
rerun fixes every kind of interruption.
## Suggested approach
Track the bytes as you delete and return early, e.g. accumulate
`getConvexSize(doc)` (or read `ctx.meta.getTransactionMetrics().bytesRead.remaining`) and
break out of the loop before exceeding a budget well under 16 MiB — the outer
`do { ... } while (batch > 0)` in `clearAll` then keeps draining the table. A budget of a
few MiB still allows at least one document per call given the 1 MiB per-document cap, so
the loop cannot stall.
## Open questions for the human
- Is this tool expected to work on production data, or only on throwaway dev deployments?
If only dev, a note in `docs/auth-setup.md` about the size limit may be enough instead
of a byte-aware batch.ℹ️ Nitpicks
docs/auth-setup.md:73lists what the reset removes but skipsverification,twoFactorand the four OAuth tables, so the enumeration reads as exhaustive. "accounts, sessions, verification and two-factor records, OAuth rows and rate-limit data" would match what the action actually clears.
Big Pickle (free) | 𝕏
| args: { table: v.string() }, | ||
| handler: async (ctx, { table }) => { | ||
| if (!(table in schema.tables)) throw new Error(`Unknown app table: ${table}`); | ||
| const docs = await ctx.db.query(table as TableNames).take(batchSize); |
There was a problem hiding this comment.
batchSize bounds this mutation by document count, but collections and backups each embed a whole snapshot tree, so the batch is bounded by bytes in practice. packages/model's own validateTree allows 2,000 nodes / 600 KB per snapshot, and Convex caps a transaction at 16 MiB read — about 168 KB per document at 100 docs, so any deployment whose snapshots average ~600 bookmarks makes this call throw Read too much data in a single function execution.
Two consequences beyond the throw: collections/devices/operations are already gone by then, so the reset stops half-finished, and the same first 100 documents fail on every rerun, which makes the documented "run the same command again to finish" recovery in docs/auth-setup.md:73 unreachable. Bound the batch by bytes instead (details in the review body).

Summary
DELETE_ALL_DATAargument and return deletion counts by table.Checks
npm run check(Chromium and Firefox builds; 165 tests passed)The reset is intentionally destructive and is only callable by a deployment operator through
npx convex run.