Skip to content

fix(storage): reopen named graphs after a crash around a pinned checkpoint - #1379

Draft
zahariash wants to merge 2 commits into
LadybugDB:mainfrom
zahariash:fix/pinned-checkpoint-named-graphs
Draft

zahariash wants to merge 2 commits into
LadybugDB:mainfrom
zahariash:fix/pinned-checkpoint-named-graphs

Conversation

@zahariash

Copy link
Copy Markdown
Contributor

This work was produced with the help of language models.

Fixes #1304. Depends on #1378, whose commit this branch includes.

After a CHECKPOINT taken while another connection had a read-only transaction open, a crash before the next commit left a database with a named graph (CREATE GRAPH g) that would not reopen: Cannot recover committed checkpoint: graph shadow file db.g.shadow is missing.. A graph shadow left by an interrupted cleanup failed the reopen too. A graph WAL written by a session that opened the graph's file on its own survived the checkpoint that applied it, and after a later commit and a crash it replayed again: Found duplicated primary key value 2, which violates the uniqueness constraint of the primary key column..

Fix. A named graph's commits go to the main WAL, and #1378's marker shows that the trailing checkpoint was applied, graph shadows included. Under the marker, graphs now load as after a completed checkpoint, without their shadows. A pinned checkpoint now also retires each graph's own WAL, which holds only the records of such a standalone session and has just been applied; only the main WAL stays pinned. A graph WAL found at recovery is therefore newer than the checkpoint and replays. This narrows ef58de2's pin to the main WAL, the replication source it was added for; graph WALs can stay pinned if replication needs them, at the cost of telling an applied graph WAL from a newer one at recovery. docs/checkpoint_recovery.md describes both rules.

Tests. NamedGraphReopensAfterPinnedCheckpoint, NamedGraphLeftoverShadowIsDiscarded, NamedGraphWALIsRetiredByPinnedCheckpoint and GraphWALWrittenAfterPinnedCheckpointReplays (PinnedCheckpointCrashTest, test/transaction/checkpoint_test.cpp) fail on #1378 with the errors above. The last one checks that a standalone session's write after the crash is kept. Full ctest: 3198/3198; storage-compat suite, every release: same results as main.

zahariash and others added 2 commits October 24, 2026 16:39
A checkpoint taken while a read-only transaction is open keeps its CHECKPOINT
record in the active WAL. Recovery treated a shadow file left next to such a
WAL as a checkpoint in progress: it replayed only the records before the first
CHECKPOINT (applying them twice and dropping every later commit), replayed the
next checkpoint's unfinished shadow, or required partition shadows that
cleanup had already removed.

- A whole transaction after the last CHECKPOINT record proves that checkpoint
  completed. Recovery then skips through the record and removes the leftover
  shadow, in the active and the frozen WAL. Other records after a CHECKPOINT
  keep the prefix replay.
- The pinned-checkpoint marker now means that every CHECKPOINT record in the
  active WAL is applied: WAL::logAndFlushCheckpoint removes it before logging
  a record, and pinned cleanup writes it (content 2) before removing shadows.
  With it, a WAL ending in a CHECKPOINT discards leftover main and partition
  shadows instead of replaying or requiring them.
- A WAL with nothing committed after its last CHECKPOINT is recovered as one
  ending in it, so it is retired instead of kept.
- A marker from an older build (content 1) only proves that a missing shadow
  was applied.

Named graphs are not covered: pinned checkpoints keep their own WALs too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…point

After a checkpoint taken with a read-only transaction open, a crash before
the next commit left a database with a named graph that would not reopen:
recovery required the graph's shadow file, which cleanup had already removed.
A graph WAL left by a session that opened the graph's file on its own also
survived the pinned checkpoint that applied it and replayed again later.

- Under the pinned-checkpoint marker, graphs load as after a completed
  checkpoint: their shadows are neither required nor replayed.
- A pinned checkpoint retires graph WALs as well; only the main WAL, where a
  named graph's commits go, stays pinned. A graph WAL found at recovery is
  newer than the checkpoint and replays.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: a crash after a CHECKPOINT with a reader open can lose commits or block reopening

1 participant