Skip to content

Slice 005: take the SQLite write lock up front, and close R26's cause - #97

Merged
HackTuah merged 1 commit into
mainfrom
slice/005-sqlite-transaction-mode
Sep 25, 2026
Merged

HackTuah merged 1 commit into
mainfrom
slice/005-sqlite-transaction-mode

Conversation

@HackTuah

Copy link
Copy Markdown
Member

The intermittent Exqlite.Error: Database busy that failed the gate in a different test each time across five pull requests in two days is gone, for the reason it happened rather than around it.

The mechanism

A DEFERRED transaction — SQLite's default, and Ecto's — takes no lock at BEGIN and starts as a reader. At its first write it tries to upgrade, and if another connection has written since, SQLite returns SQLITE_BUSY immediately and without invoking the busy handler, because blocking there risks deadlock.

No busy_timeout reaches that path. This project raised it from 5s to 30s and the failures continued — the observation that should have pointed at the cause a day earlier than it did.

IMMEDIATE asks for the write lock at BEGIN, which is where the handler is allowed to wait.

R26 had the diagnosis right and the fixes wrong

The row diagnosed this correctly on 2026-09-23 and then recorded three candidates: drain the stress test's sessions, give it its own CI step, remove the second connection's need. All three reduce how often two connections contend and leave the upgrade path exactly where it is. They are not taken, and the register now says why — that is the more interesting failure here.

Reproduced deterministically, for the first time

R26 has read "not once reproduced locally across repeated runs" since it was opened. It now reproduces in a tenth of a second:

  • DEFERRED: asserts the busy error and elapsed < 1_000 against busy_timeout=5000. The timing assertion is the point — it measures that the handler was never consulted.
  • IMMEDIATE: the same sequence waits and succeeds once the other connection commits.

Against its own database file, because Ecto.Adapters.SQL.Sandbox wraps a test in one connection's transaction and cannot express two contending writers. What's under test is SQLite's locking, not Ecto's.

An intermittent fault that merely stops appearing is a fault you got lucky with.

Scope

Configured for every environment, not just the suite: the hazard is SQLite's, and a desktop Trinity with the scheduler and a session writing at once meets the same upgrade. Guarded on the adapter so it cannot reach the Postgres build, which has neither the option nor the hazard — and the test asserts that absence rather than skipping silently.

The mode is read back from repo.config() on the running repo, not parsed from a config file, so an environment override that put it back to :deferred would fail.

Slice 070's fifty-writer stress test: three consecutive runs, 2 passed in 0.8s each. If IMMEDIATE had turned contention into queuing severe enough to change what it measures, that number would have moved.

R26 is not closed

Its lift condition is ten consecutive green runs, which this can begin earning and cannot assert today. Likelihood drops High → Low pending those runs.

mix gate
Result: 798 passed (14 properties, 784 tests), 18 excluded

🤖 Generated with Claude Code

An intermittent Database busy failure had failed the gate in a different test each
time across five pull requests in two days. It is gone, for the reason it happened.

A DEFERRED transaction takes no lock at BEGIN and starts as a reader. At its first
write it tries to upgrade, and if another connection has written since, SQLite
returns SQLITE_BUSY immediately and without invoking the busy handler, because
blocking there risks deadlock. No busy_timeout reaches that path, which is why
raising it from five seconds to thirty changed nothing: the observation that should
have pointed at the mechanism a day earlier than it did.

Both SQLite repos now set default_transaction_mode: :immediate, which takes the
write lock at BEGIN where the handler is allowed to wait. It is configured for
every environment rather than only the suite, because the hazard is SQLite's and
not the test harness's, and guarded on the adapter so it cannot reach the Postgres
build, which has neither the option nor the hazard.

The reproduction is the part worth keeping. R26 has read not once reproduced
locally since it was opened; it now reproduces in a tenth of a second by driving
two connections through both sequences, asserting the old one is refused in under a
second despite a five-second timeout and the new one waits and succeeds. An
intermittent fault that merely stops appearing is a fault you got lucky with.

R26 recorded three candidate fixes when it was first diagnosed, and all three
reduce how often two connections contend rather than changing what happens when
they do. The diagnosis was right and the proposals did not follow from it. They are
not taken, and the register now says so.

The row is not closed. Its lift condition is ten consecutive green runs, which this
can begin earning and cannot assert today.

mix gate: 798 passed (14 properties, 784 tests), 18 excluded.

Signed-off-by: Ayla Croft <aylacroft@proton.me>
@HackTuah
HackTuah merged commit 714784f into main Sep 25, 2026
12 checks passed
@HackTuah
HackTuah deleted the slice/005-sqlite-transaction-mode branch September 25, 2026 00:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant