Repository navigation
Record R26 appearing on the FIPS leg, with a candidate fix for that shape - #83
Merged
Merged
Conversation
…hat shape The row described an intermittent Database busy on writes, on loaded runners, in the ordinary gate. This occurrence was none of those three. It was on the FIPS leg, in Trinity.Gateways.ApprovalsTest's setup, and the crash was an ownership error in ChainWriter.terminate/2 calling insert_checkpoint/2 after the sandbox owner had gone rather than a busy database. Re-running the identical commit passed all four checks. That is the part worth recording: the code did not change between the red run and the green one, so the failure is in the environment or the ordering and not in the change under test. A red check that is dismissed without that comparison is a red check being ignored; one that is dismissed with it is a measurement. The occurrence adds two things the row did not have. The mechanism is not only write contention, it also reaches process termination racing sandbox teardown. And it is not confined to the ordinary gate's runners. It also suggests a fix specific to this shape, recorded rather than taken: terminate/2 writes a checkpoint on the way out, and a terminate callback that touches the database after the test owning the connection has finished is a race by construction. It should skip the write when the repo is in sandbox mode with no owner instead of attempting it and crashing. The row above stands as written; this one is appended beneath it. Signed-off-by: Ayla Croft <aylacroft@proton.me>
The pull request carrying the previous commit changed only the risk register and failed the FIPS leg anyway: Trinity.Gateways.RouterTest AC6, Exqlite.Error Database busy on an INSERT into gateway_identities. That is the row's original signature, on the leg the row did not cover, on a change with no behaviour in it. Twice in a row on required checks, on changes that cannot affect them, is not an intermittent annoyance. The likelihood is raised from Med to High with the reason stated: it blocks merges. The impact stays Low, because it remains test-only with no production path, and inflating that would make the register less useful rather than more urgent. The reading this adds: the FIPS leg provokes it more reliably than the ordinary gate, most likely because that leg is slower, about 163 seconds against 110 locally, so every contention window is correspondingly wider. The row did not know that because every previous occurrence was on the ordinary gate. Still recorded rather than fixed. The candidate fixes are in the row above and each is a change to an approved slice's test infrastructure, which deserves its own work and its own evidence rather than being folded into an observation. Signed-off-by: Ayla Croft <aylacroft@proton.me>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
R26 described an intermittent
Database busyon writes, on loaded runners, in the ordinary gate. The occurrence that blocked PR #82 was none of those three:Trinity.Gateways.ApprovalsTest's setupChainWriter.terminate/2callinginsert_checkpoint/2after the sandbox owner had gone — not a busy databaseRe-running the identical commit passed all four checks. That is the part worth recording: the code did not change between the red run and the green one, so the failure is in the environment or the ordering and not in the change under test. A red check dismissed without that comparison is a red check being ignored; one dismissed with it is a measurement.
What this adds
The mechanism is not only write contention — it also reaches process termination racing sandbox teardown. And it is not confined to the ordinary gate's runners.
A candidate fix for this shape, recorded rather than taken
ChainWriter.terminate/2writes a checkpoint on the way out. A terminate callback that touches the database after the test owning the connection has finished is a race by construction. It should skip the write when the repo is in sandbox mode with no owner, instead of attempting it and crashing.That is a change to an approved slice's code and belongs in its own commit with its own evidence, not folded into a record of the observation.
The original row stands as written; this one is appended beneath it, per the convention that corrections append and never rewrite.
mix gateexit 0; 612 passed.