From 6046f7575b083915e90e58641fa252726aa95fb2 Mon Sep 17 00:00:00 2001 From: Ayla Croft Date: Wed, 23 Sep 2026 19:33:14 -0400 Subject: [PATCH 1/2] docs: record R26 appearing on the FIPS leg, and a candidate fix for that shape The row described an intermittent Database busy on writes, on loaded runners, in the ordinary gate. This occurrence was none of those three. It was on the FIPS leg, in Trinity.Gateways.ApprovalsTest's setup, and the crash was an ownership error in ChainWriter.terminate/2 calling insert_checkpoint/2 after the sandbox owner had gone rather than a busy database. Re-running the identical commit passed all four checks. That is the part worth recording: the code did not change between the red run and the green one, so the failure is in the environment or the ordering and not in the change under test. A red check that is dismissed without that comparison is a red check being ignored; one that is dismissed with it is a measurement. The occurrence adds two things the row did not have. The mechanism is not only write contention, it also reaches process termination racing sandbox teardown. And it is not confined to the ordinary gate's runners. It also suggests a fix specific to this shape, recorded rather than taken: terminate/2 writes a checkpoint on the way out, and a terminate callback that touches the database after the test owning the connection has finished is a race by construction. It should skip the write when the repo is in sandbox mode with no owner instead of attempting it and crashing. The row above stands as written; this one is appended beneath it. Signed-off-by: Ayla Croft --- docs/06-risk-register.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/06-risk-register.md b/docs/06-risk-register.md index ac6a5ce..d582029 100644 --- a/docs/06-risk-register.md +++ b/docs/06-risk-register.md @@ -27,3 +27,4 @@ | R23 | EMA / ID-JAG is beta everywhere (vendors label it so); IETF draft still moving | Low (re-scoped 2026-09-22) | Med | Spec revision breaks the exchange at the external authorization server, which is where it happens: Trinity redeems no assertion since the owner's decision of 2026-09-22 (slice 062 NOTES) | The resource server validates whatever audience-bound token that server issues, so a draft revision is that server's upgrade, not Trinity's; the row stays until the deferral lifts | 062 | | R25 | `glib` 0.18.5 carries GHSA-wrw7-89jp-8q8g (medium, unsoundness in `Iterator` and `DoubleEndedIterator` impls); first patched in 0.20.0 | Low | Med | The advisory is open against `src-tauri/Cargo.lock`; measured 2026-09-23 | It is transitive through the GTK 0.18 stack (`gtk` 0.18.2, `webkit2gtk` 2.0.2) that Tauri 2.11.5 requires on Linux, so it is fixed by a Tauri release built on gtk-rs 0.20 and not by a direct bump. **Scope:** the desktop shell only. The `headless` release is `steps: [:assemble]` with no Burrito and no `src-tauri`, so a server deployment does not ship this dependency. Owner: the maintainer. Lift condition: a Tauri release whose Linux stack resolves `glib` >= 0.20, adopted by the desktop shell work | 100 | | R26 | An intermittent `Exqlite.Error: Database busy` fails the test suite on loaded CI runners, in a different test each time, always on a write to the main repository | Med | Low (test-only; no production path) | Failed the gate on pull requests 67, 70 and again on the commit that attempted to fix it; the same commit passed on its other run, so it is seed- and load-dependent. Not once reproduced locally across repeated runs | Ruled out: `busy_timeout`, raised from the shipped 5 s to 30 s in the suite, did not stop it. SQLite returns BUSY immediately, without the busy handler, when a read transaction upgrades to a write while another connection holds the write lock; no timeout affects that path. The suite runs two connections (one for the sandbox owner, one for Oban's boot check) and deliberately queues fifty concurrent writers at slice 070 AC7, which is the load that provokes it. Candidate fixes, not yet taken: drain the stress test's sessions before the suite continues; give that test its own tag and CI step; or remove the second connection's need. Owner: the maintainer. Lift condition: ten consecutive green CI runs of the full suite with the stress test in the default run | 070 | +| R26 | Occurrence recorded 2026-09-23: the same class of failure appeared on the **FIPS leg** rather than the ordinary gate, which the row above does not cover. `Trinity.Gateways.ApprovalsTest` AC4 failed in its setup, with the crash in `Trinity.Receipts.ChainWriter.terminate/2` calling `insert_checkpoint/2` after the sandbox owner had gone: an ownership error at termination rather than a `Database busy` on a write. **Re-running the identical commit passed all four checks**, which is the evidence that matters: the code did not change between the red run and the green one, so the failure is in the environment or the ordering and not in the change under test. What this adds to the row above: the mechanism is not only write contention, it also reaches process termination racing sandbox teardown, and it is not confined to the ordinary gate's runners. A candidate fix specific to this shape: `ChainWriter.terminate/2` writes a checkpoint on the way out, and a terminate callback that touches the database after the test that owns the connection has finished is a race by construction; it should skip the write when the repo is in sandbox mode and has no owner, rather than attempting it and crashing. The row above stands as written | Med | Low (test-only) | The lift condition in the row above, still unmet | docs/06-risk-register.md R26 | 070 | From c3ac86f108d84784ddc7f0f6ca9b371cf861787d Mon Sep 17 00:00:00 2001 From: Ayla Croft Date: Wed, 23 Sep 2026 19:41:02 -0400 Subject: [PATCH 2/2] docs: R26 fired again the same day, on a change that could not cause it The pull request carrying the previous commit changed only the risk register and failed the FIPS leg anyway: Trinity.Gateways.RouterTest AC6, Exqlite.Error Database busy on an INSERT into gateway_identities. That is the row's original signature, on the leg the row did not cover, on a change with no behaviour in it. Twice in a row on required checks, on changes that cannot affect them, is not an intermittent annoyance. The likelihood is raised from Med to High with the reason stated: it blocks merges. The impact stays Low, because it remains test-only with no production path, and inflating that would make the register less useful rather than more urgent. The reading this adds: the FIPS leg provokes it more reliably than the ordinary gate, most likely because that leg is slower, about 163 seconds against 110 locally, so every contention window is correspondingly wider. The row did not know that because every previous occurrence was on the ordinary gate. Still recorded rather than fixed. The candidate fixes are in the row above and each is a change to an approved slice's test infrastructure, which deserves its own work and its own evidence rather than being folded into an observation. Signed-off-by: Ayla Croft --- docs/06-risk-register.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/06-risk-register.md b/docs/06-risk-register.md index d582029..6dda52a 100644 --- a/docs/06-risk-register.md +++ b/docs/06-risk-register.md @@ -27,4 +27,4 @@ | R23 | EMA / ID-JAG is beta everywhere (vendors label it so); IETF draft still moving | Low (re-scoped 2026-09-22) | Med | Spec revision breaks the exchange at the external authorization server, which is where it happens: Trinity redeems no assertion since the owner's decision of 2026-09-22 (slice 062 NOTES) | The resource server validates whatever audience-bound token that server issues, so a draft revision is that server's upgrade, not Trinity's; the row stays until the deferral lifts | 062 | | R25 | `glib` 0.18.5 carries GHSA-wrw7-89jp-8q8g (medium, unsoundness in `Iterator` and `DoubleEndedIterator` impls); first patched in 0.20.0 | Low | Med | The advisory is open against `src-tauri/Cargo.lock`; measured 2026-09-23 | It is transitive through the GTK 0.18 stack (`gtk` 0.18.2, `webkit2gtk` 2.0.2) that Tauri 2.11.5 requires on Linux, so it is fixed by a Tauri release built on gtk-rs 0.20 and not by a direct bump. **Scope:** the desktop shell only. The `headless` release is `steps: [:assemble]` with no Burrito and no `src-tauri`, so a server deployment does not ship this dependency. Owner: the maintainer. Lift condition: a Tauri release whose Linux stack resolves `glib` >= 0.20, adopted by the desktop shell work | 100 | | R26 | An intermittent `Exqlite.Error: Database busy` fails the test suite on loaded CI runners, in a different test each time, always on a write to the main repository | Med | Low (test-only; no production path) | Failed the gate on pull requests 67, 70 and again on the commit that attempted to fix it; the same commit passed on its other run, so it is seed- and load-dependent. Not once reproduced locally across repeated runs | Ruled out: `busy_timeout`, raised from the shipped 5 s to 30 s in the suite, did not stop it. SQLite returns BUSY immediately, without the busy handler, when a read transaction upgrades to a write while another connection holds the write lock; no timeout affects that path. The suite runs two connections (one for the sandbox owner, one for Oban's boot check) and deliberately queues fifty concurrent writers at slice 070 AC7, which is the load that provokes it. Candidate fixes, not yet taken: drain the stress test's sessions before the suite continues; give that test its own tag and CI step; or remove the second connection's need. Owner: the maintainer. Lift condition: ten consecutive green CI runs of the full suite with the stress test in the default run | 070 | -| R26 | Occurrence recorded 2026-09-23: the same class of failure appeared on the **FIPS leg** rather than the ordinary gate, which the row above does not cover. `Trinity.Gateways.ApprovalsTest` AC4 failed in its setup, with the crash in `Trinity.Receipts.ChainWriter.terminate/2` calling `insert_checkpoint/2` after the sandbox owner had gone: an ownership error at termination rather than a `Database busy` on a write. **Re-running the identical commit passed all four checks**, which is the evidence that matters: the code did not change between the red run and the green one, so the failure is in the environment or the ordering and not in the change under test. What this adds to the row above: the mechanism is not only write contention, it also reaches process termination racing sandbox teardown, and it is not confined to the ordinary gate's runners. A candidate fix specific to this shape: `ChainWriter.terminate/2` writes a checkpoint on the way out, and a terminate callback that touches the database after the test that owns the connection has finished is a race by construction; it should skip the write when the repo is in sandbox mode and has no owner, rather than attempting it and crashing. The row above stands as written | Med | Low (test-only) | The lift condition in the row above, still unmet | docs/06-risk-register.md R26 | 070 | +| R26 | Occurrence recorded 2026-09-23: the same class of failure appeared on the **FIPS leg** rather than the ordinary gate, which the row above does not cover. `Trinity.Gateways.ApprovalsTest` AC4 failed in its setup, with the crash in `Trinity.Receipts.ChainWriter.terminate/2` calling `insert_checkpoint/2` after the sandbox owner had gone: an ownership error at termination rather than a `Database busy` on a write. **Re-running the identical commit passed all four checks**, which is the evidence that matters: the code did not change between the red run and the green one, so the failure is in the environment or the ordering and not in the change under test. What this adds to the row above: the mechanism is not only write contention, it also reaches process termination racing sandbox teardown, and it is not confined to the ordinary gate's runners. A candidate fix specific to this shape: `ChainWriter.terminate/2` writes a checkpoint on the way out, and a terminate callback that touches the database after the test that owns the connection has finished is a race by construction; it should skip the write when the repo is in sandbox mode and has no owner, rather than attempting it and crashing. **Second occurrence the same day, and it changes the severity.** The very next pull request, which changed *only this file*, failed the FIPS leg again: `Trinity.Gateways.RouterTest` AC6, `Exqlite.Error: Database busy` on an `INSERT INTO gateway_identities` - the row's original signature, on the leg the row did not cover. A change that cannot affect behaviour failing a required check twice in a row means this is no longer an intermittent annoyance: **it blocks work**, and the honest reading is that the FIPS leg provokes it more reliably than the ordinary gate does, most likely because that leg is slower (163 s against about 110 s locally) and every contention window is correspondingly wider. The row above stands as written | **High** (was Med: it now blocks merges) | Low (test-only; no production path) | The lift condition in the row above, still unmet; two failures on the FIPS leg on 2026-09-23, one of them on a documentation-only change | docs/06-risk-register.md R26 | 070 |