Repository navigation
[fix] sqlite: connection-pool semantics + cancellable operations (PKG-75/76) - #18
Merged
Merged
Conversation
Pin the hardened base (v0.1.2, adds base/util) and starlet v0.2.3 (carrying starlight v0.2.1), converging the ecosystem on one starlet version.
STAR-75 — the PRAGMAs (busy_timeout/journal_mode/synchronous/cache_size/ foreign_keys) were applied once with db.Exec on the pool, so they bound only a single borrowed connection; every later connection fell back to SQLite's defaults (notably foreign_keys OFF, silently accepting constraint-violating writes), and an in-memory database — being per-connection — lost its schema and data when the pool opened a second or recycled the first. They are now carried in the DSN (modernc _pragma=), applying to every pooled connection, and an in-memory / private-temp database is pinned to one never-recycled connection. journal_mode/synchronous are validated as bare identifiers before going into the DSN. The misuse of the connection timeout as SetConnMaxLifetime (which recycled live connections after 30s) is removed. STAR-76 — the remote Ping and the whole query surface took no context, so a remote libSQL query against an unreachable/slow host blocked the host goroutine until the TCP timeout and ignored script cancellation. Every operation now derives a context from the calling thread (via base/util OpContext) plus the config (repurposed as a per-operation deadline): Ping/Query/Exec/ QueryRow/Begin/Prepare all use their *Context variants. A transaction's context is cancellation-only and lives until commit/rollback (a per-statement deadline would abort a long transaction). Also adopts base.NewNamedConfigOption for the module's config options.
… tx leak
- openDatabase: in-memory/private-temp DBs (which are pinned to one connection)
now set their PRAGMAs with db.Exec on that single connection, so an empty
connStr is no longer misparsed by modernc into a persistent disk filename;
file DBs still carry PRAGMAs in the DSN for every pooled connection.
- isInMemoryDSN: parse the DSN properly instead of a substring/first-query scan.
A bare path (even 'foo.db?mode=memory' or 'file::memory:evil') is a disk file
and must not pass the file-access gate; valid in-memory forms (':memory:?…',
'file:', 'file:?…') are now recognized. Locked by TestInMemoryDSNClassification.
- processQueryOneRow: check rows.Err() on a false first Next(), so a
cancellation/deadline is not reported as a successful no-row result.
- Transactions get a finalizer that cancels the lifetime context if the script
abandons them (the host Machine may run under an uncancelled context), releasing
the connection they pin.
- Docs/metadata: describe timeout as the per-operation deadline (not a connection
timeout); note the libSQL driver does not cancel COMMIT/ROLLBACK or a ws/wss
dial on Go 1.20.
- Stronger pool tests (hold distinct connections; assert MaxOpenConnections==1).
The DSN classifier must never trust a mode=memory query parameter to reclassify a disk path: Go's url parsing and SQLite's DSN parsing disagree on malformed or edge queries (%00 truncation, duplicate keys), and any such disagreement on a disk path is a file-access-gate bypass. It now classifies purely on the path — empty or ":memory:" is in-memory, everything else (including a non-URI ":memory:?cache=shared", which SQLite's ATTACH treats as a disk filename) is a file. Locked by the expanded TestInMemoryDSNClassification (incl. %00 attacks).
begin() acquired the connection with the transaction's cancellation-only context (no deadline), so on a single-connection in-memory database a second begin() whose sole connection was held by an open transaction blocked the host indefinitely, ignoring the configured timeout. begin() now acquires a dedicated *sql.Conn under the per-operation deadline and starts the transaction on it with the cancellation-only lifetime context; the connection is released on commit/rollback (and by the finalizer for an abandoned transaction). Acquisition is bounded by the timeout again while a long transaction is still not aborted by a per-statement deadline.
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Complexity | 39 |
| Duplication | -8 |
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
…sis gate Extract helpers so the methods the context-threading touched stay within the cyclomatic-complexity / length limits (a Codacy gate re-scans changed methods): openDatabase splits out configureMemoryConnection; pragmaIdentifier uses isASCIIAlphanumeric; update/upsert/delete/truncate_table share execAffecting; createTable and insertMany split into buildCreateTableStatement/execCreateTable and extractInsertColumns/buildInsertManyQuery/execInsertManyRows/ buildInsertRowParams. Pure refactor — same SQL, error messages, and behavior; full suite passes with -race, Docker go1.20, and the real Turso remote.
Remove the leftover starlet v0.2.2 / starlight v0.2.0 / base v0.1.1 checksums so 'go mod tidy' is a no-op on both the go1.20 floor and go1.25 (the latter's tidy prunes them, which failed the CI tidy check).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Fixes two connection/correctness findings and adopts the shared base v0.1.2 layer.
1. Connection-pool semantics (high)
The five PRAGMAs (
busy_timeout,journal_mode,synchronous,cache_size,foreign_keys) were applied once withdb.Execon the pooled*sql.DB, so they bound only a single borrowed connection. Every later connection fell back to SQLite's defaults — notablyforeign_keysOFF, so constraint-violating writes silently succeeded — and an in-memory database (which lives inside one connection) lost its schema and data when the pool opened a second or recycled the first (no such table). Additionally, the connectiontimeoutwas misused asSetConnMaxLifetime, recycling live connections after 30s._pragma=), applying to every pooled connection.SetMaxOpenConns(1)+ConnMaxLifetime(0)+ConnMaxIdleTime(0)).journal_mode/synchronousare validated as bare identifiers before entering the DSN (no injection).SetConnMaxLifetimemisuse is removed.2. Cancellable / deadline-bounded operations (medium)
The remote
Pingand the whole query surface took nocontext, so a remote libSQL query to an unreachable/slow host blocked the host goroutine until the TCP timeout and ignored script cancellation. Every operation now derives a context from the calling thread viabase/util.OpContextplus thetimeoutconfig (repurposed as a per-operation deadline):PingContext/QueryContext/ExecContext/QueryRowContext/BeginTx/PrepareContextthroughout. A transaction's context is cancellation-only and lives until commit/rollback (a per-statement deadline would abort a legitimately long transaction).3. Pin + adoption
Bumps
base v0.1.2/starlet v0.2.3; adoptsbase.NewNamedConfigOption.Behavior changes
timeoutnow bounds each operation (per-query deadline + thread cancellation), not the connection's max lifetime.foreign_keysis now genuinely enforced on every connection (a previously-silent constraint violation now correctly errors).Tests
New regression tests: in-memory data persists across many queries;
foreign_keysreads 1 on every pooled connection under concurrency; a query on a cancelled thread aborts withcontext canceled. Full suite green with-race, on go1.20 in Docker, and against the real Turso remote (TestConnectRemote).