Skip to content

[fix] sqlite: connection-pool semantics + cancellable operations (PKG-75/76) - #18

Merged
vt128 merged 7 commits into
masterfrom
fix/pool-semantics-and-context
Jul 14, 2026
Merged

vt128 merged 7 commits into
masterfrom
fix/pool-semantics-and-context

Conversation

@vt128

@vt128 vt128 commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

What

Fixes two connection/correctness findings and adopts the shared base v0.1.2 layer.

1. Connection-pool semantics (high)

The five PRAGMAs (busy_timeout, journal_mode, synchronous, cache_size, foreign_keys) were applied once with db.Exec on the pooled *sql.DB, so they bound only a single borrowed connection. Every later connection fell back to SQLite's defaults — notably foreign_keys OFF, so constraint-violating writes silently succeeded — and an in-memory database (which lives inside one connection) lost its schema and data when the pool opened a second or recycled the first (no such table). Additionally, the connection timeout was misused as SetConnMaxLifetime, recycling live connections after 30s.

  • PRAGMAs now travel in the DSN (modernc's _pragma=), applying to every pooled connection.
  • An in-memory / private-temp DB is pinned to a single, never-recycled connection (SetMaxOpenConns(1) + ConnMaxLifetime(0) + ConnMaxIdleTime(0)).
  • journal_mode/synchronous are validated as bare identifiers before entering the DSN (no injection).
  • The SetConnMaxLifetime misuse is removed.

2. Cancellable / deadline-bounded operations (medium)

The remote Ping and the whole query surface took no context, so a remote libSQL query to an unreachable/slow host blocked the host goroutine until the TCP timeout and ignored script cancellation. Every operation now derives a context from the calling thread via base/util.OpContext plus the timeout config (repurposed as a per-operation deadline): PingContext/QueryContext/ExecContext/QueryRowContext/BeginTx/PrepareContext throughout. A transaction's context is cancellation-only and lives until commit/rollback (a per-statement deadline would abort a legitimately long transaction).

3. Pin + adoption

Bumps base v0.1.2 / starlet v0.2.3; adopts base.NewNamedConfigOption.

Behavior changes

  • timeout now bounds each operation (per-query deadline + thread cancellation), not the connection's max lifetime.
  • In-memory databases are served from one connection (data persists across queries, as documented).
  • foreign_keys is now genuinely enforced on every connection (a previously-silent constraint violation now correctly errors).

Tests

New regression tests: in-memory data persists across many queries; foreign_keys reads 1 on every pooled connection under concurrency; a query on a cancelled thread aborts with context canceled. Full suite green with -race, on go1.20 in Docker, and against the real Turso remote (TestConnectRemote).

vt128 added 5 commits July 14, 2026 13:30
Pin the hardened base (v0.1.2, adds base/util) and starlet v0.2.3 (carrying
starlight v0.2.1), converging the ecosystem on one starlet version.
STAR-75 — the PRAGMAs (busy_timeout/journal_mode/synchronous/cache_size/
foreign_keys) were applied once with db.Exec on the pool, so they bound only a
single borrowed connection; every later connection fell back to SQLite's
defaults (notably foreign_keys OFF, silently accepting constraint-violating
writes), and an in-memory database — being per-connection — lost its schema and
data when the pool opened a second or recycled the first. They are now carried
in the DSN (modernc _pragma=), applying to every pooled connection, and an
in-memory / private-temp database is pinned to one never-recycled connection.
journal_mode/synchronous are validated as bare identifiers before going into the
DSN. The misuse of the connection timeout as SetConnMaxLifetime (which recycled
live connections after 30s) is removed.

STAR-76 — the remote Ping and the whole query surface took no context, so a
remote libSQL query against an unreachable/slow host blocked the host goroutine
until the TCP timeout and ignored script cancellation. Every operation now
derives a context from the calling thread (via base/util OpContext) plus the
 config (repurposed as a per-operation deadline): Ping/Query/Exec/
QueryRow/Begin/Prepare all use their *Context variants. A transaction's context
is cancellation-only and lives until commit/rollback (a per-statement deadline
would abort a long transaction).

Also adopts base.NewNamedConfigOption for the module's config options.
… tx leak

- openDatabase: in-memory/private-temp DBs (which are pinned to one connection)
  now set their PRAGMAs with db.Exec on that single connection, so an empty
  connStr is no longer misparsed by modernc into a persistent disk filename;
  file DBs still carry PRAGMAs in the DSN for every pooled connection.
- isInMemoryDSN: parse the DSN properly instead of a substring/first-query scan.
  A bare path (even 'foo.db?mode=memory' or 'file::memory:evil') is a disk file
  and must not pass the file-access gate; valid in-memory forms (':memory:?…',
  'file:', 'file:?…') are now recognized. Locked by TestInMemoryDSNClassification.
- processQueryOneRow: check rows.Err() on a false first Next(), so a
  cancellation/deadline is not reported as a successful no-row result.
- Transactions get a finalizer that cancels the lifetime context if the script
  abandons them (the host Machine may run under an uncancelled context), releasing
  the connection they pin.
- Docs/metadata: describe timeout as the per-operation deadline (not a connection
  timeout); note the libSQL driver does not cancel COMMIT/ROLLBACK or a ws/wss
  dial on Go 1.20.
- Stronger pool tests (hold distinct connections; assert MaxOpenConnections==1).
The DSN classifier must never trust a mode=memory query parameter to reclassify
a disk path: Go's url parsing and SQLite's DSN parsing disagree on malformed or
edge queries (%00 truncation, duplicate keys), and any such disagreement on a
disk path is a file-access-gate bypass. It now classifies purely on the path —
empty or ":memory:" is in-memory, everything else (including a non-URI
":memory:?cache=shared", which SQLite's ATTACH treats as a disk filename) is a
file. Locked by the expanded TestInMemoryDSNClassification (incl. %00 attacks).
begin() acquired the connection with the transaction's cancellation-only context
(no deadline), so on a single-connection in-memory database a second begin()
whose sole connection was held by an open transaction blocked the host
indefinitely, ignoring the configured timeout. begin() now acquires a dedicated
*sql.Conn under the per-operation deadline and starts the transaction on it with
the cancellation-only lifetime context; the connection is released on
commit/rollback (and by the finalizer for an abandoned transaction). Acquisition
is bounded by the timeout again while a long transaction is still not aborted by
a per-statement deadline.
@codacy-production

codacy-production Bot commented Jul 14, 2026 •

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 39 complexity · -8 duplication

Metric Results
Complexity 39
Duplication -8

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

vt128 added 2 commits July 14, 2026 15:29
…sis gate

Extract helpers so the methods the context-threading touched stay within the
cyclomatic-complexity / length limits (a Codacy gate re-scans changed methods):
openDatabase splits out configureMemoryConnection; pragmaIdentifier uses
isASCIIAlphanumeric; update/upsert/delete/truncate_table share execAffecting;
createTable and insertMany split into buildCreateTableStatement/execCreateTable
and extractInsertColumns/buildInsertManyQuery/execInsertManyRows/
buildInsertRowParams. Pure refactor — same SQL, error messages, and behavior;
full suite passes with -race, Docker go1.20, and the real Turso remote.
Remove the leftover starlet v0.2.2 / starlight v0.2.0 / base v0.1.1 checksums so
'go mod tidy' is a no-op on both the go1.20 floor and go1.25 (the latter's tidy
prunes them, which failed the CI tidy check).
@vt128
vt128 merged commit 17ac86f into master Jul 14, 2026
10 checks passed
@vt128
vt128 deleted the fix/pool-semantics-and-context branch July 14, 2026 07:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant