From 02b3f09df87e2a9f00d0f7d8957f2ba892f3f15a Mon Sep 17 00:00:00 2001 From: Kevin Tang <73975146+vt128@users.noreply.github.com> Date: Thu, 24 Sep 2026 00:36:04 +0800 Subject: [PATCH 1/2] [fix] enforce local SQLite host policy (STAR-102) --- CLAUDE.md | 7 +- README.md | 33 ++++- docs/API.md | 51 +++++-- example_test.go | 383 +++++++++++++++++++++++++++++++++++++++++++++++- function.go | 13 ++ sqlite.go | 66 +++++++-- 6 files changed, 513 insertions(+), 40 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index c5a643d..8cd931f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -37,7 +37,7 @@ Remote tests (`TestConnectRemote`) self-skip unless `SQLITE_REMOTE_URL` + `SQLIT The module is a **two-driver, one-surface bridge**: any connection — local or remote — becomes the same Starlark object with the same ~24 methods, because both drivers yield a `*sql.DB`. -- **`sqlite.go`** — the module entry. `Module` holds a `base.ConfigurableModule`; `NewModule()` / `NewModuleWithFileAccess(allowed)` construct it. `LoadModule()` exposes three builtins: **`connect`** (local), **`connect_remote`** (libSQL), **`register_function`** (custom scalar funcs). Config keys + defaults + the `restrictFileAccess` flag + `isInMemoryDSN` live here. +- **`sqlite.go`** — the module entry. `Module` holds a `base.ConfigurableModule`; `NewModule()` / `NewModuleWithFileAccess(allowed)` / `NewModuleWithHostPolicy(policy)` construct it. `LoadModule()` exposes three builtins: **`connect`** (local), **`connect_remote`** (libSQL, with `file:` routed to the local path), **`register_function`** (custom scalar funcs, rejected when disabled by the host). Config keys, policy flags and `isInMemoryDSN` live here. - **`database.go`** — `newDatabaseInstance(db, maxRows, restrictFileAccess)` wraps a `*sql.DB` into the Starlark object and registers every method (`query`, `query_one`, `execute`, `select`, `insert`, `insert_many`, `update`, `delete`, `upsert`, `count`, `batch`, `begin`, `prepare`, `prepare_query`, `attach`/`detach`, `create_table`/`drop_table`/`truncate_table`, `tables`/`table_exists`/`table_info`/`indices`, `close`). - **`operations.go`** — the largest file: the actual SQL behind those methods (statement building, arg binding, result shaping). - **`transaction.go`** — `begin` returns a transaction object mirroring the connection's methods plus `commit`/`rollback`. @@ -51,10 +51,11 @@ Tier-2 hardening landed in v0.1.0; the iron rule is **opt-in / default-off so ol 1. **No host panics from script input.** `register_function` wraps the user callback in a `recover()` (`function.go`): a panicking custom function becomes a script-level error, never a host crash. Don't remove the deferred recover. 2. **Bounded results.** Every query helper routes materialization through `processQueryRows(rows, maxRows)` (`utils.go`). `max_rows` (config / `SQLITE_MAX_ROWS`, default `0` = unlimited) caps rows so a hostile `SELECT *` can't exhaust host memory. New query paths must go through this function, not raw `rows.Scan` loops. -3. **Opt-in file-access gate.** `NewModuleWithFileAccess(false)` sets `restrictFileAccess`; `connect` and `attach` then reject any on-disk path except the configured `database`, allowing only in-memory DSNs. `isInMemoryDSN` must parse `mode=memory` as a real query parameter (via `net/url`), **not** as a substring — a loose `strings.Contains(s, ":memory:")` was a real bypass (`file:/etc/passwd?x=:memory:`). When touching DSN classification, keep it parse-based. +3. **Opt-in file-access gate.** `NewModuleWithFileAccess(false)` sets `restrictFileAccess` and freezes the configured `database` at construction through `base.SetHostOnly`; no `set_database` is exported. `connect` (including `connect_remote` with a `file:` URL) rejects other disk paths; `attach` allows only memory. `isInMemoryDSN` trusts only the parsed empty/`:memory:` path, not query parameters such as `mode=memory`, because SQLite and Go parse malformed URI queries differently. Keep that strict classification. 4. **Backward compatibility.** `NewModule()` == `NewModuleWithFileAccess(true)`, `max_rows` defaults to unlimited. Any new safety lever must default to the historical behavior. 5. **PRAGMA config.** `journal_mode` / `synchronous` / `foreign_keys` / `busy_timeout` / `cache_size` are standard SQLite PRAGMAs applied at connect — they are not custom invented defaults. 6. **UDF register/connect serialization (PKG-29).** modernc's `Driver.Open` reads its process-global UDF map (`d.udfs`) with no lock, while `RegisterScalarFunction` (called by `register_function`) writes it. `connect` opens through `localDriverName` — a `funcMutex`-serialized wrapper (`ensureLocalDriver` / `udfSerializedDriver`) whose `Open` holds `funcMutex.RLock` while modernc reads the map, mutually exclusive with `register_function`'s `Lock`/write. This is required because a connection's `Open` is called lazily by `database/sql` (and from parallel host machines), so the "register before connect" convention doesn't hold. **Keep `connect` on `localDriverName`, never raw `"sqlite"`.** Scope/limits (documented in `ensureLocalDriver`): it protects this module's own register vs connect, not a different process-wide user of raw `"sqlite"`, nor collation/connection-hook registration (this module registers neither). +7. **Opt-in UDF exclusion (STAR-102).** `DisableCustomFunctions` uses `noFunctionsDriverName`, backed by an independent zero-value modernc driver with no registered functions, collations or hooks. It also rejects `register_function` before touching the global registry. Every local path, including the libSQL `file:` fallback and pool replacement, must preserve this choice. The legacy path still uses `localDriverName`; network server functions remain server-controlled. This is exclusion, not isolated/budgeted execution of enabled UDFs. ## Test organization @@ -70,6 +71,6 @@ Three layers must stay in sync (enforced by the doc standard, `plan/starpkg文 ## Release discipline - **Floor = go 1.20**, registered under **ENG-09 SEP** (`plan/ENG-09-SEP…`): the libSQL remote client requires go 1.20; `modernc.org/sqlite` is capped at **v1.31.x** because v1.32+ uses `runtime.Pinner` (a go 1.21 API → declares go 1.20 but won't build on it). Don't bump modernc past v1.31.x without raising the floor. -- **CI matrix** = `[1.20.x, 1.25.x]` via the centralized reusable workflow in `1set/meta`. +- **CI matrix** = `[1.20.x, 1.27.x]` via the centralized reusable workflow in `1set/meta`. - **Releases so far**: `v0.1.0` (local modernc, Tier-2 hardened) → `v0.2.0` (added remote libSQL `connect_remote`). The libSQL dependency adds **~6 MB / +70%** to a built binary vs modernc-only — a candidate for build-tag isolation in a future version. - **Bumping the version, the go floor, or tagging are user-confirmed actions** — never tag autonomously; default to patch bumps; published tags are immutable. diff --git a/README.md b/README.md index 5167e17..37bee32 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,7 @@ SQL functions — pure Go, no cgo, all platforms. - **Prepared statements** — `prepare` / `prepare_query` for repeated execution. - **Custom SQL functions** — `register_function` runs Starlark logic inside SQL queries. - **Local & remote** — local files / in-memory via `connect`; remote libSQL (self-hosted `sqld` or Turso Cloud) via `connect_remote`, exposing the same object surface. -- **Safe by default** — SQL-injection-resistant parameter binding, automatic SQLite ⇄ Starlark type conversion, plus opt-in host hardening (`max_rows`, `NewModuleWithFileAccess`). +- **Parameter binding and host controls** — SQLite ⇄ Starlark type conversion, configurable row limits, and opt-in file/function restrictions (`NewModuleWithFileAccess`, `NewModuleWithHostPolicy`). For the complete per-builtin reference — signatures, parameters, returns, errors, examples — and the configuration accessors, see @@ -135,9 +135,11 @@ errors, and examples of every builtin and method above. The module's options (`database`, `timeout`, `busy_timeout`, `foreign_keys`, `journal_mode`, `synchronous`, `cache_size`, `max_rows`) are configured via environment variables (`SQLITE_*`) or per-option `get_` / `set_` -accessor builtins, and serve as defaults for `connect` / `connect_remote`. Two -opt-in host levers (`max_rows` and `NewModuleWithFileAccess`) bound what an -untrusted script can reach; both default to off. `timeout` bounds each database +accessor builtins, and serve as defaults for `connect` / `connect_remote`. +`NewModuleWithFileAccess(false)` fixes the allowed `database` at construction +and omits `set_database`; later environment changes cannot widen that path. +`max_rows` remains a configurable row limit, with `0` meaning unlimited. +`timeout` bounds each database operation (a per-query deadline, and a cancellation point tied to the script thread) — this is what stops an unreachable remote from hanging the host. The connection PRAGMAs (`foreign_keys`, `journal_mode`, …) are applied to every @@ -146,6 +148,29 @@ so its schema and data persist across queries. See the [Configuration section of docs/API.md](docs/API.md#configuration) for the full option table, defaults, accessors, and host-hardening details. +Hosts can opt out of custom SQLite functions while retaining ordinary local SQL: + +```go +module := sqlite.NewModuleWithHostPolicy(sqlite.HostPolicy{ + RestrictFileAccess: true, + DisableCustomFunctions: true, +}) +``` + +`DisableCustomFunctions` rejects `register_function` and opens local connections +on a driver without process-wide custom functions, including functions registered +by another module. SQLite built-ins, transactions and prepared queries remain +available. A `file:` URL passed to `connect_remote` follows this same local path, +file restrictions and connection PRAGMAs. Real network libSQL endpoints retain +their existing behavior; server functions and network access are controlled by +the host/server separately. No connection is opened until a script requests one. + +`NewModule()` and the zero `HostPolicy` retain custom functions and mutable +configuration. Enabled script UDFs still use the global registry and a separate +Starlark thread: they do not inherit the caller's execution budget or cancellation, +and registrations are not released on connection close. The opt-out is suitable +for hosts that do not need UDFs; it is not a process or filesystem sandbox. + ## License MIT diff --git a/docs/API.md b/docs/API.md index 34c40aa..9eaab22 100644 --- a/docs/API.md +++ b/docs/API.md @@ -95,9 +95,10 @@ rows = db.query("SELECT * FROM notes") db.close() ``` -> Note: `connect_remote` is unaffected by the file-access restriction -> (`NewModuleWithFileAccess`) — it opens a network libSQL endpoint, not a local -> file. Gate remote access at the network/credential layer. +> A `file:` URL is a local connection and follows `connect`'s file/function +> restrictions, module defaults and connection PRAGMAs. `auth_token` is unused +> for local files. HTTP(S), libSQL and WebSocket endpoints remain remote; gate +> their access at the network/credential layer. ### `close()` @@ -997,7 +998,7 @@ accessor — never a getter — but this module has none.) | Option | Getter | Setter | Type | Env var | Default | Description | |--------|--------|--------|------|---------|---------|-------------| -| `database` | `get_database` | `set_database` | string | `SQLITE_DATABASE` | `:memory:` | Path to the SQLite database (use `:memory:` for in-memory) | +| `database` | `get_database` | `set_database` (unrestricted modules only) | string | `SQLITE_DATABASE` | `:memory:` | Path to the SQLite database (use `:memory:` for in-memory) | | `timeout` | `get_timeout` | `set_timeout` | float | `SQLITE_TIMEOUT` | `30.0` | Per-operation deadline in seconds (per-query; cancels with the script thread; 0 = none) | | `busy_timeout` | `get_busy_timeout` | `set_busy_timeout` | float | `SQLITE_BUSY_TIMEOUT` | `5.0` | Busy timeout in seconds | | `foreign_keys` | `get_foreign_keys` | `set_foreign_keys` | bool | `SQLITE_FOREIGN_KEYS` | `true` | Enable foreign key constraints | @@ -1028,23 +1029,43 @@ db = connect("app.db") # opens with journal_mode=WAL ### Host hardening (opt-in, host side) -Two **opt-in** levers let the host bound what a script can reach. Both default to -**off**, so existing scripts keep working unchanged. They are configured by the -host in Go, not from a script. +File and function restrictions are **opt-in**, fixed by the host in Go. The +default constructors continue to allow custom functions and local files. - **Bound result size — `max_rows`.** A query helper materializes every returned row into memory. Set `max_rows` (the config option above, or `SQLITE_MAX_ROWS`) to cap the number of rows any single query returns; exceeding the cap raises a script error instead of allocating without bound. `0` (the default) means - unlimited. + unlimited. This configuration is script-settable; it is not an immutable + host policy in the default module. - **Restrict file access — `NewModuleWithFileAccess`.** By default a script may `connect(path)` to any file path (or `attach` any database). `NewModuleWithFileAccess(false)` locks this down: scripts may only open - **in-memory** databases (`:memory:`, `file::memory:…`, `mode=memory`) or the - **one** `database` configured on the module — any other path, and any `attach` + **in-memory** databases (`:memory:`, `file::memory:…`) or the + **exact** `database` configured on the module — any other path, and any `attach` of an on-disk database, is rejected. `NewModule()` equals - `NewModuleWithFileAccess(true)`. - -> Note: `connect_remote` is unaffected by `NewModuleWithFileAccess` — it opens a -> network libSQL endpoint, not a local file. Gate remote access at the -> network/credential layer. + `NewModuleWithFileAccess(true)`. In restricted mode, `database` is captured + at construction, has no `set_database`, and ignores subsequent environment + changes. This also applies to `connect_remote` with a `file:` URL. Paths are + compared as configured DSNs; this is not a filesystem or symlink sandbox. + +- **Disable custom functions — `NewModuleWithHostPolicy`.** Set + `HostPolicy{DisableCustomFunctions: true}` to reject `register_function` and + exclude process-wide functions from every local connection, including pooled + replacements and local `file:` URLs passed to `connect_remote`. Functions + registered before or after construction by another module are also excluded. + The independent driver does not inherit custom collations or connection hooks + installed on the default driver. Ordinary SQLite built-ins, queries, writes, + transactions, attached databases and prepared statements remain available. + Combine with `RestrictFileAccess: true` for the file restriction above. + Policy flags have no script getters/setters or environment-variable controls. + + A zero `HostPolicy` preserves `NewModule()` behavior. When custom functions + remain enabled, registrations are process-wide and permanent, and script UDF + callbacks use a separate thread without the caller's step budget/cancellation. + Disabling UDFs avoids that callback path; it does not provide isolated or + budgeted execution for UDFs that remain enabled. + +> Network libSQL endpoints retain their existing behavior. Server-side functions +> and network authorization are controlled separately; the local UDF policy does +> not claim to restrict a remote server. All connections are explicit script calls. diff --git a/example_test.go b/example_test.go index 360c688..c095c0a 100644 --- a/example_test.go +++ b/example_test.go @@ -4,6 +4,8 @@ import ( "context" "database/sql" "fmt" + "net/http" + "net/http/httptest" "os" "path/filepath" "strings" @@ -17,6 +19,9 @@ import ( "go.starlark.net/starlark" ) +// Sections: script examples; row/file limits; UDF conversion and concurrency; +// host function policy and local/remote connection boundaries. + // funcNameCounter makes custom-function names unique within a process. The // register_function registry is process-global, so reusing a name across runs // (e.g. under -count>1) would fail with "already registered"; a per-call suffix @@ -2489,7 +2494,7 @@ main() // and data: because the pool is pinned to one connection, a later query never // lands on a second (empty) physical connection ("no such table" / lost rows). func TestMemoryDBSingleConnection(t *testing.T) { - db, err := openDatabase(":memory:", 5, true, "MEMORY", "NORMAL", -2000) + db, err := NewModule().openDatabase(":memory:", 5, true, "MEMORY", "NORMAL", -2000) if err != nil { t.Fatalf("openDatabase: %v", err) } @@ -2520,7 +2525,7 @@ func TestMemoryDBSingleConnection(t *testing.T) { // silently succeed. func TestForeignKeysAcrossConnections(t *testing.T) { path := filepath.Join(t.TempDir(), "fk.db") - db, err := openDatabase(path, 5, true, "WAL", "NORMAL", -2000) + db, err := NewModule().openDatabase(path, 5, true, "WAL", "NORMAL", -2000) if err != nil { t.Fatalf("openDatabase: %v", err) } @@ -2554,7 +2559,7 @@ func TestForeignKeysAcrossConnections(t *testing.T) { // thread's context, so a cancelled thread aborts it (the fix for uncancellable // remote/slow queries that could hang the host). func TestQueryRespectsThreadCancellation(t *testing.T) { - db, err := openDatabase(":memory:", 5, true, "MEMORY", "NORMAL", -2000) + db, err := NewModule().openDatabase(":memory:", 5, true, "MEMORY", "NORMAL", -2000) if err != nil { t.Fatalf("openDatabase: %v", err) } @@ -2609,7 +2614,7 @@ func TestInMemoryDSNClassification(t *testing.T) { // per-operation timeout — it errors instead of blocking the host indefinitely // when the sole connection is held by an open transaction. func TestNestedBeginHonorsTimeout(t *testing.T) { - db, err := openDatabase(":memory:", 5, true, "MEMORY", "NORMAL", -2000) + db, err := NewModule().openDatabase(":memory:", 5, true, "MEMORY", "NORMAL", -2000) if err != nil { t.Fatalf("openDatabase: %v", err) } @@ -2700,3 +2705,373 @@ func TestConcurrentRegisterAndConnectNoRace(t *testing.T) { close(stop) wg.Wait() } + +// Host policy: a libSQL file URL must obey the same local database restrictions. +func TestConnectRemoteFileRestriction(t *testing.T) { + path := filepath.ToSlash(filepath.Join(t.TempDir(), "restricted.db")) + script := fmt.Sprintf(` +load("sqlite", "connect_remote") +db = connect_remote(%q) +db.close() +`, "file:"+path) + requireSQLiteScriptErrorContains(t, script, func() starlet.ModuleLoader { + return NewModuleWithFileAccess(false).LoadModule() + }, "file database access is restricted") +} + +func policyModule(t *testing.T, m *Module) starlark.HasAttrs { + t.Helper() + values, err := m.LoadModule()() + if err != nil { + t.Fatal(err) + } + return values[ModuleName].(starlark.HasAttrs) +} + +func policyCall(t *testing.T, object starlark.HasAttrs, name string, args ...starlark.Value) (starlark.Value, error) { + t.Helper() + fn, err := object.Attr(name) + if err != nil || fn == nil { + t.Fatalf("missing %s: %v", name, err) + } + return starlark.Call(&starlark.Thread{Name: t.Name()}, fn, starlark.Tuple(args), nil) +} + +func policyMustCall(t *testing.T, object starlark.HasAttrs, name string, args ...starlark.Value) starlark.Value { + t.Helper() + value, err := policyCall(t, object, name, args...) + if err != nil { + t.Fatalf("%s: %v", name, err) + } + return value +} + +func policyDB(t *testing.T, m *Module, entry, dsn string) starlark.HasAttrs { + t.Helper() + db := policyMustCall(t, policyModule(t, m), entry, starlark.String(dsn)).(starlark.HasAttrs) + t.Cleanup(func() { policyMustCall(t, db, "close") }) + return db +} + +func policyFunction(t *testing.T, m *Module) string { + t.Helper() + name := uniqueFuncName("policy_udf") + fn := starlark.NewBuiltin(name, func(*starlark.Thread, *starlark.Builtin, starlark.Tuple, []starlark.Tuple) (starlark.Value, error) { + return starlark.MakeInt(42), nil + }) + policyMustCall(t, policyModule(t, m), "register_function", starlark.String(name), fn, starlark.MakeInt(0)) + return name +} + +func TestHostPolicyRegistration(t *testing.T) { + for _, disabled := range []bool{false, true} { + t.Run(fmt.Sprint(disabled), func(t *testing.T) { + policy := HostPolicy{DisableCustomFunctions: disabled} + m := NewModuleWithHostPolicy(policy) + policy.DisableCustomFunctions = !disabled // The constructor must copy the policy. + name := uniqueFuncName("registration_policy") + fn := starlark.NewBuiltin(name, func(*starlark.Thread, *starlark.Builtin, starlark.Tuple, []starlark.Tuple) (starlark.Value, error) { + return starlark.MakeInt(42), nil + }) + _, err := policyCall(t, policyModule(t, m), "register_function", starlark.String(name), fn) + if disabled { + if err == nil || !strings.Contains(err.Error(), "custom functions are disabled") { + t.Fatalf("registration must be denied: %v", err) + } + funcMutex.RLock() + _, exists := registeredFuncs[name] + funcMutex.RUnlock() + if exists { + t.Fatal("denied registration changed the global registry") + } + } else if err != nil { + t.Fatal(err) + } + }) + } +} + +func TestHostPolicyExports(t *testing.T) { + const common = "connect connect_remote get_busy_timeout get_cache_size get_database get_foreign_keys get_journal_mode get_max_rows get_synchronous get_timeout register_function set_busy_timeout set_cache_size set_foreign_keys set_journal_mode set_max_rows set_synchronous set_timeout" + for _, restricted := range []bool{false, true} { + for _, disabled := range []bool{false, true} { + m := NewModuleWithHostPolicy(HostPolicy{RestrictFileAccess: restricted, DisableCustomFunctions: disabled}) + want := common + if !restricted { + want = strings.Replace(want, "set_cache_size ", "set_cache_size set_database ", 1) + } + if got := strings.Join(policyModule(t, m).AttrNames(), " "); got != want { + t.Errorf("policy file=%v functions=%v exports = %s, want %s", restricted, disabled, got, want) + } + } + } +} + +func TestHostPolicyGlobalFunctions(t *testing.T) { + for _, entry := range []string{"connect", "connect_remote"} { + for _, dsn := range []string{"file::memory:", "file:" + filepath.ToSlash(filepath.Join(t.TempDir(), "functions.db"))} { + t.Run(entry+dsn, func(t *testing.T) { + before := policyFunction(t, NewModule()) + m := NewModuleWithHostPolicy(HostPolicy{DisableCustomFunctions: true}) + db := policyDB(t, m, entry, dsn) + after := policyFunction(t, NewModule()) + for _, connection := range []starlark.HasAttrs{db, policyDB(t, m, entry, dsn), policyDB(t, NewModuleWithHostPolicy(HostPolicy{DisableCustomFunctions: true}), entry, dsn)} { + for _, name := range []string{before, after} { + _, err := policyCall(t, connection, "query", starlark.String("SELECT "+name+"()")) + if err == nil || !strings.Contains(err.Error(), "no such function") { + t.Fatalf("global function %s remains visible: %v", name, err) + } + } + } + legacy := policyDB(t, NewModule(), entry, dsn) + got := policyMustCall(t, legacy, "query_one", starlark.String("SELECT "+before+"() AS n, "+after+"() AS m")) + if got.String() != `{"n": 42, "m": 42}` { + t.Fatalf("legacy functions changed: %s", got) + } + }) + } + } +} + +func TestHostPolicySQLPaths(t *testing.T) { + name := policyFunction(t, NewModule()) + for _, method := range []string{"query", "query_one", "execute", "batch", "prepare", "prepare_query"} { + t.Run(method, func(t *testing.T) { + db := policyDB(t, NewModuleWithHostPolicy(HostPolicy{DisableCustomFunctions: true}), "connect", ":memory:") + var query starlark.Value = starlark.String("SELECT " + name + "()") + if method == "batch" { + query = starlark.NewList([]starlark.Value{query}) + } + value, err := policyCall(t, db, method, query) + // database/sql may defer preparation until execution. + if err == nil && (method == "prepare" || method == "prepare_query") { + stmt := value.(starlark.HasAttrs) + defer policyMustCall(t, stmt, "close") + op := "query" + if method == "prepare" { + op = "execute" + } + _, err = policyCall(t, stmt, op) + } + if err == nil || !strings.Contains(err.Error(), "no such function") { + t.Fatalf("%s allowed a custom function: %v", method, err) + } + }) + } +} + +func TestHostPolicyTransactionPaths(t *testing.T) { + name := policyFunction(t, NewModule()) + for _, method := range []string{"query", "query_one", "execute"} { + t.Run(method, func(t *testing.T) { + db := policyDB(t, NewModuleWithHostPolicy(HostPolicy{DisableCustomFunctions: true}), "connect", ":memory:") + tx := policyMustCall(t, db, "begin").(starlark.HasAttrs) + defer policyMustCall(t, tx, "rollback") + result := policyMustCall(t, tx, method, starlark.String("SELECT "+name+"()")).(*OperationResult) + if result.err == nil || !strings.Contains(result.err.Error(), "no such function") { + t.Fatalf("transaction %s allowed custom function: %s", method, result) + } + }) + } +} + +func TestHostPolicyDatabaseFrozen(t *testing.T) { + allowed := filepath.Join(t.TempDir(), "allowed.db") + denied := filepath.Join(t.TempDir(), "denied.db") + t.Setenv("SQLITE_DATABASE", allowed) + m := NewModuleWithFileAccess(false) + t.Setenv("SQLITE_DATABASE", denied) + mod := policyModule(t, m) + if got := policyMustCall(t, mod, "get_database"); got != starlark.String(allowed) { + t.Fatalf("host database changed through environment: %s", got) + } + if setter, _ := mod.Attr("set_database"); setter != nil { + t.Fatal("restricted module exposes a database allowlist setter") + } + for _, entry := range []string{"connect", "connect_remote"} { + _, err := policyCall(t, mod, entry, starlark.String("file:"+filepath.ToSlash(denied))) + if err == nil || !strings.Contains(err.Error(), "file database access is restricted") { + t.Fatalf("%s bypassed the fixed database: %v", entry, err) + } + } +} + +func TestHostPolicyRemoteHTTP(t *testing.T) { + var calls int64 + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + atomic.AddInt64(&calls, 1) + w.Header().Set("Content-Type", "application/json") + fmt.Fprint(w, `{"results":[{"type":"ok","response":{"type":"execute","result":{"cols":[{"name":"n","decltype":"INTEGER"}],"rows":[[{"type":"integer","value":"42"}]],"affected_row_count":0}}}]}`) + })) + defer server.Close() + db := policyDB(t, NewModuleWithHostPolicy(HostPolicy{RestrictFileAccess: true, DisableCustomFunctions: true}), "connect_remote", server.URL) + beforeQuery := atomic.LoadInt64(&calls) + if got := policyMustCall(t, db, "query_one", starlark.String("SELECT 42 AS n")); got.String() != `{"n": 42}` { + t.Fatalf("remote result: %s", got) + } + if atomic.LoadInt64(&calls) <= beforeQuery { + t.Fatal("remote query did not reach the local HTTP fixture") + } +} + +func requirePolicyAttrs(t *testing.T, object starlark.HasAttrs, want string) { + t.Helper() + if got := strings.Join(object.AttrNames(), " "); got != want { + t.Fatalf("public methods = %s, want %s", got, want) + } +} + +func TestHostPolicyNormalSQL(t *testing.T) { + for _, restricted := range []bool{false, true} { + for _, disabled := range []bool{false, true} { + m := NewModuleWithHostPolicy(HostPolicy{RestrictFileAccess: restricted, DisableCustomFunctions: disabled}) + db := policyDB(t, m, "connect", ":memory:") + requirePolicyAttrs(t, db, "attach batch begin close count create_table delete detach drop_table execute indices insert insert_many prepare prepare_query query query_one select table_exists table_info tables truncate_table update upsert") + policyMustCall(t, db, "execute", starlark.String("CREATE TABLE t (n INTEGER)")) + stmt := policyMustCall(t, db, "prepare", starlark.String("INSERT INTO t VALUES (abs(?))")).(starlark.HasAttrs) + requirePolicyAttrs(t, stmt, "close execute") + policyMustCall(t, stmt, "execute", starlark.Tuple{starlark.MakeInt(-42)}) + policyMustCall(t, stmt, "close") + query := policyMustCall(t, db, "prepare_query", starlark.String("SELECT n FROM t")).(starlark.HasAttrs) + requirePolicyAttrs(t, query, "close query query_one") + for _, method := range []string{"query", "query_one"} { + if got := policyMustCall(t, query, method).String(); !strings.Contains(got, `"n": 42`) { + t.Fatalf("ordinary query changed: %s", got) + } + } + policyMustCall(t, query, "close") + tx := policyMustCall(t, db, "begin").(starlark.HasAttrs) + requirePolicyAttrs(t, tx, "commit execute query query_one rollback") + if result := policyMustCall(t, tx, "execute", starlark.String("UPDATE t SET n = 43")).(*OperationResult); result.err != nil { + t.Fatal(result.err) + } + policyMustCall(t, tx, "commit") + if got := policyMustCall(t, db, "query_one", starlark.String("SELECT n FROM t")).String(); got != `{"n": 43}` { + t.Fatalf("ordinary transaction changed: %s", got) + } + } + } +} + +func TestHostPolicyPooledConnections(t *testing.T) { + before := policyFunction(t, NewModule()) + m := NewModuleWithHostPolicy(HostPolicy{DisableCustomFunctions: true}) + db, err := m.openDatabase(filepath.Join(t.TempDir(), "pool.db"), 5, true, "WAL", "NORMAL", -2000) + if err != nil { + t.Fatal(err) + } + defer db.Close() + db.SetMaxIdleConns(0) + first, err := db.Conn(context.Background()) + if err != nil { + t.Fatal(err) + } + defer first.Close() + after := policyFunction(t, NewModule()) + second, err := db.Conn(context.Background()) + if err != nil { + t.Fatal(err) + } + defer second.Close() + for _, conn := range []*sql.Conn{first, second} { + for _, name := range []string{before, after} { + if _, err := conn.ExecContext(context.Background(), "SELECT "+name+"()"); err == nil || !strings.Contains(err.Error(), "no such function") { + t.Fatalf("pooled connection sees %s: %v", name, err) + } + } + var foreignKeys int + if err := conn.QueryRowContext(context.Background(), "PRAGMA foreign_keys").Scan(&foreignKeys); err != nil || foreignKeys != 1 { + t.Fatalf("pooled connection lost PRAGMAs: %d, %v", foreignKeys, err) + } + _ = conn.Close() + } + if _, err := db.Exec("SELECT " + before + "()"); err == nil || !strings.Contains(err.Error(), "no such function") { + t.Fatalf("recycled pool connection sees a global function: %v", err) + } +} + +func TestHostPolicyStoredFunctionCalls(t *testing.T) { + name := policyFunction(t, NewModule()) + dsn := filepath.Join(t.TempDir(), "stored.db") + legacy := policyDB(t, NewModule(), "connect", dsn) + policyMustCall(t, legacy, "execute", starlark.String("CREATE TABLE t(n INTEGER)")) + policyMustCall(t, legacy, "execute", starlark.String("CREATE VIEW v AS SELECT "+name+"() AS n")) + policyMustCall(t, legacy, "execute", starlark.String("CREATE TRIGGER tr AFTER INSERT ON t BEGIN SELECT "+name+"(); END")) + db := policyDB(t, NewModuleWithHostPolicy(HostPolicy{DisableCustomFunctions: true}), "connect", dsn) + for _, query := range []string{"SELECT * FROM v", "INSERT INTO t VALUES (1)"} { + if _, err := policyCall(t, db, "execute", starlark.String(query)); err == nil || !strings.Contains(err.Error(), "no such function") { + t.Fatalf("stored SQL called a global function: %s, %v", query, err) + } + } +} + +func TestHostPolicyConnectErrors(t *testing.T) { + for _, address := range []string{"", "%", "ftp://invalid.example/database"} { + m := NewModuleWithHostPolicy(HostPolicy{DisableCustomFunctions: true}) + if _, err := policyCall(t, policyModule(t, m), "connect_remote", starlark.String(address)); err == nil { + t.Errorf("invalid remote address %q accepted", address) + } + } +} + +func TestHostPolicyConcurrentConnections(t *testing.T) { + var wg sync.WaitGroup + for i := 0; i < 4; i++ { + wg.Add(1) + go func(disabled bool) { + defer wg.Done() + mod := policyModule(t, NewModuleWithHostPolicy(HostPolicy{DisableCustomFunctions: disabled})) + for j := 0; j < 20; j++ { + db := policyMustCall(t, mod, "connect_remote", starlark.String("file::memory:")).(starlark.HasAttrs) + policyMustCall(t, db, "query", starlark.String("SELECT abs(-42)")) + policyMustCall(t, db, "close") + } + }(i%2 == 0) + } + for i := 0; i < 40; i++ { + policyFunction(t, NewModule()) + } + wg.Wait() +} + +func TestHostPolicyDatabaseInitiallyUnset(t *testing.T) { + t.Setenv("SQLITE_DATABASE", "") + if err := os.Unsetenv("SQLITE_DATABASE"); err != nil { + t.Fatal(err) + } + m := NewModuleWithHostPolicy(HostPolicy{RestrictFileAccess: true}) + t.Setenv("SQLITE_DATABASE", filepath.Join(t.TempDir(), "late.db")) + if got := policyMustCall(t, policyModule(t, m), "get_database"); got != starlark.String(":memory:") { + t.Fatalf("late environment widened the default database: %s", got) + } + db := policyDB(t, m, "connect_remote", "file::memory:") + policyMustCall(t, db, "execute", starlark.String("CREATE TABLE t(n INTEGER)")) +} + +// BenchmarkLocalConnect can also be copied unchanged to the pre-policy revision. +func BenchmarkLocalConnect(b *testing.B) { + values, err := NewModule().LoadModule()() + if err != nil { + b.Fatal(err) + } + connect, err := values[ModuleName].(starlark.HasAttrs).Attr("connect") + if err != nil { + b.Fatal(err) + } + thread := &starlark.Thread{Name: "benchmark"} + b.ReportAllocs() + b.ResetTimer() + for i := 0; i < b.N; i++ { + value, err := starlark.Call(thread, connect, starlark.Tuple{starlark.String(":memory:")}, nil) + if err != nil { + b.Fatal(err) + } + closeFn, err := value.(starlark.HasAttrs).Attr("close") + if err != nil { + b.Fatal(err) + } + if _, err := starlark.Call(thread, closeFn, nil, nil); err != nil { + b.Fatal(err) + } + } +} diff --git a/function.go b/function.go index b7908f1..2af771b 100644 --- a/function.go +++ b/function.go @@ -34,6 +34,10 @@ var ( // funcMutex-serialized wrapper over modernc's "sqlite" driver (see below). const localDriverName = "sqlite-udf-serialized" +// noFunctionsDriverName uses an independent, immutable modernc driver with no +// registered custom functions, collations, or connection hooks. +const noFunctionsDriverName = "sqlite-no-custom-functions" + var registerLocalDriverOnce sync.Once // ensureLocalDriver registers the funcMutex-serialized wrapper over modernc's @@ -67,6 +71,7 @@ func ensureLocalDriver() { inner := db.Driver() _ = db.Close() sql.Register(localDriverName, &udfSerializedDriver{inner: inner}) + sql.Register(noFunctionsDriverName, &sqlite.Driver{}) }) } @@ -87,6 +92,14 @@ func (d *udfSerializedDriver) Open(name string) (driver.Conn, error) { // Function Registration // ============================================================================ +// registerFunction enforces the host policy before touching the global registry. +func (m *Module) registerFunction(thread *starlark.Thread, fn *starlark.Builtin, args starlark.Tuple, kwargs []starlark.Tuple) (starlark.Value, error) { + if m.disableCustomFunctions { + return nil, fmt.Errorf("custom functions are disabled by the host") + } + return registerFunction(thread, fn, args, kwargs) +} + // registerFunction implements the register_function Starlark builtin. func registerFunction(thread *starlark.Thread, fn *starlark.Builtin, args starlark.Tuple, kwargs []starlark.Tuple) (starlark.Value, error) { var name string diff --git a/sqlite.go b/sqlite.go index 5854c8e..fe2fd46 100644 --- a/sqlite.go +++ b/sqlite.go @@ -63,15 +63,34 @@ const ( // Module wraps the ConfigurableModule with specific functionality for SQLite operations. type Module struct { - cfgMod *base.ConfigurableModule - ext *base.ConfigurableModuleExt - restrictFileAccess bool + cfgMod *base.ConfigurableModule + ext *base.ConfigurableModuleExt + restrictFileAccess bool + disableCustomFunctions bool } -// NewModule creates a new module with default configuration. -func NewModule() *Module { - return newModuleWithOptions( - genConfigOption(configKeyDatabase, "Path to SQLite database (use :memory: for in-memory)", defaultDatabase), +// HostPolicy contains opt-in restrictions copied into a module at construction. +// The zero value preserves NewModule's behavior; scripts cannot change these flags. +type HostPolicy struct { + // RestrictFileAccess applies the same local file gate as NewModuleWithFileAccess(false). + RestrictFileAccess bool + // DisableCustomFunctions prevents registration and use of process-wide custom + // functions on local connections. Built-in SQLite functions remain available. + // Remote server functions are controlled by the server, not by this flag. + DisableCustomFunctions bool +} + +// NewModuleWithHostPolicy creates a module with fixed, host-owned restrictions. +// It does not add Starlark setters or environment options for the policy. +func NewModuleWithHostPolicy(policy HostPolicy) *Module { + // A host-only option snapshots its environment at module construction, + // before any script can change the process environment or its database. + databaseOpt := genConfigOption(configKeyDatabase, "Path to SQLite database (use :memory: for in-memory)", defaultDatabase) + if policy.RestrictFileAccess { + databaseOpt.SetHostOnly(true) + } + m := newModuleWithOptions( + databaseOpt, genConfigOption(configKeyTimeout, "Per-operation deadline in seconds (0 = no deadline)", defaultTimeout), genConfigOption(configKeyBusyTimeout, "Busy timeout in seconds", defaultBusyTimeout), genConfigOption(configKeyForeignKeys, "Enable foreign key constraints", defaultForeignKeys), @@ -80,13 +99,19 @@ func NewModule() *Module { genConfigOption(configKeyCacheSize, "Cache size in number of pages", defaultCacheSize), genConfigOption(configKeyMaxRows, "Maximum rows returned by query helpers (0 means unlimited)", defaultMaxRows), ) + m.restrictFileAccess = policy.RestrictFileAccess + m.disableCustomFunctions = policy.DisableCustomFunctions + return m +} + +// NewModule creates a new module with default configuration. +func NewModule() *Module { + return NewModuleWithHostPolicy(HostPolicy{}) } // NewModuleWithFileAccess creates a new module and optionally restricts file database access. func NewModuleWithFileAccess(allowed bool) *Module { - m := NewModule() - m.restrictFileAccess = !allowed - return m + return NewModuleWithHostPolicy(HostPolicy{RestrictFileAccess: !allowed}) } // genConfigOption creates a configuration option with common settings. @@ -132,7 +157,7 @@ func (m *Module) LoadModule() starlet.ModuleLoader { additionalFuncs := starlark.StringDict{ "connect": starlark.NewBuiltin(ModuleName+".connect", m.connect), "connect_remote": starlark.NewBuiltin(ModuleName+".connect_remote", m.connectRemote), - "register_function": starlark.NewBuiltin(ModuleName+".register_function", registerFunction), + "register_function": starlark.NewBuiltin(ModuleName+".register_function", m.registerFunction), } // Return the module @@ -200,7 +225,7 @@ func (m *Module) connect(thread *starlark.Thread, fn *starlark.Builtin, args sta maxRows := m.ext.GetInt(configKeyMaxRows, defaultMaxRows) // Create a new database connection - db, err := openDatabase(database, busyTimeout.GoFloat(), foreignKeysValue, journalMode, synchronous, cacheSize) + db, err := m.openDatabase(database, busyTimeout.GoFloat(), foreignKeysValue, journalMode, synchronous, cacheSize) if err != nil { return nil, fmt.Errorf("failed to open database: %w", err) } @@ -243,6 +268,15 @@ func (m *Module) connectRemote(thread *starlark.Thread, fn *starlark.Builtin, ar if dbURL == "" { return nil, fmt.Errorf("connect_remote: url is required") } + parsed, err := url.Parse(dbURL) + if err != nil { + return nil, fmt.Errorf("failed to create remote connector: %w", err) + } + if parsed.Scheme == "file" { + // libSQL otherwise opens the raw default SQLite driver, bypassing both + // local host policy and register/open serialization. + return m.connect(thread, fn, starlark.Tuple{starlark.String(dbURL)}, nil) + } var opts []libsql.Option if authToken != "" { @@ -310,7 +344,7 @@ func isInMemoryDSN(s string) bool { // pinned to one never-recycled connection, because its schema and data live only // inside that connection — a second (empty) connection or a recycled one would // otherwise make earlier data vanish ("no such table"). -func openDatabase(connStr string, busyTimeout float64, foreignKeys bool, journalMode, synchronous string, cacheSize int) (*sql.DB, error) { +func (m *Module) openDatabase(connStr string, busyTimeout float64, foreignKeys bool, journalMode, synchronous string, cacheSize int) (*sql.DB, error) { if !pragmaIdentifier(journalMode) { return nil, fmt.Errorf("invalid journal_mode %q", journalMode) } @@ -330,7 +364,11 @@ func openDatabase(connStr string, busyTimeout float64, foreignKeys bool, journal // (which reads modernc's global UDF map) can't race a concurrent // register_function call (see ensureLocalDriver). ensureLocalDriver() - db, err := sql.Open(localDriverName, dsn) + driverName := localDriverName + if m.disableCustomFunctions { + driverName = noFunctionsDriverName + } + db, err := sql.Open(driverName, dsn) if err != nil { return nil, err } From 8329f1bdd54b06205efe7e35f2d0efdc31f035f1 Mon Sep 17 00:00:00 2001 From: Kevin Tang <73975146+vt128@users.noreply.github.com> Date: Thu, 24 Sep 2026 00:44:54 +0800 Subject: [PATCH 2/2] [doc] normalize host policy example indentation --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 37bee32..92720ea 100644 --- a/README.md +++ b/README.md @@ -152,8 +152,8 @@ Hosts can opt out of custom SQLite functions while retaining ordinary local SQL: ```go module := sqlite.NewModuleWithHostPolicy(sqlite.HostPolicy{ - RestrictFileAccess: true, - DisableCustomFunctions: true, + RestrictFileAccess: true, + DisableCustomFunctions: true, }) ```