Add SQLite3MultipleCiphers 2.3.3 support (SQLite 3.53.0) - #1
Open
gainskills wants to merge 190 commits into
Open
Add SQLite3MultipleCiphers 2.3.3 support (SQLite 3.53.0) #1gainskills wants to merge 190 commits into
gainskills wants to merge 190 commits into
Conversation
use same workaround as authelia/authelia#6404 before mattn#1177 fixes the build otherwise
action/upload-artifact@v1 has been deprecated for a while. It seems like GitHub Actions will now cancel workflows if it is still using v1 of the action. This upgrades to the latest v4 of the action. See https://github.blog/news-insights/product-news/get-started-with-v4-of-github-actions-artifacts/ for details.
The commit removes the use of runtime.SetFinalizer to finalize
SQLiteRows since only serves to close the associated SQLiteStmt which
already has a registered finalizer.
It also fixes a race and potential panic in SQLiteRows.Close around the
SQLiteRows.s field (*SQLiteStmt) which is accessed without a mutex being
held, but modified with it held (null'd out). Further the mutex we are
holding is that of the SQLiteStmt so a subsequent call to Close will
cause a panic sine it'll attempt to dereference a nil field. The fix
here is to add a mutex for closing to SQLiteRows.
Since we now also set the s field to nil when closing this commit
removes the "closed" field (since checking if s is nil is the same) and
also changes the type of "nc" (number of columns) to an int32 so that we
can pack the nc and cls fields, and add the close mutex without making
the struct any bigger.
```
goos: darwin
goarch: arm64
pkg: github.com/charlievieth/go-sqlite3
cpu: Apple M4 Pro
│ x1.txt │ x4.txt │
│ sec/op │ sec/op vs base │
Suite/BenchmarkExec/Params-14 719.2n ± 2% 716.9n ± 1% ~ (p=0.897 n=10)
Suite/BenchmarkExec/NoParams-14 506.5n ± 3% 500.1n ± 0% -1.25% (p=0.002 n=10)
Suite/BenchmarkExecContext/Params-14 1.584µ ± 0% 1.567µ ± 1% -1.07% (p=0.007 n=10)
Suite/BenchmarkExecContext/NoParams-14 1.524µ ± 1% 1.524µ ± 1% ~ (p=0.539 n=10)
Suite/BenchmarkExecStep-14 443.9µ ± 3% 441.4µ ± 0% -0.55% (p=0.011 n=10)
Suite/BenchmarkExecContextStep-14 447.8µ ± 1% 442.9µ ± 0% -1.10% (p=0.000 n=10)
Suite/BenchmarkExecTx-14 1.643µ ± 1% 1.640µ ± 0% ~ (p=0.642 n=10)
Suite/BenchmarkQuery-14 1.968µ ± 3% 1.821µ ± 1% -7.52% (p=0.000 n=10)
Suite/BenchmarkQuerySimple-14 1.207µ ± 2% 1.040µ ± 1% -13.84% (p=0.000 n=10)
Suite/BenchmarkQueryContext/Background-14 2.400µ ± 1% 2.320µ ± 0% -3.31% (p=0.000 n=10)
Suite/BenchmarkQueryContext/WithCancel-14 8.847µ ± 5% 8.512µ ± 4% -3.79% (p=0.007 n=10)
Suite/BenchmarkParams-14 2.131µ ± 2% 1.967µ ± 1% -7.70% (p=0.000 n=10)
Suite/BenchmarkStmt-14 1.444µ ± 1% 1.359µ ± 1% -5.89% (p=0.000 n=10)
Suite/BenchmarkRows-14 61.57µ ± 1% 60.24µ ± 1% -2.16% (p=0.000 n=10)
Suite/BenchmarkStmtRows-14 60.15µ ± 1% 59.08µ ± 1% -1.78% (p=0.000 n=10)
Suite/BenchmarkQueryParallel-14 960.9n ± 1% 420.8n ± 2% -56.21% (p=0.000 n=10)
geomean 4.795µ 4.430µ -7.62%
```
A valid sqlite header must always be included (like in the other files) but sqlite3-binding.h explicitly guards against the system library case.
Because of sqlite's flexible typing, even though the column was declared as jsonb, the values are stored as the TEXT-typed json because they're converted to strings by Value/Scan. If the table is strict with a BLOB column, the example fails because of the type mismatch. This can be fixed by using the `jsonb()` function to convert incoming string-typed json and the `json()` function to convert outgoing binary-typed jsonb. The example is expanded to show both of these approaches. Note that both approaches use the same string-typed marshalling functions because the conversion to jsonb occurs within sqlite3, not within the Go code. SQLite docs state that the binary format is internal and applications shouldn't try to generate it: https://sqlite.org/json1.html#jsonb
GCP (at least Google Compute Engine) is a VM, and does not have restrictions relating to gcc. Also it appears that this whole section refers to general compiling, rather than compiling on GCP.
* Add ability to set an int64 file control * Update documentation * Remove duplicate err check in test * Update sqlite3.go Co-authored-by: rittneje <rittneje@gmail.com> --------- Co-authored-by: rittneje <rittneje@gmail.com>
Release vtable and cursor handles when SQLite destroys them
Do not clobber SQLite's default cost estimates in BestIndex
Translate SQL NULL filter arguments to nil like goVUpdate
Identify updated row by argv 0 in goVUpdate
Ignore Used for constraints SQLite marked not usable
Reject nil module and nil BestIndex result
Fail upgrade tool on download and write errors
Fix off-by-one truncating SQL in fuzz target
Fix wrong results and cursor state sharing in series example
Use the table name from xCreate args in vtable example
Close leaked rows in hook example
Close prepared statement and fail if limit is not enforced in limit example
Upgrade SQLite to version 3053004
Handle NULL and BLOB values in json example Scan
Add workflow to create releases on tag push
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
aes128cbc,aes256cbc,chacha20,sqlcipher, andrc4._cipher,_key,_kdf_iter,_hmac_check,_hmac_use,_legacy,_legacy_page_size,_mc_legacy_wal,_plaintext_header_size. Cipher PRAGMAs run beforeany schema-touching SQL;
_keyruns last in the cipher block so key derivation triggers after configuration.sqlite3mc-2.2.7), upgraded to MC 2.3.3 (adds thread-safety fix insqlite3mc_cipher_name, symbol-conflict fix when statically linking withlibsodium, and secure clearing of cipher data structures on free).
mattn/masterso the cipher integration coexists with upstream's CGO/perf rewrite, opt-in statement cache (_stmt_cache_size), Go 1.21 floor, and recent bug fixes.Test plan
go build ./...succeedsgo test -run Cipher -v ./...passes (end-to-end cipher integration test insqlite3_cipher_test.go)go test ./...passes (full suite, no regressions on the merged tree)_stmt_cache_size_keylast in the cipher block)sqlite3ext.hentries (SQLite 3.52.0+ function pointers) compile cleanly with the existing build flags🤖 Generated with Claude Code