Skip to content

[Android] Bundle SQLite 3.54.1 instead of @nozbe/sqlite 3.46.0 - #21

Merged
joker-777 merged 7 commits into
kenhubfrom
deps/sqlite-3.54.1
Nov 2, 2026
Merged

joker-777 merged 7 commits into
kenhubfrom
deps/sqlite-3.54.1

Conversation

@joker-777

@joker-777 joker-777 commented Nov 2, 2026 •

Copy link
Copy Markdown
  • On Android (JSI) and Windows, the build compiles SQLite into the app. It came from the @nozbe/sqlite package, which holds SQLite 3.46.0 from June 2024 and hasn't been updated since. That package is an unmodified copy of SQLite's official single-file source (the "amalgamation").
  • The 3.54.1 amalgamation now lives in native/sqlite, and both builds compile it from there. The build script copies it into the package. The download was checked against the SHA3-256 hash on sqlite.org, and README.md there records the version, source and update steps.
  • The @nozbe/sqlite dependency is removed. The published package stays about 3 MB, since it now carries the source that the old dependency used to bring in.
  • CMakeLists.txt: the build looks for its own test app by checking for @nozbe/simdjson instead of @nozbe/sqlite. The cache counter is bumped, so CMake builds from scratch.
  • WatermelonDB.vcxproj: WatermelonSqliteDir now defaults to native/sqlite, the same way WatermelonJsiSharedDir points to native/shared. Consumers can still override it.
  • WatermelonDB.vcxproj also defines SQLITE_UWP. SQLite 3.53.0 dropped Windows RT support, and 3.54.0 documents -DSQLITE_UWP=1 for UWP apps. SQLITE_OS_WINRT stays defined for older SQLite versions.
  • New native/windows/WatermelonDB/sqlite3_uwp.h, force-included only into sqlite3.c. SQLite 3.53 dropped its CreateFile2 path, and 3.54 still calls the desktop-only CreateFileW and AreFileApisANSI with SQLITE_UWP (reported upstream in this forum thread, unfixed). In Windows Store builds, the header routes CreateFileW to CreateFile2 with SQLite 3.46's flag mapping, and makes AreFileApisANSI return true, as 3.46 did. sqlite3.c stays unmodified.
  • iOS and the non-JSI Android bridge are unchanged; they use the system SQLite.

Verification:

  • sqlite3.c compiles with NDK 27.1 for arm64 at -Os.
  • yarn build puts the SQLite files under dist/native/sqlite, and dist/package.json no longer lists @nozbe/sqlite.
  • Windows: the fork's CI has the Windows job disabled (if: false, the port is unmaintained upstream), so it was checked with one-off runs on windows-2022 (MSVC 14.44, Windows SDK 10.0.26100) in Windows Store mode (WINAPI_FAMILY=WINAPI_FAMILY_APP, as WatermelonDB.vcxproj builds):
    • Without the header, 3.54.1 fails on CreateFileW and AreFileApisANSI (run).
    • With this PR's files, sqlite3.c compiles at /W4 without errors or warnings. It no longer references CreateFileW, uses CreateFile2, and every Windows import is in WindowsApp.lib, the set Store apps may call (run).
    • A smoke test of the same header passed: 200,000 rows, WAL mode, temp files, a checkpoint and a shared in-memory database (run). It ran as a normal desktop program, not inside a Store app's sandbox.
    • The full WatermelonDB.vcxproj / react-native-windows build was not run.
  • This PR's Android CI job builds the JSI library and runs the adapter tests on an emulator.

🤖 Generated with Claude Code

The JSI build on Android compiles SQLite into the app from the
@nozbe/sqlite package, which still holds SQLite 3.46.0 (June 2024). That
package is an unmodified copy of the official amalgamation and has not
been updated since.

The amalgamation of SQLite 3.54.1, checked against the SHA3-256 on
sqlite.org, now lives in native/android-jsi/sqlite and is compiled from
there. The @nozbe/sqlite dependency is removed. iOS keeps using the
system SQLite.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 2, 2026 15:02

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Removing @nozbe/sqlite breaks clean Windows builds because the Windows project still references its sources; migrate those references to vendored SQLite or retain the dependency.

1 open issue
What changed in this PR

This PR updates WatermelonDB’s bundled Android JSI SQLite from 3.46.0 to 3.54.1 and moves its source into the repository. iOS and the non-JSI Android bridge remain unchanged.

Changes:

  • Switch Android CMake to local SQLite sources, update test-app detection, and bump the cache counter.
  • Remove the @nozbe/sqlite dependency and lockfile entry.
  • Document the bundled version, source verification, and update procedure.
File Description
yarn.lock Removes the @nozbe/sqlite lockfile entry.
package.json Removes the @nozbe/sqlite dependency.
native/​android-jsi/​src/​main/​cpp/​CMakeLists.txt Builds local SQLite sources and updates dependency discovery.
native/​android-jsi/​sqlite/​sqlite3ext.h Adds the bundled SQLite extension API header.
native/​android-jsi/​sqlite/​README.md Records the version, checksum, source, and update steps.
docs-website/​docs/​docs/​Advanced/​ProTips.md Points users to the bundled-version README.
CHANGELOG-Unreleased.md Announces the Android JSI SQLite upgrade.

Review effort: Balanced

Comment thread package.json Resolved
The Windows project also compiled SQLite from @nozbe/sqlite, so removing
the dependency broke it. The source moves to native/sqlite, which both
the Android JSI and the Windows project build from, and the build script
copies it into the package.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 2, 2026 15:11

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The Windows Store project still defines SQLITE_OS_WINRT instead of SQLITE_UWP, causing the new SQLite amalgamation to select desktop-only APIs and break the UWP build.

1 open issue
1 resolved since last review

Review effort: Balanced

Current SQLite selects its Windows Store code paths with SQLITE_UWP
instead of SQLITE_OS_WINRT, so with 3.54.1 the project would use
desktop-only APIs. Both are defined, so a project pointing
WatermelonSqliteDir at an older SQLite still works.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 2, 2026 15:37

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The SQLite upgrade and SQLITE_UWP changes lack Windows/UWP build and adapter validation, while Windows CI is disabled.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

SQLITE_UWP was introduced in SQLite 3.54.0, not right after 3.46.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 2, 2026 15:54

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Windows/UWP compatibility requires MSVC and Windows SDK validation because CreateFileW and AreFileApisANSI are referenced unconditionally, reported MinGW checks fail, and Windows CI is disabled.

No open issues 🎉

Review effort: Balanced

@joker-777

Copy link
Copy Markdown
Author

Checked the Windows Store build with Microsoft's compiler: a one-off job on windows-2022 (MSVC 14.44, Windows SDK 10.0.26100) compiled native/sqlite/sqlite3.c with /DWINAPI_FAMILY=WINAPI_FAMILY_APP, the mode WatermelonDB.vcxproj builds in (AppContainerApplication). Run, scratch branch since deleted.

  • With SQLITE_UWP (this PR): fails. CreateFileW and AreFileApisANSI are undeclared.
  • Without it: fails on the same two plus HeapValidate.

So SQLite 3.54.1 does not compile for this Windows Store project in either configuration, while 3.46.0 does. MinGW reports the same. Not fixed yet; the options are being discussed.

🤖 Generated with Claude Code

A Windows SDK build in Windows Store mode fails on CreateFileW and
AreFileApisANSI, which SQLite 3.54.1 calls even with SQLITE_UWP. The
Windows port is unmaintained and not built on CI, so the bundled SQLite
is shared anyway, and the breakage is listed under breaking changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 2, 2026 16:21
Only 3.46.0 and 3.54.1 were checked.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The bundled SQLite migration has no active review issues.

No open issues 🎉

Review effort: Balanced

SQLite 3.53 dropped its CreateFile2 path, and 3.54 still calls the
desktop-only CreateFileW and AreFileApisANSI with SQLITE_UWP. A header
force-included only into sqlite3.c routes CreateFileW to CreateFile2 with
SQLite 3.46's flag mapping and makes AreFileApisANSI return true, as 3.46
did, so sqlite3.c stays unmodified. The breaking-change note goes away.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 2, 2026 16:43

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The Windows project and UWP file-API shim need maintainer validation through a full react-native-windows build and Store-sandbox database/WAL testing, as existing evidence covers only standalone compilation and desktop smoke tests.

No open issues 🎉

Review effort: Balanced

@joker-777
joker-777 merged commit 4dd7442 into kenhub Nov 2, 2026
8 checks passed
@joker-777
joker-777 deleted the deps/sqlite-3.54.1 branch November 2, 2026 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants