Repository navigation
Conversation
markLocalChangesAsSynced looked up each pushed record with affectedRecords.find(), and applying pulled changes checked every created and updated record with locallyDeletedIds.includes(). Both were O(n²) in the number of local changes. Records are now indexed by table and id in a Map, and the locally deleted ids are kept in a Set. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
Hi @joker-777, thank you, and I really like the Proxy trick for testing this without relying on timing! I maintain NitromelonDB, a maintained fork of WatermelonDB (New Architecture, Nitro Modules). We've done a fair bit of sync performance work there, but both of these are still quadratic: Our fork was rewritten in TypeScript (strict mode) to reduce the chance of bugs slipping through, so I think the community would really benefit if you opened your PRs against it too. If you don't have the time or would rather not, no worries at all, I'm happy to port it myself. Just let me know. Thanks a lot! |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two sync steps were quadratic in the number of local changes. This makes them linear. No behaviour change.
Changes
markLocalChangesAsSyncedlooked up each pushed record withaffectedRecords.find(). Affected records are now indexed by table and id in aMaplocallyDeletedIds.includes(). Locally deleted ids are now aSet, built once per tablesrc/sync/already uses Sets and Maps for its lookups. Not changed:addToRawSet(per record, bounded by the column count) and LokiJS adapter internalsTests
Two new Jest tests count array element reads through a
Proxyinstead of timing, so they aren't flaky: 2000 pushed records, and 1000 pulled updates to locally deleted records. Master needs 2,001,000 and 501,500 reads; this branch stays under 4000 and 3000.Verification
sqlite-nodesuite was left out because better-sqlite3 does not build on the host's Node; this change does not touch the adapters).flow checkis clean, and ESLint is clean on the changed filesapplyRemote.jsand its tests in separate places. They merge into master together without conflicts, and Jest passes on the result🤖 Generated with Claude Code