Batch group meta data updates in a single DB transaction - #357
Open
jolavillette wants to merge 1 commit into
Open
Batch group meta data updates in a single DB transaction#357jolavillette wants to merge 1 commit into
jolavillette wants to merge 1 commit into
Conversation
processGrpMetaChanges() wrote each queued group meta update with its own call to updateGroupMetaData(), i.e. one implicit SQLCipher transaction and one fsync per entry. One such write was measured at ~1 s, and services accumulating many updates (typically identity usage stamps at startup, ~3700 identity groups) froze their tick thread for as long as 86 s while draining the queue, blocking all other GXS activity of the service through mGenMtx. Mirror the existing message-side batching: add a vector variant of updateGroupMetaData() that wraps all row updates in a single transaction (one fsync total), and make processGrpMetaChanges() collect the entries that pass their mask and write them in one call. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
jolavillette
force-pushed
the
fix/gxs-grpmeta-batch-write
branch
from
August 6, 2026 23:17
dd4bdea to
4efb9cb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
processGrpMetaChanges() wrote each queued group meta update in its own implicit SQLCipher transaction — one fsync per entry, measured at ~1 s each. Services accumulating many updates (typically identity usage stamps at startup, with a few thousand identity groups) froze their tick thread for up to 86 s while draining the queue.
This adds a vector variant of updateGroupMetaData() that wraps the whole batch in one transaction (same pattern as the message side), and makes processGrpMetaChanges() write all entries in one call. The same backlog is now absorbed in about a second.
🤖 Generated with Claude Code