feat(firestore-bigquery-export): add the sync task enqueue module - #3128
Open
cabljac wants to merge 1 commit into
Open
feat(firestore-bigquery-export): add the sync task enqueue module#3128cabljac wants to merge 1 commit into
cabljac wants to merge 1 commit into
Conversation
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces a new tasks module to handle enqueuing Firestore document changes to a BigQuery sync task queue with exponential backoff and jitter, along with exporting the SerializedDocumentChange interface and adding comprehensive tests. The review feedback suggests randomizing the jitter per retry attempt rather than once per task to prevent thundering herd issues, and highlights that passing Infinity for maxAttempts silently defaults to no retries due to a strict integer check.
cabljac
force-pushed
the
feat/kits-fbe-sync-task-module
branch
from
September 7, 2026 16:45
d07ada3 to
408c6ee
Compare
cabljac
marked this pull request as ready for review
September 7, 2026 16:50
Adds enqueueSyncTask and syncQueuePath ahead of wiring them into the write path. The queue path is left unprefixed so firebase-admin resolves the kit instance prefix, and the region comes from FUNCTION_REGION with DATABASE_REGION as the local fallback. Enqueue retries in-process with bounded backoff, treats task-already-exists as success, and clamps the attempt budget to at least one. SerializedDocumentChange is exported as the task payload type.
cabljac
force-pushed
the
feat/kits-fbe-sync-task-module
branch
from
September 7, 2026 17:26
408c6ee to
7ca16c1
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Second layer of the Cloud Tasks write buffer stack. Adds
src/tasks.tswithenqueueSyncTaskandsyncQueuePath, and exportsSerializedDocumentChangefrom the handlers as the task payload type. Nothing calls the module yet; the write path is wired in a later layer.The queue path is deliberately unprefixed so firebase-admin resolves the kit instance prefix. The region comes from the CLI-set
FUNCTION_REGIONfirst, withDATABASE_REGIONmapped through the dual-region table as the fallback for local runs, and a missing region throws. Enqueue retries in-process with exponential backoff and jitter, capped at 5s, and clamps the attempt budget to at least one. The task id is derived from the event id, so a retried enqueue of an event already in Cloud Tasks returnsfunctions/task-already-exists, which is treated as success rather than buffering the event twice.tests/tasks.test.tspins all of the above with a mockedfirebase-admin/functions. 75 tests pass. No live queue has been exercised.Part of #3031.