Skip to content

feat(firestore-bigquery-export): add the sync task enqueue module - #3128

Open
cabljac wants to merge 1 commit into
chore/kits-fbe-admin-14from
feat/kits-fbe-sync-task-module
Open

feat(firestore-bigquery-export): add the sync task enqueue module#3128
cabljac wants to merge 1 commit into
chore/kits-fbe-admin-14from
feat/kits-fbe-sync-task-module

Conversation

@cabljac

@cabljac cabljac commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Second layer of the Cloud Tasks write buffer stack. Adds src/tasks.ts with enqueueSyncTask and syncQueuePath, and exports SerializedDocumentChange from the handlers as the task payload type. Nothing calls the module yet; the write path is wired in a later layer.

The queue path is deliberately unprefixed so firebase-admin resolves the kit instance prefix. The region comes from the CLI-set FUNCTION_REGION first, with DATABASE_REGION mapped through the dual-region table as the fallback for local runs, and a missing region throws. Enqueue retries in-process with exponential backoff and jitter, capped at 5s, and clamps the attempt budget to at least one. The task id is derived from the event id, so a retried enqueue of an event already in Cloud Tasks returns functions/task-already-exists, which is treated as success rather than buffering the event twice.

tests/tasks.test.ts pins all of the above with a mocked firebase-admin/functions. 75 tests pass. No live queue has been exercised.

Part of #3031.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new tasks module to handle enqueuing Firestore document changes to a BigQuery sync task queue with exponential backoff and jitter, along with exporting the SerializedDocumentChange interface and adding comprehensive tests. The review feedback suggests randomizing the jitter per retry attempt rather than once per task to prevent thundering herd issues, and highlights that passing Infinity for maxAttempts silently defaults to no retries due to a strict integer check.

Comment thread kits/firestore-bigquery-export/src/tasks.ts
Comment thread kits/firestore-bigquery-export/src/tasks.ts
Comment thread kits/firestore-bigquery-export/src/tasks.ts
@cabljac
cabljac force-pushed the feat/kits-fbe-sync-task-module branch from d07ada3 to 408c6ee Compare September 7, 2026 16:45
@cabljac
cabljac marked this pull request as ready for review September 7, 2026 16:50
@cabljac cabljac mentioned this pull request Sep 7, 2026
62 tasks
Adds enqueueSyncTask and syncQueuePath ahead of wiring them into the
write path. The queue path is left unprefixed so firebase-admin resolves
the kit instance prefix, and the region comes from FUNCTION_REGION with
DATABASE_REGION as the local fallback. Enqueue retries in-process with
bounded backoff, treats task-already-exists as success, and clamps the
attempt budget to at least one.

SerializedDocumentChange is exported as the task payload type.
@cabljac
cabljac force-pushed the feat/kits-fbe-sync-task-module branch from 408c6ee to 7ca16c1 Compare September 7, 2026 17:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant