Skip to content

[bigquery-firestore-export kit] An adopted TRANSFER_CONFIG_NAME keeps its old notification topic, so no output reaches Firestore #2970

Description

@IzaakGough

The bigquery-firestore-export kit (@firebase/bigquery-firestore-export, on the kits branch) lets you adopt an existing BigQuery Data Transfer scheduled query by setting TRANSFER_CONFIG_NAME. An adopted query gets recorded in Firestore, but its output never reaches Firestore.

Cause

handleUpsertTransferConfig in kits/bigquery-firestore-export/src/handlers.ts takes the transferConfigName branch, does a read-only getTransferConfig, stores the result and returns before reaching updateTransferConfig. Only the update path sets notification_pubsub_topic, so the adopted query keeps publishing wherever it always did, while processMessages listens on the topic the kit created.

What happens

Adopted a config on a live project and ran it. Its notificationPubsubTopic and updateTime were unchanged, the run succeeded and wrote its BigQuery destination table, the notification arrived on the old topic, processMessages was never invoked, and the runs subcollection stayed empty. Nothing logs an error, so it looks like a working install.

There is a second effect when migrating off an extension instance: uninstalling deletes the instance's ext- topic, so an adopted config that was pointed at it would publish to a topic that no longer exists.

Fix

Repoint the notification topic on the link path, or refuse to link a config whose topic does not match. #2960 takes the first option, sending an update masked to notification_pubsub_topic alone and skipping it when the topic already matches, so the adopted query's own settings are left alone.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    type: bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions