Skip to content

firestore_fn builds a new firestore_v1.Client (and calls google.auth.default()) on every event delivery #309

Description

@johanstoevlbaek

Version: firebase-functions 0.5.0 (same code on main / 0.6.0), Python 3.14, Cloud Run functions gen2, google-auth 2.53.0

What happens

firebase_functions/firestore_fn.py::_firestore_endpoint_handler does, per delivery:

if _DEFAULT_APP_NAME not in _apps:
    initialize_app()
app = get_app()
firestore_client = _firestore_v1.Client(project=app.project_id, database=event_database)

google.cloud.firestore_v1.Client(...) with no credentials= calls google.auth.default(),
which on Cloud Run pings the metadata server (google.auth.compute_engine._metadata.ping).
That ping only retries on TransportError; any non-200 + Metadata-Flavor: Google answer
returns False immediately, and gen2's sandbox has no DMI fallback, so the call raises
DefaultCredentialsError — before the user's handler runs and before any user-level error
handling can see it.

We observed this on a warm instance that had been serving for three hours and had served
nine events in the previous thirty seconds: four concurrent deliveries at
2026-08-28T21:40:13Z all died in the framework with

File "firebase_functions/firestore_fn.py", line 145, in _firestore_endpoint_handler
    firestore_client = _firestore_v1.Client(project=app.project_id, database=event_database)
...
File "google/auth/_default.py", line 748, in default
    raise exceptions.DefaultCredentialsError(_CLOUD_SDK_MISSING_CREDENTIALS)

while firebase_admin in the same process already held perfectly good cached credentials.
Thirty seconds later the same instance logged a TLS EOF talking to an unrelated host, so the
trigger was a brief network hiccup on that instance — but the per-event credential lookup is
what turned it into four dropped events (the trigger had no retry policy).

Why it matters

  • The client is only used to decode the snapshot (_firestore_helpers.decode_dict(..., firestore_client) and DocumentReference); it never talks to Firestore for that. A
    credential lookup per event is pure overhead and a per-event failure point.
  • The failure happens before user code, so retry on the trigger is the only mitigation
    available to users.

Suggested fix

Reuse the admin app's client (firebase_admin.firestore.client(app, database_id=...)) or
cache one firestore_v1.Client per (project, database) at module level, so
google.auth.default() runs once per process instead of once per event. Passing
credentials=app.credential.get_credential() into the Client constructor would also avoid
the per-event lookup.

Related gap (worth a second issue or the same one)

FirestoreOptions._endpoint hardcodes retry=False and FirestoreOptions does not accept a
retry keyword (TypeError: FirestoreOptions.__init__() got an unexpected keyword argument 'retry'), even though EventHandlerOptions.retry exists and the Node SDK exposes retry on
Firestore triggers. The only way to get an Eventarc retry policy from Python today is to
mutate fn.__firebase_endpoint__.eventTrigger["retry"] after decoration.

Repro

Any Firestore trigger; make the metadata server return a non-200 for one request (or block
169.254.169.254 briefly) on a warm instance and deliver an event. The handler never runs.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions