Version: firebase-functions 0.5.0 (same code on main / 0.6.0), Python 3.14, Cloud Run functions gen2, google-auth 2.53.0
What happens
firebase_functions/firestore_fn.py::_firestore_endpoint_handler does, per delivery:
if _DEFAULT_APP_NAME not in _apps:
initialize_app()
app = get_app()
firestore_client = _firestore_v1.Client(project=app.project_id, database=event_database)
google.cloud.firestore_v1.Client(...) with no credentials= calls google.auth.default(),
which on Cloud Run pings the metadata server (google.auth.compute_engine._metadata.ping).
That ping only retries on TransportError; any non-200 + Metadata-Flavor: Google answer
returns False immediately, and gen2's sandbox has no DMI fallback, so the call raises
DefaultCredentialsError — before the user's handler runs and before any user-level error
handling can see it.
We observed this on a warm instance that had been serving for three hours and had served
nine events in the previous thirty seconds: four concurrent deliveries at
2026-08-28T21:40:13Z all died in the framework with
File "firebase_functions/firestore_fn.py", line 145, in _firestore_endpoint_handler
firestore_client = _firestore_v1.Client(project=app.project_id, database=event_database)
...
File "google/auth/_default.py", line 748, in default
raise exceptions.DefaultCredentialsError(_CLOUD_SDK_MISSING_CREDENTIALS)
while firebase_admin in the same process already held perfectly good cached credentials.
Thirty seconds later the same instance logged a TLS EOF talking to an unrelated host, so the
trigger was a brief network hiccup on that instance — but the per-event credential lookup is
what turned it into four dropped events (the trigger had no retry policy).
Why it matters
- The client is only used to decode the snapshot (
_firestore_helpers.decode_dict(..., firestore_client) and DocumentReference); it never talks to Firestore for that. A
credential lookup per event is pure overhead and a per-event failure point.
- The failure happens before user code, so
retry on the trigger is the only mitigation
available to users.
Suggested fix
Reuse the admin app's client (firebase_admin.firestore.client(app, database_id=...)) or
cache one firestore_v1.Client per (project, database) at module level, so
google.auth.default() runs once per process instead of once per event. Passing
credentials=app.credential.get_credential() into the Client constructor would also avoid
the per-event lookup.
Related gap (worth a second issue or the same one)
FirestoreOptions._endpoint hardcodes retry=False and FirestoreOptions does not accept a
retry keyword (TypeError: FirestoreOptions.__init__() got an unexpected keyword argument 'retry'), even though EventHandlerOptions.retry exists and the Node SDK exposes retry on
Firestore triggers. The only way to get an Eventarc retry policy from Python today is to
mutate fn.__firebase_endpoint__.eventTrigger["retry"] after decoration.
Repro
Any Firestore trigger; make the metadata server return a non-200 for one request (or block
169.254.169.254 briefly) on a warm instance and deliver an event. The handler never runs.
Version: firebase-functions 0.5.0 (same code on
main/ 0.6.0), Python 3.14, Cloud Run functions gen2, google-auth 2.53.0What happens
firebase_functions/firestore_fn.py::_firestore_endpoint_handlerdoes, per delivery:google.cloud.firestore_v1.Client(...)with nocredentials=callsgoogle.auth.default(),which on Cloud Run pings the metadata server (
google.auth.compute_engine._metadata.ping).That ping only retries on
TransportError; any non-200 + Metadata-Flavor: Googleanswerreturns
Falseimmediately, and gen2's sandbox has no DMI fallback, so the call raisesDefaultCredentialsError— before the user's handler runs and before any user-level errorhandling can see it.
We observed this on a warm instance that had been serving for three hours and had served
nine events in the previous thirty seconds: four concurrent deliveries at
2026-08-28T21:40:13Zall died in the framework withwhile
firebase_adminin the same process already held perfectly good cached credentials.Thirty seconds later the same instance logged a TLS EOF talking to an unrelated host, so the
trigger was a brief network hiccup on that instance — but the per-event credential lookup is
what turned it into four dropped events (the trigger had no retry policy).
Why it matters
_firestore_helpers.decode_dict(..., firestore_client)andDocumentReference); it never talks to Firestore for that. Acredential lookup per event is pure overhead and a per-event failure point.
retryon the trigger is the only mitigationavailable to users.
Suggested fix
Reuse the admin app's client (
firebase_admin.firestore.client(app, database_id=...)) orcache one
firestore_v1.Clientper(project, database)at module level, sogoogle.auth.default()runs once per process instead of once per event. Passingcredentials=app.credential.get_credential()into theClientconstructor would also avoidthe per-event lookup.
Related gap (worth a second issue or the same one)
FirestoreOptions._endpointhardcodesretry=FalseandFirestoreOptionsdoes not accept aretrykeyword (TypeError: FirestoreOptions.__init__() got an unexpected keyword argument 'retry'), even thoughEventHandlerOptions.retryexists and the Node SDK exposesretryonFirestore triggers. The only way to get an Eventarc retry policy from Python today is to
mutate
fn.__firebase_endpoint__.eventTrigger["retry"]after decoration.Repro
Any Firestore trigger; make the metadata server return a non-200 for one request (or block
169.254.169.254briefly) on a warm instance and deliver an event. The handler never runs.